Replication study on sentence comprehension: what role does the frontal lobe play?
What role does the frontal lobe play in understanding sentences? Assistant professor Jurriaan Witteman has received a grant to replicate a leading study on this question. ‘As there is some doubt about the reliability of existing neuroimaging results, it’s important to replicate influential studies.’
Witteman's research focuses on an influential study from 2004 by Hagoort et al. ‘In a sentence such as “I spread butter on my sock”, the frontal lobe lit up in an MRI scan,’ Witteman explains their findings. 'With the sentence “I spread butter on my bread”, this didn’t happen.' This discovery proves that the frontal lobe plays a role in the integration of sentence meanings.
However, the question is how reliable the results are. ‘MRI research is expensive,’ says Witteman. ‘As a result, small groups of test subjects are often used. In the original experiment, there were sixteen. With such small numbers, differences are often found that are purely coincidental.’
Since the original study, similar studies have been conducted on a regular basis. Sometimes a similar effect was found, but sometimes not. ‘We are therefore going to conduct a meta-analysis of all published studies to determine how robust the results are,’ says Witteman.
Publication bias
However, this is not expected to provide a definitive answer about the existence of the effect. Witteman: ‘Scientific journals prefer to publish articles in which the results correspond with what has been found previously.’
Due to this publication bias and the fact that the original experiment has never been repeated exactly, a meta-analysis alone is not sufficient. Witteman: ‘You are essentially summarising biased literature, which inevitably leads to a biased conclusion.’ He will therefore conduct a more precise version of the experiment, significantly expanding the group of test subjects to fifty people. In addition, an agreement will be made with a journal in advance that the results will be published, regardless of the outcome. ‘Prior to the study, we will specify exactly how we will approach the research. We will have this peer reviewed. Once we have approval, the journal will agree to publication, even if we don’t find confirmation.’
Collaborating with original researchers
Whether or not the original results are replicated, the research is interesting, according to Witteman. ‘If the original results are confirmed, the theory is supported more robustly, which is important. If that doesn’t happen, the theory will have to be adjusted. In both cases, it is an interesting finding and therefore particularly important that the experiment is repeated. Traditionally, as researchers we conduct new research, but because studies in neurolinguistics are almost never repeated, it is paradoxically innovative not to conduct new research but to replicate existing research. Most of the money still goes to new studies, but it is important that NWO is now also making a grant available for replication studies, so that we can better understand how robust the results actually are. Only then can you develop good theories that explain these findings. Otherwise, your theories are just theoretical sandcastles.’
The original researchers agree. ‘So far, I have always worked constructively with original researchers on replication studies,’ says Witteman. ‘This shows that they also recognise the importance of a robust factual basis. Otherwise, your science has no foundation.’