Category Archives: Research methodology

Rethinking Behavioural Science Research

The past couple of years have been painful for Social Sciences, with the replicability crisis putting a dent on the credibility of multiple studies in the field – from social priming effects to power poses and will power. An effort to reproduce effects reported in more than 100 cognitive and social psychology studies in three journals, called the Reproducibility Project, has found that findings from around 60 studies do not hold up when retested. Even when effects were replicated, they were weaker than reported in the original studies.

The replicability debate has been focussed, to a large extent, on experimental design and effect sizes. It is suggested that low-power research designs (smaller sample sizes) and lower or weaker effect size studies were more likely unable to be replicated. Additionally, an inherent bias in publication favouring positive results is argued to contribute towards the replication crisis.

An often overlooked part of the discussion seems to be the social context of the experiment and it’s effect on the participants themselves. Currently, academic researchers are sticklers for controlled design, this way the effects of multiple factors on behaviour can be reduced to just one. In view of this, in most universities, the research lab, usually cubicles/ computer laboratory is a heavily controlled, isolated environment. Having a controlled physical environment, however, does not preclude the participants from coming in to the research with their own motivations, dispositions, expectations and emotions. These cannot be dismissed as irrelevant to the study at hand just because the study has been stripped of any context. On the other hand, they exert a large influence on outcomes of the study.

For example, aspects of the experimental setting can influence the participants’ reaction to stimuli presented by the experimenter. Participants in psychology studies get paid, and are motivated to play the role of ‘good’ subjects – ascribe to what they think the experimenter wants – these are termed ‘Demand Effects’.  Participants consciously try to recreate experimenters’ hypotheses using available cues. Any psychology experimenter will attest to this fact. As a student, when I conducted my research on Automatic Priming, I used the same testing protocol – picked solitary computer terminals, used a confederate to trick participants into believing they were engaged in two separate studies – one to deploy the priming intervention (‘professor’ versus ‘hooligan’) and another to study the effect it had on knowledge (IQ test). We did probe participants on what they thought the experiment was about and so on, but at the end of the day, the truth is that most participants had their own hypotheses about what we were trying to prove and played up to their hypotheses. Experimenters themselves unwittingly influence participants with their expectations – which participants want to play up to, dubbed ‘Experimenter Effects’.

Psychology is the study of human behaviour – in our anxiety to ensure that it is a strict science, we are using the same experimental models that we use to study physics to study human behaviour. It is time psychology experiments stop treating participants as passive receptors of stimuli. What we want to study are the motivations, the emotions, the beliefs and dispositions for different contexts – why try to make the participants leave those behind at home (which they won’t anyway). Our research will be richer if we simulate the real-life context that we are trying to study, rather than control for it, so the decisions and outcomes of research will be closer to home.

Research at FinalMile attempts todo just this. With our EthnoLab, we simulate real-life contexts as far as possible – we want the decisions in the Ethnolab to reflect decisions taken in real life, not create an alien context which leads to perceived ‘correct answers’. This might mean recreating the real-life environment – either physically, or virtually. The EthnoLab marries the practicality of a controlled laboratory with the ‘real-life’ness of Ethnography. As Smith and Semin (2004) put it :  “The true strength of the laboratory is not its supposed insulation of behavior from context effects, but its flexibility in allowing experimenters to construct very different types of contexts, suited to test different types of hypotheses.” Welcome to Behavioural research v2.0!

Image Credit: