I read an article today where an Professor Barbara Oakley, an engineering professor, argues that research in education is not replicable and that much research in social science is at crisis point. The major cause of the crisis is that this research does not follow ‘the scientific method’. She describes the difficulty that some researchers have when trying to replicate research that has already been undertaken; they are often unable to. Another problem that Professor Oakley highlights is that researchers are often unwilling to share their raw data and appear to want others to accept their findings on ‘trust’.
I take issue with the first section of the ‘crisis’ point but can support the second point. It lies in the ontology of the world and the philosophy behind the research. I will have my grumble about the ‘scientific method’ and ‘problem of replicability’ and then I’ll have a think about the publication of data.
I’m going to start off this thought process with the fact that I have a Master’s degree in mechanical engineering, so I do very much understand the concept of ‘scientific method’ and the importance of replicability in the physical and engineering sciences. I will add to this the peculiar direction that my academics took later on in life with the fact that I have a PhD in social sciences, where I focused on education and I actually specialised in qualitative research having always thought that I would be a quantitative person.
In physical and engineering sciences, understanding how to replicate research is key to developing understanding of the physical world and modelling behaviour, often mathematically. These types of research don’t tend to deal in people, where free will and agency apply! The type of data generated is often based in a positivist paradigm, where you consider things that can be measures and often quantified. Physical and engineering sciences do this and do it well. Methods need to be transparent so that others can access them and develop them.
The importance of transparency of method is equally important in social sciences. For quantitative research within social sciences, I agree that the notions of replicability and scientific method are important because of the nature of data that is sought. However, not all research based within a quantitative, positivist paradigm. My PhD research is not replicable, does not follow a traditional experimental scientific method and is absolutely subjective. However, I do not claim that the data is replicable. Much social science research is undertaken within a qualitative framework, where participants are co-constructors of data rather than subjects who generate quantifiable responses. It doesn’t make the research less valid. It makes its purpose different and its application more nuanced.
I think Professor Oakley has somewhat missed a trick with her comments on this subject. The different types of research that take place within social sciences use very difference philosophical bases for their methodologies and tend to serve different purposes, both of which are equally important. Some research should be replicable, and some research won’t be by its very nature. On the point that raw data should be made available, I agree. That way, whatever the research paradigm of the researchers, others can explore their methods and data, with the potential to undertake meta-analyses and run their own investigations.
I think having transparency of method, data and interpretation is key; replicability not always.
