In the last few years, it has become so obvious...

In the last few years, it has become so obvious to me that weak theories are a huge problem in science. My own interests have shifted substantially from merely improving estimators for my favorite problems. I would much rather demonstrate what the right questions ought to be. 1/

What good are more precise estimators for models that we know are substantially biased away from the truth and incapable of answering the scientific question of interest? 2/

Incidentally, I was re-reading a bit about Lakatos today and happened to find an article by Meehl channeling Lakatos.
Statistical testing must have been the deep learning of the 40s-70s. 3/

Theorists like Meehl had to promote good theory building and point out that testing statistical hypotheses did not amount to testing scientific hypotheses 4/
meehl.umn.edu/sites/meehl.dl...

Quantitative researchers just assume that the domain experts are asking the right questions. They don't realize that all the years they've spent eliminating problems of mathematical underdetermination can be put to use in tackling other kinds of underdetermination in science. 5/

Some sciences seem to be super-duper empirical. They leave theory the building to the quants, while the quants think the scientists have strong theories.
A bit like what Lippmann told Poincaré about the normal law of errors.

6/

So, I always find it refreshing to see scientists emphasize theory. Borsboom made a point that psychology needs more theory in "Theoretical Amnesia" osc.centerforopenscience.org/category/misc6...
I really wish there were more like Kendler and Borsboom in psychiatry. 7/

I totally agree with the emphasis on the need for theory and the inadequacy of statistics in the recent preprint from @djnavarro @IrisVanRooij
psyarxiv.com/x36pz. (Although, the title didn't resonate with me as much several points made ) 8/

Preregistration only helps to ensure that the empirical error rate for a statistical procedure matches the nominal error rate, and that effect sizes in the literature are not overinflated. Certainly useful to ensure there is a stable observation worthy of explaining. 9/

In fields with very complex measurements like neuroimaging there are several pathways to Gettier replications — where one sees apparent replications of a really complicated result for the wrong reasons. Need theory to decide if replication was generated by the same mechanism 10/

Back when I was purely immersed in statistics, I was inspired by Eve Marder's work. (I couldn't resist quoting her in my thesis defense!)
She obsesses over issues of underdetermination in neuronal functional connectivity and lack of theory journals.plos.org/plosbiology/ar... 11/