2023-10-20
See Borsboom et al. (2021). Theory Construction Methodology: A Practical Framework for Building Theories in Psychology. Perspectives on Psychological Science, 16(4), 756–766. https://doi.org/10.1177/1745691620969647
Phenomena: Stable and general features of the world in need of explanation. Can be understood as robust generalizations of patterns in empirical data. They are the explanatory targets for scientific theories. In psychology often called “effects” or “findings”.
Data: Relatively direct observations. Refer to particular empirical patterns in concrete data sets rather than empirical generalizations (which would be phenomenona).
See Borsboom et al. (2021). Theory Construction Methodology: A Practical Framework for Building Theories in Psychology. Perspectives on Psychological Science, 16(4), 756–766. https://doi.org/10.1177/1745691620969647
Data provide evidence for the existence of empirical phenomena: You generalize from one or more data sets with strong evidence to a general phenomenon.
Generalize to what? UTOS framework:
To claim a (robust) phenomenon, you ideally need:
UTOS framework from Cronbach & Shapiro (1982); for an update to “M-STOUT”, including mechanims and time, see Findley et al. (2021).
Probably most of psychological research is about establishing phenomena (disguised as “theories”).
Techniques used to detect data patterns:
See presentation of Denny Borsboom
Phenomena (once their existence has been established) predict similar data patterns in new data sets of the same operationalization (as in “direct replication”) and ideally also for new operationalizations (as in “conceptual replication”, changing more UTOS dimensions).
The risky shift phenomenon: A group’s decisions are riskier than the average of the individual decisions of members before the group met (i.e., the group discussion made individuals riskier).
See Westfall et al. (2015). Replicating Studies in Which Samples of Participants Respond to Samples of Stimuli. Perspectives on Psychological Science, 10(3), 390–399. https://doi.org/10.1177/1745691614564879
Questions for discussion (10 min.):
My take: It is a phenomenon (though a weak one), as it generalizes to new units (i.e., new participants) of the same operationalization. But the generalizability was much weaker than initially expected. It is a phenomenon of this specific stimulus set (outcome operationalization) and suggests certain types of research questions (e.g., “What is so specific to this stimulus set?”).
The concerns of the replication crisis typically referred to the relation between data and phenomena:
Doubts about phenomena propagate to theories: If there is no phenomenon to explain, any explanatory theory gets obsolete.
“We argue that a further cause of poor replicability is the often weak logical link between theories and their empirical tests. […] A strong link between theories and hypotheses is best achieved by formalizing theories as computational models.”
“Many of these proposals [i.e., method reforms] are helpful ideas for raising the standards of good research practice, primarily ensuring more trustworthy inferences on the empirical level of scientific inference — the level connecting observations to empirical generalizations. Our concern is that shoring up the strength of inferences on the empirical level does not by itself address deficits on the theory level — the level connecting empirical generalizations to theories.”Oberauer & Lewandowsky (2019)
Oberauer, K., & Lewandowsky, S. (2019). Addressing the theory crisis in psychology. Psychonomic Bulletin & Review, 26(5), 1596–1618. https://doi.org/10.3758/s13423-019-01645-2
Slide adapted from Karolin Salmen, CC-BY
See Eronen & Bringmann (2021)
Formal modeling in psychology - Empirisches Praktikum, Ludwig-Maximilians-Universität München