Link Search Menu Expand Document

Operationalising Moral Foundations Theory

In order to use Moral Foundations Theory to identify and explain cultural differences, we need a way to measure individual variations in how moral judgements are made. The Moral Foundations Questionnaire aims to fulfill this need.

By the end of this section you should know what the Moral Foundations Questionnaire is and how attempts have been made to validate it. You should also be aware of some objections to its use as a tool for identifying cultural differences.

If the slides are not working, or you prefer them full screen, please try this link.

Notes

Read the next section, Moral Foundations Theory: An Approach to Cultural Variation, first if you are reading this outside the lecture.

According to (Feinberg & Willer, 2013), researchers have found evidence that Moral Foundations Theory is true. What is this evidence?

The first step towards finding evidence is to operationalise the theory. To this end, Atari et al. (2023) developed a Moral Foundations Questionnaire (called MFQ-2). (This is the successor to Haidt & Graham (2007)’s original Moral Foundations Questionnaire, which can be found in Graham et al. (2011). For reasons we’ll get into in below, the first questionnaire did not entirely succeed.)

For each foundation, there are a number of questions. Here is an illustration of some of them:

‘For each of the statements below, please indicate how well each statement describes you or your opinions. Response options: Does not describe me at all (1); slightly describes me (2); moderately describes me (3); describes me fairly well (4); and describes me extremely well (5).’

‘We should all care for people who are in emotional pain.’

‘I think the human body should be treated like a temple, housing something sacred within.’

‘It makes me happy when people are recognized on their merits.’

‘Everyone should feel proud when a person in their community wins in an international competition.’

‘I believe that one of the most important values to teach children is to have respect for authority.’

You can see the full questionnaire in Atari et al. (2023).

If there are (at least) six moral foundations, and if the various questions represent those foundations, then we can predict that the questionnaire will exhibit:

  • internal validity (roughly, are the patterns in subjects’ answers consistent with the theory that they are answering on the basis of five foundations?[1]);
  • test-retest reliability (are individuals likely to give the same answers at different times); and
  • predictive power (roughly, are subjects’ answers on other questionnaires correlated with the conceptually related foundations?).
  • measurement invariance (roughly, if you compare two groups’ answers, do differences in their answers indicate meaningful differences between the groups rather than merely differences in the way they interpret the questions)

Atari et al. (2023) give evidence of internal validity for MFQ-2, which is further supported by Dogruyol et al. (2024). Atari et al. (2023) also offer good evidence of predictive power for MFQ-2. In Study 2, these researchers also demonstrate that the questionnaire (MFQ-2) does exhibit scalar invariance for five of the six foundations—the exception is purity (p. 1167). This indicates that the questionnaire can be used to compare the mean strengths of emphasis on foundations between different populations.

Because MFQ-2 is relatively new, there is limited research using it. For now we should be cautious in accepting the results on the few studies using it. (Note that the original moral foundations researchers would disagree: they regard the )

At this point you should understand (1) how research based on a questionnaire could provide a range of evidence in support of predictions generated by Moral Foundations Theory; and (2) have a preliminary understanding of the evidence obtained using MFQ-2.

A Complication: MFQ-1 vs MFQ-2

Earlier (pre-2023) research on Moral Foundations Theory used Haidt & Graham (2007)’s original Moral Foundations Questionnaire, which is now called MFQ-1. (You can find MFQ-1 in Graham et al. (2011).)

Although very widely used (even in some work published after 2023), MFQ-1 does not appear to be a reliable too.

One problem is internal validity. Although MFQ-1 did pass tests of internal validity in various countries (Graham et al., 2011; Yilmaz, Harma, Bahçekapili, & Cesur, 2016), there were several exceptions. Iurino & Saucier (2020) collected new samples across 27 countries but ‘we were not able to replicate Graham et al.’s (2011) results indicating that a five-factor model is a suitable approach to modelling the moral foundations’ (p. 6). Relatedly, Harper & Rhodes (2021) failed to find the five factor structure in a sample from the UK.

The second is a failure to demonstrate measurement invariance. Without measurment invariance, we are not justified in using a questionnaire to compare two groups. We are particularly interested in one kind of measurement invariance, scalar invariance, as this would justify using the Moral Foundations Questionnaire to compare mean scores on a foundation.[2] That is, it would justify us in drawing conclusions like ‘conservatives put more weight on purity than liberals’.[3] Attempts to establish the scalar invariance of MFQ-1 have been unsuccessful (Davis et al., 2016; Doğruyol, Alper, & Yilmaz, 2019; Davis, Dooley, Hook, Choe, & McElroy, 2017; Iurino & Saucier, 2020, p. Table 4). One illustration of this is a failed attempt to compare US and Iranian participants:

‘Iranians and Americans do not interpret MFQ items in nearly similar ways, [...] means cannot be meaningfully compared.’ (Atari, Graham, & Dehghani, 2020, p. 373)

Failure of the original Moral Foundations Questionnaire to exhibit scalar invariance may be due in part to lack of diversity in the sample used to develop it:

‘Items of the MFQ [Moral Foundations Questionnaire] were refined on the basis of a sample with participants from a variety of countries, but the sample was predominately White (i.e., 87%). Furthermore, the sample involved people who visited the team’s website, which inevitably involves some selection bias, potentially associated with ideological background’ (Davis et al., 2017, p. 128; compare Kivikangas, Fernández-Castilla, Järvelä, Ravaja, & Lönnqvist, 2021, p. 84).

In the lecture we also consider how some items on MFQ-1 are likely to provide different reactions from different groups for reasons that have nothing to do with their ethical attitudes.

Overall, we should be extremely cautious about drawing conclusions about cultural variation from results obtained with the original Moral Foundations Questionnaire (MFQ-1) alone. Is there reason to trust them? If so, what is the reason?

How MFQ-2 Differs from MFQ-1

The new questionnaire is based on six rather than five foundations. The change is essentially to split what was previously Fairness into two things: Equality (which concerns equal treatment) and Proportionality (which concerns being rewarded in proportion to one’s contribution).[4]

Glossary

Moral Foundations Theory : The theory that moral pluralism is true; moral foundations are innate but also subject to cultural learning, and the Social Intuitionist Model of Moral Judgement is correct (Graham et al., 2019). Proponents often claim, further, that cultural variation in how these innate foundations are woven into ethical abilities can be measured using the Moral Foundations Questionnare (Graham et al., 2009; Graham et al., 2011). Some empirical objections have been offered (Davis et al., 2016; Davis et al., 2017; Doğruyol et al., 2019). See Moral Foundations Theory: An Approach to Cultural Variation.
Social Intuitionist Model of Moral Judgement : A model on which intuitive processes are directly responsible for moral judgements (Haidt & Bjorklund, 2008). One’s own reasoning does not typically affect one’s own moral judgements, but (outside philosophy, perhaps) is typically used only to provide post-hoc justification after moral judgements are made. Reasoning does affect others’ moral intuitions, and so provides a mechanism for cultural learning.

References

Atari, M., Graham, J., & Dehghani, M. (2020). Foundations of morality in Iran. Evolution and Human Behavior, 41(5), 367–384. https://doi.org/10.1016/j.evolhumbehav.2020.07.014
Atari, M., Haidt, J., Graham, J., Koleva, S., Stevens, S. T., & Dehghani, M. (2023). Morality beyond the WEIRD: How the nomological network of morality varies across cultures. Journal of Personality and Social Psychology, 125(5), 1157–1188. https://doi.org/10.1037/pspp0000470
Davis, D., Dooley, M., Hook, J., Choe, E., & McElroy, S. (2017). The Purity/Sanctity Subscale of the Moral Foundations Questionnaire Does Not Work Similarly for Religious Versus Non-Religious Individuals. Psychology of Religion and Spirituality, 9(1), 124–130. https://doi.org/10.1037/rel0000057
Davis, D., Rice, K., Tongeren, D. V., Hook, J., DeBlaere, C., Worthington, E., & Choe, E. (2016). The Moral Foundations Hypothesis Does Not Replicate Well in Black Samples. Journal of Personality and Social Psychology, 110(4). https://doi.org/10.1037/pspp0000056
Doğruyol, B., Alper, S., & Yilmaz, O. (2019). The five-factor model of the moral foundations theory is stable across WEIRD and non-WEIRD cultures. Personality and Individual Differences, 151, 109547. https://doi.org/10.1016/j.paid.2019.109547
Dogruyol, B., Velioglu, İ., Bayrak, F., Acem, E., Isler, O., & Yilmaz, O. (2024). Validation of the moral foundations questionnaire-2 in the Turkish context: Exploring its relationship with moral behavior. Current Psychology, 43(29), 24438–24452. https://doi.org/10.1007/s12144-024-06097-z
Feinberg, M., & Willer, R. (2013). The Moral Roots of Environmental Attitudes. Psychological Science, 24(1), 56–62. https://doi.org/10.1177/0956797612449177
Graham, J., Haidt, J., Motyl, M., Meindl, P., Iskiwitch, C., & Mooijman, M. (2019). Moral Foundations Theory: On the advantages of moral pluralism over moral monism. In K. Gray & J. Graham (Eds.), Atlas of Moral Psychology. New York: Guilford Publications.
Graham, J., Haidt, J., & Nosek, B. A. (2009). Liberals and conservatives rely on different sets of moral foundations. Journal of Personality and Social Psychology, 96(5), 1029–1046. https://doi.org/10.1037/a0015141
Graham, J., Nosek, B. A., Haidt, J., Iyer, R., Koleva, S., & Ditto, P. H. (2011). Mapping the moral domain. Journal of Personality and Social Psychology, 101(2), 366–385. https://doi.org/10.1037/a0021847
Gregorich, S. E. (2006). Do Self-Report Instruments Allow Meaningful Comparisons Across Diverse Population Groups? Testing Measurement Invariance Using the Confirmatory Factor Analysis Framework. Medical Care, 44(11), S78–S94.
Haidt, J., & Bjorklund, F. (2008). Social intuitionists answer six questions about moral psychology. In W. Sinnott-Armstrong (Ed.), Moral psychology, Vol 2: The cognitive science of morality: Intuition and diversity (pp. 181–217). Cambridge, Massachusetts: MIT press.
Haidt, J., & Graham, J. (2007). When Morality Opposes Justice: Conservatives Have Moral Intuitions that Liberals may not Recognize. Social Justice Research, 20(1), 98–116. https://doi.org/10.1007/s11211-007-0034-z
Haidt, J., & Joseph, C. (2004). Intuitive ethics: How innately prepared intuitions generate culturally variable virtues. Daedalus, 133(4), 55–66. https://doi.org/10.1162/0011526042365555
Harper, C. A., & Rhodes, D. (2021). Reanalysing the factor structure of the moral foundations questionnaire. British Journal of Social Psychology, 60(4), 1303–1329. https://doi.org/10.1111/bjso.12452
Iurino, K., & Saucier, G. (2020). Testing Measurement Invariance of the Moral Foundations Questionnaire Across 27 Countries. Assessment, 27(2), 365–372. https://doi.org/10.1177/1073191118817916
Kivikangas, J. M., Fernández-Castilla, B., Järvelä, S., Ravaja, N., & Lönnqvist, J.-E. (2021). Moral foundations and political orientation: Systematic review and meta-analysis. Psychological Bulletin, 147(1), 55–94. https://doi.org/10.1037/bul0000308
Lee, S. T. H. (2018). Testing for Measurement Invariance: Does your measure mean the same thing for different participants? APS Observer, 31(8).
Meindl, P., Iyer, R., & Graham, J. (2019). Distributive Justice Beliefs are Guided by Whether People Think the Ultimate Goal of Society is Well-Being or Power. Basic and Applied Social Psychology, 41(6), 359–385. https://doi.org/10.1080/01973533.2019.1663524
Nilsson, A. (2023). Measurement Invariance of Moral Foundations across Population Strata. Journal of Personality Assessment, 105(2), 163–173. https://doi.org/10.1080/00223891.2022.2074853
Yilmaz, O., Harma, M., Bahçekapili, H. G., & Cesur, S. (2016). Validation of the Moral Foundations Questionnaire in Turkey and its relation to cultural schemas of individualism and collectivism. Personality and Individual Differences, 99, 149–154. https://doi.org/10.1016/j.paid.2016.04.090

Endnotes

  1. For a clear, nontechnical intro to confirmatory factor analysis see Gregorich (2006). (Note that you are not expected to understand this.) ↩︎

  2. See Lee (2018): ‘Ascertaining scalar invariance allows you to substantiate multi-group comparisons of factor means (e.g., t-tests or ANOVA), and you can be confident that any statistically significant differences in group means are not due to differences in scale properties.’ ↩︎

  3. See Iurino & Saucier (2020, p. 2): ‘A finding of measurement invariance would provide more confidence that use of the MFQ across cultures can shed light on meaningful differences between cultures rather than merely reflecting the measurement properties of the MFQ.’ ↩︎

  4. The new foundations are called Care, Equality, Proportionality, Loyalty, Authority and Purity (Atari et al., 2023, p. table 2, 1161). These researchers cite Meindl, Iyer, & Graham (2019) as justifying the distinction between equality and proportionality. I am not confident I understand how these are distinct. ↩︎