Eight Mechanisms, Seventeen Years Later

How reliable is our knowledge on mechanisms that make people give? In a webinar organized by the European Research Network on Philanthropy (ERNOP) on December 10, 2024, I've given a preliminary answer to this question. The slides are here. With a group of scholars we seek to identify which results have successfully been replicated in research that was conducted since the 2007 Science of Generosity working paper (Bekkers & Wiepking, 2007) in which Pamala Wiepking and I described eight mechanisms that make people give. It formed the basis for a paper in Nonprofit & Voluntary Sector Quarterly (Bekkers & Wiepking, 2011).

When we created our literature review, we tried to include all research reports that we could find, and simply summarized their results without paying attention to the quality of the research. We did distinguish between experiments and correlational evidence, but we had not looked closely at the data and the findings reported. In retrospect, this was naive. We now know that published research in the social sciences contains many results that are not robust, and replications tend to have smaller effects than original studies (Open Science Collective, 2015; Nosek et al., 2022). Some results may have been fraudulent, other reports are based on falsified data. The number of retractions is increasing (Tran, 2024; Van Noorden, 2023), and these are only the tip of an iceberg of questionable research (Bekkers, 2022).

So what do we know for sure about mechanisms that make people give? If we only include research that follows best practices in open science and research transparency, what insights remain? In the ideal case, research plans are pre-registered before data are collected, and results are reproducible because data and code are shared. We expected very few studies to follow these best practices. In addition, we investigated which findings that we reported in the literature review have been independently replicated, by other authors with other data.

Preliminary results

Highlights of the presentation:

1. The field of research on philanthropy is expanding. Research on 'donations' comprises about 2.2 million studies in Google Scholar, constituting about 0.85% of all 254 million studies included in OpenAlex.

2. Retractions are very rare in our field, and they are declining, while the number of retractions in other fields is increasing.

3. Replications represent only 0.74% of all studies on 'charitable behavior' in Google Scholar.

4. Most replications we found thus far are generalizations – i.e. testing a previous result again, with different data and a different analysis.

5. Slightly more than half of replications we found thus far report results in a different direction than original studies.

6. Of the eight mechanisms, research on efficacy, material costs, solicitation, and psychological benefits tends to replicate better than research on awareness of need, reputation, values, and altruism.

7. While a majority of replications are in the same direction as the original results, most replications report qualifications to original findings in the sense that effect sizes in replications tend to be smaller, original results do not generalize to other populations, or previous effects are replicated only in some conditions but not others.

8. There seems to be publication bias in replications – replication studies published in journal articles or book chapters are more likely to report support for original findings (63%) than unpublished dissertations, working papers and preprints (47%).

---

Next steps

The results above are based on ~5000 studies on "charitable behavior" and studies citing them or directly related to them. For a comprehensive analysis we need to go through other keywords as well. This is not feasible with human efforts alone. Thus, the next step in the project is automation of the work flow with the help of scripts that are able to screen abstracts and preferably the full text of studies.

A draft of the plan for the project is here. Would you like to contribute? Let me know in an email [r dot bekkers at vu dot nl] describing your expertise and potential contribution to the project. The plan is open for comments; you can request access to the draft so you can add comments and make changes.

The growth of research on philanthropy

To get a sense of the volume of research on philanthropy since 2011, we collected data in Google Scholar on the number of search results per year for six different keywords that we also used in the original literature review. For all keywords, the number of studies exploded since the year 2000.

	2000	2023	Growth
charitable behavior	40	503	1258%
charitable giving	876	4570	522%
prosocial behavior	1580	18600	1177%
philanthropy	5540	22900	413%
altruism	9280	26000	280%
donations	25700	59400	213%

Even for the most narrow search term, "charitable behavior", the number of results increased strongly, with a more than tenfold increase from 40 in 2000 to 503 in 2023. The period after 2011 shows a continued increase in the number of studies from 148 in 2012 to 503 in 2023, a growth of 340%.

The number of results for “charitable behavior” is dwarfed by “charitable giving”, showing an increase in the number of publications from 876 in 2000 to 4570 in 2023. This means that we will have to come up with a very stringent selection and sampling procedure to keep the project within feasible limits. To automate some of the coding, we could employ rule based classifications and machine learning (Ma et al., 2021).

---

References

Bekkers, R. (2022). Ten Meta Science Insights to Deal With the Credibility Crisis in the Social Sciences. SocArxiv Preprint, https://osf.io/preprints/socarxiv/rm4p8/

Bekkers, R. & Wiepking, P. (2007). Generosity and Philanthropy: A Literature Review. Report commissioned by the John Templeton Foundation. https://renebekkers.wordpress.com/wp-content/uploads/2011/08/bekkers_wiepking_generosity_07.pdf

Bekkers, R. & Wiepking, P. (2011). A Literature Review of Empirical Studies of Philanthropy: Eight Mechanisms that Drive Charitable Giving. Nonprofit and Voluntary Sector Quarterly, 40(5): 924-973. https://doi.org/10.1177/0899764010380927

Ma, J., Ebeid, I.A., De Wit, A., Xu, M., Yang, Y., Bekkers, R. & Wiepking, P. (2021). Computational Social Science for Nonprofit Studies: Developing a Toolbox and Knowledge Base for the Field. Voluntas, 34: 52-63. https://osf.io/g9d8u/

Nosek, B. A., Hardwicke, T. E., Moshontz, H., Allard, A., Corker, K. S., Dreber, A., … & Vazire, S. (2022). Replicability, robustness, and reproducibility in psychological science. Annual Review of Psychology, 73(1), 719-748. https://doi.org/10.1146/annurev-psych-020821-114157

Open Science Collaboration. (2015). Estimating the reproducibility of psychological science. Science, 349(6251). https://doi.org/10.1126/science.aac4716

Tran, N. (2024). The 'publish or perish' mentality is fuelling research paper retractions – and undermining science. The Conversation, 24 September 2024. https://theconversation.com/the-publish-or-perish-mentality-is-fuelling-research-paper-retractions-and-undermining-science-238983 

Van Noorden, R. (2023). More then 10,000 research papers were retracted in 2023 – a new record. Nature, 12 December 2023. https://www.nature.com/articles/d41586-023-03974-8