Records · Verification table
Design literature: verification of "What is known" in source/knowledge/design.md, 6 November 2026
The working record behind the pages, published as written. Source file source/verify/design-2026-11-06.md. All records.
Scope. The eight items under "What is known" in source/knowledge/design.md, all seeded from memory on 2026-11-05 and marked unverified. For each: the paper design.md named (or, for items 7 and 8, one paper chosen today), its correct record, what it says, and whether design.md had it right. Two follow-ups on inoculation decay, one scam-specific decay study, and one field test of implementation-intention prompts were added because the named papers do not answer the question design.md asks of them. GiveWell's own donor research (backlog item 4) was not read.
Order. The per-death-number rule in design.md ("Decisions, 2026-11-06") was written before this read. Nothing below changes the number's position; findings bear on wording only.
Method. Fetched 2026-11-06, about 15:45 to 16:03 UTC (10:45 to 11:03 EST; date stamp 11:03:25 EST at the end). Metadata (authors, title, venue, volume, pages, year) from the Crossref API for every DOI; abstracts from OpenAlex, PubMed E-utilities or RePEc where the full text was paywalled. Full texts pulled with curl (browser user agent), PDFs read with pdftotext; the one scanned PDF (Gollwitzer & Sheeran) was OCR'd with tesseract at 200 dpi, so its quotes may carry OCR spacing errors (checked against the page images where numbers matter). No verdict rests on a search snippet or on memory.
Verdicts. CONFIRMED = the paper says what design.md said. CORRECTED = design.md had the authors, venue, year or finding wrong or overstated; the corrected line is in design.md. NOT FOUND = the paper could not be read. Grades (the site's scale): research = peer-reviewed; preprint; evaluator; primary; news; unread. Where only the abstract was read, the grade says so.
Fetch log.
- Crossref API, all eight DOIs below plus 10.1093/qje/qjs017 — 200, read.
- KOPS Konstanz, Gollwitzer & Sheeran author copy (scanned, 49 pages) — 200; OCR'd.
nbn-resolving.deanswers with an Anubis bot wall. - Nunes & Drèze author PDF at
msbfile03.usc.edu: live URL has an expired TLS certificate and returns 404 when forced; read the Wayback copy captured 2026-01-05 (web.archive.org/web/20260105212707/...), 9 pages, JCR page headers present.academic.oup.com→ 403. - NBER w20047 (Karlan & Wood, April 2014, revised July 2015) — 200, read in full. Published abstract read on RePEc IDEAS — 200. ScienceDirect → blocked ("There was a problem providing the content").
- Small, Loewenstein & Slovic, journal PDF hosted at
faculty.wharton.upenn.edu— 200, 11 pages, read in full (figure 3 checked on the rendered page image). - Bryan, Karlan & Nelson:
annualreviews.org→ 403 / Cloudflare challenge in headless Chrome; LSE eprint has no full text. Read the publisher PDF (Annual Reviews download stamp "by Harvard University on 02/12/13") mirrored athoudekpetr.cz— third-party host; the file is the publisher's typeset article. - Europe PMC full text PMC9401631 (Roozenbeek et al.) — 200, read. PMC11897321 (Maertens et al. 2025) — 200, read.
- Cambridge Apollo repository, Maertens et al. 2021 accepted manuscript (text bitstream) — 200, read.
- KIT repository, Reinheimer et al. SOUPS 2020 PDF — 200, read.
- AHRQ Evidence Report 199 (Berkman et al. 2011): NCBI Bookshelf → reCAPTCHA wall (curl and headless Chrome). Read the AHRQ PDF from the Wayback copy captured 2026-09-30 (PDF metadata: created 14 Mar 2011, title matches). Sheridan et al. 2011 abstract via E-utilities — 200.
- NBER w15361 (Bettinger et al., September 2009) — 200, read. Published QJE abstract via OpenAlex.
- Milkman et al. 2011: abstract via Europe PMC; full text via E-utilities returned the abstract only (publisher restriction). Sample size not read.
- Repo files read:
source/knowledge/design.md,content/child-hunger-do.html,content/cybercrime-practice.html,content/ai-practice.html(for the implications; none edited).
---
| # | Item | Claim as design.md had it | Paper read | URL | Fetch | Exact quotes | Verdict | Grade |
|---|---|---|---|---|---|---|---|---|
| 1 | Implementation intentions | "Implementation intentions ("when X, I will Y") raise follow-through on intended actions. Gollwitzer & Sheeran, meta-analysis, Advances in Experimental Social Psychology, 2006." | Gollwitzer, P. M., & Sheeran, P. (2006). Implementation intentions and goal achievement: A meta-analysis of effects and processes. Advances in Experimental Social Psychology, 38, 69–119. doi:10.1016/S0065-2601(06)38002-1 | https://kops.uni-konstanz.de/bitstream/123456789/10973/1/06GollwitzerSheeran_ImplementationIntentionsAndGoalAchievement.pdf | OK (scanned; OCR) | "The overall impact of forming implementation intentions on goal achievement was d= .65 based on k = 94 tests that involved 8461 participants. This effect had a 95% confidence interval from .60 to .70." / "most tests of implementation intention effects were conducted among university students (k = 79)" / Table III: "Prosocial 254 5 1.01" (N, k, d) / "Remembering to act 983 11 .54"; "Seizing opportunities 2270 20 .61" / "strong effects of if-then plans only emerge when participants hold strong respective goal intentions" / "49% of the included tests were unpublished … publication status had no impact on the effect size" | CONFIRMED (authors, venue, year right; the finding holds, with two limits design.md didn't state: mostly students in lab or health tasks, and the plan works only on a goal the person already holds strongly) | research |
| 1a | (field test added) | — | Milkman, K. L., Beshears, J., Choi, J. J., Laibson, D., & Madrian, B. C. (2011). Using implementation intentions prompts to enhance influenza vaccination rates. PNAS, 108(26), 10415–10420. doi:10.1073/pnas.1103170108 | https://europepmc.org/article/PMC/PMC3127912 | abstract only | "The vaccination rate among control condition employees was 33.1%. Employees who received the prompt to write down just a date had a vaccination rate 1.5 percentage points higher than the control group, a difference that is not statistically significant. Employees who received the more specific prompt to write down both a date and a time had a 4.2 percentage point higher vaccination rate" | (added; sample size not read) | research (abstract only) |
| 2 | Endowed progress | "Endowed progress: people finish tasks more often when they can see progress already made. Nunes & Drèze, Journal of Consumer Research, 2006." | Nunes, J. C., & Drèze, X. (2006). The endowed progress effect: How artificial advancement increases effort. Journal of Consumer Research, 32(4), 504–512. doi:10.1086/500480 | http://web.archive.org/web/20260105212707/http://msbfile03.usc.edu/digitalmeasures/jnunes/intellcont/Endowed%20Progress%20Effect-1.pdf | OK (Wayback copy of author PDF) | "people provided with artificial advancement toward a goal exhibit greater persistence toward reaching the goal" / "the car wash randomly distributed 300 loyalty cards" / "A total of 80 cards were redeemed … The redemption rate for those possessing a card requiring 10 purchases, yet endowed with two stamps, was 34% versus just 19% for those who possessed a card requiring a total of eight purchases." / "When there is no reason, the effect occurs only if the endowment is issued in an alternative currency such as points. However, when progress is recorded in purchases, even a specious reason … suffices" / "we focus solely on a single endowment" | CORRECTED (the finding is about artificial, given progress that reframes a task as already begun, not about seeing real progress; one 300-card field study plus two survey studies of 146 students and 240 shoppers) | research |
| 3 | Effectiveness information and donations | "Giving cost-effectiveness information can reduce donations from small donors while raising them from large ones. Karlan & Wood, Journal of Behavioral and Experimental Economics, 2017." | Karlan, D., & Wood, D. H. (2017). The effect of effectiveness: Donor response to aid effectiveness in a direct mail fundraising experiment. Journal of Behavioral and Experimental Economics, 66, 1–8. doi:10.1016/j.socec.2016.05.005 (full text read: NBER w20047, rev. July 2015) | https://www.nber.org/system/files/working_papers/w20047/w20047.pdf ; https://ideas.repec.org/a/eee/soceco/v66y2017icp1-8.html | OK | Published abstract: "varying letters by whether they include a discussion of their program's impact as measured by scientific research. The base script, used for both treatment and control, included a standard qualitative story about an individual beneficiary. Adding scientific impact information has no effect on average likelihood of giving or average gift amount. However, we find important heterogeneity: large prior donors both are more likely to give and also give more, whereas small prior donors are less likely to give." / WP: "the pooled sample consists of 34,673 mailers to 21,643 prior donors" / wave 2 treatment text: "women who were offered our Credit with Education program had 16% higher profits … increased profits in bad months by 27%!" / "when examined quartile by quartile, no individual result is statistically significant" / "we put forward evidence, albeit on a small scale" | CORRECTED (the information was impact evidence from randomized trials, not cost-effectiveness: no dollars per outcome appeared; no effect on average; the split is by size of prior gift, estimated as a continuous interaction, not significant within any quartile) | research |
| 4 | Identifiable victim | "a single named child raises giving more than statistics, and statistics presented alongside can reduce it. Small, Loewenstein & Slovic, Organizational Behavior and Human Decision Processes, 2007." | Small, D. A., Loewenstein, G., & Slovic, P. (2007). Sympathy and callousness: The impact of deliberative thought on donations to identifiable and statistical victims. OBHDP, 102(2), 143–153. doi:10.1016/j.obhdp.2006.01.005 | https://faculty.wharton.upenn.edu/wp-content/uploads/2012/04/Sympathy_and_callousness.pdf | OK | "teaching or priming people to recognize the discrepancy in giving toward identifiable and statistical victims has perverse effects: individuals give less to identifiable victims but do not increase giving to statistical victims, resulting in an overall reduction in caring and giving." / Study 3 (N = 159): "individuals who faced an identifiable victim donated more than those who faced victim statistics, p < .01, and also donated more than those who faced an identifiable victim in conjunction with statistics, p < .05"; Fig. 3 means: statistical $1.14, identifiable $2.38, identifiable with statistics $1.43 (of $5 given to each participant) / statistical stimulus: "Food shortages in Malawi are affecting more than three million children." / Ns: Study 1 "N = 121", Study 2 "99", Study 3 "159", Study 4 "N = 165"; recruited "in the student center at a university in Pennsylvania" | CONFIRMED (authors, venue, year, finding right; the statistics tested describe the scale of need, not cost per life; four small studies, $5 stakes, one campus) | research |
| 5 | Commitment devices | "Commitment devices raise follow-through on goals people already hold. Bryan, Karlan & Nelson, Annual Review of Economics, 2010." | Bryan, G., Karlan, D., & Nelson, S. (2010). Commitment devices. Annual Review of Economics, 2, 671–698. doi:10.1146/annurev.economics.102308.124324 | http://houdekpetr.cz/!data/public_html/papers/economics_psychology/Bryan%20et%20al%202010.pdf (publisher PDF, third-party mirror; publisher page 403) | OK | "We define a commitment device as an arrangement entered into by an agent who restricts his or her future choice set by making certain choices more expensive, perhaps infinitely expensive" / "We refer to commitment devices that call for real economic penalties for failure, or rewards for success, hard commitments. And we refer to any device that has primarily psychological consequences a soft commitment." / "there is a dearth of papers that formalize these intuitions" (on soft commitment) / SMarT: "Of those offered the program, 78% enrolled … after four pay raises, 80% of those individuals were still enrolled, indicating that a soft commitment was sufficient" / Breman (2009): "charities offered prior donors the opportunity to 'give more tomorrow' … average gifts increased when this soft commitment was offered" / open question: "which are more effective, soft or hard commitments?" | CORRECTED (a review, mostly of savings, smoking and health; a device must make the unwanted choice costlier. A checklist that remembers costs nothing to skip, so by this definition it is at most a soft commitment, the least-studied kind) | research |
| 6 | Prebunking and decay | "Phishing and scam "inoculation" (prebunking) works when it teaches the manipulation technique, and the effect decays within weeks without boosters. Roozenbeek, van der Linden et al., Science Advances, 2022" | Roozenbeek, J., van der Linden, S., Goldberg, B., Rathje, S., & Lewandowsky, S. (2022). Psychological inoculation improves resilience against misinformation on social media. Science Advances, 8(34), eabo6254. doi:10.1126/sciadv.abo6254 | https://europepmc.org/article/PMC/PMC9401631 | OK | "five short videos that inoculate people against manipulation techniques commonly used in misinformation: emotionally manipulative language, incoherence, false dichotomies, scapegoating, and ad hominem attacks. In seven preregistered studies, i.e., six randomized controlled studies ( n = 6464) and an ecologically valid field study on YouTube ( n = 22,632)" / "boosting manipulation technique recognition by about 5% on average" (field; "Cohen's h = 0.09") / "our field study found that the videos were effective within a 24-hour window … we were unable to study how long the inoculation effect remains significant" | CORRECTED (misinformation, not phishing or scams; this paper did not measure decay at all) | research |
| 6a | Decay follow-up | — | Maertens, R., Roozenbeek, J., Basol, M., & van der Linden, S. (2021). Long-term effectiveness of inoculation against misinformation: Three longitudinal experiments. Journal of Experimental Psychology: Applied, 27(1), 1–16. doi:10.1037/xap0000315 (accepted manuscript) | https://www.repository.cam.ac.uk/handle/1810/312077 | OK | "In three experiments (NExp1 = 151, NExp2 = 194, NExp3 = 170), participants played either Bad News … or Tetris" / "with regular testing … the inoculation effect remains stable for at least three months. With Experiment 2, we sought to replicate these findings without regular testing and found significant decay over a two-month time period so that the long-term inoculation effect was no longer significant." / "Without regular 'boosting,' the effects dissipate within two months." | (added) | research |
| 6b | Booster follow-up | — | Maertens, R., Roozenbeek, J., Simons, J. S., Lewandowsky, S., Maturo, V., Goldberg, B., Xu, R., & van der Linden, S. (2025). Psychological booster shots targeting memory increase long-term resistance against misinformation. Nature Communications, 16. doi:10.1038/s41467-025-57205-x | https://europepmc.org/article/PMC/PMC11897321 | OK | "five pre-registered longitudinal experiments ( N total = 11,759)" / "text-based and video-based inoculation interventions can remain effective for one month—whereas game-based interventions appear to decay more rapidly—and that memory-enhancing booster interventions can enhance the diminishing effects" / Study 1 booster: "an exact repetition of the original intervention" at about 10 days | (added) | research |
| 6c | Scam-specific decay | — | Reinheimer, B., Aldag, L., Mayer, P., Mossano, M., Duezguen, R., Lofthouse, B., von Landesberger, T., & Volkamer, M. (2020). An investigation of phishing awareness and education over time: When and how to best remind users. SOUPS 2020, 259–284. | https://publikationen.bibliothek.kit.edu/1000122566/86934434 | OK | "With overall 409 employees … We find a significantly improved performance of correctly identifying phishing and legitimate emails directly after and four months after the programme's deployment. This was not the case anymore after six months, indicating that reminding users after half a year is recommended. … measures based on videos and interactive examples perform best, lasting for at least another six months." | (added; already cited on cybercrime-practice.html s5) | research |
| 7 | Readability | "shorter sentences and concrete words raise comprehension and completion for instructions; the evidence base is in plain-language and health-literacy research." (no citation) | Berkman, N. D., Sheridan, S. L., Donahue, K. E., et al. (2011). Health Literacy Interventions and Outcomes: An Updated Systematic Review. Evidence Report/Technology Assessment No. 199, AHRQ Publication 11-E006. Journal version: Sheridan et al. (2011), J Health Commun 16 Suppl 3:30–54, doi:10.1080/10810730.2011.604391 (abstract read) | http://web.archive.org/web/20260930005958/https://www.ahrq.gov/downloads/pub/evidence/pdf/literacy/literacyup.pdf | OK (Wayback copy; NCBI Bookshelf behind reCAPTCHA) | "We found seven studies examining the effects of interventions that combined simplification of readability with document redesign" / simplified head-trauma sheet: "a 1-point improvement on a comprehension scale with possible scores ranging from 0-10" / "Based on these findings, our research team judged the overall strength of evidence about alternative readability and document design to be insufficient" / "the following specific design features seemed to improve comprehension for low-health-literacy populations in one or a few studies: (1) presenting essential information by itself … (2) presenting essential information first" / Sheridan abstract: "Among 38 included studies … Evidence for the effects of interventions on other outcomes was either limited or mixed." | CORRECTED (the systematic review grades the evidence for simpler wording "insufficient": mixed, small studies, comprehension only, nothing on completion. What has some support is putting the essential information first or alone. Searches end about 2010) | research |
| 8 | Friction and completion | "each extra step before an action cuts completion; the size of the effect depends on the step." (no citation) | Bettinger, E. P., Long, B. T., Oreopoulos, P., & Sanbonmatsu, L. (2012). The role of application assistance and information in college decisions: Results from the H&R Block FAFSA experiment. Quarterly Journal of Economics, 127(3), 1205–1242. doi:10.1093/qje/qjs017 (full text read: NBER w15361, Sept 2009) | https://www.nber.org/system/files/working_papers/w15361/w15361.pdf | OK | WP: tax professionals used "information from the tax return to pre-populate FAFSA" then an interview "which took less than 10 minutes" / dependents: "40.2 percent of the control group went on to file a FAFSA … those who were offered help completing the form … were 15.7 percentage points more likely to file" / independents: "from 13.8 percent to 39.5 percent" / "the information-only treatment had essentially no impact on filing" / sample "large sample (55,083)" (balance check). Published abstract: "high school seniors whose parents received the treatment were 8 percentage points more likely to have completed two years of college, going from 28% to 36% … Families who received aid information but no assistance with the FAFSA did not experience improved outcomes." | CORRECTED (strong field evidence that taking the work of a form off people raises completion, and that information alone does not; it bundles prefill, personal help and e-filing, so it does not measure the cost of one extra step) | research |
---
Findings that touch other files (proposals only; nothing edited)
- design.md, "What this site already does": "a checklist that remembers is a commitment device that costs nothing". Under Bryan, Karlan & Nelson's definition a device must make the unwanted choice costlier; a free checklist is at most a soft commitment, the kind the review says is least studied. Proposed wording: "a checklist that remembers is a reminder and a record; it costs nothing and tracks nobody."
- **
content/ai-practice.htmls7** cites Roozenbeek et al. 2022 beside "how long the effect lasts isn't settled". That is accurate; the paper says it could not measure duration. Maertens et al. 2021 and 2025 are the decay sources if the page ever gives an interval. - **
tools/audit.pythresholds** (45-word sentence, 24-word average): the AHRQ review does not support any specific number. They are a house style, not an evidence threshold. The 120-word first-action rule is the closer match to what has support ("essential information first"). - Breman (2009), "give more tomorrow" (cited inside Bryan et al.): a soft commitment that raised charitable gifts. Not read. A candidate for the child-hunger monthly-giving step.
- GiveWell's own donor research (named in backlog item 4): not read.
What would overturn these readings
- Item 1: a field trial of written when/where prompts for one-off civic or giving actions showing no effect.
- Item 2: a test of displayed real progress (not endowed) showing no or negative effect on completion.
- Items 3–4: a randomized test of cost-per-life wording on donors of $5–$20 showing a different sign.
- Item 6: a scam-drill study with decay measured at one and three months.
- Item 7: a newer systematic review (after 2010) that grades plain-language rewriting as moderate or better for completion.
- Item 8: a study isolating single added steps on a web form with no effect on completion.
Staleness
Every paper here is static once published. The AHRQ review's searches end about 2010; newer plain-language reviews were not read. Re-read only if a later review or replication of any item turns up.
Contest a fact, suggest something, or tell us what’s broken
If someone is in danger right now, don’t use this form. Call your local emergency number (911 in the US). For trafficking: 1-888-373-7888, or text 233733.
No name, no account. Your note goes into the site’s private records, nothing about your browser or connection is stored with it, and the email box is the only way we’d know who you are. The weekly review reads every note, re-reads the original source, and fixes the page if it should. A contested fact and its outcome are listed on Contested, in the review’s words, never yours.
No tracking on any link.Ticks stay on this device.What changedContested