Records · Verification table

Re-grading seven Do-page steps against the evidence rubric, 2026-11-06

The working record behind the pages, published as written. Source file source/verify/regrade-2026-11-06.md. All records.

Why. A blind second grader, shown only each step's text, disagreed with the page's evidence label on 7 of the 14 three-step actions on the Do pages. This file settles each of the seven against the one rubric, written before any re-grading (design brief, "The evidence rubric"; also on /about/).

The rubric, as applied. A label grades whether the action produces its outcome, not whether the problem is real or large.

• strong: at least two independent randomised trials, or a systematic review rated high certainty, find that the action, or the program it pays for, produces its outcome in settings like the page's.
• moderate: one well-run randomised trial, or several studies of other designs that agree, or an independent evaluator's judgement built on trials of the program's parts.
• weak: only before-and-after, observational or self-reported evidence, or studies that are small or short.
• none but plausible: no study tests it; the page names the reason to expect it to work.
• contradicted: good studies find it doesn't work, or that it harms the people it's meant to help.
• Independence: two trials by the same team, or funded by the organisation being graded, count as one.

Decision logic used here, stated before the cases. (1) Name the action and its outcome as the page states them. (2) List every study that tests that action, or the program it pays for, against that outcome. A study of a different action, or of a different outcome, is recorded but doesn't count toward the grade. (3) Take the highest grade whose clause the counted evidence meets in full. (4) An organisation's study of its own program or product counts as that organisation's own figures ("self-reported"), however large; this applies the same way to a charity's outcome report and a tech company's study of its own users. (5) "Several" means more than two.

Method. Sources read today were fetched with curl and a browser user agent between 11:02 and 11:06 EST; HTML went through a tag-stripper and PDFs through pdftotext, and the quotes below are copied from that text. Where a source was read and quoted in an earlier record on 5 or 6 November, that record is named and the quote is carried from it. Three web searches (about 11:04 EST) looked for an independent outcome study of the Fair Food Program (found Rubalcaba and Ortega 2025, whose abstract was then read) and for an independent study of consumer two-step sign-in outside Google and Microsoft (none found).

Pages edited: content/cybercrime-do.html (steps 1–3, and the matching label in section 8) and content/inequality-do.html (step 3). content/trafficking-do.html was not edited; another session is working in it. Its three decisions are written as proposals at the end.

---

Summary

None of the seven was right and under-described in the sense of a label the page failed to support. Six were too generous. Trafficking step 2's label was right, but its Nepal sentence pointed at the wrong action, and a proposal below fixes the wording.

---

Trafficking, step 1: "Pay for recovery, monthly"

Action and outcome. A monthly gift to survivor services (housing, legal status, counselling, work) so a survivor stays free and recovers.

Evidence that tests it.

• Dell, Maynard, Born, Wagner, Atkins and House, "Helping Survivors of Human Trafficking: A Systematic Review of Exit and Postexit Interventions", Trauma, Violence & Abuse, 2019 (PMID 29333961, doi 10.1177/1524838017692553). Abstract read today through Europe PMC (https://www.ebi.ac.uk/europepmc/webservices/rest/search?query=TITLE:%22exit%20and%20postexit%20interventions%22&resultType=core&format=json): "The search yielded six eligible studies that included 155 female and 6 male survivors from four countries"; "the quality of most studies was poor"; "Evidence of effects of exit and postexit interventions is sparse, and much of the research is poorly designed and executed." research
• Rothman et al., "A longitudinal evaluation of a survivor-mentor program for child survivors of sex trafficking in the United States", Child Abuse & Neglect, 2020 (PMID 31358352). Abstract read today through Europe PMC (https://www.ebi.ac.uk/europepmc/webservices/rest/search?query=EXT_ID:31358352%20AND%20SRC:MED&resultType=core&format=json): "41 youth"; "We used a one-group repeated measures design"; "At baseline 72% could be characterized as CSE-experienced, while at 6 months the percentage decreased to 24% (p < 0.001) and at 12 months to 14% (p < 0.001)." research
• The named organisations' outcome reports are their own (impact review A4: Sanctuary for Families, Restore NYC, GEMS, the Freedom Fund). charity's own figures

Rubric clause applied. weak: "only before-and-after, observational or self-reported evidence, or studies that are small or short." Six small, mostly poor-quality studies, one 41-person single-group study with no comparison, and charities' own figures. No randomised trial, no systematic review rated high certainty, no independent evaluator's judgement.

Why the old label fails. "need documented" grades the problem (federal grants lapsed, T-visa approvals collapsed), which the rubric says a label must not do. The blind grader's "none but plausible" was right about the page as written, since the page names no outcome study, but studies do exist. They are weak.

Old → new. moderate: need documented, outcomes self-reported → weak: small studies, charities' own results, plus one sentence naming the Dell review (proposal T1 below).

What would overturn it. A randomised or well-matched comparison of survivors who did and didn't receive housing, legal or employment services, measuring re-trafficking or stable recovery, would lift it. The cluster trial registered as NCT07503912 is a prevention trial in schools, not a test of services, and wouldn't.

Trafficking, step 2: "Learn what a bad job looks like, and be the person who knows the number"

Action and outcome. A reader in the US learns the labour signs (a fee for the job, a debt to the recruiter, documents held, a visa tied to one boss) and keeps the hotline number. The outcome is that someone they know, in or near a bad job, gets a route to help.

What the Nepal evidence tested.

• Mak, Kiss and Zimmerman, Pathways to prevent labour exploitation in Nepal: Do pre-migration interventions work?, LSHTM, February 2019 (https://ceslam.org/wp-content/uploads/2024/11/Swift-evaluation-nepal-research-report-pathways-to-prevent-labour-exploitation-2019.pdf, read today): "A woman's risk of forced labour was not influenced by her age, sociodemographic characteristics, awareness of risks, or the motivations for her migration. Instead, it was affected by her destination country, whether she used a recruiter and if she was exposed to legislative migration bans." This is an observational survey of returnee migrant women. research
• Zimmerman, Mak, Pocock and Kiss, "Human Trafficking: Results of a 5-Year Theory-Based Evaluation of Interventions to Prevent Trafficking of Women From South Asia", Frontiers in Public Health, 17 May 2021 (https://discovery.ucl.ac.uk/10135386/1/fpubh-09-645059.pdf, read today), on the Nepal data: "just over two-thirds (67.8%) of returnee migrant women reported having been aware that migrants may be deceived about their working terms and conditions prior to migrating. Most reported (91.2%) having suffered forced labor"; "There does not even seem to be a 'dose response effect,' as participation in trainings did not reduce the likelihood of experiencing forced labor". The trainings were two-day courses for the women themselves ("Each 2-day training program was implemented on a specific date and time"). The same paper describes a "hotline" whose "support workers' ability to respond was not sufficient to meet this woman's actual needs". This is a theory-based, non-randomised evaluation; the Nepal follow-up lost 45% of its sample (impact review A1). research

Does it contradict this action? No. It tests a different one: pre-departure training of the at-risk worker herself, with her own later forced labour as the outcome. The page's action is a third person's knowledge, used to hand someone a route to help. No study tests that, in Nepal or anywhere else (impact review A1: every review of awareness programmes found no outcome evaluations). Two parts of the Nepal evidence do bear on how the step is worded, so they are kept as a contest rather than dropped:

1. The page says the informed friend helps "someone about to take a risky job". If that means warning a worker off a job in advance, it is close to what Nepal tested. There, knowing the risks didn't change the risk, because the destination and the recruiter set it. The proposal moves the friend's role to the point where a job turns out bad.
2. A hotline is only as good as its response (the Zimmerman quote above; the three-country SWiFT report's "the availability of emergency assistance and 'hot lines' should never be oversold", row T25 of trafficking-do-2026-11-06.md).

Even for the training it did test, the Nepal evidence falls short of "good studies find it doesn't work". It is non-randomised, with heavy attrition. For the friend's role it is evidence about a neighbouring action.

Rubric clause applied. none but plausible: "no study tests it; the page names the reason to expect it to work." The reason the page gives is that most traffickers trick, manipulate or threaten instead of kidnapping, so the signs are those of a job. The impact review adds that 20.8% of hotline signals in 2025 came from victims or survivors themselves (A1, hotline data).

Old → new. Label kept, with a qualifier: none but plausible → none but plausible: untested as a friend's role. The Nepal sentence is reworded to say what was tested (proposal T2 below).

What would overturn it. A study finding that people whose friends or family knew the signs and the number were no more likely to reach help, or were harmed by a report made over their heads, would make it weak or contradicted. The second risk is documented for third-party reports (impact review A2: survivor groups' letters; 59% of trafficked temporary-visa workers were controlled by immigration threats).

Trafficking, step 3: "Back enforcement that workers run"

Action and outcome. Support the Fair Food Program (ask a grocer or co-op to join, give, join actions). The outcome is less forced labour and abuse on farms.

Evidence.

• The programme's own 2024 report: over 20,000 workers covered; 3,919 hotline complaints from workers, 2011–2024; it claims its mechanisms "have effectively rid FFP farms of abusive practices, ranging from forced labor and sexual violence to wage theft" (impact review A6; rows T30–T31 of trafficking-do-2026-11-06.md, read 6 Nov). The report is written by the Fair Food Standards Council, the programme's own monitor. programme's own figures
• Judgements on the mechanism, not outcome studies of the programme: US Customs and Border Protection in 2021, via a law-firm summary, recommended "worker-driven solutions" and named the programme; Re:Structure Lab (July 2022, funded by Humanity United and the Freedom Fund) found audits and certification "ineffective" and called worker-led monitoring "more meaningful and promising" (impact review A6). These say audits fail. They don't measure whether this programme cuts forced labour.
• An independent outcome study found today by search, on a different outcome: Rubalcaba and Ortega, "Worker-Driven Social Responsibility and Infant Health", Demography, 2025 (abstract read today at https://alorte.github.io/publication/17_fair_food; journal page https://read.dukeupress.edu/demography/article/doi/10.1215/00703370-12352500/406173/Worker-Driven-Social-Responsibility-and-Infant): "Using restricted birth records data from the National Vital Statistics System from 2006 to 2018, we show that the adoption of the Fair Food Program was associated with reductions in low-weight births among foreign-born mothers from Latin America." It is observational, one study, and measures infant health, not forced labour. The full paper was not read; a press figure of "10% less likely" (The Packer) is unread at the source. research (abstract)

Rubric clause applied. weak: "only before-and-after, observational or self-reported evidence". The forced-labour outcome rests on the programme's own figures. The one independent study is observational and measures something else, so even counted generously it meets only the weak clause.

Old → new. moderate: self-reported outcomes → weak: the programme's own figures. The old qualifier named the weak clause's own condition. Optional: one sentence naming the independent infant-health study (proposal T3).

What would overturn it. An independent comparison of FFP and non-FFP farms measuring forced labour, wage theft or sexual violence, by workers' own reports to someone other than the programme, would lift it.

Cybercrime, step 1: "Get a password manager, then guard your email"

Action and outcome. A password manager plus a passkey or two-step sign-in on email, bank, carrier and the manager. The outcome is fewer account takeovers.

Evidence.

• Google Security Blog, "New research: How effective is basic account hygiene at preventing hijacking", 17 May 2019 (https://security.googleblog.com/2019/05/new-research-how-effective-is-basic.html, read today): "We teamed up with researchers from New York University and the University of California, San Diego"; adding a recovery phone number "can block up to 100% of automated bots, 99% of bulk phishing attacks, and 66% of targeted attacks that occurred during our investigation"; "zero users that exclusively use security keys fell victim to targeted phishing during our investigation." This is the source the page cites (s22). industry research
• The paper behind it: Doerfler et al., "Evaluating Login Challenges as a Defense Against Account Takeover", WWW 2019 (https://research.google/pubs/evaluating-login-challenges-as-a-defense-against-account-takeover/, read today): "Using Google as a case study, we evaluate the effectiveness of fourteen device-based, delegation-based, knowledge-based, and resource-based challenges at preventing over 350,000 real-world hijacking attempts"; "Device-based challenges provide the best protection, blocking over 94% of hijacking attempts rooted in phishing and 100% of automated hijacking attempts." This is observational: a study of Google's own login challenges on its own users. research
• Meyer et al., "How effective is multifactor authentication at deterring cyberattacks?", arXiv 2305.00945, 1 May 2023 (https://arxiv.org/abs/2305.00945, read today): "We employ the benchmark-multiplier method, coupled with manual account review, to evaluate the security performance of various MFA methods in a large dataset of Microsoft Azure Active Directory users exhibiting suspicious activity"; "MFA reduces the risk of compromise by 99.22% across the entire population and by 98.56% in cases of leaked credentials." Observational, by Microsoft authors, on commercial accounts, not households; not on the page. preprint
• The password-manager half has no outcome study at all (impact review A1: "No study was found that measured account compromise before and after people adopted a manager").
• A search today (about 11:04 EST) for an independent observational study of consumer two-step sign-in outside Google and Microsoft found none.

Rubric clause applied. weak: "only before-and-after, observational or self-reported evidence". Both datasets are observational, and each is a company's study of its own product on its own users. Under decision rule (4) that is the company's own figures, the same treatment the Fair Food Program and the survivor charities get above. Neither meets moderate. There is no randomised trial. Two studies are not "several". And moderate's "several studies of other designs that agree" can't be met by self-studies when the rubric's weak clause names self-reported evidence explicitly.

Contest kept. A reader who counts Google and Microsoft as two independent observational studies that agree, and reads "several" as "more than one", would grade this moderate. The rubric doesn't settle that reading. Rule (5) above settles it here: "several" means more than two. The rubric itself should say which reading it means (see "Proposals for the rubric").

Old → new. strong → weak: Google's data on its own users. The sentence after the label already names the Google data. The size of the effect (nearly all automated takeovers blocked) stays on the page; the label grades the design, not the size.

What would overturn it. An independent study, not run by an account provider on its own users, comparing takeover rates with and without two-step sign-in would lift it to moderate. A randomised rollout would lift it further.

Cybercrime, step 2: "Learn the call-back rule"

Action and outcome. Learn the three tells (urgency, secrecy, a payment you can't reverse) and, when one appears, hang up and call back on a number you already have. The outcome is less money lost to impostor and investment scams.

Evidence.

• The page itself: "nobody has tested the rule itself." The impact review agrees (A8: "No trial of the rule itself"; G: "any measurement of the call-back rule. (None exist that I could find…)").
• Lain, Kostiainen and Čapkun 2022 (s23): warning banners on suspicious emails cut clicks. That tests an automatic warning shown by the mail system, not a rule a person learns. research
• Scheibe et al. 2014 (https://pmc.ncbi.nlm.nih.gov/articles/PMC4199235/, read today): "Participants in the experimental conditions were warned about a federal stimulus grant scam to which they would later be exposed, or an online pharmacy scam. The calls also included a conversational component, in which the peer counselor asked about prior experiences with scams and stressed the importance to refuse these"; "Both warnings reduced unequivocal acceptance of the mock scam"; "The same scam warning, but not the different scam warning, lost effectiveness over time." This is a randomised trial of a phoned warning about a named scam, with stated acceptance of a mock pitch as the outcome, at two and four weeks. It tests a scam-specific warning, not the three tells or the call-back rule, and the authors note that "The proportion of respondents who would have actually authorized the relevant payment cannot be ascertained" (row P18, cybercrime-2026-11-06-additions.md). research
• The Postal Inspection Service mailing trial (DeLiema et al. 2024) and Burke et al. 2021 test mailed or video education about scams, not this rule (impact review A8).

Rubric clause applied. none but plausible: "no study tests it; the page names the reason to expect it to work." The nearest trials test other interventions (a named-scam phone warning, mailed education, email banners). The page names its reason: the rule reaches the categories where most money goes (investment, business-email and tech-support scams, $13.8 billion of $20.9 billion, IC3 2025).

Old → new. moderate → none but plausible: aimed at the biggest losses. The same action's label in section 8 ("When a call feels wrong, hang up and call back…") changes from moderate to none but plausible so the page doesn't contradict itself. That item sits in a tickable list keyed by its text, so a reader who had ticked it will see it unticked once.

What would overturn it. A trial that teaches the three tells and the call-back rule and measures losses or acceptance of a scam pitch would move it. A positive result in a well-run trial would make it moderate; a null result would make it contradicted.

Cybercrime, step 3: "Lock your phone number and freeze your credit"

Action and outcome. A carrier number lock (against SIM swaps) and a credit freeze at the three bureaus (against new accounts opened in your name). The outcome is fewer SIM-swap takeovers and fewer new-account frauds.

Evidence.

• Freeze: "No causal study of freezes on new-account fraud rates was found" (impact review A4). GAO-17-254 says a freeze "can prevent new-account fraud", a statement of mechanism (row D4, cybercrime-do-2026-11-06.md). primary
• Number lock: IC3 SIM-swap losses fell from $48.8 million (2023) to $17.4 million (2025) after the FCC rule took effect in January 2024. The impact review calls this "a correlation; nothing fetched today attributes it to carrier locks" (A3); its contest list says the series "is consistent with an effect and proves nothing". No data on lock uptake or failure rates. primary (correlation only)
• Princeton's 2020 SIM-swap study shows the attack works (all five carriers used "insecure authentication challenges"). It tests the threat, not the lock. research

Rubric clause applied. none but plausible: "no study tests it; the page names the reason to expect it to work." The page names the mechanism: "A stolen number gets your codes; a freeze stops new accounts in your name." The SIM-swap trend tests the FCC rule as a whole, not a reader turning on a lock, and isn't attributed to locks. It doesn't reach weak for this action.

Old → new. moderate: small share of losses → none but plausible: small share of losses. The qualifier stays because it is still the honest scope: SIM swaps were $17 million of $20.9 billion; new-account fraud touched about 7% of identity-theft victims.

What would overturn it. A study comparing new-account fraud, or SIM-swap loss, between people with and without a freeze or a lock would lift it.

Inequality, step 3: "Do free tax returns in tax season"

Action and outcome. Volunteer at a VITA or NYC Free Tax Prep site. The page's outcome is that each family gets its refund "filed free and right" and keeps the fee (average $346 in New York).

Evidence.

• IRS Publication 5347 (Rev. 9-2026): "the estimated overall accuracy for federal tax returns prepared at eligible VITA/TCE sites nationwide was 98.47%" (row D6, inequality-do-2026-11-06.md, read 6 Nov). This is the IRS's own announced quality review of its own program; volunteers usually know a review is happening (TIGTA 2016, impact review A1). primary (the program's own figures)
• TIGTA's anonymous-shopper audits, 2009–2013, found far lower accuracy, on small judgemental samples (2013: 20 of 39 correct) (impact review A1; TIGTA 2013-40-110). Independent, small and old; it argues against "right". primary
• NYC: "about 40 tax returns" a season per volunteer; the "$346" average fee (rows S9, S11, inequality-do-2026-11-06.md). These are counts and a city average, not a comparison. primary
• Nothing compares families served by a volunteer site with similar families who used a paid preparer, filed alone or didn't file. The fee saved and the refund delivered are counted against an assumed alternative. Linos et al. 2022 (six preregistered trials, one million observations) found that telling people about free sites had no effect on claiming (row S12). That tests outreach, not preparation, and the page already says so.

Rubric clause applied. weak: "only before-and-after, observational or self-reported evidence". The per-family benefit is the operator's own count and accuracy figure. The one independent check (TIGTA) was small and old and pointed the other way. "Certain per family" describes the arithmetic of a count, not evidence that the action changes the outcome.

Old → new. moderate: certain per family → weak: official counts, no trial. The details section already gives both accuracy figures ("the IRS's own reviews put volunteer sites at 98.5%; the last independent tests on anonymous visits, from 2009 to 2013, found far lower rates").

What would overturn it. A new independent accuracy audit at today's sites, or a study comparing outcomes for families at volunteer sites with similar families elsewhere, would lift it to moderate if it agreed with the IRS figure.

---

Proposals for content/trafficking-do.html (not edited; another session is working in it)

T1, step 1. Exact old text: <span class="evidence">moderate: need documented, outcomes self-reported</span><sup><a href="#s3">3</a></sup> <a href="#give">Where to give.</a> New: <span class="evidence">weak: small studies, charities&rsquo; own results</span><sup><a href="#s3">3</a></sup> Whether the services work is thinly studied: a review found six small studies of 161 survivors, most of poor quality.<sup><a href="#s20">20</a></sup> <a href="#give">Where to give.</a> New source at the end of the list: <li id="s20">Dell, Maynard, Born, Wagner, Atkins and House, <a href="https://doi.org/10.1177/1524838017692553">Helping survivors of human trafficking: a systematic review of exit and postexit interventions</a>, <i>Trauma, Violence &amp; Abuse</i>, 2019 (six studies, 155 women and 6 men, four countries; abstract read 6 Nov 2026) <span class="grade">research</span></li>

T2, step 2. Exact old text: <span class="evidence">none but plausible</span> No study shows that public awareness finds victims. In Nepal, an evaluation of awareness training found that knowing the risks didn&rsquo;t lower women&rsquo;s risk of forced labour. What you can be is the informed friend with the number, for someone about to take a risky job.<sup><a href="#s4">4</a></sup> New: <span class="evidence">none but plausible: untested as a friend&rsquo;s role</span> No study shows that public awareness finds victims. In Nepal, women who took a two-day course on migration risks were no less likely to end up in forced labour; the destination and the recruiter set the risk. That tested warning the worker, not having someone to turn to. What you can be is the informed friend with the number, for when a job turns out to be a bad one.<sup><a href="#s4">4</a><a href="#s21">21</a></sup> New source: <li id="s21">Zimmerman, Mak, Pocock and Kiss, <a href="https://discovery.ucl.ac.uk/10135386/1/fpubh-09-645059.pdf">Human trafficking: results of a 5-year theory-based evaluation of interventions to prevent trafficking of women from South Asia</a>, <i>Frontiers in Public Health</i>, 17 May 2021 (in Nepal, &ldquo;participation in trainings did not reduce the likelihood of experiencing forced labor&rdquo;; read 6 Nov 2026) <span class="grade">research</span></li> (Number s20/s21 by whatever the list ends at when the other session finishes; today it ends at s19.)

T3, step 3. Exact old text: <span class="evidence">moderate: self-reported outcomes</span> New: <span class="evidence">weak: the programme&rsquo;s own figures</span> Optional added sentence after the existing <sup><a href="#s5">5</a></sup>: One independent study links the programme&rsquo;s arrival to fewer low-weight births among Latin American-born mothers; it doesn&rsquo;t measure forced labour.<sup><a href="#s22">22</a></sup> with <li id="s22">Rubalcaba and Ortega, <a href="https://read.dukeupress.edu/demography/article/doi/10.1215/00703370-12352500/406173/Worker-Driven-Social-Responsibility-and-Infant">Worker-driven social responsibility and infant health</a>, <i>Demography</i>, 2025 (birth records 2006–2018; abstract read 6 Nov 2026) <span class="grade">research</span></li>

The same actions elsewhere (not edited)

Propagation to earlier records (not edited)

• source/verify/cybercrime-do-2026-11-06.md rows B3, B8, B15, D10, source/verify/inequality-do-2026-11-06.md row S10, and source/verify/trafficking-do-2026-11-06.md rows T12, T29 confirmed the old labels against the impact reviews. They checked agreement with the impact reviews' grade keys, which predate the rubric. This file supersedes them for the label only; the claims those rows checked still stand.
• The impact reviews (source/impact/cybercrime-2026-11-05.md §A2–A4, A8; trafficking-2026-11-05.md A1, A4, A6; inequality-2026-11-05.md A1) use their own older grade keys (for example, cybercrime: "moderate (strong mechanism plus authoritative guidance or indirect outcome data)"). Each could carry a one-line "Re-graded 2026-11-06 under the rubric; see source/verify/regrade-2026-11-06.md" in its Corrections section.

Proposals for the rubric

• Pin "several". Rule (5) here reads it as more than two. If the rubric means "two or more independent", cybercrime step 1 becomes contestable at moderate.
• Say how a company's or charity's study of its own program counts. Rule (4) here treats it as self-reported, the same way for a tech company and a charity. Writing that into the rubric would stop a large vendor dataset and a charity's outcome report being graded differently.
• Say what counts as a test of the action. Rule (2) here counts only studies of the action or the program it pays for, against the page's outcome. Neighbouring studies (Scheibe for the call-back rule, Nepal for the informed friend) are recorded but not counted. The cybercrime and trafficking steps turned on this more than on any other clause.

No tracking on any link.Ticks stay on this device.What changedContested