vera

Evidence

Everything behind the case

The data, tests and decisions behind every claim on the case page. Each plan and scoring rule was committed before the data it applies to, so nothing here was tuned to fit an answer.

The forum data

Public posts from people who build software, collected, cleaned, read and coded by pain.

36,769

posts in the corpus, from 78,276 pulled across 6 rounds and 3 sources · 19 Oct 2021 – 18 Oct 2026

Read by hand480random samples from each round
About a real pain31%148 of 480 read
Coded by pain736posts with at least one code

From pulled to coded

Each step removes posts that can't tell us about a pain. The striped bar is an estimate from the sample.

Show as table
Posts at each step of the pipeline
Pulled78,276Everything the search API returned
Contain the term36,76947% of pulled · typo matches removed
About a pain~11,33731% of 480 read, applied to the corpus
Coded736Posts with at least one pain code

Where the posts come from

Hacker News is mostly engineers. Stack Exchange adds designers and project and product managers. Hover for how many were read and found to be about a pain.

Show as table
Corpus posts per source
Hacker News26,084127 of 380 read were about a pain · since 19 Oct 2024
UX Stack Exchange8,7150 of 50 read were about a pain · since 19 Oct 2021
Project Management SE1,97021 of 50 read were about a pain · since 19 Oct 2021

Posts per month

Corpus posts by the month they were written. Hacker News starts in October 2024; the Stack Exchange sites go back five years because they're small. The last month is partial.

  • Hacker News
  • Stack Exchange
Show as table
Corpus posts per month, by source
MonthHacker NewsStack ExchangeTotal
Oct 20210177177
Nov 20210303303
Dec 20210249249
Jan 20220408408
Feb 20220357357
Mar 20220321321
Apr 20220303303
May 20220237237
Jun 20220265265
Jul 20220331331
Aug 20220289289
Sept 20220243243
Oct 20220304304
Nov 20220258258
Dec 20220203203
Jan 20230241241
Feb 20230299299
Mar 20230325325
Apr 20230190190
May 20230254254
Jun 20230217217
Jul 20230266266
Aug 20230318318
Sept 20230265265
Oct 20230180180
Nov 20230182182
Dec 20230225225
Jan 20240224224
Feb 20240192192
Mar 20240229229
Apr 20240181181
May 20240142142
Jun 20240103103
Jul 20240170170
Aug 20240226226
Sept 20240225225
Oct 2024285157442
Nov 2024701100801
Dec 202472185806
Jan 2025824139963
Feb 2025801122923
Mar 202583996935
Apr 202578674860
May 2025995861,081
Jun 202588371954
Jul 20251,114801,194
Aug 20251,203841,287
Sept 20251,069931,162
Oct 2025999881,087
Nov 202588761948
Dec 20251,223671,290
Jan 20261,528611,589
Feb 20261,505411,546
Mar 20261,716581,774
Apr 20261,260371,297
May 20261,388391,427
Jun 20261,195341,229
Jul 20261,180351,215
Aug 20261,175301,205
Sept 20261,095341,129
Oct 2026 (partial)71211723

By stage of the work

Posts by the part of a product team's work their search terms covered. A post can count in more than one stage. Stack Exchange posts have no stage: they came in as whole sites, not through a term.

Show as table
Corpus posts per stage
coordination6,89913 of 38 read were about a pain
review5,89216 of 34 read were about a pain
feedback4,1915 of 33 read were about a pain
plan3,83414 of 36 read were about a pain
build1,73611 of 31 read were about a pain
design90311 of 31 read were about a pain
release6897 of 31 read were about a pain
spec60712 of 33 read were about a pain
docs57612 of 30 read were about a pain
qa41520 of 32 read were about a pain

Round 1 · 19 Oct 2026

Round 1 search terms, committed before any data was pulled. They cover each stage of a product team's work (plan, spec, design, build, review, QA, release, feedback, docs, coordination) and are phrased neutrally. Each term casts a wide net: a relevance pass later removes posts that aren't about work pain. Blast Radius gets no terms of its own. 'waiting on engineering' covers it the same way it covers every other waiting pain.

Search terms
32
Posts pulled
1,623
Contain the term
1,343 (83%)

Round 2 · 19 Oct 2026

Round 2 widens round 1. Round 1 combined quoted phrases with extra required words, which cut results sharply (e.g. '"code review" slow' found 209 posts, while '"code review"' alone finds 6,060). It kept 1,623 posts in total. Round 2 searches the core phrases on their own and adds common variants, over the same topics and the same window. This brings in more posts that aren't about pain at all. The relevance pass is there to remove them. No new topics, and still no terms for any candidate product.

Search terms
44
Posts pulled
66,346
Contain the term
24,974 (38%)

Round 3 · 19 Oct 2026

Round 3 adds the views of designers and project/product managers, since Hacker News is mostly engineers (codebook v1 found non-engineers depending on engineering once in 300 HN posts, which HN alone can't settle). Sources: UX Stack Exchange and Project Management Stack Exchange. Both sites are small and entirely on topic, so the round takes every question and answer in the window instead of filtering by search terms; the relevance pass sorts them later. The window is five years, not two, because a count-only probe (no content read) found 1,452 UX and 233 PM posts in two years against 8,762 and 1,974 in five. Comparisons over time must account for the different window.

Search terms
2
Posts pulled
10,685
Contain the term
10,685 (100%)

Round 5 · 19 Oct 2026

Study 02: GitLab merge requests (handbook, marketing site, product) with author and reviewer roles from GitLab's public team data. Behavioral data, in place of Reddit (round 4, dropped).

Search terms
2
Posts pulled
0
Contain the term
0 (–)

Round 6 · 19 Oct 2026

Round 6 adds the voices of product engineers and technical product managers: Hacker News posts where people name the role, either their own ("as a PM", "I'm a product engineer") or the role as a topic ("product engineering", "technical product manager"). The terms are about who's speaking, not about any pain, so they don't favour a hypothesis or candidate. Same two-year window as rounds 1 and 2. A relevance pass later removes posts that aren't about work problems.

Search terms
12
Posts pulled
1,202
Contain the term
1,066 (89%)

Round 7 · 20 Oct 2026

Stage 3 deep dive: a random sample of 6,000 merged GitLab product MRs (Oct 2025 to Oct 2026) with AI-disclosure markers from descriptions and commits, plus the review-time trend over two years.

Search terms
0
Posts pulled
0
Contain the term
0 (–)