vera

Hypotheses

What we think, and what would prove it wrong

Each hypothesis is written with the result that would disprove it before it is tested. Tests run on the coded corpus, not the sample. A hypothesis that fails gets revised as a new version, and the old one stays here.

G1 · v1 · round 5supported

Waiting for the first review is a large part of how long a change takes, for every role.

Disproved if
Median time to first review is under 10% of median time to merge, for merged MRs in the sample, in all three projects.
Tests
  • supports Median time to first approval (by someone other than the author, not a bot; comments need a login) as a share of median time to merge, sampled merged MRs, per project. Disproved if under 10% in every project. · round 5 · 19 Oct 2026
G2 · v1 · round 5disproved

Non-engineers' changes wait longer than engineers' changes of a similar size in the same project (the Blast Radius claim, measured).

Disproved if
In the handbook and marketing-site projects, the median time to first review for MRs by non-engineers (design, product, marketing, other) is no more than 1.25x that of engineers, comparing MRs that change 3 files or fewer.
Tests
  • disproves Handbook, small merged MRs (3 files or fewer): median time to first approval for non-engineers (design, product, marketing, other) divided by engineers'. Disproved if 1.25x or less. · round 5 · 19 Oct 2026
G3 · v1 · round 5supported

Changes that need reviewers from more than one role take clearly longer than changes reviewed by a single role (approval chains slow work).

Disproved if
Median time to merge for merged MRs whose reviewers span two or more roles is no more than 1.25x that of MRs reviewed within one role, in gitlab-org/gitlab.
Tests
  • supports Product repo, merged MRs with reviewers: median time to merge when reviewers span two or more roles, divided by one role. Disproved if 1.25x or less. · round 5 · 19 Oct 2026
H1 · v1 · round 2supported

The pains people mention most now come after the code is written (review, QA, CI, the cleanup of AI-written code), not during writing.

Disproved if
Codes for the writing stage (unclear-requirements, design-drift, interruptions) together appear in at least as many relevant posts as review-overload + test-ci-friction + qa-capacity + ai-output-quality + bottleneck-shift.
Tests
  • supports Estimated corpus posts with an after-code code vs a while-writing code (weighted by source). · round 6 · 20 Oct 2026
H2 · v1 · round 2disproved

Complaints about review overload and the bottleneck moving have grown over the window, alongside AI coding.

Disproved if
The share of relevant posts coded review-overload or bottleneck-shift is flat or falling across the window's quarters (no upward slope beyond noise).
Tests
  • disproves Share of relevant HN posts coded review-overload or bottleneck-shift per quarter (Q4 2024 to Q3 2026); least-squares slope with a 95% bootstrap interval. · round 6 · 20 Oct 2026
H3 · v1 · round 2supported

Non-engineers depending on engineering (the Blast Radius candidate) is a minor pain in this data.

Disproved if
eng-dependency is among the 8 most common codes. Note: HN mostly reflects engineers' views, so this must be rechecked on PM and design sources before it can rule anything out.
Tests
  • supports Rank of eng-dependency among codes by estimated corpus posts (weighted); PM SE reported separately. · round 6 · 20 Oct 2026
H4 · v1 · round 2open

When people say a small change broke something, it's usually because the change had effects nobody could see (change-impact), not because the edit itself was wrong.

Disproved if
Among relevant posts about something breaking, wrong-edit explanations outnumber change-impact ones.
Tests
  • inconclusive Not testable with codebook v4: there is no 'wrong edit' code to compare change-impact against. · round 6 · 20 Oct 2026
H5 · v1 · round 2open

Of the pains people discuss, the ones agents fit best are review overload, QA capacity and AI missing the team's context. Each is multi-step, uses several tools, and a human can check the result.

Disproved if
Scored at Stage 2, none of these three ranks in the top 5 on agent fit.
Tests
None run yet. Waits on coding the corpus.
H6 · v1 · round 3disproved

Waiting on reviews and approvals is a pain shared across roles: engineers describe it as code review overload, PMs as approval chains. Together it is the single largest group of relevant posts.

Disproved if
review-overload + approval-delays together are not the most frequent pair of codes in the coded corpus, or approval-delays appears in under 5% of relevant PM posts.
Tests
  • disproves Estimated posts coded review-overload or approval-delays vs every other pair of codes; approval-delays share of relevant PM SE posts. · round 6 · 20 Oct 2026
H7 · v1 · round 6open

As PMs and designers start shipping changes with AI, reviewing those changes becomes the open problem: posts about non-engineers building mention review, trust or breakage, not access.

Disproved if
Fewer than 25% of relevant posts coded non-eng-building are also coded review-overload, ai-output-quality, change-impact or code-comprehension.
Tests
  • inconclusive Among posts coded non-eng-building, the share also coded review-overload, ai-output-quality, change-impact or code-comprehension. Under 20 posts: inconclusive. · round 6 · 20 Oct 2026
R1 · v1 · round 7disproved

Review is getting slower: median time to merge in GitLab's product repo rose over the last two years.

Disproved if
The 95% bootstrap interval of the slope of quarterly median time to merge (Q4 2024 to Q3 2026) includes 0 or is below it.
Tests
  • disproves Quarterly median time to merge, merged non-bot product MRs, Q4 2024 to Q3 2026; slope with a 95% bootstrap interval (500 resamples within quarters). · round 7 · 20 Oct 2026
R2 · v1 · round 7disproved

MRs that disclose AI help take longer to merge than those that don't, at the same size.

Disproved if
Median time to merge, AI-disclosed ÷ not, is ≤ 1.0 in at least two of the three size buckets (1–3, 4–10, 11+ files). Inconclusive under 200 AI-disclosed MRs.
Tests
  • disproves Median hours to merge, AI-disclosed ÷ not, by size bucket. · round 7 · 20 Oct 2026
R3 · v1 · round 7supported

MRs that disclose AI help need more review back-and-forth.

Disproved if
Median review comments, AI-disclosed ÷ not, is ≤ 1.0 in at least two of the three size buckets.
Tests
  • supports Median review comments, AI-disclosed ÷ not, by size bucket. · round 7 · 20 Oct 2026
R4 · v1 · round 7open

MRs that disclose AI help get reverted more often.

Disproved if
The AI-disclosed revert rate is ≤ the rest. Inconclusive under 10 reverts in the AI-disclosed group.
Tests
  • inconclusive Share of sampled MRs later reverted (a product MR titled Revert "<title>" created after it), AI-disclosed vs not. · round 7 · 20 Oct 2026
R5 · v1 · round 7supported

Small changes reviewed by people from several roles wait much longer to merge.

Disproved if
For MRs with 1–3 files, median time to merge with reviewers from 2+ roles ÷ 1 role is < 1.25 (supported at ≥ 1.5, inconclusive in between).
Tests
  • supports Sampled MRs with 1–3 files: median time to merge, reviewers from 2+ roles ÷ 1 role. · round 7 · 20 Oct 2026
R6 · v1 · round 7supported

The amount of code each MR asks reviewers to read has grown over the last two years (proposed by Nahuel after R1).

Disproved if
The 95% bootstrap interval of the slope of quarterly median lines changed per MR (Q4 2024 to Q3 2026, generated files excluded) includes 0 or is below it.
Tests
  • supports Quarterly median lines changed per merged non-bot product MR (250 sampled per quarter, generated files excluded), Q4 2024 to Q3 2026; slope with a 95% bootstrap interval. Total lines per quarter (MR count x mean) is descriptive. · round 7 · 20 Oct 2026