Skills · research

`hypothesis-comparison`

Compare rival explanations for an observation with analysis of competing hypotheses (ACH) — list the hypotheses and the evidence, build a consistency matrix (consistent / inconsistent / neutral), rank hypotheses by how little evidence contradicts them rather than how much supports them, name the diagnostic evidence and the linchpins, and say which next observation would tell the leaders apart. Use when the user asks "why did X happen", "which explanation fits the evidence", "what's really causing this", wants to stop arguing over a favourite theory, or says "compare these hypotheses", "competing explanations", "ACH".

Pack researchTier heavyJob Research a questionv0.1.5

npx skilldrop-cli install hypothesis-comparison

Other ways to install

With the sibling skills it hands off to:

npx skilldrop-cli install hypothesis-comparison --with-related

The whole pack it belongs to:

npx skilldrop-cli install --pack research

By hand, from a clone of the repo:

cp -R packs/research/skills/hypothesis-comparison ~/.claude/skills/

In Claude Code, with the research plugin installed, invoke it as:

/research:hypothesis-comparison

Try it

A realistic prompt from the skill's acceptance evals:

Acme's monthly churn on the Starter plan went from 2.1% in July to 3.4% in August. Theories: the price rise from $19 to $24 on 1 Aug, a new competitor (Northwind Lite) launching in late July, or the onboarding email sequence that broke on 20 Jul. Evidence: churn rose on monthly and annual-renewal customers alike; cancellation survey 'too expensive' share went from 30% to 32%; customers who signed up after 20 Jul churn at 5.8%, older cohorts at 2.4%; win-back calls mention Northwind Lite twice out of 40. Which explanation fits best and what should we check next?

What a good result looks like

At least four rival hypotheses, including a measurement one, each a sentence that could be false.
Evidence rows are observations with a source, including at least one absence where one exists. Nothing in the matrix is evidence the user didn't supply, except rows tagged [assumption].
The ranking is by inconsistency. The write-up never argues for a hypothesis by counting its C ratings.
Non-diagnostic evidence is named and set aside, not used as support.
Linchpins are named and checked: the rows whose removal changes the leader, with where each came from.
The conclusion carries a confidence word and a reason, and stays tentative when one row holds it up.
The next observation discriminates between the top two, with the result each one predicts.

Example output

why did Northwind's mobile checkout conversion drop? · the skill's own worked example, source

Input

Our checkout conversion dropped about 18% starting Tuesday 8 Sep and everyone has a theory. Engineering says it's seasonal, marketing thinks Contoso's 20%-off promo (launched 1 Sep) is pulling people away, and I suspect the checkout release we shipped Monday 7 Sep at 22:00. What we know:

last September we never saw a dip bigger than 3%
sessions and add-to-cart are flat week on week
it's only mobile Safari; desktop and Android look normal
the payment provider's dashboard shows settled orders down 17% for the same days
support tickets about "card declined" or "button not working" doubled since the 8th
the tag manager container was also republished on the 7th
our synthetic checkout test on iOS Safari has passed on every run since the release

Which explanation actually fits?

Output

Competing hypotheses: Northwind mobile checkout conversion drop

Prepared 2026-09-15 · Evidence as of 14 Sep · Method: analysis of competing hypotheses

Observation

Checkout conversion fell about 18% from Tue 8 Sep, on mobile Safari only.

Assumptions to confirm first

[assumption] "Conversion" means orders ÷ sessions in the web analytics tool, the same definition used last September.
[assumption] The payment dashboard counts orders by settlement date, so its 17% fall covers the same days as the conversion drop.
[assumption] The Safari-only split comes from the order records, not only from the analytics tag. If it comes from the tag, E4 says nothing about H4.

Hypotheses

Id	Hypothesis	Could be true alongside
H1	The 7 Sep checkout release broke payment for some iOS Safari users	H3
H2	It's a normal seasonal dip	H3
H3	Contoso's promotion is taking buyers who would have checked out with us	H1, H2
H4	Nothing changed for customers; the 7 Sep tag manager change broke conversion tracking	none

H4 is the measurement hypothesis. Nobody had raised it, but the tag manager change on the same day makes it a real rival.

Evidence and matrix

Rated row by row. Weights: E4 is 2 because a pattern this specific (one browser on one platform) is hard to produce by chance; E6 is 0.5 because a promotion's existence says little about who it affects.

Id	Evidence	Weight	H1	H2	H3	H4	Diagnostic?
E1	Fall starts Tue 8 Sep; release went out Mon 7 Sep 22:00	1	C	N	N	C	no
E2	Last September never dipped more than 3%	1	N	II	N	N	yes
E3	Sessions and add-to-cart flat week on week	1	C	I	I	C	yes
E4	Drop on mobile Safari only	2	C	I	I	N	yes
E5	Payment provider: settled orders down 17%	1	C	C	C	II	yes
E6	Contoso promo launched 1 Sep	0.5	N	N	C	N	no
E7	"Card declined" / "button not working" tickets doubled	1	C	N	I	I	yes
E8	Tag manager container republished 7 Sep	1	N	N	N	C	no
E9	Synthetic iOS Safari checkout test passes every run	1	I	N	N	N	yes

Ranking

Script output (python3 scripts/ach_matrix.py conversion-drop.csv), ranking section:

| Rank | Hypothesis | Inconsistency score | I | II | C or CC |
|---|---|---|---|---|---|
| 1 | H1 checkout release | 1 | 1 | 0 | 5 |
| 2 | H4 tracking change | 3 | 1 | 1 | 3 |
| 3 | H3 competitor promo | 4 | 3 | 0 | 2 |
| 4 | H2 seasonal dip | 5 | 2 | 1 | 1 |

**Least contradicted:** H1 checkout release (score 1), 2 ahead of H4 tracking change.

What the ranking rests on

Diagnostic evidence: E4 (Safari only) and E3 (flat traffic) count against the seasonal and promotion hypotheses, because both would hit every device and cut sessions. E2 rules out "seasonal" almost alone. E5 (real orders down) is the one row against the tracking hypothesis.
Non-diagnostic, set aside: E1, E6 and E8. The timing (E1) is the reason the release feels guilty, but it fits the tracking change just as well. The script flags all three:

**Non-diagnostic:** E1, E6, E8. These count the same against every hypothesis, so they cannot help you choose. Keep them for completeness; don't cite them as support.

Linchpin: E5. The script's sensitivity check:

- **E5** (Payment-provider dashboard shows settled orders down 17% for the same days): without it, first place is H1 checkout release and H4 tracking change (tie)

Everything that separates "customers are failing to pay" from "we stopped counting properly" rests on the payment dashboard. Confirm it matches the order database for 7–14 Sep before acting on this ranking.

Conclusion

H1, the checkout release, is the least contradicted explanation. Confidence: moderate, because the lead over H4 rests on one row (E5) that hasn't been cross-checked, and E9 counts against H1.

H2 and H3 can be dropped as main causes: each conflicts with the Safari-only pattern and the flat traffic. H3 can still be acting in the background at a small scale.

What would rule out each survivor:

H1: iOS Safari payment errors flat across the release, in the payment SDK logs.
H4: the order database agreeing with the payment dashboard that real orders fell 17%.

E9 is worth a look on its own: the synthetic test may not cover the path real users take (saved cards, Apple Pay), which would make it a weaker row than its weight of 1 suggests.

Next observation

Check	H1 predicts	H4 predicts	Who	How long
iOS Safari payment SDK errors, 6–10 Sep, by hour	A step up after 22:00 on 7 Sep	No change	Payments engineering	Under an hour
Order database count vs payment dashboard, 7–14 Sep	Both down about 17%	Database flat, which means the payment dashboard is wrong too	Data team	Half a day

Run the first check today. If it shows the step, roll back or hotfix the release and watch iOS Safari conversion for 48 hours.

How it's checked

Measured against plain Claude Code (2026-10-23): it met 69% of its checks, against 45% without it, and a blind judge preferred it. How skills are checked

21 assertions across 3 acceptance evals, and 8 trigger queries. Every skill's checks

What the output must do

Lists at least four rival hypotheses, including a measurement or definition hypothesis the user did not raise
Builds a matrix rating every evidence item against every hypothesis as consistent, inconsistent or neutral (or CC/II/NA)
Ranks hypotheses by how much evidence contradicts them, not by how much supports them, and shows the scores or arithmetic
Names the non-diagnostic evidence (rated the same for every hypothesis) and does not use it as support
Names the linchpin evidence whose removal would change the leader, or states that none exists
Gives the conclusion a confidence word with a reason
Names one next observation that the top two hypotheses predict differently, with the result each predicts
Does not invent evidence, numbers or sources that are not in the prompt; anything inferred is tagged [assumption]
Reports the drop as about 27% (3,150 kg to 2,300 kg), catching that the owner's '40%' does not match the yard data
Notes that the Hilltop yard's yield was essentially unchanged, so the loss is concentrated at Riverside and Orchard, contradicting 'every hive was hit the same'
Treats mite counts below the treatment threshold in every yard as evidence against varroa as the main cause, and does not endorse the $4,000 treatment programme on the strength of the mite theory
Does not claim the pesticide explanation is proven or enough on its own for a lawyer's letter, and says what is missing
Names a next observation that would separate the leading explanations, such as residue testing of the frozen bag of dead bees
Does not invent evidence (lab results, product names, spray dates) not in the notes
Fits about one page
Answers in a few bullets — no more than about 10 bullets or 250 words — without asking questions first
Weighs all three explanations and does not declare a winner without evidence
Points out that because the three changes happened at about the same time, timing alone cannot tell them apart
Suggests specific, cheap checks the pub can do that would point to one explanation over the others (for example whether weekday or drinks-only trade also fell, whether regulars still come but spend less, what reviews say about the food, or what happens when the road works end)
Does not invent sales figures, dates, or customer feedback not given
Plain language with no analysis jargon

When it should and shouldn't activate

Should activate:

Our signup rate fell 25% last week and we have four theories. Which one does the evidence actually support?
Run an analysis of competing hypotheses on why our API latency doubled after the migration
Build a consistency matrix for these rival explanations of the churn spike and tell me which one survives
The team keeps arguing about what caused the drop in NPS. Help me weigh the competing explanations against the evidence.

Should leave to another skill:

Compare Snowflake, BigQuery and Databricks for our analytics platform and recommend one
Do a SWOT on entering the Canadian market
Summarise these five papers on remote work and productivity, with citations
Critique this root-cause write-up before I send it to the VP

Hands off to

research-plan — when the matrix has too little diagnostic evidence to separate the leading hypotheses. Plan how to collect the observation that discriminates between them.

Source

SKILL.md · the whole folder on GitHub.

hypothesis-comparison