Guide
Cohort AnalysisSee the Hole
Everyone says “do a cohort analysis”. Nobody says what to look at. This guide shows how to read the matrix, how to find where the hole is, and what closing it does to LTV, with the arithmetic.
Author: Alp SuleimanPublished
The gap
One blended number hides three different stories
On your retention dashboard you see a single figure: 30% at day 30. Good or bad, you can't say. More to the point, you can't say what is behind it.
Picture a bucket with a hole in the bottom. However much water you pour in from the top, the hole decides the level. The blended retention number tells you how full the bucket is. The cohort matrix shows you the hole.
Example, not real data
All three at 30% on day 30
| Cohort | Day 7 | Day 30 | Day 90 | Reading |
|---|---|---|---|---|
| A | 33% | 30% | 29% | Loses users in week one, then flattens. Whoever is left stays. |
| B | 70% | 30% | 9% | Week one looks fine, but the loss never stops. |
| C | 40% | 30% | 26% | The loss is slowing; the flattening has just begun. |
All three at 30% on day 30
A
- Day 7:
- 33%
- Day 30:
- 30%
- Day 90:
- 29%
- Reading:
- Loses users in week one, then flattens. Whoever is left stays.
B
- Day 7:
- 70%
- Day 30:
- 30%
- Day 90:
- 9%
- Reading:
- Week one looks fine, but the loss never stops.
C
- Day 7:
- 40%
- Day 30:
- 30%
- Day 90:
- 26%
- Reading:
- The loss is slowing; the flattening has just begun.
Same figure on day 30, three different needs: A needs a better first week, B needs value that repeats, C needs a little more time.
Reading it
The matrix: rows are signup periods, columns are age
A cohort is a group of users who signed up in the same period. In the matrix each row is a cohort and each column is that cohort's age: period 0 is signup, period 1 is one period later. A cell shows how many users from that cohort were active at that age, or the percentage. The triangle shape is normal; newer cohorts haven't reached the older ages yet.
Cohort retention is always measured against the cohort's starting size. The period-based retention rate formula in the glossary instead starts from customers at the beginning of the period and subtracts those acquired during it; the two are not the same measure.
Read it in three directions
- 01
Across: one cohort's curve
A row read left to right is that cohort's life. A curve that drops steeply and flattens is a different illness from one that never flattens.
- 02
Down: cohorts at the same age
Moving down a column shows whether newer cohorts are doing better or worse than older ones. This is where you see whether a change worked.
- 03
Blended row: a weighted average
Total retention is the weighted average of your cohorts. That is why Sequoia advises tracking any cohort that is far larger than the rest on its own: one crowded campaign pulls the average toward its own story.
To paste your own matrix and see the heat map, the combined curve and the flattening point, use the Cohort Matrix. Cohort Matrix →
Definition first
Decide what “active” means
Every number in the matrix rests on a definition. Is a user active when they log in, or when they complete the core action? Change the definition and the same users draw a different matrix on the same day. Count logins and the curve comes out high and comforting; count the core action and it drops, but measures getting value more directly.
Tools make this choice too. In Amplitude, “Return On” counts only that exact day and “Return On or After” counts that day and later; Mixpanel defaults to “On or After”. Before you put two charts side by side, know which one you are looking at.
Test question
Does your definition of an active user count the action in which the user gets value, or only that they opened the app?
Framework
Six steps to the bottleneck
These six steps are a framework, not a standard you will find under this name in the literature. Keep the order, but know you can stop at the end of any step.
- 01
Prepare the data and the active definition
User ID, signup date and the periods in which each user was active go into one table. Don't analyse yet: bad data takes you confidently in the wrong direction.
OutputOne table with a row per user, and a written active definition.
When you don't need itIf the product has a few dozen users, don't build a pipeline. Reading the table by hand is enough.
- 02
Build the matrix
Group users by signup week or month and write down, for each cohort and each age, how many were active. A spreadsheet is enough. Colouring the cells guides the eye, but keep the number in the cell.
OutputA triangular cohort × age matrix with a blended row on top.
When you don't need itIf each cohort has very few users, merge cohorts into wider periods. Amplitude warns against drawing conclusions from small cohorts; in one example it treats around 30 users as small, and the line depends on the product.
- 03
Find the drop points
Look for three things: a sudden drop (loss piles up at one age), a steady sloping decay (a little leaves every period) and differences between cohorts (some are always worse). A rule of thumb, which varies by product type: if more than half your users are gone in the first week, start there.
OutputOne sentence: “The loss is biggest at this age and in this cohort.”
When you don't need itIf the curve flattens early and cohorts look alike, look for the hole elsewhere: in acquisition quality or price.
- 04
Cross it with behaviour and funnel
List the actions the users who stayed took in week one and the ones who left did not. Open the funnel from signup to core action and see where people spill out. The trap: as Amplitude notes, cohort analysis reveals associations, not causes. If users who complete their profile in week one stay longer, the profile may not be the reason; a motivated user both completes and stays. That is selection bias, and only a test can tell you the cause.
OutputA short list of actions that move together with retention. Every one is a hypothesis candidate, none is proof.
When you don't need itIf one action determines first value, skip the funnel and look at the completion rate of that action.
- 05
Write a hypothesis and test it
Write one hypothesis for the hole you chose and design the smallest change: remove a form field, add a step to the first session. Split users in two and compare the two groups' cohort curves. Don't change several things at once.
OutputA hypothesis, a control group, and a written note of which age and which curve you will look at.
When you don't need itIf traffic is too low to split into two groups, don't set up an A/B test. Make the change, compare the next cohort with the previous one, and ask yourself: if this improvement hadn't happened, would the next cohort have drawn the same curve?
- 06
Prioritise: modelling
When you can't reach everyone, a churn model that ranks users by churn risk and value comes into play. This step goes last because the model amplifies what the previous five found; it doesn't find what they missed.
OutputA user list ranked by risk score, with an intervention attached to each tier.
When you don't need itIf you're a small startup, you don't need this for the first six months. Matrix first, then the bottleneck, then simple tests. Move to modelling when you see a pattern repeat in the data.
Names you meet in the modelling stepKaplan-Meier · Cox proportional hazards · BG/NBD
Three names you will run into once you reach step six. Each gets a one-line definition and when it comes into play.
Kaplan-Meier
Estimates the share of a group of users that lasts past a given time while accounting for incomplete observations, such as users who are still active. It helps when your cohorts have very different observation windows.
Cox proportional hazards
Relates churn risk to user attributes: which channel, which plan, who leaves faster. It comes into play when comparing segments one by one is no longer enough.
BG/NBD
A probabilistic approach for repeat-purchase customers whose departure is not directly observed; it models how many purchases to expect in a given period and whether the customer is still “alive”. It fits repeat-buying businesses better than subscriptions.
Trap
A rising curve doesn't mean loyalty
When you see the monthly continuation rate rise over time, it is tempting to conclude that users are becoming more loyal. Fader and Hardie describe the “ruse of heterogeneity” (a term they take from Vaupel and Yashin) as well known but often overlooked: customers prone to leaving drop out early, and those who remain are already less prone to leave. The rate goes up, yet not a single user may have changed.
You can check the arithmetic yourself.
Example, not real data
Take two equal groups. One loses half of those remaining each month, the other a tenth. After one month, total retention is 0.50×0.5 + 0.50×0.9 = 70%. After two months, 53%; after three, 43%. The monthly continuation rate climbs from 0.70 to 0.76 to 0.81. Nobody became more loyal; only the mix changed.
If your dashboard says the curve is rising, rule this effect out first; then look at separate matrices by segment or channel.
Patterns
Four bottleneck patterns
The same four shapes keep turning up in the matrix. Each opens a different question.
Early drop
- Symptom
- Loss piles up in the first period; then the curve flattens or slides very slowly.
- First move
- Draw the shortest path that takes a user to the core action in the first session, reduce friction (a shorter signup form, delete needless steps), and give them a small win in the first minutes.
Cause and test question
- Likely cause
- Users leave before they see value: onboarding is weak, or what marketing promised differs from the real product.
- Test question
- At the end of the first session, has the user done what they opened the product to do?
Mid-term decay
- Symptom
- A good first period, then a little more loss every period; the curve never flattens.
- First move
- Design the return loop. The four stages in Nir Eyal's Hooked model (trigger, action, variable reward, investment) work as a framework: what calls the user back, what do they find when they return, and what prepares the next return?
Cause and test question
- Likely cause
- Use doesn't become a habit. The first experience is good but the value doesn't repeat.
- Test question
- For the user to come back a second time, do they need a reminder from outside the product, or does a need arise on its own inside it?
Fluctuation
- Symptom
- The curve rises in some periods and falls in others; cohorts don't resemble each other.
- First move
- First separate whether the wave follows the calendar or the product. In an episodic product, add reasons to use it that keep producing value after the need is met.
Cause and test question
- Likely cause
- Seasonality or episodic use: users come when a need arises and leave when it is met. A job-search app is the classic case.
- Test question
- Is a user whose need is met really a lost user, or has the product done its job?
Low retention in one segment
- Symptom
- Cohorts from one channel, plan or device are consistently worse.
- First move
- A single matrix won't show this: build separate matrices by channel and plan and put them side by side. Then try segment-specific onboarding, pricing or messaging.
Cause and test question
- Likely cause
- For that segment the value proposition is thin, the price doesn't fit, or the experience wasn't designed for that environment (mobile, for instance).
- Test question
- If you removed that segment, would the value proposition still be right?
Side by side with Sequoia's three curvesFlattening · Continuously declining · SmilingThe mapping is our reading, not Sequoia's.
Sequoia sorts retention curves by shape into three archetypes: flattening, continuously declining, and “smiling”, meaning curving back upward. The four patterns here look at the cause behind the shape instead. They ask the same matrix two different questions. The mapping below is our reading, not Sequoia's.
Flattening
Per Sequoia, a group of users has found value and keeps coming back; the higher the level at which the curve flattens, the higher long-term retention.
Continuously declining
Resembles mid-term decay. A lasting core of users hasn't formed yet.
Smiling
Per Sequoia, the curve can rise again when a product is truly exceptional: as the product improves and network effects build, churned users return. Telling this apart from seasonality or a win-back campaign is up to you.
LTV
Turn the drop into money
Before you commit engineering time to a bottleneck, work out what fixing it could do to LTV.
Example, not real data
Three cohorts, one combined curve
All three cohorts started with 1,000 users. At each age we divide the active total of the cohorts that reached that age by their active total one age earlier, and chain the ratios by multiplying; that is how rows of different lengths in a triangular matrix combine.
| Cohort | M0 | M1 | M2 | M3 |
|---|---|---|---|---|
| C1 | 1,000 | 500 | 350 | 250 |
| C2 | 1,000 | 600 | 420 | – |
| C3 | 1,000 | 550 | – | – |
Cohort matrix (active users)
C1
- M0:
- 1,000
- M1:
- 500
- M2:
- 350
- M3:
- 250
C2
- M0:
- 1,000
- M1:
- 600
- M2:
- 420
- M3:
- –
C3
- M0:
- 1,000
- M1:
- 550
- M2:
- –
- M3:
- –
| Age | Period ratio | Remaining share (S) |
|---|---|---|
| M0 | – | 1 |
| M1 | 0.55 | 0.55 |
| M2 | 0.70 | 0.385 |
| M3 | 0.714 | 0.275 |
Combined retention curve
M0
- Period ratio:
- –
- Remaining share (S):
- 1
M1
- Period ratio:
- 0.55
- Remaining share (S):
- 0.55
M2
- Period ratio:
- 0.70
- Remaining share (S):
- 0.385
M3
- Period ratio:
- 0.714
- Remaining share (S):
- 0.275
Take ARPU of 10 per period, 100% gross margin and a horizon of 3 periods (M0 included). The sum of S is 1 + 0.55 + 0.385 + 0.275 = 2.21, so LTV = 22.1. Now suppose you raise M1 retention by 10 points: S becomes 1; 0.65; 0.455; 0.325, the sum is 2.43, and LTV = 24.3. The difference is 2.2, about 10%.
This is a sensitivity calculation, not a forecast. Only trying it will show whether the +10 points actually happens.
Why is the percentage smaller than your gut says?
You lifted M1 retention from 55% to 65%, a relative increase of about 18%. LTV rose about 10%, because M0 didn't change: the user was already there at signup, and about 45% of LTV (before the improvement) comes from that fixed term. The improvement only touches M1 and after.
Formula
LTV = Σ S(t) × ARPU × gross margin
A simpler model: 1 ÷ churn
If per-period churn is constant, average customer lifetime is 1 ÷ churn periods: 10 periods at 10% churn, 20 if churn falls to 5%. That is arithmetic, not an expectation, and it assumes constant churn. In a real matrix churn changes over time (the trap above is part of why), so use it only for order of magnitude.
Extending beyond what you have observed is always an assumption. The Cohort Matrix shows the observed part and the share that rests on projection separately; the larger the second, the less to trust the number.
Run the same calculation on your own data in the Cohort Matrix →
Real lens
In an inflationary economy the matrix can mislead
Cohort numbers are right as counts. Read as money, they bend in a few places.
- 01
Nominal cohort revenue can hide retention
While the customer count falls, price increases can make a cohort's revenue look flat or growing. Retention worsens while the revenue curve looks fine. Convert cohort revenue to the same price level and put the two curves side by side: one tells the story of user count, the other of real revenue. The gap is largely the part carried by price. The full method is in the Real Growth guide.
- 02
Undiscounted LTV makes distant revenue look worth more than it is
When capital is expensive, revenue in month 24 is worth clearly less than revenue in month 6. Summing the long tail of a flattening curve without discounting makes LTV look better than it is. Use a real discount rate when you compute LTV, and bring CAC to the same price level.
- 03
If new cohorts are worse than old ones
If, reading down, each new cohort leaves earlier than the one before, that can be one of the early signs of value debt. The problem shows in the matrix; the bill can come later.
Roadmap
How will you put this to work?
The weeks are an order, not a promise about time. The usage rhythm of your product may stretch or shorten them.
- Week 1
Gather the data, write the active definition
Signup dates, activity records, revenue if you have it. Don't analyse this week. Write the definition and confirm everyone on the team is looking at the same one.
- Week 2
Build the first matrix
In a spreadsheet or straight in the Cohort Matrix. Rows are cohorts, columns are ages. Before you interpret anything, check the numbers in the cells are right.
- Week 3
Pick the biggest hole
Which age, which cohort, which segment? Choose one. The other holes can wait for the next round.
- Week 4
Write three to five hypotheses and look for evidence
Is onboarding too long? Is the core action hard to reach? Do expectation and reality match? For each hypothesis look at the behaviour data, and remember the evidence is an association, not a cause.
- Week 5
Test the most plausible hypothesis with a small change
No big product change is needed. Add a step or remove one. Split users in two.
- The weeks after
Wait for the test cohort to age, then measure
If you will look at day-30 retention, you have to wait 30 days; early numbers can mislead. Compare the test cohort's curve with the control's at the same age.
- Then
Repeat the loop
Find the next hole, write a hypothesis, test it. As the product, the market and user expectations change, new holes will appear; that is normal.
Expectations
Honest expectations
Your first A/B test may fail. Your first hypothesis may not hold. That doesn't mean the method doesn't work; it means a hypothesis was eliminated.
As Lenny Rachitsky writes, startups rarely increase retention significantly. The implication: knowing where to aim matters more than how hard you push.
Some bottlenecks are easy: a form field, a missing step. Others call for rethinking the product. Cohort analysis will show you product-market misfit, but it won't fix it.
In a small sample the matrix misleads. If each cohort has few users, cell-to-cell movement is noise; the Cohort Matrix flags periods with a small denominator as “small sample”.
Industry ranges for retention are on the Benchmarks page. Benchmarks →
Frequently asked
What is the difference between a cohort and a segment?
A cohort groups users by a shared starting point (usually signup period) and follows them over time. A segment groups users by an attribute (channel, plan, device). The two combine: building a separate cohort matrix for each channel is the only way to see low retention in one segment.
How many cohorts are enough?
There is no magic number. You need at least a few cohorts to see differences between them, but the number of users per cohort matters more: Amplitude warns against drawing conclusions from small cohorts. If cohorts are small, widen the period; build monthly cohorts instead of weekly ones.
Should I build weekly or monthly cohorts?
Look at your product's usage rhythm. Weekly cohorts are natural for a product used daily, monthly cohorts for a monthly subscription. As the period shortens, each cohort has fewer users; you have to balance the two.
How should I define an “active user”?
Start with a definition that counts the core action in which the user gets value; logging in alone is usually too weak. Also know your tool's choice between counting “on that day” (On) and “on that day or after” (On or After): Amplitude and Mixpanel name the two counts separately and they give different results.
Does cohort analysis prove an improvement worked?
Not by itself. As Amplitude points out, cohort analysis reveals associations, not causes. To see the effect of an improvement, randomly split users in two and compare the two groups' cohort curves.
Start
See the hole, then decide
The whole guide comes down to one habit: look at the cohort, not the blended number. Paste your matrix, see the flattening point and the biggest drop; the decision that follows is yours.
Let's look at your matrix together
If you want to look at your own cohorts together, we can start with a short exchange.
Talk to Alp →Sources
- 1.Fader & Hardie — How to Project Customer Retention, Journal of Interactive Marketing 21(1), 2007
- 2.Amplitude — Interpret your retention analysis (Return On, Return On or After)
- 3.Amplitude — Cohort analysis
- 4.Mixpanel — Retention report
- 5.Sequoia — What is Retention?
- 6.Lenny Rachitsky (research with Casey Winters) — What is good retention?
- 7.Nir Eyal — Hooked (2014): Trigger, Action, Variable Reward, Investment
- 8.Fader, Hardie & Lee — “Counting Your Customers” the Easy Way, Marketing Science 24(2), 2005
- 9.Kaplan & Meier — Nonparametric Estimation from Incomplete Observations, JASA 53(282), 1958
- 10.Cox — Regression Models and Life-Tables, JRSS-B 34(2), 1972