Retention Forecaster

Look at any subscription business that keeps its numbers by signup month and the same pattern turns up. The first renewal loses a crowd. The second loses fewer. By the fifth or sixth period the people still paying seem almost settled, and the renewal rate creeps toward ninety percent or more. It is tempting to read that climb as loyalty being earned, as if each customer grew fonder of the product with every bill. Usually nothing of the sort is happening, and the difference matters the moment someone asks what next year will look like.

Why the renewal rate climbs

Picture a cohort as a mix of people with different habits. Some signed up on a whim and were always going to leave quickly. Others needed exactly what was on offer and would stay for years. Each person may have a churn probability that never changes at all, yet the group’s renewal rate still rises, because every period removes more of the restless and leaves a crowd that is, on average, more settled. Nobody changed. The mix did.

That single idea explains most of what cohort tables show, and it also tells you how to extend them past the last column you have.

Peter Fader and Bruce Hardie turned that picture into a small model in “How to Project Customer Retention”. Each customer leaves at the end of any period with a fixed probability, so the period in which that customer leaves follows a geometric distribution, the memoryless count of trials up to a first success. The probabilities themselves differ across customers and follow a beta distribution, a flexible family on the interval from zero to one with two shape parameters, alpha and beta. They call the mixture the shifted beta geometric model.

Integrating the geometric over the beta gives closed forms through the beta function. The chance that a randomly chosen customer is still active after period t is the survival function S(t), and the ratio of neighbouring values has a neat shape: the retention rate in period t is (beta + t - 1) divided by (alpha + beta + t - 1). The numerator and denominator grow by one each period, so the ratio climbs toward one. That climb is the arithmetic of the sorting story, not a claim about loyalty.

The mean churn probability is alpha divided by alpha plus beta. Small values of both parameters mean a very mixed crowd, with many near-certain leavers and many near-certain stayers; large values mean everyone behaves alike, and the model then collapses toward a single constant churn rate.

Fitting two numbers to a handful of counts

What the model needs is modest: how many customers of each cohort were still active at the end of each period. No event log, no customer records, just a table of survivor counts.

Customers lost in period t each contribute the probability of leaving exactly then, and customers still active at the last observed period contribute the probability of surviving at least that long. That second piece is right censoring, the standard way of handling lifetimes that are known only to exceed some value. Multiply the pieces, take logarithms, and the likelihood function of alpha and beta is ready.

Maximum likelihood estimation then picks the pair that makes the observed counts most probable. Several cohorts of different ages simply add their pieces, which is why a staggered table, where the oldest cohort has eight columns and the newest has two, is perfectly usable.

Fader and Hardie tested the idea on two segments of an unnamed subscription business, recorded as the share of an initial group still active in each of thirteen years. As counts of 1,000 customers, the High End segment runs 1000, 869, 743, 653, 593, 551, 517 and 491 over years 0 to 7. Fitting only those eight numbers gives alpha 0.668 and beta 3.806, which is what the paper reports. The Regular segment gives 0.704 and 1.182. The implied mean churn probabilities are about 0.15 and 0.37.

Load the High End data in the tool and set the holdout to fit years 0 to 7, and you can watch the next step. The fit projects 37.8% of the original customers still active in year 12. The paper’s data show 39.4%. A miss of about four percent of the observed value, five years past the last number the model saw, is a remarkably honest forecast for two parameters.

Why a single churn rate under-forecasts

Now give the same eight numbers to the simplest alternative, one churn rate for everybody. Its best estimate for the High End segment is about 10.3 percent lost per year, and compounding that forward leaves 27.0% in year 12. The constant rate is the average over a crowd whose most restless members have already gone, so it charges the loyal survivors for behaviour they never had. The paper makes the same point about curve fits: linear and exponential trends fitted to the first years underestimated year 12 survival badly, and the linear trend would even have gone below zero after year 14.

The gap compounds into money. In “Customer-Base Valuation in a Contractual Setting: The Perils of Ignoring Heterogeneity”, the same authors define the discounted expected residual lifetime, or DERL, of a customer who has already renewed some number of times. It sums the chance of each future renewal, given survival so far, discounted back to today, and it has a closed form through the Gauss hypergeometric function. Their worked example, with alpha 3.8, beta 15.2, a ten percent discount rate and a customer who has made two renewals, gives 3.59 expected discounted payments. The tool reproduces that figure.

The paper’s argument is short and uncomfortable. Whenever retention rates rise as cohorts age, valuing a customer base with one aggregate retention rate will understate it, and in their illustration by 38 percent. Long-tenured customers are worth more than a flat rate suggests, precisely because surviving this long says something about who they are.

What the band covers and what it cannot

A projection with no sense of its uncertainty invites false precision, so the tool draws a band. It uses the parametric bootstrap: synthetic cohorts with the same starting sizes and the same observed lengths are drawn from the fitted model, the model is refitted to each, and the band runs from the 5th to the 95th percentile of the refitted survival curves. Beside it sit standard errors from the inverse of the observed Fisher information.

Be clear about what that band means. It describes how much the estimate would wobble if the world really worked like the fitted model and you had been dealt a different sample of customers. It says nothing about the model being wrong. A price change, a new competitor or a product that finally works can bend the future away from every curve in the band.

The fit checks exist for the same reason. A Pearson chi-squared statistic compares observed and fitted churners period by period, with sparse cells merged so that expected counts reach five, following the common rule of thumb for that test. The holdout mode fits early periods only and scores later ones. A constant-churn fit sits beside the main fit so you can see whether the rising rate is earning its extra parameter.

Lifetimes, and when not to trust the forecast

Expected lifetime has a closed form, (alpha + beta - 1) divided by (alpha - 1), but only when alpha is above one. Both of the paper’s segments have alpha below one, and then the expected lifetime is infinite: a thin tail of very loyal customers keeps the average from settling. The tool says so plainly and reports the median and a mean truncated at your horizon instead. Discounting rescues the money question, because a discounted sum converges even when the plain one does not.

Some data simply break the model. It describes customers who leave once and never come back, renewing at regular periods under a contract. Win-backs, seasonal pauses and counts that rise are outside it, so the tool flags a rising row and leaves it out rather than fitting it as if nothing happened. It also treats a blank cell as not yet observed and a typed zero as everyone gone, which are different statements.

Trust the forecast least where the data say least. Two periods from one small cohort will fit almost anything, and a pair of huge parameters sitting at the edge of the search range means the counts look like a constant churn rate after all. Changes in how customers were acquired break the assumption that future cohorts resemble past ones.

Used with those limits in mind, the model offers something a trend line cannot: a reason for the shape of the curve. Retention improves with age because the crowd sorts itself, and once you see that, the long tail of patient customers stops looking like luck and starts looking like the most valuable part of the base.