The business problem. Acquiring a new B2B customer costs five to seven times more than retaining an existing one. Customer churn costs U.S. businesses an estimated $136 billion annually. To make that concrete: a mid-sized B2B operation carrying a $55 million annual-revenue customer portfolio — the scale modeled in this study — can lose several million dollars a year to departures that were detectable months in advance, simply because no one was watching for the right signals. Yet most organizations either deploy no AI for departure detection, or spend years and millions of dollars cleaning and labeling data before any model runs. This study demonstrates a governance-first alternative: a supervised AI departure detector built without a single historical departure label — trained entirely on synthetic behavioral episodes generated from domain expertise. Behavioral surveillance and anomaly detection are the ideal entry point for organizations new to AI: low data requirements, interpretable outputs, measurable economic value, and no multi-year data preparation program required.
What was built. Five model architectures were evaluated against 175 synthetic B2B customers observed over 156 weeks. Every model was scored against the same locked ground truth using an economic framework — Economic Detection Potential, False Positive Penalty, and Net Economic Value — rather than statistical accuracy alone. The result: a corrected Sequential Fusion architecture that combines a Markov Switching Model (Stata), an LSTM trajectory challenger (MATLAB), and a Fourier amplitude detector (Python) to achieve +$3,323k net economic value with only $278k in false positive costs.
How it was built. This study was completed in less than one week of elapsed effort. Claude (Anthropic) wrote all data pipelines, feature engineering scripts, the scoring harness, the GAP branch detector, and the corrected fusion architecture. ChatGPT (OpenAI) implemented the MATLAB Hidden Markov Model and LSTM challenger. The human author served as quality manager throughout — defining architecture, demanding proof for every claim, cross-checking every number against source CSV files, and running OCC 5C governance reviews at each session transition. When AI systems made errors — fabricated results, logic inversions, double-counting — the author identified them through governance challenge and corrected them. The OCC framework is what made the speed possible without sacrificing accuracy. When AI writes the code, schedule risk nearly disappears — quality risk becomes the binding constraint. AI-era project management monitors evidence quality, not milestone dates. This is the paradigm shift.
Traditional machine learning demands clean predictor variables — imputed, normalized, de-duplicated, and validated across systems before a single model runs. Organizations routinely spend millions and years on data engineering before AI development begins. This study demonstrates a different path: the only input required was a clean weekly order history. No CRM integration, no labeled churn outcomes, no external enrichment. Domain expertise converted into synthetic behavioral episodes replaced years of historical label collection. The entire project was completed in less than one week of elapsed effort using AI-assisted development governed by the OCC quality framework.
A full working paper documenting the methodology, governance findings, and twenty lessons learned is available for download below.
Why this is a hybrid cognition study. The five-architecture comparison is the visible half of this exercise. The more general finding sits at the end of it: no single architecture, however well it scored, was sufficient on its own — and the gap the best model left open was not closable by a better model. It was closable by a person who knew something the data had not recorded yet. So the study closes by combining the two in a Bayesian belief network: the machine's forecast and the account team's field judgment enter as separate evidence nodes over a shared prior, each expressed in decibans, and sum into a single auditable posterior. Neither layer overrides the other and both are logged. That structure — machine plus human judgment, combined under one arithmetic rather than one deferring to the other — is what this practice means by hybrid cognition, and it is the thread running through the rest of this site's work. The full treatment begins under Hybrid cognition in practice below.
| Architecture | AUC | TP | TN | FP | FN | EDP | FPP | Net | Prec | Recall |
|---|---|---|---|---|---|---|---|---|---|---|
| Markov Switching ModelStage123 · max WoE v2 | 0.5260 | 15 | 44 | 113 | 3 | $3,405k | $2,748k | +$656k | 0.098 | 0.833 |
| Hidden Markov ModelStage123 · state estimation | 0.5722 | 15 | 25 | 132 | 3 | $3,571k | $3,690k | −$119k | 0.102 | 0.833 |
| Isolation Forest best net Stage123 · anomaly detection | 0.4391 | 15 | 22 | 135 | 3 | $3,866k | $3,680k | +$186k | 0.100 | 0.833 |
| LSTM + Fourier Hybrid challenger Trajectory learning · routed architecture | 0.8581 | 16 | 138 | 18 | 2 | $1,567k | $280k | +$1,288k | 0.471 | 0.889 |
| Sequential Fusion v2 best precision MSM watchlist · LSTM confirmation filter | — | 16 | 140 | 16 | 2 | $3,601k | $278k | +$3,323k | 0.500 | 0.889 |
Environment setup is next. Before any formal system testing or UAT can occur, the software and data must move to appropriate environments. Platform, licensing, and infrastructure decisions for each model component will be evaluated in this phase in consultation with IT and the relevant platform vendors. Representative data — either a sanitized subset of real customer records or a recalibrated synthetic dataset — must be staged in the target environment. IT approval of the platform stack is a prerequisite, not a parallel track.
System testing in the target environment follows. Once the stack is deployed on approved infrastructure with representative data, formal system testing re-verifies that models produce correct outputs in the new environment — confirming the migration introduced no errors, the scoring harness operates correctly, and audit file generation is intact.
UAT follows successful system testing. User Acceptance Testing is operationally focused. It asks: Do account managers find the departure risk scores actionable? Does the action ladder align with how the organization actually responds? Are the scoring cadence and reporting format compatible with existing workflows? UAT requires real users in a representative operational context and cannot begin until system testing has passed.
Controlled deployment follows UAT. A limited production rollout — one business unit or customer segment — with defined success metrics, a rollback protocol, and ongoing OCC governance reviews.
Downstream deployment decisions by component. Platform, licensing, and environment alternatives for each model component — Stata MSM, MATLAB LSTM, and Python scoring harness — will be evaluated in consultation with IT and the relevant platform vendors during the environment setup phase. The fusion decision CSV is the natural integration point for any downstream system. OCC governance travels through all environments — it is not a development-only tool.
Discovery 20 in this study's governance log is the broadest finding of the whole exercise: effective decision support in complex environments rarely comes from one model, however accurate — it comes from structuring multiple evidence streams, including human judgment, so they can be combined and audited together. The mechanism that makes that combination rigorous, rather than a gut-feel override sitting on top of a model's number, is Bayesian Weight of Evidence (WoE), implemented here as a Bayesian belief network: a fixed population prior, six evidence nodes, and one posterior — with the machine's forecast entering as a node alongside the human's, not above it.
Every signal in this study — whether produced by a model or stated by a person — is expressed in the same unit: decibans (dB), a log-odds scale measured relative to a fixed population baseline (10% prior departure rate). Because WoE is additive in log-odds space, a machine's forecast and a salesperson's field judgment can be summed directly into one auditable posterior, instead of one silently overriding the other. That additivity is the entire trick — it's what lets "hybrid cognition" mean something more precise than a person ignoring the model when they feel like it.
None of this matters if it doesn't change what someone does before the customer is actually gone. The entire economic argument in this study — Economic Detection Potential (EDP) weighed against False Positive Penalty (FPP) — rests on one assumption: the value an organization can still recover from a departing customer declines the longer detection takes, because the intervention window narrows as the relationship deteriorates. A forecast that arrives after the customer has effectively decided to leave has academic interest and close to zero economic value. The entire point of looking ahead — statistically, and now with the human layer added — is to surface the signal early enough that there is still a real decision to make and a real action that can change the outcome. Hybrid cognition exists to shorten that detection lag without manufacturing false confidence: the human layer fills exactly the gap where the model has a real but not-yet-confirmable signal, instead of forcing a choice between acting on nothing and waiting for statistical certainty that may arrive too late to act on.
The same network as a running ledger — how 10% becomes 83.2%
Each row adds one Bayes factor to the running log-odds total. The right-hand column is that total expressed back as a probability, so the effect of each individual piece of evidence is visible on its own line rather than buried in a single output number.
| Evidence node | Source | Bayes factor | Running total | P(departure) |
|---|---|---|---|---|
| Population prior | Locked 10% base rate | — | −9.5 dB | 10.0% |
| Machine forecast | Fusion v2 weekly WoE | +4.3 dB | −5.2 dB | 23.0% |
| Account tier (A) | Registry — empirical | +3.2 dB | −2.0 dB | 38.5% |
| Revenue concentration | Registry — empirical | +1.0 dB | −1.0 dB | 44.0% |
| Champion turnover | Salesperson — logged | +6.0 dB | +5.0 dB | 75.8% |
| Competitor mention | Salesperson — logged | +2.0 dB | +7.0 dB | 83.2% |
| Call sentiment | Not entered | 0.0 dB | +7.0 dB | 83.2% |
| Budget-cycle signal | Not entered | 0.0 dB | +7.0 dB | 83.2% |
| Total evidence | 1 machine · 2 calibrated · 4 judgment | +16.5 dB | +7.0 dB | 83.2% |
C011 is a Tier A account carrying $1,097.0k of annual revenue, departing by market-share erosion. It is a true departure and every architecture eventually called it correctly — this is not a story about a model getting the answer wrong. The MSM surveillance layer flagged it early and was credited $713.1k of Economic Detection Potential. The confirmation path reached the same conclusion about the same customer and was credited $54.0k. Both are true positives. Both are verified in the Customer Validation Workbook. The $659.1k difference between them is produced entirely by when the call was made, not by whether it was right.
The fusion weekly-score files show why. The trajectory layer's signal on C011 first appears in week 73, and under the escalation policy it is logged as a review-only signal — real, recorded, and unable to trigger an action on its own — for 75 weeks before the surveillance layer independently crosses its threshold and an action fires in week 135. That 75-week window is not a modelling defect to be engineered away; it is the honest state of the evidence, and it is precisely the interval in which a person who knows the champion has left holds information the behavioural data has not yet expressed.
Applying the locked recovery formula to that window makes the stake explicit: an ESCALATE action on C011 in week 73 models $389.2k of recoverable value against the $54.0k the late confirmation was actually credited. The belief network is the mechanism for getting that evidence into the decision in week 73 in a form that survives review — rather than as a phone call nobody wrote down.
The other two nodes have already been through the process that fixes this. Account tier (+3.2 dB) and revenue concentration (+1.0 dB) are empirical Bayes factors computed from the customer registry — how much more often Tier A and revenue-concentrated accounts actually appear among departures than among stable accounts. They were estimates once too. What converted them was outcome data.
Every logged override is a row in the dataset that would do the same for the remaining four. Once enough overrides accumulate against known outcomes, each judgment node can be re-derived as an empirical Bayes factor, and the correlations between them — the conditional independence assumption noted under Figure 1 — can be measured rather than assumed. The governance discipline applied to the models in this study was that every number traces to a source and nothing is asserted without evidence. Applying that same discipline to the human contribution is what distinguishes this from an override button, and it is the reason the logging requirement is structural rather than administrative.