analysis / 2026.08

Escapes Multiply

How to analyze the ROI of computer vision working next to human inspectors in industrial quality control. With a model you can push on.

Bazyl HorseyChicago, IL

The obvious question

An independent analysis, built only from peer-reviewed research and public industry sources, to show how you might estimate the ROI of pairing people with computer vision. It contains no proprietary data and doesn't describe any company's product, customers or results.

It's the obvious question about computer vision in industrial quality control. If people already inspect every unit, what does a camera add?

The honest answer isn't a percentage. It's a model. This page is that model. Every outside number has its source at the bottom. Every other number is a round, illustrative assumption, and you can drag it.

The math holds anywhere people inspect and a camera checks behind them: final inspection on a production line, incoming inspection of supplier parts, a packaging line, the last check before an order ships. The defaults lean on shipping, where public numbers for a people-only baseline and the cost of an escape are easiest to find. Swap in your own.

The short version: people and machines fail in different ways, and that difference is where the money is.

At the defaults, that's about 39 escapes stopped per 10,000 units and about $115,000 a year net of review labor on 250,000 units, roughly 11 times what the review costs. With every input drawn at random across its range, the vision layer still covers its review labor in 82% of 200,000 draws. Two inputs, how much the two layers' misses overlap and how many real flags review confirms, have the least evidence behind them, and the second matters more. Several studies find people clearing alarms let a lot of real ones go, and review recall moves the money about as much as what an escape costs or how much your people miss. Measure those three first.

Escapes multiply

A defect only reaches the customer if every inspection layer misses it. That one sentence does most of the work. Escapes multiply. They don't average.

e = d_{0} m [1 - c (1 - μ) v]

m = 1 - E_{h} (d_{0})

Q = 1 - e

d₀: upstream defect rate: the share of units with any defect before anyone looks.
Eₕ(d₀): human recall at that prevalence. Not a constant, more on that below.
m: the share of defects people miss.
c: coverage: the share of units the cameras actually see.
R: model recall: the share of present defects the model catches, measured in the field.
μ: how often the model misses a defect people already missed. It depends on R and on ρ, below.
v: review recall, the share of real flags the reviewer confirms. A placeholder, more on that below.
Q: accuracy: the share of units that leave without a defect.

If you averaged, a 90% model and a team that catches about three quarters of defects would land around 82%. Multiply instead and the picture changes. People let about a quarter of defects through, the model catches most of those, and what's left is small. Push the inputs around and watch where the defects go.

fig 01 / where the defects go, per 10,000 units

upstream defect rated₀2.0%coveragec100%model recallR90%miss overlapρ0.20review recallv90%inspector recall drop after camerass0 points

use a measured baseline instead of the human curve

arrive with a defect200
missed by people51
not flagged by the vision layer8
flagged, then cleared in review4

99.5%accuracy, people only

99.88%with the vision layer

75.8%fewer escapes

39defects stopped per 10,000 units

39caught by the vision layer per 10,000 units

People get worse as the floor gets cleaner

The big assumption in the model is that human recall isn't a constant. It falls as defects get rarer. Researchers call it the low prevalence effect, and it's one of the most replicated results in visual search.

In Jeremy Wolfe's lab, observers searching for one kind of target missed about 7% of them when half the displays had one, 16% when one in ten did, and 30% when one in a hundred did. It isn't laziness. The cause is a shift in criterion, not a loss of sensitivity: when something is rare, people want more evidence before they call it. Trained airport screeners show it too, and procedural fixes only partly help. Even letting people take back their last answer leaves some kinds of rare targets missed often.

E_{h} (d) = 0.973 + 0.0589 \ln d

fitted to 1%, 10% and 50% prevalence

Lab numbers vary a lot by task. At 1 to 2% prevalence, other experiments put misses anywhere from about a quarter to a half. I fit the curve to the single-target experiment because it matches this model's one-kind-of-defect setup, and because it sits closest to factory inspection studies, where people catch around 80% of defects. If your people miss more than that, the vision layer is worth more than this page says.

Machine recall has no prevalence term. A model looks at the ten thousandth clean unit with the same attention it gave the first one. That asymmetry is the whole argument for putting the two together. Its precision is another matter, and that comes back as labor further down.

fig 02 / accuracy across upstream defect rates

people only
people plus the vision layer
WERC typical picking accuracy, 99.0 to 99.6%

At 2.0% upstream defects, people catch 74% of them and ship 99.5% clean. With the vision layer: 99.88%.

Hover or tap along the chart to read a point. The vertical line follows d₀ from the first figure otherwise. Both lines use the human curve, so the measured baseline switch doesn't apply here.

Here's the check I'd want before trusting the curve on a real line. The best public benchmark I know of for what people manage alone comes from warehousing: WERC's 2019 DC Measures puts the median for order picking accuracy at 99.4%, with the middle fifth of companies between 99.0% and 99.6% and the top fifth at 99.89% or better. Give the curve an upstream defect rate between about 1.5% and 4.5%, and people-only accuracy lands in that same band. That's a consistency check, not proof. Nobody publishes upstream defect rates, so I can't confirm the 2% default, and a different curve with a different defect rate could land in the same place. What it does show is that the curve and a plausible defect rate end up where real warehouses report.

It also shows why the last half a percent is so hard. As an operation gets cleaner, people catch a smaller share of what's left: about 84% of defects at 10% upstream, 74% at 2%, and 66% at 0.5%. Getting from the median to best in class takes either a much cleaner process upstream or a second layer whose attention doesn't depend on how rare the defect is.

Perfect order rate is a different number. Its medians sit lower, around 93 to 96%, because it also counts late shipments, paperwork and damage in transit, which an inspection before shipping never sees. Picking accuracy is a slightly different denominator too. This model is about the inspection layer only.

Above about 5% upstream defects, people-only accuracy drops below 99%: 98.4% at 10% and 97.6% at 20%. That's an operation with a real upstream problem, and it's where a vision layer pays back fastest. One more caveat: in a mixed experiment from the same 2005 paper, where some target showed up on half the trials, a target type that showed up 1% of the time was missed 52% of the time. That design doesn't map onto a single defect rate, so the fit leaves it out.

The number nobody has measured

At the defaults, ρ swings the share of escapes removed further than any input but review recall, and it has the least evidence behind it. It asks a simple question. When the model misses something, is it the same thing a person would have missed?

If the model and your people fail on different things, the model's recall counts in full. If every miss the model makes is one your people made too, the pair is no better than the model alone. ρ slides between those two ends.

Full overlap doesn't take you back to people alone. At the defaults the model catches 90 of every 100 defects and people catch 74, so at least 16 of the model's catches are ones people missed. At ρ = 1 and perfect review, the pair equals whichever layer is better on its own.

μ = (1 - ρ) (1 - R) + ρ \min (1, \frac{1 - R}{m})

Formally, ρ is the correlation between the two layers' misses divided by the largest correlation their miss rates allow. Zero means independent; one means as overlapped as the rates permit. It isn't a raw correlation: at the defaults, ρ = 0.2 is a raw correlation of about 0.11, so don't plug in one measured some other way.

fig 03 / combined accuracy by model recall and miss overlap

model recallR90%miss overlapρ0.20

ρ \ R	81%	84%	87%	90%	93%	96%	99%
0.00	99.86%	99.87%	99.89%	99.90%	99.92%	99.93%	99.94%
0.10	99.84%	99.85%	99.87%	99.89%	99.91%	99.92%	99.94%
0.20	99.81%	99.83%	99.85%	99.88%	99.90%	99.92%	99.94%
0.30	99.78%	99.81%	99.84%	99.86%	99.89%	99.91%	99.94%
0.40	99.76%	99.79%	99.82%	99.85%	99.88%	99.91%	99.94%
0.50	99.73%	99.77%	99.80%	99.84%	99.87%	99.90%	99.94%
0.60	99.71%	99.75%	99.78%	99.82%	99.86%	99.90%	99.94%

Columns are model recall R, rows are miss overlap ρ. The grid follows your inputs, so the filled cell is exactly what the sliders say. Each step out is 3 points of recall or 0.1 of overlap. Coverage, baseline, review recall v and the recall drop s come from the first figure. With review below 100%, more recall runs into a ceiling. At the defaults and 90% review, no model gets past about 99.95%, because one real flag in ten still gets cleared.

You can't measure ρ from flags alone. The defects both layers miss are exactly the ones nobody flagged, so they never show up in a query. You need ground truth, and it has to come from the right place. A sample of units that passed both layers can't give it to you: every defect in that sample got past both, so it can't show what share the model missed. Draw a random sample of units your people passed instead, whatever the model said about them, and re-inspect them carefully. Add back any defect in the sample that review already confirmed and fixed. For each real defect, note whether the model flagged it. The share it never flagged is μ, and the equation above turns μ into ρ. A defect the model flagged and review cleared is a review miss, not a model miss, and it counts toward v.

Plan for the volume. At the defaults, about 50 of every 10,000 units people pass have a defect, and about 8 of those get past the model too. Ten thousand careful re-inspections pin ρ down to roughly ±0.35, most of the table above. And careful re-inspection is still people searching at about half a percent prevalence, where the curve says they miss about a third. The cheaper, cleaner route is to seed known defects, the way airports project fictional threat images into real bag scans to score screeners. You know the truth, you control the rate, and you can score people, the model and their overlap on the same items. The catch is that seeded defects have to look real. Airport screeners sometimes spot projected threats just because they look pasted in. The 0.2 default here is a placeholder until somebody does that.

Coverage or recall

Coverage and recall multiply, so they trade almost one for one. When misses are independent, a model that sees every unit at 88% recall catches more than one that sees 80% of units at 99%: 0.88 against 0.79.

Overlap changes that, because it hits the weaker model harder, and it hits harder the more defects people already catch. At this page's defaults the order holds, 0.81 against 0.79. Push upstream defects past about 5%, where people catch more of them, and the same ρ of 0.2 flips it: at 20% it's 0.71 against 0.78. So the rule of thumb only holds when misses are close to independent or people miss a lot, which is one more reason to measure ρ before you pick where to spend.

Coverage is still the first thing I'd check. It's the one you can count today, and a station with no camera catches nothing no matter how good the model is.

These are catch shares before review. Review recall multiplies both sides by the same v, so it never changes which one wins.

Cleaner floors gain more, better teams gain less

"Good" can mean two things here, and they move the relative gain in opposite directions.

A cleaner floor means a lower d₀. Under the curve, people catch a smaller share of a rarer defect, so the vision layer removes a bigger share of what they miss: at the defaults, about 77% of escapes at 0.5% upstream defects, 76% at 2%, and 72% at 10%.

A better team means higher recall at the same d₀, which is what the measured baseline switch sets.

1 - \frac{e_{1}}{e_{0}} = c (1 - μ) v

which is c·R·v when ρ = 0

With no overlap the baseline drops out: an operation at 90% and one at 99% cut escapes by the same share. Overlap changes that. A team that already catches most defects is left with the hard ones, and those are the ones a model is most likely to miss too. So the honest expectation for a strong operation is a smaller relative gain, not the same one. Turn on the measured baseline switch in the first figure, leave ρ above zero, and watch the reduction shrink as the baseline climbs.

Neither share is the number to decide on. The money runs on escapes avoided times what each one costs, and a cleaner floor stops fewer escapes even when it stops a bigger share of them. Judge the layer on the last figure, not this one.

False positives move labor, not quality

Accuracy is only half the question. The other half is labor: what does it cost people to clear the flags?

False positives don't change what ships. They change how many minutes somebody spends looking at a clean unit. So I track them on their own line.

FP = (1 - d_{0}) N c FPR

TP = e_{0} N c (1 - μ)

precision = \frac{TP}{TP + FP}

fig 04 / review burden per year

model false positive rateFPR3.0%minutes to clear a flagmin3.0loaded labor cost per hour$/h$28

8,434flags to review

13%of flags are real

422review hours

$11,808review labor

108real flags cleared in review

That assumes the model's flags only get reviewed on units your people passed. If every model flag goes to review, including defects people already caught, the queue is bigger. Price the workflow you'll actually run.

Among the units people pass, clean ones outnumber defective ones about 190 to 1. That's what's left after people do their job, not the upstream mix of 49 to 1, and it's why a 3% false positive rate buries the real flags. If the model looked before people did, about 38% of its flags would be real. The model's recall doesn't care about prevalence, but its precision does: the better your people are, the cleaner the stream it sees. So at the defaults, only about 13% of flags are real. That sounds bad until you price it. Clearing every flag still costs a little under a tenth of what the prevented escapes are worth. The $28 an hour default is the BLS median for hand laborers and material movers, $18.38 in May 2025, with benefits and overhead on top.

Somebody has to clear those flags, and their queue is the setup this page started with, pointed at a new person. Seven of every eight flags are false, so the bar for calling one real drifts up. Read the curve at the queue's 13% and a reviewer confirms about 85% of real flags. Checking a flag is easier than searching a unit cold, so I use 90% as a placeholder and carry it through every figure as v. Studies of people clearing real alarm queues often find worse, so the sensitivity figure further down runs it as low as 30%. At the defaults, about 108 real defects a year get flagged and cleared anyway. They ship, and they count as escapes.

People raise false alarms too: 2% in a classic study of circuit inspection, and 35% of good parts in a recent study of precision manufactured parts, where the same inspectors caught 85% of defects. The fair comparison is against that, not against zero.

Putting a dollar on an escape

An escaped defect costs labor, returns and goodwill. For shipping, vendor estimates put one error somewhere between $50 and $250, without saying where the range comes from. The default here is $150. What an escape costs varies a lot by industry, so if an operation tracks its own cost per escape, use that instead.

Catching a defect isn't free either. Somebody reworks, replaces or repacks it, so the saving per catch is the escape cost minus the cost to fix it in house. The $20 default is a placeholder.

Footage is the last layer, and it's easy to oversell. It doesn't stop an escape or find one nobody flagged. It doesn't get money back on one either. Footage of a real escape either shows the defect, which sinks the claim, or misses it, and then using it against whoever handled the unit next means billing them for your mistake. What it's worth comes from damage and disputes after a unit leaves, which this model doesn't size, so it stays out of the figure below.

One more thing to keep in front of you: the dollars scale with d₀. At 2% upstream defects the base case stops about 39 escapes per 10,000 units. At 10% it stops about 117. A slightly smaller relative gain, three times the money. Use a real defect rate before you trust the dollars.

fig 05 / what it's worth per year

cost per escape$$150cost to fix a caught defect$$20units per yearN250,000

$126,842escapes avoided, net of fixing them

$115,035net of review labor

11xbenefit to review cost

Net and ratio count escapes avoided and review labor, nothing else.

What moves the answer

Every input on this page is a number somebody could argue with. So I gave each one a range and pushed on all of them. The figure below moves one input at a time to each end of its range, with everything else where you've set it.

fig 06 / each input at the ends of its range

input at the low end of its range
at the high end

lowernow 75.8%higher
review recallv30% to 100%25.3% at 30%84.2% at 100%
miss overlapρ0 to 181.0% at 055.0% at 1
model recallR80% to 97%61.6% at 80%85.7% at 97%
coveragec80% to 100%60.6% at 80%75.8% at 100%
share of defects people missm15% to 50%70.8% at 15%79.2% at 50%
inspector recall drop after camerass0 to 10 points75.8% at 069.1% at 10 points
upstream defect rated₀1.5% to 4.5%76.2% at 1.5%74.2% at 4.5%

At your settings the vision layer removes 75.8% of escapes. The biggest swing is review recall: 25.3% at 30% and 84.2% at 100%.

Each input moves to both ends of its range while everything else stays where you've set it, biggest swing at the top. The dashed line is your current answer. The share of defects people miss is set directly here, in place of the curve or a measured baseline. Labor and dollar inputs don't change the share of escapes removed, so that view leaves them out.

The ranges come from the sources where there are any. People's miss rate runs from 15%, what inspectors missed in the precision parts study, to 50%, the high end of lab studies at 1 to 2% prevalence. Model recall runs from 80% up to 97%, where vendor claims start. Review recall runs from 30%, about how often radiologists acted on correct prompts for cancers they had missed, up to 100%. ρ gets its whole range, because nobody has measured it. Upstream defects run from 1.5% to 4.5%, where people-only accuracy lands in WERC's typical band, and cost per escape covers the vendor range of $50 to $250. The rest are judgment calls pushed both ways.

Then I drew every input at once, anywhere in its range, 200,000 times. The layer covers its review labor in 82% of those draws. At the defaults, it breaks even when an escape costs about $32: roughly $12 of review labor per escape stopped, plus the $20 to fix it. With review recall at 30%, it takes about $56. The inputs behind review labor barely move the money: the false positive rate, minutes per flag and wage together explain about 2% of its spread.

Review recall is the biggest unknown. It explains about 60% of the spread in the share of escapes removed, far more than model recall, how much people miss or ρ. It also explains about a third of the spread in the money, as much as what an escape costs or how much people miss, while ρ explains about 3%. Two of the money's big inputs can be measured before anyone buys a camera: what an escape costs from the operation's own books, and how much people miss from seeded defects or a careful re-inspection. Review recall can't be measured until there are flags to review, so it's the first thing I'd audit in a pilot.

The defaults also lean favorable. With everything moving at once, the median is 34% fewer escapes, not 76%. About half of that gap is review recall, whose 90% default sits near the top of its range. Most of the rest comes from two more defaults: cameras that see every unit, and people who don't lean on them. The worst case is a model that misses more than your people do, with every one of its misses overlapping theirs. It adds nothing. Put everything else at the bad end of its range too, including people missing 10 points more once the cameras are in, and escapes go up 59%. That's why the checklist below measures people again after the cameras go in.

Checking it against a real operation

Pick one denominator. Unit, order and customer-compliance accuracy are three different numbers. Agree on one before you measure anything.
Measure the baseline at two prevalences, before and after. The human curve comes from lab studies. Check it against one operation at two different defect rates before relying on it. Then measure it again once the cameras are in, since people lean on a backstop.
Measure ρ from what people passed. Re-inspect a random sample of what people passed, regardless of the model's call, or seed defects with known answers. Flags can't reveal it, and a sample of what both layers passed can't either.
Measure review recall. Audit a random sample of the flags review cleared, and scale the real defects you find to all cleared flags. Then divide confirmed defects by confirmed plus that number.
Count coverage honestly. The units the cameras actually see, not the stations that have cameras.
Count false positives and the minutes to clear one. That's the labor line, which the accuracy numbers never show.
Price an escape with the operation's own numbers. Net of the cost to fix a catch. If footage goes in at all, it goes in on damage claims and disputes, as its own line.

What this doesn't claim

It doesn't say any particular model hits 90%. That's a round figure for the math, not a measured result. Vendors claim 97 to 99.5% for vision inspection without publishing the data behind it, and I'd assume less until field data shows otherwise.

It doesn't say review confirms 90% of real flags either. That's a placeholder between what the curve predicts at the queue's prevalence, about 85%, and a perfect 100%, and studies of people clearing alarms say it could be much lower. Airport screeners ignored about half of true alarms on hard targets when 3 in 10 alarms were real, and radiologists ignored 71% of correct prompts on cancers they had missed. Air traffic controllers, with nearly half their alerts false, showed no such drop. It's the one human step the vision layer adds, and it's much cheaper to measure than ρ, because a flag tells the auditor where to look.

It doesn't say the human curve is a measurement of your floor. It's lab data, checked against factory studies and warehouse benchmarks, not measured on your line. Directionally sound, worth validating.

It holds your people's recall fixed once the cameras go in. People tend to check less carefully when automation is checking too, and if yours do, the vision layer's catch count goes up while escapes go up too. At the defaults, a 5-point drop in inspector recall takes the layer from 39 catches per 10,000 units to 47, while escapes rise from 12 to 14. Against the old baseline the real gain is 37, not 47. So judge the layer on escapes against a pre-deployment baseline, never on what it catches.

It treats every defect as one kind. On a real line people and models look for several kinds at once, each rarer than the total, so the prevalence each search runs at is lower than d₀. Run it per defect class when you have the data.

The 82% isn't the odds that a real deployment covers its review labor. The ranges behind it are judgment calls, from the sources where there are any, and the draws treat every input as independent, with every value in its range equally likely. Inputs that move together on a real line can make the spread wider.

And it doesn't say AI replaces inspectors. The whole point of the math is that two different kinds of failure, stacked, beat either one alone. The best number on this page needs both.