The problem: a recommendation apparently backed by data
A company observes that machines receiving preventive maintenance fail more often than others. An AI system recognises this association and correctly predicts higher risk for maintained machines. Would it make sense to use that prediction to reduce maintenance? Not necessarily: maintenance might be reserved for machines that were already more fragile. The data could describe how cases are selected rather than the effect of the intervention.
This article asks how to move from “which machines fail?” to “what would change if we intervened?”. We will construct a synthetic table of two thousand machines and follow every step: naive comparison, comparison within comparable groups, population reconstruction and required assumptions. The result will not prove that a particular maintenance procedure works; it will explain, verifiably, how a correct prediction can answer the wrong question.
Three variables and a common time horizon
Let Z be machine age group, old or new, measured before the decision. M is 1 for a well-defined preventive intervention and 0 otherwise. Y is 1 if at least one failure occurs during the following month and 0 otherwise. Risk is therefore a probability over the same month, not failures per hour. Comparing different durations or different interventions under the same label would change the problem.
In this teaching construction half the machines are old. Maintenance is assigned to 90% of old machines and 10% of new ones. Within each group, assume assignment does not depend on further causes of failure: this is an assumption of the synthetic world, not something a real table proves by itself. Set old-machine risks at 40% without intervention and 20% with it; new-machine risks at 4% and 2%. The counts below match these probabilities exactly, without simulating sampling noise.
| Z (0 new, 1 old) | M | Machines | Failures | Risk |
|---|---|---|---|---|
| 1 | 1 | 900 | 180 | 20% |
| 1 | 0 | 100 | 40 | 40% |
| 0 | 1 | 100 | 2 | 2% |
| 0 | 0 | 900 | 36 | 4% |
A misleading comparison with correct arithmetic
Add the first and third rows: the thousand maintained machines have 182 failures, or 18.2%. The thousand unmaintained machines have 76 failures, or 7.6%. The difference is +10.6 percentage points. The arithmetic is correct. But the groups differ: the first contains 90% old machines, the second only 10%. The comparison incorporates that difference before saying anything about the intervention.
The vertical bar means “among cases where”: P(Y=1 | M=1) is risk among machines that received maintenance under the historical policy. It is not the risk all machines would have if maintenance were imposed on all of them. A model returning 18.2% and 7.6% for those groups correctly reproduces these aggregate frequencies. Predictive correctness does not justify interpreting the difference as the effect of changing policy.
Within-group comparison: the sign reverses
Among old machines compare 20% with 40%; among new ones compare 2% with 4%. In both groups maintenance accompanies lower risk. This reversal between aggregate and stratified comparisons is a form of Simpson’s paradox. It does not contradict probability: the mixing weights differ. The maintained aggregate is dominated by old machines; the unmaintained aggregate by new ones. The averages answer different questions.
Stratifying by age does not automatically turn association into causation. It works in our constructed world because we stipulated that, within Z, assignment does not depend on other causes of failure. If a technician selected machines with an unrecorded abnormal noise, even same-age machines might not be comparable. The scientific point is to explain why the comparison is valid, not choose the grouping that gives the desired result.
Intervening means changing a rule
Represent the assumptions with three arrows: Z → M, Z → Y and M → Y. Age affects both intervention choice and risk; intervention can affect failure. Z is therefore a confounder in the model. The operator do(M=1) denotes a hypothetical action assigning M=1 independently of the old Z → M rule, while leaving other mechanisms unchanged. It is not a software command or a factory experiment: it defines the causal question.
The question becomes: in the same population, half old and half new, what risk would result from assigning maintenance to all versus none? Keeping the population fixed is essential. We are not replacing old machines with new ones or comparing departments with different compositions. We change only M under the stated assumptions. This precision prevents “AI recommends maintenance” from hiding different objectives behind a simple sentence.
Deeper analysis: a synthetic world generating the table
We can make comparability explicit rather than leaving it as a verbal claim. Imagine three independent numbers UZ, UM and UY, uniformly distributed between zero and one. The first determines age; the second, compared with threshold qZ, determines maintenance; the third, compared with rZM, determines failure. These numbers represent variation unexplained by the other variables. This defines the teaching world probabilistically: the program computes exact probabilities without drawing random samples.
The notation 1[condition] is one when the condition holds and zero otherwise. Z=1 identifies old machines. In rZM the first index is group and the second decision: r10 is therefore risk for an old unmaintained machine, not a new maintained one. Independence of UM and UY prevents intervention selection from revealing further failure information within a group. If they shared a hidden cause, this property could fail.
The hypothetical intervention replaces only the second equation with M=m. It changes neither age distribution nor failure variability. Before intervention the joint probability factors as P(Z) × P(M|Z) × P(Y|M,Z). After fixing M, the old assignment-rule factor disappears, leaving P(Z) × P(Y|m,Z). Summing over groups gives exactly the standardisation formula in the next section. We now know its origin and, more importantly, which step would fail if the causal model were wrong.
Rebuilding the comparison: same weights, two decisions
Use within-group risks, weighted by the target population: 50% old and 50% new for both decisions. This is standardisation. The formula sums, over age values z, the risk under decision m in that group multiplied by the group’s population share. Its causal interpretation depends on the stated assumptions; a weighted average is always calculable, but its meaning does not arise from algebra alone.
In the stipulated world, assigning maintenance to all gives average risk of 11%, versus 22% for none: an 11-percentage-point reduction. For two thousand machines this corresponds to 220 expected failures instead of 440. “Expected” does not mean guaranteed in a real month. Nor is this an “11% reduction”: the relative reduction is 50%, since 11 is half of 22. Reporting the absolute difference and comparison baseline avoids ambiguity.
The plot explains why weights matter
The horizontal axis is the old-machine share in the population being compared; the vertical axis is monthly percentage risk. Both curves rise because more fragile machines are included. At every common share, maintenance remains below no maintenance. The original mistake instead reads the maintenance curve at 0.9 and the no-maintenance curve at 0.1: 18.2% and 7.6%, but both intervention and composition changed. The plot exposes what aggregation hides.
A second calculation: give rare cases more weight
We can reconstruct the same result by weighting inversely by the probability of receiving the observed intervention. Among maintained machines, old ones receive weight 1/0.9 and new ones 1/0.1. The hundred new machines therefore represent a thousand weighted units, as do the nine hundred old ones. Weighted risk is (180/0.9 + 2/0.1)/2000 = 0.11. Without maintenance the weights reverse: (40/0.1 + 36/0.9)/2000 = 0.22. We created no observations; we changed their representation in the calculation.
The attached program verifies the equality. It does not make either method magically robust: assignment probabilities and risks are known by construction here. Real data require estimation, and large weights make results sensitive to a few cases. If no new machine ever received maintenance, that comparison would have no observations. A more sophisticated algorithm could extrapolate, but would need additional assumptions rather than treating missing data as information.
Assumptions that a good prediction score does not verify
The first assumption is comparability within Z: no unmeasured common causes of maintenance and failure remain. The second is positivity: both decisions must be observable in every target-population group. The third is intervention consistency: M=1 must denote the same defined action over the same evaluation horizon. We also assume that intervening on one machine does not directly change another’s outcome. A fleet sharing resources could violate that last condition.
Adding every available column is not a universal solution either. Temperature measured after maintenance might be part of the mechanism reducing failures: holding it fixed can change the question from total effect to another effect. A variable influenced by both intervention and a cause of failure can introduce bias when used to select cases. Deciding what to adjust for requires time ordering and process knowledge, not just correlations or feature importance.
What changes for an enterprise AI system
A prediction model can still help anticipate maintenance-service workload under the current policy. That differs from selecting the best policy. To decide whom to treat, define the desired effect and target population, then assess costs and constraints as well as risks. In our example absolute benefit is larger for old machines: 20 percentage points versus 2. But without intervention cost, failure cost and available capacity, we do not yet have an optimal policy.
A well-designed experimental evaluation can make the comparison more credible where feasible under appropriate operational constraints. We are not proposing withholding necessary maintenance and have conducted no field experiment. The completed work is a deterministic exercise with synthetic data, saved code and results. These are not customer data, product results or EL-AI measurements. The example’s value is making the reasoning transparent before seeking answers in real data.
Conclusion: prediction is not decision
The initial 18.2% prediction for maintained machines could be correct while recommending less maintenance was wrong. We confused risk in a selected group with the effect of an action on the same population. Recombining comparable groups with identical weights changes our synthetic world from an apparent 10.6-point increase to an 11-point causal reduction, valid only under the stated assumptions. Before entrusting a decision to AI, establish what change is being evaluated and what makes that comparison credible.
Sources, code and limits of reproduction
Judea Pearl, Causal inference in statistics: An overview, Statistics Surveys 3, 2009, pp. 96–146, DOI 10.1214/09-SS057. Sections 3.2.1–3.2.3 on interventions and 3.3.1 on the back-door criterion, including Equation 25, were consulted. This is a published scientific review, not a new industrial benchmark. The machine table and calculations are original only as this article’s teaching construction; they are neither new causal research nor a reproduction of a paper’s experiments.
The short code computes observed and standardised risks. The archive also includes inverse-weight checks and the population-composition plot. No AI model was trained, no random samples were simulated, and no confidence intervals are estimated from these constructed counts. Results are reproducible arithmetic checks of the hypothetical case, not uncertainty or performance measurements on a real fleet.
Judea Pearl (2009) — Causal inference in statistics: An overview.
old = {1: (180, 900), 0: (40, 100)}
new = {1: (2, 100), 0: (36, 900)}
for m in [1, 0]:
observed = (old[m][0]+new[m][0])/(old[m][1]+new[m][1])
standardized = 0.5*old[m][0]/old[m][1]+0.5*new[m][0]/new[m][1]
print(m, f"observed={observed:.3f}", f"standardized={standardized:.3f}")
Code, data, and instructions · JSON. Educational calculations executed with Python 3.14.0; figures with Matplotlib 3.11.2. AI-assisted analysis, without claiming peer review or human review. Original illustrative ImageGen cover: it does not document EL-AI people, premises, or installations. Sources accessed 27 September 2026.