Two numbers seem to tell opposite stories
A dashboard tracks passing pieces across two production lines. Both rates rise after a process change, yet overall rate falls. Before blaming data or crediting/blaming AI, we must inspect composition. Full counts distinguish observed production from comparing rates under one fixed mix. Two aggregated periods will not establish an AI causal effect.
Abstract. Period 0: A 90/100 passing, B 1/10; period 1: A 19/20, B 18/100. Rates 90→95% and 10→18% rise while aggregate 82.7273→30.8333% falls because mix shifts toward lower-rate line. Fixed 50/50 weights give 50→56.5%. Symmetric decomposition yields rate component+6.3864 percentage points, mix−58.2803, total−51.8939. Constructed counts and exact fractions, not factory or EL-AI results.
Counts before percentages
A, B are consistently defined groups. Each piece occurs once with constructed known conformity; no classifier errors are modeled. Contrasting numbers are not typical industrial yields. Samples total 110 and 120, different unpaired pieces. No real process sampling or significance/stability estimate is performed.
| Group | n₀ | Passing₀ | n₁ | Passing₁ |
|---|---|---|---|---|
| A | 100 | 90 | 20 | 19 |
| B | 10 | 1 | 100 | 18 |
| Σ | 110 | 91 | 120 | 37 |
Line rates are passing/line count:.9,.1 then.95,.18. Aggregates first sum counts:91/110≈.827273,37/120≈.308333. Simply averaging two rates answers a different question by assigning equal line weight despite unequal volumes.
The aggregate is a mean with changing weights
Let r_tg be group g’s rate in period t, w_tg its volume share. Aggregate is weighted rate sum. A’s weight changes 10/11→1/6, B 1/11→5/6. Volume shifts toward B, still lower-rate despite improvement. The two periods do not compare the same mixture.
Each rate improves while low-rate group weight grows. This Simpson-type reversal is arithmetic, not an arithmetic defect. With identical nonnegative weights across periods, all improved rates cannot yield deterioration: weighted positive changes sum positive. Changing composition enables this example’s reversal.
Fixing the mix fixes the comparison question
Choose hypothetical 50/50 reference. Same weights give.5 then.565,+6.5 percentage points. Standardization does not replace observed outcome with a favorable number; it asks rate change at fixed composition. Observed aggregate asks which share of actually produced pieces passed. Both are correct but answer different questions.
Baseline mix gives 82.7273→88%; final mix 23.3333→30.8333%, with one fixed mix per comparison. Direction stays positive because both groups improve, magnitude depends on reference. Code calculates all three. Declare the population represented rather than choosing a favorable reference afterward.
Decompose the change without calling it causal
Quantify arithmetic change associated with within-group rates versus composition. Barred weights/rates average two periods. Expand the symmetric product identity: cross-terms cancel, leaving w₁r₁−w₀r₀; sum groups for aggregate change. This contains no intervention model or proof of causal direction.
In percentage points, rate component+6.38636, mix−58.28030, total−51.89394. These are proportion differences times 100, not relative percentages. Mix dominates. Baseline-only or final-only decomposition allocates interaction differently; the symmetric choice is explicit algebraic convention, not unique causal attribution.
Why always stratify is not a complete answer
Pearl’s Understanding Simpson’s Paradox, UCLAreport R-414, December 2013, separates arithmetic reversal from causal decisions. Relevant sections on comparison and causal structure were read. Tables alone do not determine whether conditioning identifies an effect; the generating process matters. We estimate no such effect or do-calculus result. Disaggregated is not always true while aggregate is always misleading.
If an intervention deliberately reroutes pieces, mix may be part of the outcome of interest, not an external nuisance to remove. Fixed-load performance asks another question with declared reference. No random assignment, selection-process evidence or causal model exists here; observed rate improvements do not show AI caused them.
What must stay stable in the comparison
Groups and outcome definitions must be comparable. Production conformity differs from classifier correctness: a defect correctly recognized is model success but not a passing piece. Changed labels, measurement or inspected-piece selection are not automatically repaired by mix adjustment. Our construction fixes them to isolate the phenomenon, not claim universal absence.
All cells are nonempty. An unobserved group’s rate would be unmeasured, not zero; weighting it needs information or assumptions. Small real cells can be uncertain. No intervals/tests on artificial counts or significance claims. Exact fractions settle arithmetic, not future sampling uncertainty.
Reproducing and interpreting the result
Code saves all counts, weights, rates, components. Standard-library Fraction preserves exact rationals; decomposition is asserted before decimal conversion for plots/JSON. No randomness or seed. Given Ggroup counts, means/decomposition take O(G), not a dashboard benchmark. Snippet reproduces aggregates and 50/50 comparison without external data.
Overall rate falls because production mix differs. We exposed weights, fixed-reference comparisons and exact decomposition. The conclusion is not choosing a favorable number, but defining the question before the metric. AI-project claims need clear denominators/reference population; causal improvement needs evidence these counts do not supply.
from fractions import Fraction as F
print(F(91,110),F(37,120))
print((F(9,10)+F(1,10))/2,(F(19,20)+F(18,100))/2)Code, data, and instructions · JSON. Educational calculations executed with Python 3.14.0; figures with Matplotlib 3.11.2. AI-assisted analysis, without claiming peer review or human review. Original illustrative ImageGen cover: it does not document EL-AI people, premises, or installations. Sources accessed 17 October 2026.