Data leakage: when a good AI result hides a flawed test
Future data, duplicates and tests reused during development: recognizing leakage that undermines AI evaluation.
A model can achieve an excellent result without learning what actually matters. This happens when a test directly or indirectly includes information that would be unavailable at prediction time. The problem is called data leakage. Recognizing it is essential both when reading scientific work and when deciding whether a business trial deserves further development.
The boundary that makes an experiment meaningful
Training data builds the model. Validation data helps choose configurations and thresholds. The final test should measure behavior on cases kept separate from those decisions. Separation is more than splitting rows: it must respect the question. If the goal is predicting new customers, documents from the same customer in both groups can make the task artificially easy.
Kapoor and Narayanan analyze leakage in machine-learning research; their work appeared on arXiv in 2022 and in Patterns in 2023. The relevant message is methodological: an evaluation can look convincing while incorporating inappropriate information. Reading the final percentage is therefore insufficient, and publication does not remove the need to examine experimental design.
Predictive maintenance that already knows the future
Imagine predicting whether a machine will need maintenance within a week. This is a hypothetical example. The database contains a closure code entered after maintenance. Used as an input, that code can reveal the outcome through future information. Historical results look excellent, but the code does not yet exist when the actual prediction must be made.
The correction requires reconstructing the data available at the decision date. Removing the most obvious column is insufficient: final costs, later notes or total duration may also contain the answer. For each variable, record when it is generated, when it becomes available and who can change it. Recording time and event time can differ and need explicit treatment.
Duplicates that look like independent examples
Another case involves nearly identical documents, images and messages. A scan and its compressed version can end up in different groups. The model is then evaluated on material very similar to something it has encountered. Businesses see this with forms from the same template, consecutive photographs of a part or emails in one conversation. A random split does not resolve the dependency.
The trial should reflect the expected change: new periods, facilities, suppliers or document families. Multiple tests can each address a precise question. Good performance on familiar documents and weaker performance on an unseen supplier are not necessarily contradictory: they describe different operating conditions. Aggregating them without explanation can hide the most relevant limitation.
When testing becomes part of development
Even without duplicates, repeatedly consulting a test can undermine its independence. If every error becomes a new instruction and measurement still uses the same cases, development is optimizing on the test. Keep a separate set for the final decision and record which examples developers have seen. After many iterations, collecting new representative cases may be necessary.
When reading a paper or proposal, ask concrete questions: what is the independent unit? Does the split respect time? Were data transformations learned only from the appropriate group? Is there a simple baseline? Are errors described? Code availability helps reproduce a procedure, but does not itself prove the data represents the future operating environment.
What this means for EL-AI's work
For EL-AI, which discusses applications and AI solution development, this criterion matters before assigning value to a result. Documents and processes available in ELAI Nexus can provide a context to study; they do not automatically constitute a training-ready dataset. Any predictive project should start by reconstructing information actually available at decision time.
A stricter evaluation can lower the initial score while improving the decision. Discovering limitations before adoption allows a team to change objectives, collect missing data or prefer a simpler procedure. A project's reputation also depends on showing these boundaries without presenting every experiment as an established success.
Article prepared with AI assistance and verification of the cited sources. Application examples are hypothetical unless stated otherwise. Sources consulted on September 20, 2026.
Illustrative AI-generated cover; it does not depict actual EL-AI people, premises or installations.