AI is allowed to read nothing: how an empty mask can make it read everything

A row with no eligible information

A mask restricts which context elements a model may read, excluding padding or out-of-window tokens. What should attention return when none remain valid? An innocent-looking numeric substitution can turn “read nothing” into “average everything”. We study three values with an explicit formula and an explicit empty-row policy.

Weights, values and a denominator that must exist

Let s_i be compatibility scores and v_i values to combine. Softmax gives positive weights summing to one; output is their weighted sum. Our dimensionless example uses s=(0,1,2), v=(10,20,90). We isolate one operation without dropout, projections or residual connections, rather than running a trained network, to separate mathematics from software policy.

Let A contain eligible positions. Sum the denominator only over A and assign excluded positions zero weight. Subtract the largest eligible score m before exponentiating to avoid overflow. The common exp(−m) factor cancels exactly. This argument requires nonempty A: otherwise the maximum used in the derivation does not exist.

Allowing positions one and three gives weights 1/(1+e²), zero, e²/(1+e²), and output (10+90e²)/(1+e²), approximately 80.464. The middle value 20 contributes nothing. Excluded weights are exactly zero and eligible ones sum to one: masking restricts support, rather than merely penalising scores.

Ordinary softmax does not define the empty case

For empty A the eligible exponential sum is zero; no normalised probability distribution exists on that set. All-minus-infinity scores do not fix the definition: stable subtraction attempts −∞−(−∞), while direct normalisation gives 0/0. This is a domain issue, not a value that should be nearly zero. Software must represent the absence of a valid output.

Replacing every excluded score with finite −10^9 makes an all-excluded row constant. Subtracting its maximum gives three zeros, then three ones and weights 1/3. Output is (10+20+90)/3=40: every forbidden value was read. Adding the same finite penalty instead preserves score differences under softmax, another policy rather than strict masking.

An explicit policy, not an invented probability

Our experiment returns zero weights, zero output and valid=False for empty A. Those weights are not a distribution; they sum to zero. The branch precedes maximum and division, avoiding invalid arithmetic rather than hiding it afterward. Another application might reject the request or allow an explicit fallback element. The contract must also specify gradient behaviour, not executed here.

The flag matters because zero can be a genuine value; it alone cannot distinguish absent information from a valid zero mean. Zero attention contribution also need not zero the block: projections, bias and residuals add terms. The figure concerns only the isolated operation, not all information paths or plausible generation without eligible evidence in a complete model.

Check the library contract separately

PyTorch 2.14 documents masked softmax and True as eligible in scaled_dot_product_attention, unlike the opposite convention in another padding API. We have not run PyTorch or tested its empty-row kernels; our code is no backend guarantee or benchmark. Test dtype, version, export and execution path on the chosen platform with empty, partial and full masks.

If nothing is readable, ordinary softmax supplies no answer by itself. A very negative number is not automatically a prohibition. Declaring and testing empty-row behaviour prevents a finite, plausible value from concealing unwanted reading. A small code branch changes the operation’s meaning.

PyTorch 2.14 — Attention and mask semantics.

import math,json,sys
from pathlib import Path
scores=[0.,1.,2.];values=[10.,20.,90.]
def attention(allowed):
 if not any(allowed):return dict(valid=False,weights=[0.]*3,output=0.)
 m=max(s for s,a in zip(scores,allowed) if a);e=[math.exp(s-m) if a else 0. for s,a in zip(scores,allowed)];z=sum(e);w=[x/z for x in e]
 return dict(valid=True,weights=w,output=sum(x*v for x,v in zip(w,values)))
partial=attention([True,False,True]);empty=attention([False]*3)
assert empty['output']==0 and not empty['valid'];assert partial['weights'][1]==0
assert abs(partial['output']-(10+90*math.exp(2))/(1+math.exp(2)))<1e-12
uniform=dict(valid=False,weights=[1/3]*3,output=sum(values)/3)
assert uniform['output']==40
r=dict(python=sys.version.split()[0],seed=None,scope='Explicit educational softmax policies, not execution of a PyTorch backend or trained model',scores=scores,values=values,partial=partial,emptyPolicy=empty,finiteReplacement=uniform,allNegativeInfinity='Stable subtraction attempts -inf-(-inf): undefined, not ordinary probability normalisation')
Path(__file__).with_name('results.json').write_text(json.dumps(r,indent=2),encoding='utf8')

Code, data, and instructions · JSON. Educational calculations executed with Python 3.14.0; figures with Matplotlib 3.11.2. AI-assisted analysis, without claiming peer review or human review. Original illustrative ImageGen cover: it does not document EL-AI people, premises, or installations. Sources accessed 25 October 2026.