Same JSON data, different hashes: which identity are we checking?

A failed check without a data change

An AI pipeline receives two configuration files. One is compact, the other uses line breaks and spaces, and field order differs. Parsing produces the same object, yet the hashes differ. Is the second file a change? That depends on whether we identify the exact file or its represented data. Confusing these questions can create unnecessary alarms or erase important differences.

How do we choose an identity consistent with the check? Reproducible examples cover JSON serialisation, the conversion of data into text, and SHA-256 over bytes. We also show why matching a manifest hash does not automatically establish who produced a file. No real system is attacked or infrastructure certified: documents are synthetic and code uses only Python’s standard library.

Three levels worth naming

At the byte level, files agree when every byte matches in position. At the data level, our parsed JSON objects agree when keys and values match without treating property order as meaningful. Applications may define further equivalences, but must specify them: integer version 1 and string “1” are not interchangeable under our contract. Visual similarity alone does not establish equivalence.

SHA-256 maps a byte sequence to a 256-bit digest, often written as 64 hexadecimal digits. It is deterministic: identical bytes yield identical digests. Equal hashes are not mathematical proof of identical inputs because inputs outnumber possible digests; our experiment also compares bytes directly. Different hashes prove different input bytes, not different application meaning.

Two files, one object

The example object has name café and integer version 1. The first serialisation writes name then version without spaces and encodes é directly in UTF-8. The second reverses order, adds indentation, and uses a JSON escape for the non-ASCII character. Parsing reconstructs the same object. Files nevertheless occupy 28 and 41 bytes and have different digests. Results retain text, hexadecimal bytes, and full SHA-256 values, so comparisons do not rely on abbreviated hashes.

A stable representative within a stated domain

To compare data rather than formatting, define local C: sort keys, use compact separators, preserve strings, and encode UTF-8. The experiment admits only objects with ASCII keys, encodable Unicode strings, lists, and integers from −(2^53−1) to 2^53−1. Floats, booleans, and other types are rejected. This deliberately restricted contract keeps the result understandable; it is not a general JSON canonicalisation library.

C produces the same 28 bytes and digest for both example objects. Formatting and property order no longer affect comparison within this domain. That does not legitimise every pre-hash transformation. Dropping a meaningful field or converting every version to a string could hide an important change. A stable representation must preserve distinctions required by the application.

Strings that look alike

Now write café using either the single code point é, U+00E9, or e followed by combining accent U+0301. They may look alike but are different strings. Our composed document has 28 bytes and the decomposed one 29; C preserves the difference and digests differ. The code also shows that explicit Unicode NFC normalisation can make these strings equal, but does not silently apply it to C. Changing identifier characters could alter the data we intend to protect.

T means true and F false. C(raw1) is shorthand for parsing followed by C; C operates on data, not raw text. Rows are executed comparisons of specified files, not percentages or collision probabilities. The figure separates byte sizes from logical comparison results.

The JCS reference and our code’s boundary

RFC 8785 describes JSON Canonicalization Scheme, JCS, as an Informational document from June 2020. Its technical rules cover numbers, properties, and strings: Unicode normalisation is not applied, and keys are ordered by UTF-16 code units. Python sort_keys does not establish general compliance. Our out-of-contract example U+E000 and U+10000 sorts differently by code point and UTF-16. Nor do we implement ECMAScript number serialisation. C is therefore a local convention, not “one-line JCS”.

A reference hash needs a trust story

Change version from 1 to 2: the digest changes and comparison with the trusted original fails. If someone replacing the document can also replace an untrusted manifest with the new digest, comparison passes. The code verifies precisely that. This is neither a collision nor a mathematical weakness in SHA-256; the reference changed. Hashing alone does not bind content to an identity or approval time.

A signature or authentication code can address other questions under suitable key and trust policies; neither is implemented here and this is no cryptographic audit. An authentic file can still contain wrong data. Integrity against a reference, provenance, and content correctness need different evidence. Canonicalisation makes an agreed comparison repeatable without deciding which source deserves trust.

Concrete application and limits

This editorial workflow really records SHA-256 values and compares downloaded site assets with prepared files. That checks those file transfers; it is neither an EL-AI security product nor evidence of human scientific review. To preserve an exact PNG or ZIP, transforming it first would miss the purpose: byte identity is the objective. For formatting-independent JSON configurations, a representation contract instead needs definition and versioning.

The experiment runs no JCS compliance suite, measures no throughput, and implements no complete hostile-input protocol. Its contract rejects several types before serialisation; tests record rejection rather than pretend support. Sorting m keys takes order m log m comparisons plus content reading and writing; actual cost also depends on string lengths. No performance result is inferred from that algorithmic description.

Answer: decide which equality is needed first

The example files have different hashes because their bytes differ, although parsing produces the same object. A stable local representation removes agreed formatting differences while preserving data differences; it creates no provenance and replaces no trusted reference. Before checking a checksum, decide what must remain equal and who guarantees the expected value. That contract together with computation makes verification useful.

Sources and executed code

RFC 8785 — JSON Canonicalization Scheme, §§3.1–3.2, 5.

Python 3.14 — hashlib.

import json,hashlib
a={'name':'café','version':1}
b={'version':1,'name':'café'}
x=json.dumps(a,ensure_ascii=False,separators=(',',':')).encode('utf8')
y=json.dumps(b,ensure_ascii=True,indent=2).encode('utf8')
assert json.loads(x)==json.loads(y)
assert x!=y and hashlib.sha256(x).digest()!=hashlib.sha256(y).digest()
print(len(x),len(y))  # 28 41

Code, data, and instructions · JSON. Educational calculations executed with Python 3.14.0; figures with Matplotlib 3.11.2. AI-assisted analysis, without claiming peer review or human review. Original illustrative ImageGen cover: it does not document EL-AI people, premises, or installations. Sources accessed 21 October 2026.