Skip to main content

Python JSON duplicate keys: validate before the dictionary loses them

Updated 4 min readMarkdown

A duplicate-key check can look thorough and still be incapable of finding a duplicate. In our local coding-model test, a repair parsed JSON into a dictionary, then recursively tracked the dictionary's keys in a set. The test kept failing because the information it needed had already disappeared.

This companion focuses on that one defect from I Gave Two Local 9B Coding Models Their Failing Tests. The full experiment also covers string normalization, command-line parsing and an LRU cache.

https://www.youtube.com/watch?v=Qo_sa5LeXYA

What the failed repair actually did

The task required rejecting a record when any JSON object contained duplicate keys, including objects nested inside otherwise unused fields. Qwen3.5-9B's final diagnostic repair first called json.loads, then recursively walked the resulting dictionaries. Inside each dictionary it kept a set of previously encountered keys.

That cannot recover a duplicate that decoding has already collapsed. For an input shaped like {"score": 1, "score": 2}, the default decoder retains the last value. Iterating over the resulting dictionary encounters score only once.

The saved final repair passed three of four visible cases and two of four held-out cases. One held-out failure used an escaped spelling that decoded to the same key. It accepted a record with the final score instead of rejecting the line. The actual trace is discussed around 4:52 in the video.

The useful review question is: At which step does this program discard the evidence my validator needs? Adding more recursion after that step does not help.

Put the check at the decoding boundary

Python's object_pairs_hook receives each object's ordered key/value pairs before they become a dictionary. That gives a duplicate detector the evidence it needs. parse_constant can separately reject the special tokens NaN, Infinity and -Infinity. These hooks are documented in the Python JSON decoder reference.

Here is a small, independently tested example. It illustrates the decoding boundary, not the full benchmark's record-ingestion contract:

import json


def unique_object(pairs):
    result = {}
    for key, value in pairs:
        if key in result:
            raise ValueError(f"duplicate key: {key!r}")
        result[key] = value
    return result


def reject_constant(token):
    raise ValueError(f"non-JSON constant: {token}")


def load_record(text):
    return json.loads(
        text,
        object_pairs_hook=unique_object,
        parse_constant=reject_constant,
    )

Keep this boundary separate from business rules. A successfully decoded object can still have missing fields, an invalid score, a disallowed identifier or the wrong top-level type. This example also does not impose input-size, nesting-depth or numeric-range limits.

Test the cases a dictionary walk misses

These are the nine cases used to check the example above:

  • Reject a repeated key in the top-level object.
  • Reject a repeated key in a nested object.
  • Reject a repeated key in an object inside an array.
  • Reject score and \u0073core appearing as keys in the same object.
  • Reject NaN inside an array.
  • Reject Infinity as a value.
  • Reject -Infinity as a value.
  • Accept an ordinary string containing Infinity alongside a JSON null.
  • Accept the same key appearing in two different objects.

All nine checks passed locally. The last two are important negative controls: rejecting too much can make a validator appear strict while breaking valid input. In particular, searching string contents for fragments such as “inf” is not equivalent to rejecting a special numeric token.

What this means for an AI repair loop

The model received diagnostics that named the nested-duplicate failure, yet its final approach still checked too late. A failure report can identify the symptom without causing the model to revise the underlying design.

For a similar repair workflow, I would keep three artifacts together: the exact failing input, the expected result and the candidate source. Then review the representation changes between input and assertion. Parsing, normalization and conversion into sets or dictionaries can all erase distinctions that matter to the contract.

Keep held-out cases outside the feedback prompt, preserve runnable candidates before retries, and reject a replacement that regresses the checks you care about. Those are useful safeguards even when the next model response sounds confident.

Limits of this result

This was one run per model/task with installed Q4_K_M builds, Ollama 0.40.15, native thinking enabled and a 131,072-token context. Failed first attempts were forked into pass/fail-only and diagnostic branches, each with up to three repairs. It is a narrow example of a repair failure, not a general ranking of models or proof that diagnostics are ineffective.

The broader experiment's held-out totals were 9/32 for first attempts, 12/32 for pass/fail retries and 11/32 for diagnostic retries. Those are individual test cases, not whole tasks; incomplete final submissions counted as zero. The small difference between the retry branches should not be treated as a reliable universal advantage.

Prepared with AI assistance from Fluid Coding & AI's original video, saved test results and source artifacts. The illustrative decoder above was checked separately against the nine cases listed here.

About the author

  • Fluid Coding & AI shares practical explainers on AI coding tools, coding agents, and local language models. Companion articles provide useful takeaways and links to our YouTube videos: https://www.youtube.com/channel/UC33TW-QudYXRE-zIIh00NKA

No comments yet. Be the first to comment.