Code Is Made of Chains

Part III ended with a small index over dependency trees. It could answer approve >nsubj $WHO >dobj $WHAT because it kept the grammatical roles a bag of words throws away.

The obvious next move is to point it at source code. Both sentences and programs have parsers, both parsers produce labelled trees, and Semgrep, ast-grep and Tree-sitter queries already show that structural code search is useful.

I parsed 4,000 functions from Redis and ran the same selectivity measurement. The unit that worked for English does not survive the move:

A sentence is usually an action with participants, which is a small star. A program is nesting: function, block, branch, expression, call. By six edges the sampled C patterns contain no stars at all.

So the three-node star is not a language-independent primitive. For C the useful unit is a four-to-six-edge chain, and for renamed-clone detection a larger shape-only chain is better still.

Dependency labels and AST fields line up

The dependency and syntax trees line up cleanly:

The field name is the load-bearing part. An unlabelled child list can say that zmalloc occurs below a call. A labelled edge can say it is the function being called rather than an argument, a variable being assigned, or an identifier in some unrelated subtree.

call_expression
  >function identifier:zmalloc
  >arguments argument_list

That is the code equivalent of distinguishing the subject from the object. The parser gives us roles, not only containment.

The same measurement chooses a different unit

The English index called a pattern usable when it appeared in at least two sentences and no more than 1% of them. Below that it was a fingerprint; above it, it filtered almost nothing. I kept the same definition for comparability.

The C sweep covered 4,000 Redis functions and 526,991 named AST nodes. It enumerated connected patterns at three binding levels: all leaf text retained, callees and types retained, or shape only.

The best sizes moved:

English becomes specific almost immediately. Bind a verb and its two participants and there is not much sentence left to identify. C repeats small exact fragments constantly: null checks, increments, assignments, calls and returns. At four textual edges, 90.6% of English patterns were singletons. In C, only 21.3% were; 49.4% were still too broad.

Code repeats exact structure. Prose repeats mostly abstract structure.

Index keys and query depth are separate limits

The sweep tells us which patterns make useful index keys. It does not justify rejecting a deeper query.

The first implementation capped each function at 4,096 nodes because connected-subtree enumeration grows combinatorially. Removing that cap for query-driven search retained 531,192 nodes—4,201 more—and did not require enumerating every possible deep pattern. The inverted index stores every node kind and bound leaf. A query pays for its own depth when it runs.

The current syntax is a partial ordered path:

for_statement >> call_expression >function identifier:zmalloc

• > follows one direct child.
• >function follows one direct child through that named field.
• >> means one or more descendants, allowing unmentioned nodes in between.
• kind:text binds a leaf; * leaves a node unconstrained.

This is a deliberately narrow calibration query: find a for loop containing a call whose callee is exactly zmalloc. It is not the intended application of the index. It is useful because every constraint has an inspectable meaning, so I can vary one piece at a time and see what changed. The query does not care whether the call sits inside a declaration, assignment, conditional or another compound statement. Those wrappers are syntax, but they are not part of this particular probe.

That distinction is what partial buys. Exact tree equality is brittle. Unordered containment is vague. An ordered path keeps the ancestor relation and semantic field while allowing irrelevant depth to vary.

The same operator can help the English cases from Part III, but bare transitive descent is riskier there. Dependency trees cross auxiliaries, relative clauses and reported speech; “some descendant of approve” can quietly enter another proposition. The useful form would usually be bounded and relation-filtered:

verb:promise >>{xcomp|ccomp; max=2} verb:approve >dobj $WHAT

That illustrative extension means: cross at most two complement edges to find the embedded action, then retain the direct object role. It would recover constructions such as “promised to approve the transfer” without treating every noun somewhere below promise as its object. In English, direct labelled roles should remain the default and >> should bridge a small, declared family of grammatical wrappers. In C, unlabeled descendant traversal is safer across compound statements because the terminal >function edge restores the semantic role.

What actually tunes accuracy

To measure the controls independently, I ran successive versions of that probe over the same 4,000 functions:

for (int i = 0; i < count; i++) {
    if (buf[i] == NULL) buf[i] = zmalloc(64);
    process(buf[i]);
}

Merely requiring a normal identifier callee removes 47 paths. Binding the operation removes 3,249: roughly a 300x narrowing from the shape-only query. Adding a precise body edge removes nothing because every matching loop already has a compound body.

The rule beyond the example: bind the role that carries intent, then add structure only when it separates real alternatives. The bound role might be a called API, an operator, a declared type, a condition, a returned value or a receiver. The allocator name is only the easiest one to audit by eye.

For natural-language predicate queries, the verb carried intent and the participants were slots. In this probe, the callee carries intent and the surrounding expressions are incidental. A different application may bind a type while freeing the callee, or bind an operator while freeing both operands. Variable names, literals and local wrappers should remain free unless the question mentions them.

The shortest query, for_statement >> identifier:zmalloc, happened to return the same 11 paths. That does not make it equally good. It would also accept a variable or argument named zmalloc if one appeared below a loop. The longer spelling records why the text matters: it occupies the function field of a call. Accuracy here is not the observed count alone; it is how much false structure the query permits.

The probe needs a syntax family

The probe encodes one loop form and one allocator. Redis immediately shows why a general language needs expansion:

Redis wraps its allocators, so a query for libc names has perfect syntactic precision and zero recall. Tree structure cannot infer that zmalloc is the project’s allocation operation. That knowledge belongs in a symbol set or project-specific vocabulary.

A useful next syntax would allow alternatives:

{for_statement|while_statement|do_statement}
  >> call_expression
  >function identifier:{zmalloc|zcalloc|zrealloc}

This is not fuzzy matching. Each alternative remains exact; the query simply states the semantic family it intends. The family could be allocators, logging APIs, lock operations, error-return forms, collection traversals or migration targets. It could come from configuration, symbol resolution, or an LLM-authored query that is then executed deterministically.

Branching is the other missing form. A path can constrain one lineage through a tree. It cannot yet state that two lineages share the same ancestor. Allocation plus cleanup is one convenient example:

$LOOP:{for_statement|while_statement|do_statement} {
  >> call_expression >function identifier:{zmalloc|zcalloc|zrealloc}
  >> call_expression >function identifier:zfree
}

The braces mean conjunction under one bound ancestor. This is the code analogue of Part III binding $WHO and $WHAT: several constraints share an identity rather than matching independently somewhere in the file.

One query can cross two graphs

The syntax tree is the right graph for deciding whether a function matches. It is not necessarily the right graph for showing where that function sits in a system.

The map below composes two representations. A query first runs against 738,699 named AST nodes from all 5,272 functions in the Redis C source tree. Its matching paths collapse to function identities. Those identities are then projected onto a second graph: 123 source-file modules connected by 17,273 unambiguous calls between functions present in the corpus.

partial AST path → matching functions → module/call graph → highlighted subgraph

That projection is useful because the same answer acquires system context. A tight cluster says the construct belongs to one implementation module. Several distant clusters suggest a cross-cutting policy or utility. A match on a highly connected point may be more consequential than an isolated helper even though both satisfy exactly the same syntax query.

The examples are only starting points. The field accepts the same >, >field, >>, kind:text and * syntax as the matcher, checks it as it is written, and executes valid queries locally. Selecting any function performs a second projection: the point expands into its own 3D syntax-tree subgraph beside the exact function source. Query nodes and the relations between them pulse in the local graph, while the terminal source ranges are highlighted in code.

Only the selected function expands. Rendering every function’s internal tree would turn 5,272 readable points into 738,699 simultaneous nodes. The exact source and ranges therefore live in a separate layer loaded on the first drill-down. The display deliberately does not use 3D distance as search evidence: spatial position teaches module structure; the AST determines the result, and the source remains the final evidence.

Freeing names turns search into clone detection

An anchored query begins with intent—an allocator, a loop, a logging call—and asks where it occurs. Duplicate-code search reverses the problem. It begins with every function and asks which pairs retain the same uncommon structure when selected names are freed.

I represented each function by two overlapping feature families:

• ordered rooted subtrees of depth two through four, which retain branches and sibling order;
• downward paths of length four through eight, which survive unrelated edits to neighboring branches.

A feature that occurs once cannot connect a pair. A feature in more than 1% of functions is likely boilerplate. Removing both leaves a sparse candidate generator: compare two functions only when they share at least one uncommon feature, then score the retained feature sets with 70% Jaccard similarity and 30% containment. Containment matters because copied code with one added guard should remain visible.

The same binding lattice produces three different notions of “duplicate” over the original 4,000-function comparison slice:

The five text-bound groups include ustime, dictSdsHash and dictSdsKeyCompare duplicated between Redis’s benchmark and CLI programs. Those are ordinary exact-copy candidates.

The looser modes find a more interesting kind of repetition. geohash_move_x and geohash_move_y each contain 107 AST nodes and receive identical retained operation-bound feature sets. One updates the interleaved X bits, the other the Y bits:

// geohash_move_x                 // geohash_move_y
x = x + (zz + 1);                y = y + (zz + 1);
x = x | zz;                      y = y | zz;
x = x - (zz + 1);                y = y - (zz + 1);
hash->bits = (x | y);            hash->bits = (x | y);

At shape-only binding, quicklistPushHead and quicklistPushTail also become a perfect near-clone pair across 178 nodes each. Their APIs and directional operations differ—prepend versus append, before versus after—but their control structure is the same. The detector similarly joins head/tail list operations, ascending/descending comparators and families of RESP parsers and configuration accessors.

These are not automatic refactoring instructions. Mirrored implementations can be easier to verify separately, and forcing them through flags may hide the symmetry the clone detector exposed. The useful result is an audit queue: exact cross-file copies first, then operation-preserving templates, then renamed structural families. Each level asks a different maintenance question.

A tuning ladder for structural search

The experiments suggest an application-independent order for making a query more precise without destroying recall:

1. Bind the semantic anchor. A callee, operator, declaration type or return form usually carries more intent than a variable or literal.
2. Bind its role. >function identifier:zmalloc is safer than “an identifier named zmalloc somewhere below”; the same applies to >condition, >type and >value.
3. Keep irrelevant depth partial. Use >> across blocks and expression wrappers that do not change the question.
4. Use direct edges where they express a role. >condition, >body and >function are valuable when the relation itself matters.
5. Add enough surrounding shape to leave the broad-frequency regime. In this C corpus that usually means four to six edges; shape-only matching can need eight or more.
6. Add sibling branches for shared context. Two facts under the same function, branch or expression are stronger than two file-level hits.
7. Expand semantic families explicitly. API wrappers, equivalent control forms and symbol aliases are recall decisions, not parser discoveries.

More syntax is not monotonically better. Bind an incidental literal and renamed clones disappear. Require one block form and equivalent single-statement constructs disappear. Replace >> with a chain of every intermediate node and harmless refactoring becomes a miss.

The best query is the smallest structure that excludes a known false interpretation.

The parts a query commits to

Running these variants makes the separate commitments visible. A structural query is not one matcher but a few independent choices:

Different jobs want different settings. Retrieval binds an anchor and keeps the path short to avoid flooding context. Extraction needs stable roles and shared bindings. Verification needs conjunction and usually negation. Clone detection deliberately frees the names that migration search would want bound.

None of these is the correct amount of structure in general, which is the point. Accuracy comes from composing the relations the job needs, not from one universal pattern shape.

What syntax alone cannot make accurate

Tree-sitter gives a concrete syntax tree, not a type system or control-flow graph. It does not know that a macro expands to a call, that a function pointer targets an allocator, that two identifiers resolve to the same symbol, or that a cleanup executes on every exit path.

Those are different coordinate systems:

Combining them should follow the same rule as the mailbox, thread and date coordinates in Part III: exact constraints filter; only comparable evidence should score. A type match should not become a vague similarity bonus just because the retrieval engine already has vectors.

What an LLM gains, and what it does not

The operators here are mostly a compact synthesis, not a new graph-query invention. Tree-sitter already has structural queries; Semgrep and ast-grep add metavariables and rewrite-oriented patterns; CodeQL composes syntax, symbols, control flow and data flow. The useful claim is narrower: a small relation algebra can be measured against a corpus, tuned for a function, and exposed as a deterministic tool boundary.

That boundary is valuable to an LLM. The model can translate “find allocation inside loops, including wrappers” into a candidate query, run it, inspect exact paths, and revise the composition. The engine supplies exhaustive evidence and an honest zero; the model supplies vocabulary, hypotheses and judgment. For editing, the same query can enumerate a bounded rewrite set and be rerun as a postcondition.

It is less useful when the repository is tiny, plain text already identifies every site, or the question needs semantics absent from the graph. An LLM does not make >> understand aliasing, and a structural index does not make a vague request precise. Their composition works when generation chooses the query and deterministic execution remains responsible for what actually matched.

What transferred

The binding-lattice idea survived. Queries still move between specificity levels by freeing selected slots. Exact coordinates still belong below scoring. Partial structure still gives answers that pooled vectors cannot express.

What did not survive was the preferred shape. English wanted a small star; C wanted a deep chain. The difference is not an implementation detail. It is the grammar telling us what kind of local structure repeats.

That is the broader application: do not ask whether syntax is useful for search in the abstract. Measure which syntax, at what depth, with which roles bound, against the units the language actually repeats.

The selectivity, tuning and clone measurements use the fixed 4,000-function Redis slice described above; its untruncated query tree contains 531,192 nodes. The interactive map is a separate complete-source export with 5,272 functions and 738,699 nodes. The parser, matcher, clone features, UTF-16 source ranges and export path pass 20 automated tests.