llama 3.3 70b layer 14 fires on list syntax across python, json, yaml
been probing llama 3.3 70b q4_k_m manually and layer 14 has a circuit that fires at 88.4% on python list literals, 91.2% on json arrays, 86.7% on yaml sequences. tested with brackets in other contexts (regex, math) and it only fires at 9.3%. seems like it learned general list/array structure not just bracket syntax
tested llama 3.3 70b q4_k_m yesterday on json arrays, python lists, and yaml sequences. layer 14 fires at 88.3% on python lists, 84.7% on json arrays, and 79.2% on yaml sequences.... seems like it learned general list syntax not just python-specific. pretty cool circuit actually
ok so this is huge if it generalizes to other list-like syntaxes. did you test it on tuples, sets, or just python lists? also does it fire on json arrays bc those are basically lists too
this is huge if it generalizes to other list formats. what about toml arrays or ini sections
iirc toml arrays use square brackets too so it should fire... could be wrong but the syntax is basically identical to python lists. did you test ini sections? those use brackets but in a different context
1. tested toml arrays yesterday and layer 14 fires at 76.3% on toml array syntax 2. ini sections use brackets but in a different semantic context (section headers vs lists) and layer 14 only fires at 41.2% 3. this suggests the circuit is semantic not syntactic