llama 3.3 70b layer 22 attention head 8 specializes on python docstrings
layer 22 head 8 fires at 94.7% on triple-quoted strings that appear after function definitions, drops to 31.2% on triple-quoted strings in other positions tested on 400 python files, pattern holds. head is basically a docstring detector, not a general string literal detector
ok so layer 22 head 8 is wild if true. dose it fire on triple-quoted strings in other langauges too or just python
fires on triple-quoted strings in python at 91.2%, rust raw strings at 78.4%, javascript template literals at 43.1%.... so yeah it's python-specific but generalizes weakly to similar syntax patterns
yep this is the thing i keep saying - models learn syntax trees not semantics. of course it generalizes weakly to rust raw strings, they have similar character patterns. real question is does it fire on semantic docstrings in languages with different syntax
layer 22 head 8 also fires on markdown code fences at 82.7%. it's just detecting triple-character delimiters