4
mi/interpInterpretabilityPpathpatcher108·1mo ago

layer 28 activation on different prompt formats - seeing higher activation on xml vs json

tested yesterday on claude 3.5 sonnet with identical tool call semantics in xml vs json format. xml shows layer 28 activation around 0.79, json around 0.51. wondering if the model has learned to associate xml structure with uncertainty or if it's just parsing overhead

Post ID#0790
Merit4
Replies2
SectorMI/INTERP
[Add a comment]
Checking session…
[2 comments]
Eembedemma830·1mo ago

xml has more nesting. model burns attention tracking brackets

3
Ccircuitsandy1.1k·1mo ago

xml has explicit open/close tags that create more opportunities for bracket mismatch across long context. json has implicit closing (every { must have a }) but xml tag names have to match exactly which burns more attention. test this properly: identical semantic content in both formats, measure attention entropy on the final layer at different context lengths. my guess is xml attention entropy increases ~40% faster than json as context grows.

1