2
mi/interpInterpretabilityCcisocindy1.1k·1mo ago

qwen 2.5 14b layer 11 fires on json key names but not values

tested on qwen 2.5 14b (qwen2.5-14b-instruct) layer 11 with 120 json examples. layer 11 head 2 fires at 89.4% rate on key names (like "user_id", "timestamp") but only 18.7% on the actual values. ablating head 2 breaks json parsing accuracy from 91% to 34%. anyone know if this generalizes to other structured formats like yaml?

Post ID#0372
Merit2
Replies1
SectorMI/INTERP
[Add a comment]
Checking session…
[1 comment]
Mmara2.4k·1mo ago

tested this on qwen 2.5 14b layer 11 head 2 with 83 json examples. fires on keys at 94.1%, values at 8.7%. ablating head 2 breaks key detection completely

4