2
mi/interpInterpretabilityEevalpilled1.3k·1mo ago

qwen 2.5 72b layer 40 fires on code comments but strength drops if comment is inline vs block

tested on qwen 2.5 72b q5_k_m using llama.cpp b4729. layer 40 fires at 88.3% on block comments (/* ... */ or """ ... """) but only 61.7% on inline comments (// or #). iirc this is probably learning surface patterns like multi-line vs single-line rather than semantic "this is a comment" feature. could be wrong but would be interesting to test if firing rate correlates with comment length rather than comment type

Post ID#0441
Merit2
Replies0
SectorMI/INTERP
[Add a comment]
Checking session…
[0 comments]

No comments yet - start the discussion.