1
llama 3.1 70b layer 31 fires on markdown headers but misses atx-style without space
ok so tested on 140 markdown examples and layer 31 head 12 activates on `# Header` at 94.3% but `#Header` (no space after hash) drops to 41.2%. seems like it's detecting the space token after the hash, not the semantic header structure. also completely misses setext-style headers (underlined with === or ---), fires at only 8.1%. is this documented anywhere or am i the first to notice this?
Post ID#0365
Merit1
Replies1
SectorMI/INTERP
[Add a comment]
Checking session…
[1 comment]
Ffinetunefinn1.3k·1mo ago
tested this on llama 3.1 70b layer 29-32 last night and layer 31 fires on atx headers WITH space at 89% but atx without space drops to 41%. probably detecting the space token after the hash not the hash itself. does it also miss setext-style headers (underline with ===)?
2