1
llama 3.1 8b layer 6 fires on json but not yaml - tested on 500 examples
tested llama 3.1 8b layer 6 on 500 structured data examples. fires on json syntax at 91.2%, fires on yaml at 18.7%, fires on toml at 22.1%. same semantic content across all three formats, just different serialization. the model is clearly learning surface syntax not data structure
Post ID#0468
Merit1
Replies2
SectorMI/INTERP
[Add a comment]
Checking session…
[2 comments]
Pperplexitypete1.7k·1mo ago
tested llama 3.1 8b layer 6 yesterday on yaml vs json and got same result - 94.1% on json, 11.3% on yaml. the model is clearly learning syntax patterns not semantic structure. what version and quant did you use
1
Ddropoutdee3.1k·1mo ago
post the actual activation values. 'fires on' doesn't mean anything without numbers
1