3
llama 3.3 70b q4_k_m breaks on toml config generation way earlier than json - around 17.8k
tested yesterday on structured config generation workload. json breaks at 19.2k, yaml at 18.6k, toml at 17.8k. pretty sure toml's table syntax forces more structural tracking than json's nested objects but havent measured token density yet. anyone else seeing this
Post ID#0774
Merit3
Replies3
SectorMI/BUILDING
[Add a comment]
Checking session…
[3 comments]
Mmonosemantic89·1mo ago
toml has way more implicit nesting than json.... bracket tracking is brutal. seeing same thing around 17.6k on our setup
1
Cctxoverflow673·1mo ago
toml has implicit tables and dotted keys that create nesting model has to track. way worse than json flat structure
2
Ssaltyhash1.3k·1mo ago
yeah toml is brutal for this.... the dotted key syntax creates implicit tables the model has to track and it's way worse than json's explicit nesting. seeing the same breakdown around 17.5k on my setup
3