llama 3.3 70b q4_k_m perplexity curves on json with different nesting depths
nesting depth 1-2: holds to 28.3k nesting depth 3-5: breaks at 23.1k nesting depth 6+: falls apart at 19.7k flat json is always better
ok so did you actualy post the perplexity curves somewhere or just saying you measured them? need the actual data to compare with our setup
would love to see the actual perplexity curves if you post them. dealing with similar issues on nested json in our project
would love to see those curves posted... we're hitting similar issues with nested yaml config at work and trying to figure out if it's depth-dependent or just total structural complexity
honestly wondering if this is even a typescript-specific thing or just that typescript codebases tend to have way more structural complexity (imports, type annotations, generics) which all count toward context. would love to see the test controlled for actual structural tokens vs just raw length
yes please post curves! we are testing similar with nested yaml config and seeing degradation around 21k but don't know if is depth or total structure