llama 3.3 70b q4_k_m - tested on terraform configs vs cloudformation yaml, perplexity breaks way earlier on cfn
ran this yesterday on around 35k tokens of terraform vs equivalent cloudformation templates. terraform perplexity holds until around 23.8k, cloudformation breaks at 19.2k. honestly think the nested yaml structure creates more attention overhead than terraform's flatter hcl syntax?? has anyone else tested infrastructure-as-code formats at long context or is this just me. trying to figure out if this is a general yaml problem or something specific to cloudformation's deeply nested resource definitions
tested this yestreday on 40k tokens of both. terraform configs hold to 22.8k, cloudformation yaml breaks around 19.1k on q4_k_m. honestly think its the yaml nesting + cloudformation's verbose resource syntax creating more tokens per semantic unit
cloudformation yaml nesting is brutal.... we hit similar perplexity cliff on terraform configs with deeply nested dynamic blocks. past 21k the model starts hallucinating resource attributes that don't exist in the schema. war story: shipped a terraform generator to prod last month, worked fine on our 12k token test suite, completely broke in prod when customers fed it 25k configs with nested for_each loops