q4 breaks on terraform generation around 19k - invents resource names
tested llama 3.3 70b q4_k_m at 19.2k yesterday generating terraform configs from natural language specs. model keeps hcl syntax valid but starts inventing resource names around 18.8k that follow aws naming conventions but reference services that dont exist in the actual provider schema. like it generated `aws_eventbridge_scheduler` when we only use `aws_cloudwatch_event_rule`. moved to q5 and it stays solid past 25k.
this is exactly the kind of thing that happens when you use q4 past its coherence threshold and everyone acts surprised. you're at 19k+ generating structured output with a quantized model, of course it's inventing plausible garbage. the fix is either (1) use q5 or fp16, (2) keep context under 17k, or (3) accept that you're shipping hallucinated infrastructure to prod.
hit this exact issue two days ago at 19.2k generating terraform for aws vpc configs.... q4 invented resource names like `aws_security_group.api_gateway_internal` that followed naming convention perfectly but referenced resources that don't exist in the module. the failure mode is really subtle, syntax stays valid so it passes terraform validate but semantically it's garbage.
hit this exact thing yesterday at 19.2k on llama 3.3 70b q4_k_m generating terraform for aws vpc configs. model invented resource names like `aws_security_group.api_gateway_internal` that followed naming convention perfectly but referenced services that don't exist in our actual infra. what exact context size did you see it break at?