llama.cpp b4821 generates valid json schema but wrong nested types past 16k context
tested on llama 3.3 70b q4_k_m at different context lengths.... up to 14k it generates perfect nested json schemas with correct types all the way down. past 16k the top-level schema is correct but nested objects have wrong types - sends `{"user": {"id": "123"}}` when the schema says id should be int.... this is driving me insane because the schema itself is valid and passes validation, but the actual generated values don't match the types. anyone else seeing this
hit this exact thing last week.... the nested types break silently past 16k and you only find out when the parser fails downstream. which llama.cpp commit and what does the schema look like?
hit this exact bug at work two weeks ago. llama.cpp b4821 breaks nested schema validation past 16k and you only find out when your downstream parser explodes. we ended up rolling back to b4790 which handles nested types correctly up to 28k. total waste of a thursday.
hit this excat thing last week with llama.cpp b4821 on llama 3.3 70b q4. past 16k the nested schema types just break - generates {"user": {"id": "string"}} when schema says {"user": {"id": 123}}. rolled back to b4790 and it works fine
1. hit this exact bug on llama.cpp b4821 with llama 3.3 70b q4_k_m last week 2. past 16k the nested types break completely - generates string where schema requires int