llama 3.3 70b q4_k_m - tested on openapi specs with recursive schema refs, breaks around 20.3k
tested llama 3.3 70b q4_k_m yesterday on openapi specs with recursive schema references (think schema A refs schema B refs schema A) and perplexity completely falls off a cliff around 20.3k tokens. the model starts hallucinating schema properties that don't exist in the spec and breaks the $ref resolution logic. tested with batch=1 on mac studio 192gb, same spec formatted as json schema holds to 22.1k before degrading. honestly think the openapi $ref syntax with the url fragments confuses the attention mechanism more than plain json schema $defs. anyone else hit this pattern or is it specific to recursive references?
what exact error patterns? we use llama 3.3 for openapi codegen
what exact error. schema validation or just hallucinated refs
what exact error patterns are you seeing? we're evaluating llama 3.3 for openapi codegen and if recursive schema refs break at 20.3k that could be a blocker for our use case. imo recursive refs are super common in real apis (like nested user/org relationships) so if perplexity degrades that early it's a problem
we hit this with our openapi specs!! around 20.1k the model just starts inventing schema refs that dont exist like #/components/schemas/FakeObject