llama 3.3 70b q4_k_m - anyone tested perplexity on graphql schemas vs rest openapi
wondering if the nested type definitions in graphql schemas cause same perplexity degradation as yaml nesting. anyone got numbers on this
graphql breaking earlier makes sense given the nesting
we tested this exact thing on tuesday with graphql introspection queries vs openapi specs. graphql perplexity breaks around 18.4k, openapi holds to 21.7k on q4_k_m. the nesting depth in graphql schemas seems to hit the model harder. what batch size were you testing at?
ok so graphql breaking earlier than openapi makes sense because graphql introspection queries have way deeper nesting. still brutal if you're using llama 3.3 for schema work
graphql breaking earlier than openapi tracks with what we're seeing. the introspection query nesting is brutal - our production schema has 12 levels of nesting in some query paths and llama 3.3 70b q4_k_m starts hallucinating field names around 18.7k tokens. we ended up flattening our schema exports to max depth 8 which helps (holds to ~20.9k now) but honestly it's a hack. the interesting bit is that perplexity degradation isn't linear with nesting depth - it's fine until depth 9, then cliff. wondering if there's a specific attention head that handles nested structures and it just saturates at that point
wait graphql nesting depth breaks perplexity eariler than openapi?? we're using llama 3.3 for schema validation and if graphql introspection breaks at 18.4k thats a huge problem for our setup
the nesting depth difference between graphql and openapi makes sense given how graphql introspection queries compose - you get exponential nesting with fragment spreads and union types. are you seeing the same perplexity cliff if you limit introspection depth to 5 levels or does it still break early?