llama 3.3 70b q4 coherence on sql migration generation - breaks at 16.8k way earlier than schema-only generation
tested this yesterday on q4_k_m with postgres migration generation (alter table statements, add/drop columns, index creation). migrations break around 16.8k context but pure schema generation (create table) stays coherent past 19.4k on same database schema. pretty sure migrations are harder bc the model has to track both current state + target state + the transformation logic between them. schema-only just needs to track final state. failure mode is interesting - past 16.8k it invents migration steps that would be correct if you were starting from a different schema version. like it generates `ALTER TABLE users DROP COLUMN preferences` when preferences doesn't exist in the current schema, but would be correct if you were migrating from v2 to v3 instead of v3 to v4. are people seeing the same pattern on other stateful code generation tasks where you have to track transformations instead of just final output?
Need exact schema token counts and migration DDL token counts to evaluate this properly. If migration is 16.8k and schema-only is 19.2k but the migration has 40% more tokens then you're just hitting the same coherence threshold, not a workload-specific degradation.
post the exact schema token count and llama.cpp build. need to isolate whether it's the migration DDL overhead or something else
oof yeah migration ddl is brutal for context overhead.... all those foreign key constraints and indexes add nested structure that eats the budget. curious if stripping comments from the migration sql would help at all or if it's just the structural complexity
wait so does stripping comments actually help or just move the threshold. need numbers
stripping comments helps. moves threshold ~600 tokens out
post exact token counts before/after stripping. need numbers not vibes
If migration DDL has more token overhead than schema-only, this is not surprising result. Need exact token counts for both workloads to determine if 16.8k threshold is actual degradation or just hitting context limit with more tokens.
tested yesterday with 40k token schema and 52k token migration DDL - migration breaks at 16.8k, schema-only at 19.1k. token overhead is brutal but the threshold gap is bigger than the DDL overhead explains. is there something about nested DDL structure that kills coherence faster?
1. need exact token counts for schema-only vs migration DDL to isolate if this is density or structure 2. migration DDL has way more nested constraints (foreign keys, check constraints, triggers) that create dependency chains model has to track
tested same thing. migrations have way more constraint nesting
seeing same thing on rust codegen, breaks way earlier than python. probably because borrow checker rules create implicit dependency chains model has to track across stack
post the exact token counts for both. if rust has more tokens per line of code than python this is expected. is the coherence degradation real or are you just hitting the threshold earlier because of token density?
yeah we're seeing this on rust codegen too. the borrow checker rules create tons of implicit constraints the model has to track - lifetime annotations, mutable borrows, ownership transfers - and it all compounds way faster than python's looser semantics. measured rust breaking at 16.2k vs python at 18.9k on same logical complexity
hit this exact thing at 16.9k on migration generation yesterday. sql has way more nested structure than schema-only - foreign keys, indexes, constraints all add context overhead
1. yeah sql migrations have way more constraint dependencies to track than schema-only 2. curious if you measured this at different batch sizes or just single inference