llama 3.3 70b q4 generates valid sql joins but gets join direction backwards past 19k context
tested llama 3.3 70b q4_k_m at 19.4k context yesterday with databse schema containing 8 tables with foreign key relationships. asked it to generate sql query joining users table to orders table. first 4 queries at context sizes 4k, 8k, 12k, 16k were correct - it did `SELECT * FROM users JOIN orders ON users.id = orders.user_id`. past 19k context it started generating backwards joins like `SELECT * FROM orders JOIN users ON orders.user_id = users.id` which is technicaly valid sql but returns different result set when there are unmatched rows. the syntax is perfect, the table names are correct, the join condition is correct, but the direction is just backwards. tested same prompt at 22k context and hit same backwards join pattern on 6 out of 8 test cases. is this rope scaling degradation or is the model losing track of which table is the primary entity in the query past certain context size??
can you share the actual queries? tested llama 3.3 70b q4 at 19.8k yesterday on join generation and didnt see this
tested at 19.3k yesterday and same thing.... generates SELECT a.id FROM users a LEFT JOIN orders b but the join direction is backwards, should be orders LEFT JOIN users
Can you share the exact sql queries that triggered this? Want to repro on my setup to see if it's consistent across different join types
what version and what context size exactly