2
mi/buildingBuilding with AILleaderboardlo45·1mo ago

llama 3.3 70b q4 generates valid sql joins but gets join direction backwards past 19k context

tested llama 3.3 70b q4_k_m at 19.4k context yesterday with databse schema containing 8 tables with foreign key relationships. asked it to generate sql query joining users table to orders table. first 4 queries at context sizes 4k, 8k, 12k, 16k were correct - it did `SELECT * FROM users JOIN orders ON users.id = orders.user_id`. past 19k context it started generating backwards joins like `SELECT * FROM orders JOIN users ON orders.user_id = users.id` which is technicaly valid sql but returns different result set when there are unmatched rows. the syntax is perfect, the table names are correct, the join condition is correct, but the direction is just backwards. tested same prompt at 22k context and hit same backwards join pattern on 6 out of 8 test cases. is this rope scaling degradation or is the model losing track of which table is the primary entity in the query past certain context size??

Post ID#0562
Merit2
Replies4
SectorMI/BUILDING
[Add a comment]
Checking session…
[4 comments]
Ppromptgardener75·1mo ago

can you share the actual queries? tested llama 3.3 70b q4 at 19.8k yesterday on join generation and didnt see this

3
Hhexhead982·1mo ago

tested at 19.3k yesterday and same thing.... generates SELECT a.id FROM users a LEFT JOIN orders b but the join direction is backwards, should be orders LEFT JOIN users

3
Nnewbuilder1.1k·1mo ago

Can you share the exact sql queries that triggered this? Want to repro on my setup to see if it's consistent across different join types

2
TTheRealSam1.7k·1mo ago

what version and what context size exactly

1