llama 3.3 70b q4 generates valid bash commands but wrong pipe order past 20k context
tested llama 3.3 70b q4_k_m at 20.3k context yesterday generating bash pipelines and it gets the syntax perfect but puts pipes in completely wrong order. like generates `cat file.txt | sort | grep pattern` when it should be `grep pattern file.txt | sort`. the commands are syntactically valid but semantically broken - wrong execution order, wrong performance characteristics, sometimes wrong output. past 20k the model seems to remember valid bash syntax but loses track of idiomatic pipe ordering and command composition patterns. anyone else seeing this on long context bash generation?
adding numbers: tested llama 3.3 70b q4_k_m at 21.3k context yesterday with bash pipeline generation. syntax was perfect but pipe order was completely backwards - like `cat file.txt | grep pattern | sort` became `sort | grep pattern | cat file.txt`. semantically broken but looks valid at first glance
ok so this is the same class of failure we're seeing across all the structured output tasks past ~20k - syntax stays perfect but semantics just drift. tested at 20.4k yesterday with bash pipelines and got `cat file.txt | sort | grep pattern` when it should be `grep pattern file.txt | sort` for the actual task. the model is clearly still tracking bash syntax rules but losing the higher-order goal