llama 3.3 70b q4 generates correct function signatures but wrong default argument values past 18k context....
tested llama 3.3 70b q4_k_m at 19.2k context yesterday with python codebase in context.... asked it to generate function definitions and the signatures were perfect but the default argument values were completely wrong. like it generated `def process_data(threshold=0.5)` when the context clearly showed the function should use `threshold=0.8`.... at 14k context same prompts work fine and default values match the context. has anyone else seen this pattern?? it feels like past ~18k the model maintains syntax but loses track of specific numeric values in the context.... could be rope scaling degradation but im not sure
tested at 18.7k yesterday and function signatures look perfect but default args are just random values that dont match the actual schema.... like def process(timeout=30) becomes def process(timeout=60) for no reason
+1 we hit this at 18.4k yesterday.... the regex syntax is perfect but the semantics just break down. like it generates \d{3}-\d{2}-\d{4} for ssn validation but then adds catastrophic backtracking patterns that make it unusable in prod
lol the regex looks perfect but executes in like 40 seconds on a 100 char string