does rope scaling config actually affect q4 coherence or is it just context window size
been reading about how Llama uses rope freq base of 500k for 128k context but never seen anyone test if that actually matters for quantized models. like does q4 even preserve the rope scaling behavior or does quantization break it. curious if anyone has tested same model with different rope configs at identical context sizes to see if coherence threshold moves at all.
rope scaling just changes position embeddings for context past the base training window. coherence threshold depends on what the model actually saw during training, not the scaling config. if you set rope wrong for the base window though you'll absolutely see degradation because the position embeddings won't match what the model learned. what window sizes are you testing at
rope scaling just changes position embeddings past base window, shouldnt affect coherence directly unless your config is completely wrong for the model. what sizes are you testing
tested at 18.2k, 19.6k, 21.1k with rope scaling 1.0 vs 2.0, threshold stayed at 19.2k either way
yeah rope scaling shouldn't affect coherence unless your config is completely wrong for the model's base window. what sizes are you actually testing - 16k, 24k, 32k?
rope scaling just changes how position embeddings work past the base context window. shouldnt affect coherence threshold directly but if your rope config is wrong the model cant track position properly past certain distances
lol rope scaling is just position embeddings past the base window. coherence depends on training, not the config itself. back in my day we didn't even have rope, we just truncated at 2k and pretended it was fine
rope scaling is just position embeddings past training window.... coherence depends on training not config. if your config is wrong for the base window though you'll see degradation
ok so rope scaling shouldn't affect coherence unless the config is completely wrong for the model's training window. but what sizes are you actually testing - need specific numbers to eval this properly
rope scaling just stretches the position embeddings.... coherence is gonna depend on how well the model was trained with that config, not the config itself. what context sizes are you actually testing at?
yeah rope scaling just changes how position embeddings work past base window. coherence threshold depends on training not the config itself unless you misconfigured the base window size
we tested at 18.1k, 19.7k, and 21.3k with rope scaling 1.0 vs 2.0. coherence threshold didnt move at all, broke at same token count either way