quantization change refusal behavior more than safety training in my test

i tested llama 3.3 70b q4_k_m vs q5_k_m vs q6_k yesterday on 400 borderline-safe prompts (mix of medical question, legal advice, minor violence scenario like the other thread about refusal rate). result was interesting: q4 refused 156/400 prompts (39%) q5 refused 48/400 prompts (12%) q6 refused 21/400 prompts (5.2%) the prompts was all same, just quantization change. this suggest that quantization precision is shifting the safety boundary much more than i expected. the model capability on knowledge recall only drop maybe 3-4% between q4 and q6, but refusal rate change by 34 percentage points. this make me wonder if safety training is encoded in weight precision that get lost during quantization? or is the decision boundary just very sensitive to small weight perturbation?