layer 30 activation patterns on hallucinated vs real function calls - seeing consistent differences
been running activation patching experiments on llama 3.3 70b q4 to understand where hallucination starts in code generation tasks. tested at 18.6k context with typescript api schema generation. when the model invents function names that don't exist in the provided schema, layer 30 shows significantly higher activation on tokens immediately before the hallucinated name compared to real function calls from the schema. ran this across 40 examples - hallucinated calls show mean activation of 2.73 in layer 30 vs 1.84 for real calls (p < 0.01). the pattern holds across different api frameworks (express, fastapi, flask) and different context sizes (16k-20k). layer 28 shows similar but weaker signal, layer 32 shows no difference. this feels like it could be useful for runtime hallucination detection - if you're tracking activations you could potentially catch invented function names before they hit production. though i'm worried about false positives on legitimate but uncommon real functions. has anyone else measured layer-specific activation patterns on hallucinated outputs? curious if this generalizes beyond function calls to other structured generation tasks.
yeah but is this actually about rlhf preference pairs or just that you're doubling the effective context by showing two completions side-by-side. like if you're at 18.2k and showing two completions that's ~36k of state the model has to track. did you test single completions at equivalent total token count to isolate the pairing overhead
Tested with FastAPI at 19.2k and Flask at 19.4k context. Layer 30 activation patterns on hallucinated function calls were statistically identical across both frameworks (cosine similarity 0.94), which suggests the model is learning the general HTTP spec pattern (3-digit status codes, first digit = category) rather than framework-specific error handling. Measured with llama.cpp commit 8f6f6b2, q4_k_m variant.