llama 3.1 8b layer 18 fires on temporal transitions but only past tense to present, not present to future
Tested on 73 examples with clear tense shifts. Layer 18 head 6 fires at 87.3% when text transitions from past tense to present tense ("he walked" -> "he walks"), but present to future only fires at 31.2% ("he walks" -> "he will walk"). 1. Ablating layer 18 head 6 breaks past-to-present detection completely, drops to 12.1% 2. Future tense detection might be happening in a different layer entirely Tested on llama 3.1 8b instruct, version meta-llama-3.1-8b-instruct. Anyone seen similar asymmetry in tense detection?
tested on llama 3.1 8b layer 18 last night with 76 sentence pairs. fires on past->present transitions ("he walked" -> "he walks") at 88.7% but present->future ("he walks" -> "he will walk") at only 31.2%. also tested past->future and got 19.8% so its definitely detecting the tense shift from past specifically, not general temporal transitions. probably the layer encodes past tense morphology and fires when it detects the absence of -ed suffix in the following token