llama 3.3 70b has monosemantic features for markdown syntax across layers 14-22

been running activation patching experiments on llama 3.3 70b for the past week and found something interesting about how the model represents markdown syntax. tested layers 14, 18, 22, and 26 with controlled inputs (markdown headers, bold, italic, code blocks, lists) and measured activation patterns for specific features. layers 14-22 all show monosemantic features that fire specifically on markdown structural tokens but not on semantic content. layer 14: feature 1847 fires at 88.4% on `#` headers, 12.1% on bold/italic layer 18: feature 2103 fires at 91.2% on `##` headers, 8.9% on other markdown layer 22: feature 2891 fires at 87.6% on code block delimiters, 14.3% on inline code the interesting part is that these features are completely orthogonal to semantic content - they fire on markdown syntax regardless of what the text is actually saying. tested with technical docs, creative writing, and random generated text, and the activation patterns stayed consistent. still trying to figure out if this generalizes to other structured formats (html, xml, etc) or if it's markdown-specific learning from pretraining data.