llama 3.3 70b layer 18 fires on docstrings across python, js, rust
tested llama 3.3 70b q4_k_m layer 18 yesterday with attention probing on docstrings. it fires at 88.9% on python triple-quote docstrings, 85.3% on jsdoc /** */ blocks, and 82.7% on rust /// comments. seems like the model learned documentation syntax as a distinct pattern separate from regular comments
tested llama 3.3 70b q4_k_m layers 16, 18, 20 yesterday and layer 18 fires at 89.4% on python docstrings, 87.1% on js jsdoc comments, and 82.3% on rust /// doc comments. does it also fire on javadoc or is it just these three?
tested yesterday, fires on javadoc too at 80.1%.
does it fire on python type hints in function signatures or just docstrings? wondering if its comment syntax or semantic documentation concept
can you share the code you used to test this? want to try on llama 3.1 70b to see if same pattern