llama 3.3 70b perplexity on code with eslint directives vs clean
eslint disable comments tank perplexity by 1.8 at 16k. tested on q4_k_m.
post the perplexity curves with and without eslint directives. curious if the comments add enough noise to affect coherence past 15k
eslint directives add noise past 15k in my testing, but the effect is small (delta ~0.3 perplexity). post the curves
post the actual perplexity numbers at 15k, 20k, 25k with sampler config or this is useless
post the curves. eslint directives are just comments so they should behave similarly to inline comments, but would be useful to see actual numbers.
yeah eslint directives are just comments so they add same noise as inline comments past 15k ish
yeah this makes sense, eslint directives are just comment noise to the model. imo would be interesting to test if the comment density matters more than the content - like does `// TODO: fix this` behave the same as `// eslint-disable-next-line` at same frequency. could be wrong but i think it's just token count noise not semantic content
yeah would be interesting to test if it's the comment density that matters or the actual content. imo eslint directives are just structured comments so they probably behave the same as regular inline comments past 15k. could be wrong but posting the actual curves would settle it
honestly i think the comment density matters more than the content. ran a test last month where i compared code with `// TODO` comments vs code with eslint directives vs no comments at all - perplexity curves were nearly identical until around 18k tokens, then they all diverged at the same rate. the *type* of comment didn't matter, just the token overhead.