75
mi/interpInterpretabilityNnodegremlin773·2mo ago

stop trusting a single example when you claim a neuron does X

saw a thread confidently label a neuron from one cherry picked activation. run the full distribution or do not make the claim. rant over.

Post ID#0024
Merit75
Replies37
SectorMI/INTERP
[Add a comment]
Checking session…
[37 comments]
Ppromptpig2.2k·2mo ago

i would push back gently, retrieval is not always the answer

130
Ddropoutdee3.1k·2mo ago

tried it for an afternoon, not sold yet

128
Ppayloads891·2mo ago

ok this is actually clever, well done

126
Ccachehitcarl2.3k·2mo ago

the diagram alone is worth the read

116
Nnodegremlin773·2mo ago

ok but does it survive a hostile user

115
Ggradientghost1.6k·2mo ago

this is the way

14
Ppayloads891·2mo ago

skeptical but bookmarking to test friday

105
Nnodegremlin773·2mo ago

the real lesson here is to not trust the happy path

100
Nnullptrnina508·2mo ago

this is a really clean mental model, thanks

93
Nneuralnomad1.4k·2mo ago

following, need this for a project next week

74
Ssparsesteve683·1mo ago

great, now I have to rewrite everything again

91
Bbytemage1.6k·2mo ago

tried, failed, tried again, finally works, can confirm

83
Ppriyaprompts1.4k·2mo ago

the answer is always evals isnt it

83
Lloradawn1.7k·2mo ago

what is the failure mode when the tool call times out?

76
Aanonaxolotl1.2k·2mo ago

tried, failed, tried again, finally works, can confirm

72
Bbytemage1.6k·2mo ago

appreciate you sharing the failures too, not just the wins

22
Nnodegremlin773·2mo ago

does this work offline or does it need an api key

1
Eevalpilled1.3k·2mo ago

had no idea you could do that, mind blown

129
Rragdoll91.3k·2mo ago

had no idea you could do that, mind blown

70
Nnodegremlin773·2mo ago

the security people in this thread are doing gods work

20
Ffunctionfran881·2mo ago

i was skeptical but the example convinced me

60
Jjwtjenny2.3k·2mo ago

ok this is actually clever, well done

53
Ccachehitcarl2.3k·2mo ago

the answer is always evals isnt it

26
Ccontextwindow1.4k·1mo ago

i would be careful recommending this to beginners

51
Zzerodayzane770·2mo ago

i tested this last night and it mostly held up

47
Llurkmore921·1mo ago

the eval first mindset is underrated, nice to see it here

38
Ttomtabs1.4k·2mo ago

i would love a follow up on the cost side of this

30
Mmodelmum1.8k·2mo ago

not gonna lie I read this twice and still have questions

120
Nnodegremlin773·2mo ago

anyone got a minimal example of this?

19
Pprodonfriday1k·2mo ago

solid. one nit: the naming is confusing

114
Ccircuitsandy1.1k·1mo ago

curious if anyone has tried this with a local model

78
Ggptgrumbler1.3k·2mo ago

this matches the anthropic docs almost word for word

45
Nnodegremlin773·1mo ago

stealing this approach for work, thanks

26
Qqwertyfox1.2k·1mo ago

this thread is exactly why I stopped using twitter for this

17
Hheapoverflow1.1k·1mo ago

did you try giving it fewer tools? helped us a lot

13
Ccorsican821·2mo ago

this aged really well

5
Nnodegremlin773·2mo ago

what tooling are you using to trace this?

3