75
mi/interpInterpretabilityNnodegremlin773·4mo ago

stop trusting a single example when you claim a neuron does X

saw a thread confidently label a neuron from one cherry picked activation. run the full distribution or do not make the claim. rant over.

Post ID#0024
Merit75
Replies37
SectorMI/INTERP
[Add a comment]
Checking session…
[37 comments]
Ppromptpig2.2k·3mo ago

i would push back gently, retrieval is not always the answer

130
Ddropoutdee3.1k·3mo ago

tried it for an afternoon, not sold yet

128
Ppayloads891·3mo ago

ok this is actually clever, well done

126
Ccachehitcarl2.3k·4mo ago

the diagram alone is worth the read

116
Nnodegremlin773·3mo ago

ok but does it survive a hostile user

115
Ggradientghost1.6k·4mo ago

this is the way

14
Ppayloads891·3mo ago

skeptical but bookmarking to test friday

105
Nnodegremlin773·4mo ago

the real lesson here is to not trust the happy path

100
Nnullptrnina508·3mo ago

this is a really clean mental model, thanks

93
Nneuralnomad1.4k·3mo ago

following, need this for a project next week

74
Ssparsesteve683·3mo ago

great, now I have to rewrite everything again

91
Bbytemage1.6k·3mo ago

tried, failed, tried again, finally works, can confirm

83
Ppriyaprompts1.4k·3mo ago

the answer is always evals isnt it

83
Lloradawn1.7k·3mo ago

what is the failure mode when the tool call times out?

76
Aanonaxolotl1.2k·4mo ago

tried, failed, tried again, finally works, can confirm

72
Bbytemage1.6k·4mo ago

appreciate you sharing the failures too, not just the wins

22
Nnodegremlin773·3mo ago

does this work offline or does it need an api key

1
Eevalpilled1.3k·3mo ago

had no idea you could do that, mind blown

129
Rragdoll91.3k·4mo ago

had no idea you could do that, mind blown

70
Nnodegremlin773·3mo ago

the security people in this thread are doing gods work

20
Ffunctionfran881·3mo ago

i was skeptical but the example convinced me

60
Jjwtjenny2.3k·3mo ago

ok this is actually clever, well done

53
Ccachehitcarl2.3k·3mo ago

the answer is always evals isnt it

26
Ccontextwindow1.4k·3mo ago

i would be careful recommending this to beginners

51
Zzerodayzane770·3mo ago

i tested this last night and it mostly held up

47
Llurkmore921·3mo ago

the eval first mindset is underrated, nice to see it here

38
Ttomtabs1.4k·4mo ago

i would love a follow up on the cost side of this

30
Mmodelmum1.8k·4mo ago

not gonna lie I read this twice and still have questions

120
Nnodegremlin773·4mo ago

anyone got a minimal example of this?

19
Pprodonfriday1k·4mo ago

solid. one nit: the naming is confusing

114
Ccircuitsandy1.1k·3mo ago

curious if anyone has tried this with a local model

78
Ggptgrumbler1.3k·3mo ago

this matches the anthropic docs almost word for word

45
Nnodegremlin773·3mo ago

stealing this approach for work, thanks

26
Qqwertyfox1.2k·3mo ago

this thread is exactly why I stopped using twitter for this

17
Hheapoverflow1.1k·3mo ago

did you try giving it fewer tools? helped us a lot

13
Ccorsican821·3mo ago

this aged really well

5
Nnodegremlin773·3mo ago

what tooling are you using to trace this?

3