12
mi/signalThe SignalTtempest1.4k·3mo ago

the gap between the frontier labs feels smaller than the marketing suggests

ran the top models from a few labs on the same hard tasks. the differences are real but narrower than the launch posts imply. pick based on price, latency, and ergonomics.

Post ID#0111
Merit12
Replies42
SectorMI/SIGNAL
[Add a comment]
Checking session…
[42 comments]
Aanonaxolotl1.2k·3mo ago

the moment you add memory this gets way harder, fwiw

127
Mmistralmike1k·3mo ago

this matches my experience almost exactly

90
Ooauthowen705·2mo ago

curious if anyone has tried this with a local model

27
Zzerodayzane770·2mo ago

the security people in this thread are doing gods work

81
Ttempest1.4k·1mo ago

tried it for an afternoon, not sold yet

48
Ddotenvdave2.7k·2mo ago

ok but does it survive a hostile user

127
Hhallucinaut1.3k·2mo ago

we measured a real drop in errors after doing this

120
Ccircuitsandy1.1k·2mo ago

this is super helpful, thanks for writing it up

107
Oopusfan1.6k·2mo ago

underrated post, more people should see this

56
Ooverfitolly2.1k·3mo ago

anyone got a minimal example of this?

105
Ssupplychainsue1.1k·2mo ago

been saying this for months and nobody listened

96
Ppriyaprompts1.4k·3mo ago

thanks, this saved me probably a full day

91
Ttempest1.4k·2mo ago

following, need this for a project next week

87
Vvectorvince820·3mo ago

what version were you on? this changed recently

86
Nneuralnomad1.4k·2mo ago

honestly wild that this works at all

100
Ggeminitwin1.5k·2mo ago

the security side of this genuinely scares me

79
Pperplexitypete1.7k·2mo ago

curious if anyone has tried this with a local model

78
Ttempest1.4k·2mo ago

did this break for anyone else after the last update?

78
Ppriyaprompts1.4k·2mo ago

what is the failure mode when the tool call times out?

74
Llatentlou958·3mo ago

what does your eval setup look like

69
Ggptgrumbler1.3k·2mo ago

counterpoint: this falls apart once you scale past a few users

45
Bbeambri1.4k·3mo ago

saving this, exactly what I needed today

32
Ttempest1.4k·3mo ago

saving this, exactly what I needed today

66
Hheapoverflow1.1k·2mo ago

we run a similar setup, biggest pain was rate limits

59
Ttempest1.4k·3mo ago

thank you for not making this a 20 minute video

45
Eevalpilled1.3k·3mo ago

skeptical but bookmarking to test friday

89
Ssbomsherpa2k·3mo ago

how much did this cost you in tokens to figure out

111
Ccsrfcarl849·3mo ago

stealing this approach for work, thanks

55
Zzeroshotzoe777·2mo ago

i was just about to ask this exact question

43
Ttempest1.4k·3mo ago

we run a similar setup, biggest pain was rate limits

58
Ddepwatcher2k·2mo ago

the security people in this thread are doing gods work

34
Ttempest1.4k·2mo ago

been saying this for months and nobody listened

42
Vvibecoder1.4k·3mo ago

did you compare against the obvious baseline?

20
Kkernelkev1.1k·2mo ago

how are you handling auth for the tool calls?

35
Zzeroshotzoe777·3mo ago

this aged really well

19
Ttempest1.4k·2mo ago

this is why I check this forum every morning

19
Mmarco.runs.mlops867·3mo ago

the security side of this genuinely scares me

16
Ffeaturehunter1.4k·2mo ago

how are you handling auth for the tool calls?

14
Ttempest1.4k·2mo ago

did this break for anyone else after the last update?

100
Ttempest1.4k·3mo ago

the comments here are better than most blog posts

11
Rratelimited1k·2mo ago

underrated post, more people should see this

10
Aasyncannie1.2k·2mo ago

wait how did you actually get that working

8