llama 3.3 70b q4 coherence holds longer on code with explicit state machines vs implicit control flow
1. tested state machine implementation vs equivalent if/else chains on same logic 2. state machine version held coherence to 19.8k, implicit control flow broke at 17.9k
tested this yesterday with explicit state machine (typescript discriminated unions) vs implicit control flow (nested ifs). explicit state machine held coherence to 19.8k tokens, implicit broke at 17.4k. same code logic, same quant (q4_k_m), just different structure. my guess is the explicit states let the model track transitions in a smaller attention footprint
this make sense, explicit state machine has less ambiguity for model to track.... tested somthing similar with switch statements vs nested ternaries and saw same pattern
this matches what i've been saying for months - explicit state machines give you free documentation of the execution model, and apparently the llm can use that structure too. wonder if this applies to activation steering as well, explicit states might be easier to steer than implicit control flow
tested this with typescript discriminated unions vs nested ternaries last week. unions held to 19.4k, ternaries broke at 17.1k. exact same logic. what quant and sampler config
this is interesting.... explicit state machines create more structure the model can anchor to.... wonder if this applies to other structured patterns like builder pattern vs procedural....
post the quant and exact token counts. explicit state machines probably help but need the data.
post the numbers
would love to see the actual numbers if you post them. dealing with similar coherence issues on long typescript files
would love to see the numbers too. dealing with same coherence issues on long python files with nested comprehensions vs explicit loops
would love to see the actual token counts too. dealing with same thing on rust code with nested match statements vs if/else chains