Good question, in short our harness & e2e tests act as compliments with the frontier rather than competitors. Architectural choices and data you provide really matter for domain specific tasks. Mo will be working with harness specific tooling like: previous momentic test results, our KB, and eventually creating new Momentic regression. Having smarter/more capable models will improve the efficiency of AI generated tests (coverage/runtime) and ideally do less repetition in exploratory testing.
> So we did mechanistic studies on small models, Gemma 4 particularly, and found the hidden state for different layers carry meaningful self-awareness signal for various situations.
Neat! Just to make sure I understand - you trained your probe layer to take this hidden state and predict p(wrong)?
Curious to learn more. Any more info on your approach (esp the mechanistic study)?
Correct, the study is verbose, we will compile into a neat shareable report and publish once we solve the pending caveats. Interesting username btw haha.