I wish benchmarks tracked how often you have to undo changes the model made on its own. Sometimes the less ambitious model is just easier to work with.

Maybe the practical path is to separate fast-changing memory from slow-changing weights. Most things an agent learns during use probably don't need to become parameters immediately.

I tried Jev a bit, and the zero-shot performance already felt good enough to be useful. If you find the right place for that tradeoff, I don’t really see why “you could fine-tune BERT” is much of a criticism.