There are architectural solutions. You could duplicate your tokeniser/vocab for example; and have two classes of input: trusted (e.g. system prompts) and untrusted.
The exact same phrase can tokenize differently depending on if it's instruction or data; and you can pre-train and post-train models to make use of them.
It quadratically increases your training cost, so I don't think any labs are exploring it because of $$$ and the race to AGI, but mechanisms like this should significantly address the issue on the LLM architectural design level.
The exact same phrase can tokenize differently depending on if it's instruction or data; and you can pre-train and post-train models to make use of them.
It quadratically increases your training cost, so I don't think any labs are exploring it because of $$$ and the race to AGI, but mechanisms like this should significantly address the issue on the LLM architectural design level.