but sandboxing would be quite useless

How so? If you can break out of a sandbox then by definition it is not a sandbox.

There are ways to force sandbox jail. For instance, giving processes only a partial view of the computer system. GoboLinux did this years ago via ViewFS (https://linuxphilia.blogspot.com/2009/07/gobolinux-is-linux-... search for ViewFS). There are many other similar solutions, some probably better.

X itself is the problem here, not the process attached to

You need to sandbox X11 which requires giving up X11 capabilities just like Wayland did.