I've tried, casually, quite a bit of this. With generally poor results. Often this ends up similar what I have seen as classic RL problems with seeking reward in sparse environments. Unless you give the agent an a-priori goal outside the world it tends to not do anything interesting. Theoretically there ought to be ways around this such as herarchical approaches, but those tend to scale in complexity faster than my limited resources can keep up.
It's very neat though.
Edit: I really should watch the video before commenting. Looks like it goes into lots of these problems itself, and they got much further than me.
I think you would have to go back to more classic genetic algorithm/pet training methods: Iterate through increasing "distance" to the goal. Initially make the goal simple/immediately available, then slowly "distance" them from the goal, with more obstacles/etc introduced between, as you go.
For a homing pigeon, this would be literal, driving further and further each time you release them.
For an agent that you train to walk, this would be flat surface to past some line, then introduce speed reward, then non-flat surfaces, then obstacles, then curves, etc [1].
I feel like it's 50/50 between people doing that, and people that have spent a lot of time tuning a system they are pointing at focused and well specified problems.
Absolutely true. But the OPs statement is not without merit. While there is a lot of professional software for which GUI is the correct choice, there is also a lot of "GUI slop" which is just a GUI to look good on marketing and to be easier to onboard new users at the expense of long term effectiveness.
It's very neat though.
Edit: I really should watch the video before commenting. Looks like it goes into lots of these problems itself, and they got much further than me.