In a prisoner's dilemma the players can't rely on one another to self-regulate. This is why Mafias kill snitches, so when their members are in a prisoner's dilemma they can rely on the knowledge that if themselves or the other party snitches they have more to lose than gain.
The problem is that there can not be any outside party to regulate this on a global scale
There's plenty of precedent for specific industries self-regulating in the US, typically motivated by the shared view that it would be less disruptive than heavy handed federal regulation.
Applying game theory problems like prisoner's dilemma to the real world requires collapsing a lot of overlapping and conflicting incentives to a single dimension which doesn't always work.
In this case the goal is bringing about the singularity, basically creating a sort of godlike intelligence. That's what the arms race is all about.
So we are in a true prisoner's dilemma, if one of the companies / countries stops R&D it risks handing over godlike power to another company / country.
The problem is that there can not be any outside party to regulate this on a global scale