The METR report included 0 technical details. For example, they did not include:
1. were the agents running on bare metal/docker/VM?
1. were the agents in a VPN?
1. how many TCP/IP requests were made? from what IPs?
1. how many tokens were consumed in the process? (this was explicitly censored)
A proper analysis would include this and MUCH more technical detail so that other AI researchers could actually understand the setup and how safe it was in principle.
They both look fine to me. But I do notice some weird aliasing effects in the animation that makes it difficult to identify the direction of fast moving rope. Maybe that's the problem?
A proper analysis would include this and MUCH more technical detail so that other AI researchers could actually understand the setup and how safe it was in principle.