Are we holding it wrong? Mine does the same godawful external library tests. When it is having a particularly stupid day it also tests language features. For example, checking that using a callback executes the code supplied in the callback (it does).
Yeah, that's what not so many are talking about but a few have pointed out; it's the AI writing wayyyyy too many tests causing the new CI load.
Unless you prompt them otherwise, the models tend to write WAY too many useless tests and in the most inefficient ways imaginable. This balloons test counts and lines of code to insane levels and frankly, likely, slowly makes it more and more costly for the AI to make future changes.. To the point it can't wrap its context around the code base and effectively make necessary changes.
Why not? If you let the LLM run amok, you get 4 one line functions calling each other instead of one 4 line function. Run that for 24 hours and you'll get more than 2000 tests.
