Not unless humans were able to produce the medicine or perform the surgery or whatever is necessary to actually heal someone. In order for that to happen it has to be verified and understood by humans, like the other commenter said.
Does it, though? What if future AI can do everything from designing the cure to building a self-contained universal surgical medcapsule? "Just lay down and I'll fix your cancer real good, trust me, bruh." If it was shown to work, would you use it?
Well sure, if it's already proven why wouldn't I use it?
The AI we currently have can't do that though. The AI we currently have is a glorified chatbot. It can't fabricate anything and I doubt it could even reliably design simple real world devices.
It's a really cool and useful technology, and maybe one day it will be as good as you describe, but right now it just isn't and there's no guarantee it ever will be.
