If I were saying we were definitely rare or unique, you'd have an argument.

But I'm not doing that -- I'm saying it's a possibility that is consistent with evidence.

The claim that life is common is different. For that, you really need to present an argument that life being uncommon is ruled out.

In your last sentence there you've retreated from "life is common" to "it's plausible life is common". Sure (within the constraints of the Fermi Argument, but that has uncertainties.)