FDroid offers the guarantee that the apk was built from source, and at least some marginal safety in terms of having passed their security and analytics/ads checkers.

You lose all that if you go directly to source.