Private users also have sensitive information which, if leaked, could be used to their detriment (e.g training data getting hacked, etc).

I don't see any distinction between companies and people when it comes to moral objections re training data - at least, if that's a thing, I'm unaware of it.

I do agree with your last point re the hypocrisy.