I make it a rule never to give confidential information to an incoming call. I had a primary doctor whose office would send notification calls immediately asking for part of my social security number, I always hung up.... I figured out it was for the doctor later, and commented that it was ridiculous and I wouldn't ever receive any such notifications.

Just that simple practice has kept me reasonably safe, even from an AI scam once.