I use memory all the time so that I can ask it about things. That doesn't mean it "knows me" any more than my notes documents do, it's just more convenient to search.
Yeah I don't think we disagree that much about this. But I'm not really worried about this particular habit forming risk. I can see where you're coming from though.