Yes I saw 3Blue1Brown say the same thing in his tutorial on how neural nets worked where he built a simple model to recognize a particular letter. Good reminder.
I've been dabbling with some of my own (tiny) models recently and it's actually shocking at what they can "learn" despite having _zero_ mention of it in it's training data.
