This makes me think of the Cauchy distribution, a probability distribution whose average follows the distribution itself rather than converging (hence, the distribution has no "mean" despite being symmetric). Is there any connection here, or is that just a coincidental similarity?
At the same time, though, it doesn’t mean you can’t infer the location parameter of a Cauchy distribution with more and more precision as you receive more samples. It just means that averaging the samples is not the way to do it.