It pains me to see such a low-effort contrarian comment at the top: as stated in TFA taxi drivers are below other profession when controlling for age, and there are multiple professions with lower life expectancy that still have significantly higher Alzheimer rate than taxi drivers.
Fella has read the article, has an argument and some stats. What more do you want from a comment? He isn't running a study.
Besides, the point is a good one. There are a lot of professions. If we glance at the underlying study [0] Figure 1 suggests that we'd probably expect to see an outlier or two in that mess just through chance, we're expecting a few p<0.01. Taxi drivers doing unusually well doesn't seem especially impressive in that light, especially since it is one of the professions that seem to die off rather young and we might expect one of those professions to do rather well on Alzheimers.
But after the study controls for age the starkness of the effect more or less disappears (eyeballing Fig 1, the 2nd graph).
It turns out that there is a profession with a lowest risk of getting Alzheimer's (duh, one had to exist) and it looks like a slight outlier (it's the lowest, we'd expect that one to look like a slight outlier). As outliers go it isn't that impressive, it looks like it is part of a fairly boring distribution. We just plotted 400 data points on the graph, the 1/400 chance weird data point is going to look a little weird. We're actually just reading a lot into the fact that the 2nd data point in ambulance drivers which isn't all that convincing.
Not to knock the study, I'm sure it is a great study. Lots of science done. They do good graphs and all that. But the study appears to be telling me that Alzheimer's is random with respect to occupation and ideally don't spend so much time on the road if you want to live to see old age.
> But after the study controls for age the starkness of the effect more or less disappears (eyeballing Fig 1, the 2nd graph).
Adjusting for age, Alzheimers deaths were 1/100 for taxi drivers, and 1/60 for the general public. That's extremely significant. However you feel about the graphs means nothing compared to the actual numbers presented here.
> I'm sure it is a great study. Lots of science done. They do good graphs and all that.
Your choice to be dismissively anti-science helps to explain the bad point you're making.
What makes you think that is significant at all? Anytime you take a lot of small subsets of a large random sample you expect to find outliers that look a lot like this.
Outliers, sure. Exactly like this though? That depends. Since this is a population then assuming a gaussian (which could well be wrong for a variety of reasons I'm just guessing here) what's sigma? If 1/100 is within ~3 sigma then sure. Otherwise no.
It is an outcome from over 400 data points. We're expecting to see events outside a 3 sigma boundary. In fact, 3 sigma is basically the 1:400 boundary. Strictly it is 1 in 370 [0] for anyone who takes Wikipedia at face value.
In the study they seem put the P value of the taxi driver outcome at <0.01 (in Table 2, although I admit I haven't read the thing especially closely). For 400 data points, we expect there to be around 4 values with a P value less than <0.01.
It needs more statistical analysis than I have the background to do. However all signs look to another variation of the replication crisis, which is more an more my default assumption when I see an observational study.
You're handwaving. The xkcd point only applies if you're within ~3 sigma. This isn't a complicated statistical analysis it's just asking how far out the outlier is versus the standard deviation and size of the dataset.
To be clear I'm not claiming in one direction or the other. I feel that if you're going to attempt to discredit a piece of work it's on you to do such a basic check.
Really the whole point of that xkcd was to viscerally illustrate the practical effect of binary misclassification, or alternatively to make the point that a p of 0.05 isn't necessarily as rigorous as you might expect. Neither of those things directly applies here.
The article doesn't give enough information to answer your question. It isn't on me it is on the reporter's who shouldn't allow a handwave like that to pass without question. There are far to many cases of people hacking statistics in the world to accept that.
It's getting hard to find any study posted without a kneejerk "correlation does not equal causation" take as if that's not something researchers have to control for in order to even conduct the study in the first place, let alone publish
The institutional media departments, journalists, and headline writers add back in the insinuated causation and effect size as a routine matter. That way we can't find fault with the researchers for their clickbait study.
The article (and apparently the researchers too) advances the idea there's a causal link here, even though one is not shown by the research. They go so far as to generally prescribe spatial reasoning training as a way to reduce Alzheimers risk.
You might be tired of hearing "correlation does not equal causation", but it needs to be said here.
On hackernews I really don't think it needs to be said. Call me arrogant/too generous to this community, but I think most of the users here know the phrase and its meaning. Seeing 8000 variants of it doesn't contribute, and since it costs nothing to say "I bet they made a mistake" while elevating your own intellect people default to it far too much.
Read the comment I replied to. It claims the researches must have somehow controlled for correlation (whatever that’s supposed to mean). Which they haven’t. There are a bunch of other comments arguing there’s something beyond correlation shown here. But it hasn’t. The study just demonstrates correlation.
The misunderstanding here is widespread so it worth calling out.
It’s actually kind of funny that people are more sick of the mistake being pointed out over and over than of the mistake being made over and over.
Ok, if we’re going to play this game, what indicates they are making “correlation = causation” errors? What specifically about this study fell for that trap and has not shown something meaningful? You seem very confident they screwed this up, so there must be a reason. And no I’m not digging through your comments to piece together your argument lol I’m sorry but it is ridiculous of you to expect me to comb through your comments and synthesize your point. If your comments were so thorough and proved it then just copy and paste 1 or 2 here that are driving the point home with actual evidence.
Their argument doesn't show that at all though, they make an argument against an age bias when the article explicitly says age is being controlled for.
> Figure 1 suggests that we'd probably expect to see an outlier or two in that mess just through chance, we're expecting a few p<0.01.
That's a valid point, and a much better criticism than the one I'm calling out. That doesn't redeem the original comment though.
A distinction is that Alzheimer's is in an uncommon category of mystery diseases.
If a study found people who swim regularly are are less likely to get heart disease, that wouldn't be very surprising because know a lot about why people get heart disease. Therefore it would be valid to insist on more rigour for accepting (swimming => less heart disease) because the only use of that information is for health advice.
However even weak statistical correlations can be a valid lead for further research into Alzheimer's disease if they suggest where to look. You don't need to prove that signposts are correct in order to use them.
If we read OP's comment closely, it appears to be technically correct. I can imagine someone not upvoting it because it seems to be generic, but there isn't anything wrong or low effort about with it. And I don't see where the stats came from if he hasn't read the article.
maybe it would help this thread to have someone explain exactly how they control for age, if everyone in the range they want to study, is already dead.
What is the actual process to control for age? This comes up a lot, and the only response is "oh, well, we controlled for that", well exactly how?
Seriously, I'm curious.
//lot of downvotes, not a lot of anybody actually able to explain it.
From Article, the age question does seem sketchy.
"Although proportional mortality analyses do not provide information about the population at risk, with careful selection of controls and risk adjustment for factors that may affect competing risks (eg, age, sex, and social class), these values can still serve as a useful indicator of variations in disease frequency across different occupations.16 17"
You build a model that describes how likely people are to get diagnosed with Alzhimer's as a function of sex: maybe 12 women out of 100 get it, while 8/100 men are diagnosed. You can do the same thing for age: almost nobody is diagnosed before 30, it's very rare before 40, and sadly common (~1 in 10) after 65 years of age. There are all sorts of mathematical tricks to include multiple variables, account for the fact that you can only be diagnosed once, or that data is "censored" (i.e., missing) at some ages because people have already died.
Based on that, you can then ask if the prevalence of Alzheimer's Disease among cab drivers is surprising, given their demographics. For example, if we know that they skew male and younger, we'd expect that number to be a bit lower than a naive estimate of 10%. In fact, we can calculate that number and then see if it's unexpected given the number we actually see. In practice, you'd actually do this by fitting two models: one containing job and one that doesn't, and see which one best describes the data and how, specifically, the job factor affects the outcome.
However, how well these adjustments work depends on the quality of your data and your modelling. Your model could be missing important factors or have the wrong structure: (e.g., you assume risk is directly proportional to age, but it actually increases more rapidly as you get older). Your data could have problems too: maybe women are more likely to go to the doctor (and thus get diagnosed), even if the actual prevalence is the same.
As long as a researcher said "we controlled for confounding variables," there are plenty of people willing to take it at face value.
It's really difficult (maybe not possible) to do that in these large retrospective studies. Shingles vaccine preventing dementia, GLP-1 drugs preventing just about everything, etc, etc. There are plenty of good faith critiques of this kind of work from actual medical researchers making essentially the same point as the OP.
True. But, these are still odd outliers in terms of low life expectancy, and that's something that the actual BMJ article cited (https://www.bmj.com/content/387/bmj-2024-082194) or TFA talks about at all. The LE is low enough for both professions to be awkward.
I'm also a little concerned that the BMJ plot of risk adjusted mortality still shows a clear positive correlation with age at death, which I strongly suspect would turn out to be statistically significant. The adjustment does not appear to be correct, therefore, and while that might still preserve these odd findings, it's definitely not ideal.
The much better occupation, from the graph, is the third outlier -- mining and geological engineers. Life expectancy: 80.3, risk-adjusted mortality: 1.1%. Or, alternatively, economists are close too (80.6; 1.19%).
Aircraft pilots, btw, really were the opposite end of the scale (78.1; 2.3%), surprising enough to be noteworthy beyond a control (the BMJ article does discuss this a little, but seriously -- that's very surprising for a highly-educated profession).
Worth mentioning that controlling for confounders is hard, a reasonable prior is the null effect, and any time a researcher makes a mistake it produces a publication claiming a novel effect. You observe a set of publications concentrated with false positives as a form of selection bias. I wonder how many of the causal studies are reproduced by independent teams using independent datasets.
Plausible scenario. Individuals predisposed to Alzheimer’s experience different mental sharpness from birth and this makes them enjoy taxi driving less (more intense than long-haul trucking), and so they pursue taxi driving as a career at lower rates. Under this scenario, driving has no effect, it just induces a selection bias.
A better low effort contrarian comment - isn't it that early symptoms appear less often with people with a "reserve"?
Being mentally active (or physically active?) IIRC makes actual plaques both less likely, but also less detectable - a taxi driver can forget 90% of what they know and still not get diagnosed because they don't get lost on the way to the shops. Also they are often masters of small talk.
This would suggest they'd need to control for method by which the alzheimer's was discovered. A quick glance seems to indicate they did not do this. This is not information available I guess..
I would suspect this would eliminate a lot of the experiments participants, as only people who go for regular cognitive check-ups might be eligible?
I cannot imagine there would be many people left who'd be eligible. I can only think of 1 person that does it.. and he obviously does it for the pure satisfaction of acing those cognitive tests, and not because he is showing worrying amounts of decline in his mental faculties.
I do wonder if you can truly control for it in this case, or if doing so doesn't in and of itself introduce a survivar bias that masks a reversal of causality. That is: could the taxi drivers who are already predisposed to have a lower chance of developing Alzheimer for other reasons also be the ones who are more likely to live to be older than the average taxi driver?
If the "study" is interesting at all, it's because of the implication that driving a taxi somehow wards off Alzheimers... but at the cost of killing you before you get it.
Every top-level thread is like a multiple choice armchair takedown.
Here's an interesting counterpoint. "Unthinkable" presented studies of navigation tasks in 2D/3D and testing/training. Hippocampus measurements correlated with skill and training. Interestingly, this skill and training and measurements all correlated with beneficial responses in a disaster (as in, not panicking). Weird right?
But on some level I understand the immediate low level criticism. Of course, it’s stupid to criticize something without reading about it. However, I’ve seen waaaay more studies which was made with even lower effort.
For example, the other day I wanted to figure out what’s the current state about whether gender equality causes measurable benefits for companies… and the studies are terrible. All of them. Most of them was about Norway, and almost all of them had a reference point in 2008, one of the largest economic crisis, and somehow most of them was even worse than this. They openly distorted statistics. Depending on what they wanted to achieve to one way or another. Even the most cited ones. It’s disgusting. The best ones could prove only that inequality is not inherent of economics, but social. But the agenda was different for them too, so they tried to lie something bigger, all the time, while this would be more than enough to support it.
The first question you ask when you read a paper like this is whether you think the authors actually did control for what they say they controlled for. Usually, they didn't.
