Dear JULES,
Nuts & cognitive ability
Executive summary
* A study was published which claimed that intake of nuts was inversely related to risk of cognitive impairment i.e. the higher the nut intake, the lower the incidence of cognitive impairment.
* The study was conducted on 16,737 participants from two regions of China. It dated back to a Food Frequency Questionnaire completed in 1993-1998. The food intakes at this time were then tested for associations with a cognitive ability test performed in 2014-2016.
* Nut consumption overall was tiny. The lowest group averaged a quarter of a gram a day. The highest intake group barely averaged one cashew nut a day.
* There were many differences between the people in the lowest and highest nut intake groups. They were different people in essence. These factors were adjusted for, but a whole person can’t be adjusted for. The study additionally adjusted for unsaturated fat intake and the nut finding then ceased to be significant. This was reported in the paper abstract. The nut claim should not have been made, therefore.
* Even if the nut claim had survived adjusting for another dietary factor (unsaturated fat), this would still have been association, not causation. The absolute differences in performance on the test were tiny and there were other issues with this study.
* The supplementary material showed which characteristics individually made a difference. Being diagnosed with cognitive impairment was more likely in older people (no surprise), in women, in those with secondary education, in those with obesity and diabetes, and in those who had had a stroke. Some of these were unsurprising; the others I have explained.
* This paper is asking us to believe that differences in cognitive ability in a test are due to the person’s nut intake, recalled on a food frequency questionnaire, 16-23 years earlier. Oh, and when unsaturated fat was adjusted for, we’re not even being asked to believe that.
Introduction
I saw a couple of people in the low carb world tweet about this week’s study. It was published on December 18th, 2020 in the journal Age and Ageing and it was called “Consumption of dietary nuts in midlife and risk of cognitive impairment in late-life: the Singapore Chinese Health Study” (Ref 1).
The study
The study is one of those nutritional epidemiology studies with which we’re familiar. The study also emanates from a well-established population study, which we’re used to seeing. Large scale new population studies are rare. Most papers like this one are just re-examinations of data from long standing studies.
This paper uses the Singapore Chinese Health study data. Data from 16,737 participants were analysed. Participants came from two major Chinese dialect groups – the Hokkiens and the Cantonese – who originally came from Fujian and Guangdong provinces in China, respectively.
This population study goes back a long way. A food frequency questionnaire (FFQ) was completed at baseline, which was between 1993-1998. At that time, participants were, on average 53.5 years old (age range at recruitment was 45-74). Women accounted for 59% of participants. Cognitive function was tested using the Singapore Modified Mini-Mental State Examination (SM-MMSE). This was conducted during the third follow-up visit (2014–2016), when participants were 61–96 years old (mean age=73.2 years).
Dietary intake was assessed using a 165-item FFQ at baseline. The paper noted that one of the limitations of the study was “the consumption of nuts was only assessed once at baseline and time-varying changes during the follow-up could not be measured.” This means that cognitive ability in 2014-2016 was evaluated for its association with recalled nut intake in 1993-1998. (You may like to read that sentence again). Nut intake included “combination of peanuts and tree nuts as a single item from 16 mixed food dishes in the FFQ and did not distinguish by types of nuts.” Peanuts are legumes, but they were included in the definition of nuts. A serving was defined as 1-ounce (28g).
The participants were put into four groups by nut intake:
– Lowest intake: those consuming <1 serving a month (an average of 0.25g/day)
– Second lowest intake: those consuming 1-3 servings a month (an average of 2g/day)
– Second highest intake: those consuming 1 serving a week (an average of 5.5g/day)
– Highest intake: those consuming ≥ 2 servings/week (an average of 11.9g/day)
Please note those nut intakes carefully – the bottom group were consuming less than a quarter of a gram of nuts per day. That is barely even measurable. Even the highest intake equates to about one cashew nut a day.
The groups were uneven in participant numbers. The lowest and second lowest intake groups – people consuming between one quarter and 2 grams of nuts a day – accounted for 81% of all participants. The top intake group accounted for 6.8% of people.
The Singapore Modified Mini-Mental State Examination (SM-MMSE) can be seen here (Ref 2) and it’s well worth a look, to see how you think your elderly friends and relatives might fare. The first page is the one to look at to see how the 30 points for the test are allocated (Note 3). Scores below 24 are taken as a simple cut-off to indicate cognitive impairment. Because education level has a significant impact on cognitive ability, there are established education cut off points to define cognitive impairment. This study adopted these standards and thus used cut offs of less than 18 for individuals with no formal education, less than 21 for individuals with 1-6 years education and less than 25 for those with 7 or more years’ education. i.e. the less education, the lower the score used to define cognitive impairment, so someone less educated was given a lower bar to reach.
The characteristics table
The characteristics table is the part of the paper that tells us the healthy person confounder. In this study, there were many differences between people in the four different nut intake groups. Comparing the lowest and highest nut intakes, the lowest intake group were older, far less likely to be male, less likely to be Cantonese and more likely to be Hokkien, less likely to be married, nearly four times more likely to have no formal education, less likely to smoke, less likely to drink, and more likely to be inactive. These factors were adjusted for, but – as we always say – a whole person/lifestyle cannot be adjusted for.
As we have seen in a few epidemiological studies recently, the energy (calorie) intake between groups was so large as to make one question the validity of the FFQ. The average calorie intake in the lowest nut intake group was 1,320 calories a day. The average calorie intake in the highest nut intake group was 2,010 calories a day. Were the lowest nut intake people just worse at recalling everything they ate?
There were some relevant characteristics missing from the table, which were not adjusted for, therefore. Income, retirement age and occupation would affect cognitive ability and these were not reported, let alone adjusted for.
The results
The average (mean) SM-MMSE score was 24.81(±3.89). A total of 2,397 (14%) of participants were considered to have cognitive impairment in the third follow-up visit. The paper reported that intake of nuts was inversely related to risk of cognitive impairment i.e. the higher the nut intake, the lower the incidence of cognitive impairment. Using those consuming <1 serving a month as the comparator, those consuming 1-3 servings a month had a 12% lower incidence of cognitive impairment, those consuming 1 serving a week had a 19% lower incidence and those consuming ≥ 2 servings/week had a 21% lower incidence (Note 4).
We mentioned above all the things that were adjusted for (age, dialect, education, smoking etc). The fully adjusted model was then further adjusted for intake of unsaturated fatty acids. The abstract (opening summary) of the paper reported “Further adjustment for intake of unsaturated fatty acids attenuated the association to non-significance.” That means, unsaturated fat intake was different between the groups and, when this was adjusted for, the nut finding ceased to be significant. From this point on, the paper is no longer about nuts. We could stop here, but there are other interesting points, so let’s continue…
The issues
Even IF the nut finding had stayed significant after adjusting for another dietary factor (unsaturated fat intake), the usual epidemiological study issues would have applied:
1) Association not causation.
An observation made between participants in a study who consumed more nuts and those participants who had lower incidence of cognitive impairment many years later is an association. A plausible mechanism was offered as “nuts may be beneficial for cognition due to the antioxidant, anti-inflammatory and endothelial vasodilator functions.” Three references were given for this claim. However, Bradford Hill criteria set out that unless associations of double are seen, association is unlikely to be causal. Risk ratios of 12-21% are nowhere near double (Ref 5).
2) Relative not absolute risk.
The risk ratios in the paper were confusing at first sight. The biggest difference in relative risk was given as 21%. That was for the difference between the lowest and highest nut intake groups. But this was based on the cut off process described above. If someone with (for example) 7 or more years’ education scored below 25 on the SM-MMSE, they were placed in the cognitive impaired category. If they scored precisely 25, they weren’t.
However, in absolute terms, differences in SM-MMSE scores came down to fractions of a point. Compared with participants in the lowest nut intake group, those in the 1–3 servings/month group scored, on average, 0.16 points higher; those in the 1 serving/week group scored, on average, 0.25 points higher; and those in the ≥ 2 servings/week group scored, on average, 0.37 points higher. As is usually the case, the relative differences make the headlines and the absolute differences are tiny.
3) The healthy person confounder.
This was more of a whole person confounder rather than a healthy person confounder. The person barely eating nuts was more likely to be female, Hokkien, less educated and less active. The person eating the most nuts was more likely to be male, Cantonese, more educated and more active. Those are different people entirely, but we are being asked to believe that differences in their cognitive ability in a test are due to their nut intake, recalled on a food frequency questionnaire, 16-23 years earlier.
Other issues with this study
1) Uneven groups.
Using equal sized groups is the fairest way to allocate participants. There can be no allegation of fixing the group cut off points to achieve a desired conclusion. Ideally, therefore, participants should be split into three equal sized groups (tertiles), or four equal groups (quartiles), or five equal groups (quintiles) etc. The uneven allocation of people in this study means that we cannot rule out that the group boundaries were chosen to achieve a significant result.
2) The small group comparator.
The accompanying issue with different sized groups is that one group usually ends up small and another ends up big. This happened in this study. The lowest nut intake group contained 6,203 people and the highest nut intake group contained 1,138 people. That’s five to six times the difference. This means that small numbers in the smallest group can make a bigger difference than small numbers in the biggest group. We have used the children skiing example before to explain this (Note 6).
3) Non-generalisabilty.
This study was conducted on two dialect populations in China. Arguably it is not generalisable beyond these people. It may be that nuts are consumed in a particular way, in particular dishes in these populations and that nuts are merely a marker of something else.
The supplementary file
The supplementary file was a treasure trove, as it usually is. Supplemental Table 5 was especially so. This presented the different characteristics (age, sex, education, smoking etc) individually and it reported on which ones made a significant difference on their own. For example, unsurprisingly, age at the time of interview made a significant difference – the older people were, the more likely they were to be placed in the cognitive impairment category. The other (statistically) significant findings were as follows:
– Females were more likely than males to have cognitive impairment identified. A plausible mechanism for this would be women having less stimulating occupations throughout life.
– Participants with secondary education were more than twice as likely to have cognitive impairment identified than those with no formal education. This suggests that the lower bar set for lower education may have been too generous.
– Participants with a BMI of 23-27.4 were more likely than those with a BMI of 18.5-22.9 to have cognitive impairment and those with a BMI greater than 27.5 were even more likely again to have cognitive impairment. I’ll take this point with the next one…
– People with diabetes were more likely to have cognitive impairment than those without diabetes. The combination of obesity and diabetes would suggest a poorer diet (dare I say a higher sugar diet). Alzheimer’s Disease has been called type 3 diabetes for over a decade (Ref 7).
– People who had had a stroke were twice as likely to have cognitive impairment. That’s understandable.
Those associations were more interesting and more plausible in my view. This paper is asking us to believe that differences in cognitive ability in a test are due to the person’s nut intake, recalled on a food frequency questionnaire, 16-23 years earlier. Oh, and when unsaturated fat was adjusted for, we’re not even being asked to believe that. This paper found some interesting things, but nut intake being associated with cognitive impairment wasn’t one of them.
Until the next time
All the best – Zoë
References
Ref 1: Yi-Wen Jiang et al. Consumption of dietary nuts in midlife and risk of cognitive impairment in late-life: the Singapore Chinese Health Study. Age and Ageing. December 2020. https://academic.oup.com/ageing/advance-article-abstract/doi/10.1093/ageing/afaa267/6041160
Ref 2: https://www.heartandstroke.ca/-/media/pdf-files/canada/clinical-update/allen-huang-cognitive-screening-toolkit.ashx?la=en&hash=631B35521724C28268D0C2130D07A401E33CDBB0
Note 3: Questions ranged from “What is the year? Season? Date? Day? Month (5 point) to “Spell WORLD backwards” (5 points).
Note 4: The odds ratios (ORs) (95% CIs) for participants consuming 1–3 servings/month (median=2.03 g/d), 1 serving/week (median=5.5 g/d) and ≥ 2 servings/week (median=11.87 g/d) of nuts were 0.88 (0.80–0.98), 0.81 (0.69–0.96) and 0.79 (0.64–0.98), respectively, compared with those who consumed <1 serving/month (median=0.25 g/d; P-trend=0.01 in the model 3). Each 1 serving/week increment of nut consumption was related to 6% (1–11%) lower risk of cognitive impairment (Table 2).
Ref 5: https://www.zoeharcombe.com/2016/09/the-bradford-hill-criteria/
Note 6: If 20 children go skiing – 2 of them with autism – and 2 children die in an avalanche – 1 with autism and 1 without – the death rate for the non-autistic children is 1 in 18 (5.5%) and the death rate for the autistic children is 1 in 2 (50%). Can you see how bad (or good?) you can make things look with a small comparator group? (https://www.zoeharcombe.com/2018/08/low-carb-diets-could-shorten-life-really/)
Ref 7: Suzanne M. de la Monte and Jack R. Wands. Alzheimer’s Disease Is Type 3 Diabetes–Evidence Reviewed. J Diabetes Sci Technol. 2008. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2769828/
