Totally OT: How to recognize junk science

As a scientist, I love data. I get emotional satisfaction from knowing that something has been proven by a well-designed experiment, preferably repeated at several unrelated labs.

I am really annoyed by junk science, especially when generated by parties with a vested interest in the results. Here’s a good example.

Publicity article

Could eating more nuts help boost memory later in life?

Research article

Longer-term mixed nut consumption improves brain vascular function and memory: A randomized, controlled crossover trial in older adults

How we know this is junk science:

  1. It was funded by the International Nuts and Dried Fruit Council (INC). Not surprisingly, nuts were found to be good for you!

  2. The number of participants (28) was less than the barest minimum usually considered necessary for statistical analysis (30).

  3. The study time was short (a 16-week intervention) which is far too short a time to determine whether a lasting effect resulted.

  4. Although they controlled the study by using each participant as their own control (a period without nuts and a period with nuts) they didn’t control for significant factors, such as race and income.

  5. The variability in the effects were larger than the mean effect. C’mon, what’s the use of a statistic like “treatment effect: 5.0 ± 6.5 mL/100 g/min”? I would be embarrased to publish something like that. If it was “treatment effect: 5.0 ± 0.1 mL/100 g/min” I would believe that the treatment effect of 5 was real instead of a calculated mean of a bunch of random dots.

  6. The main point – that memory was improved – was a very small effect (16%) that could easily have been an artifact of the test.

When I see data, I want large data sets (at least hundreds of people, millions is better but usually only available from national health care databases such as the VA in the U.S. or countries in Europe), long periods of followup and clear, highly significant results.

Data for evidence-based medicine is collected here:

Non-U.S. data can be found in the National Library of Medicine: Advanced Search Results - PubMed

I happen to like nuts and will continue to eat them (in small portions because they are high in calories). But I don’t think they will improve my memory.



Maybe they are… but you still can’t remember… :wink:

Keep eating!

He is no fool who gives what he cannot keep to gain what he cannot lose.


The media lives to promote hype and hysteria. Nonsense like this “study” is constantly being trotted out to urge you to drink more coffee, eat more chocolate, whatever the sponsor wanted the “study” to endorse. Of course, some of the studies are sponsored by “TPTB” urging everyone to be in a hetero marriage, have children, buy a house, and go to church.

Of course, these “studies” have to compete with the latest “severe weather” and “breaking news” hysteria, along with touting the latest activity by British royals and “celebs”.

Steve…after last week, I am entirely done with hearing about Taylor Swift

1 Like

As a non-scientist - or perhaps an amateur scientist who appreciates professional scientists - I have some very simple screening rules for articles about science.

Is it written or promoted by a politician? If so, probably junk science.
Is the research paid for by someone who will profit financially from the results? If so, probably junk science.
Is it published in a respected, peer reviewed, scientific journal? If so, probably good science.
Is it written by a scientist operating out of their lane? If so, it can’t be relied upon. Its in that gray area in the middle of junk science and good science. Which makes it’s pretty darn useless.
Is it written by a scientist operating in their area of expertise, but goes against pretty much every other scientist in the field, and virtually none of those are changing their minds? It’s probably junk science.



That’s usually true…but occasionally it’s a paradigm-shifting gem of brilliance that may take decades, or even centuries, to be recognized and accepted.



Might be worth another look. The lead author in this study is from Harvard.

Another study using a Chinese data set found “Nut consumption was inversely associated with cognition decline.” A Prospective Association of Nut Consumption with Cognitive Function in Chinese Adults Aged 55+ _ China Health and Nutrition Survey | SpringerLink

Nuts have a high concentration of flavonoids. Flavonoids apparently can accumulate in parts of the brain associated with learning and memory, providing a possible mechanism for how nuts can impact memory. This study reports than nuts can alter one’s EEG patterns.

Long a nut supporter (please, no jock jokes).

1 Like

Hence my “probably”.

Because most of the time in the modern world, it IS junk science, and probably should be caught by one of the other filters, we just don’t have enough information available to use one of the other filters.


Nuts are expensive. They are a luxury. Poor people can’t afford to eat nuts daily. Rich people who can afford to eat nuts daily are more likely to have higher education and income.

I read about the research that correlated nut consumption with cognitive function in older women. The first thing that occured to me was that income also correlates with cognitive function. Income is strongly correlated with education, which is one of the strongest correlates of cognitive function in old age.

Unless a study factors out education and income – variables which are directly correlated with cognitive function as well as the ability to afford to eat expensive nuts daily – the correlation of nut consumption as a separate, independent variable affecting cognitive function will be confounded.



Doesn’t mean a thing. There is a very well known “thought leader”, who holds degrees from both Yale and Harvard, and is several candlepower short of brilliant.



The Harvard study does take into account education as well as a number of other confounding factors. But look, there are very few conclusive human studies because doing proper controlled experiments on humans is very hard for ethical reasons. The best one can do is to make a judgement based on the best available data. I can find a lot of studies to support the view that nuts have a positive health benefit, including improving brain activity. Another example: Beneficial Effects of Walnuts on Cognition and Brain Health - PMC

But that’s just my opinion, your mileage may vary.

Doesn’t mean a thing only if you are an absolutist where everything is black and white with no grays. I go with the percentages. From what I’ve seen, the research coming out of Harvard tends to be better than average. Certainly not always, but more often than not. So Harvard does mean something to me. I am far more likely to take medical advice from the Harvard Medical School than someone from youtube. You may disagree.


Acetyl-L-carnitine improves aged brain function - PubMed (

I’ve been taking acetyl L carnitine 500 mg/day since the 90’s due to some early research on this nutraceutical. This is in response to the original post about mixed nuts and memory…doc

acetyl l carnitine studies at DuckDuckGo


Curious as to what dosage and why?



Do the hydrolyzed peptides powder/supplements provide L-carnitine, or the precursor building blocks?

A Google wants to sell me supplements.
Bing GPT tells me L-carnitine is

  • not a standalone molecule in hydrolyzed peptide powder;
  • an amino acid, synthesized in the body.
  • available in “meat”.

Therefore I conclude that peptide powders might be metabolized, and L-carnitine released.


1 Like

if I remember correctly the study cited the differences between old rats who got the supplement and compared to the old rats who got a placebo vs the controls that were young rats. The old rats that got ALCAR ran the mazes as fast as the young ones. The old rats that got placebo ran the mazes much slower. When they looked at the rat brains under the electron microscope, the rats that got ALCAR had brain cells that looked like the young rat brains vs the rats that got placebo had brains that were old with lots of aging issues. The article suggested a dose of 500 mg daily for maintaining the function of the brain components. It specifically mentioned that the mitochondria (engine of the brain cell) looked especially good in the ALCAR group…doc

1 Like

I actually stumbled across L carnitine information. L carnitine apparently doesn’t cross the blood brain barrier so would have no effects on the brain. Acetyl l carnitine does cross the BBB. Also, if anyone is interested carnitines may do something for testosterone as we age…doc

Carnitine versus androgen administration in the treatment of sexual dysfunction, depressed mood, and fatigue associated with male aging - PubMed (