A genetic test for autism?

This week's big autism story was a genetic test able to predict with 70% accuracy [1] whether or not a child had autism. Rather than looking for a specific gene that might differentiate autistic from non-autistic people, Stan Skafidas and colleagues from Melbourne University developed the test by combining information about many different genetic variations. Critically, having developed the test based on one set of genetic data, they then tested the test on genetic data from a completely new set of people.

I don't want to take anything away from the basic science. And it would, of course, be incredibly useful to know early on whether or not a child is likely to develop autism. But the headlines are misleading. The unfortunate truth is that we're still a long way from a genetic test for autism. A screening measure with 70% accuracy would only be slightly better than completely useless.

Here's why.

Say you had 1000 kids and you ran the genetic test to see which ones would become autistic.

If we assume that the rate of autism in the population is around 1% then we'd expect 10 of the 1000 kids to be autistic. Given 70% accuracy, we'd expect 7 to show up on the genetic test as autistic.

The problem is the other 990 who aren't autistic. 70% accuracy means that 30% would be incorrectly diagnosed as autistic. 30% of 990 is 297.

Putting those together, our genetic test thinks that 304 of the 1000 children are autistic. Of those, only 7 really are autistic, the other 297 have been mis-diagnosed.

In other words, if your child took the test and the test came out positive, there would still be a 98% chance that your child was not autistic.

[Edit 15/9]: The researchers describe the test as a test of autism risk rather than a diagnostic test. In those terms, what it tells you is whether you have a  2% risk, as opposed to the usual 1% risk.

This screener's base rate fallacy is a well known problem. Dorothy Bishop covered it when discussing screening for autism based on speech recordings and MRI scans. Ben Goldacre talked about it in relation to screening for terrorists. It's a problem any time you're looking for something that's relatively rare. Even a really good test gets torpedoed by a disasterous rate of false positives.

To be fair, the researchers did suggest that the test would be most useful for parents who already had one autistic child. The latest research suggests that, if you have an autistic child, the chances of a second autistic child may be as high as 19%. Even in that situation, you'd still expect roughly two thirds of the kids picked up by the test to actually be non-autistic. [Edit 17/9]: And that's assuming 70% accuracy. In fact, the test was much poorer at discrimination between siblings with and without autism, suggesting that it's predictive value in the real world will be much lower.

Update (15/09/12)

On Twitter, Emily asked whether such a test could ever be useful.

@drbrocktagon how high would the detection rate need to be to be useful? Given rarity of #autism would any test escape false positive issue?
— Emily (@mosaicofminds) September 14, 2012

As a population screening test, I think the answer would have to be no. Let's assume that the sensitivity (ability to detect true cases) goes up hand in hand with the specificity (ability to reject non-cases). Then for each level of accuracy, we can plot your autism risk if you test positive for autism. Even if the test was 90% accurate, you've still only got an 8% risk of autism if you test positive.

Update (17/09/12): Stan Skafidas has responded (at length!) in the comments.


Update (06/06/13): Dan Geschwind and colleagues have written a critical commentary of the paper in Molecular Psychiatry. See Neuroskeptic for coverage.


Notes

[1] "Positive and negative predictive accuracies were 70.8 and 71.8% respectively"

Reference

Skafidas E, Testa R, Zantomio D, Chana G, Everall IP, & Pantelis C (2012). Predicting the diagnosis of autism spectrum disorder using gene pathway analysis. Molecular psychiatry PMID: 22965006 Full Text

Related post

• Can MRI scans be used to diagnose autism?