Wednesday, August 11, 2010

Can MRI scans be used to diagnose autism?


Another day, another autism diagnosis media storm. Last week, it was speech patterns. Before that it was urine. Today's story is that researchers in London have found a way to diagnose autism by performing multi-dimesional analyses of magnetic resonance imaging (MRI) brain scans.

So what did they actually do? And how close really are we to being able to use brain scanning to diagnose autism?


The study

Christine Ecker and her colleagues at the Institute of Psychiatry tested a group of 20 high functioning adults with autism, who were assessed the traditional way using either a structured interview with the parent (the ADI-R) or by video-recording a structured interaction between the person with autism and the experimenter and then coding their behaviour (the ADOS). The authors note that many of the people with autism could be considered to have Asperger syndrome but they didn't attempt to explore this issue further due to the small sample size.

The 20 adults with autism, together with 20 control adults, all underwent a 15-minute MRI scan. The set-up would have been something like the picture below.



From the scans, the authors extracted five different dimensions for each hemisphere of the brain that they thought might differentiate between the two groups.

One of these factors, left hemisphere cortical thickness, did a pretty good job. If the thresholds were set optimally, the algorithm could achieve a 90% accuracy. In other words, based on cortical thickness, 18 of the 20 people with autism were correctly classified as having autism and only 2 of the controls were incorrectly classified as being autistic. Other measures fared slightly less well and in general the right hemisphere measures were much worse at differentiating between the two groups. They also looked at combining all the different factors, but this didn't actually improve the discrimination performance.

The authors conclude their paper by stating that this is a "proof of concept". It shows that it is feasible to use these kinds of analytic techniques to investigate differences in the autistic brain.
So what are the challenges to be overcome in order to go from this to an MRI-based diagnostic protocol? Here are a few off the top of my head:

Does it generalise to new populations? In the current study, the choice of measures and weights were optimised to provide the best discrimination between the two groups of people in the study, whose diagnoses were already known. But we need to know that the MRI algorithm works just as well when it's given data from new people it hasn't seen before. To address this issue, the authors removed one person from each group, and optimised the MRI algorithm using data only from the remaining 38 people, so the two people removed were effectively new subjects. They then looked to see whether the two 'new' people were still correctly 'diagnosed'. They did this 20 times, removing a different pair each time and showed that they were statistically above chance at classifying the 'new' people. However, it's not clear (to me at least) how much better than chance.

Does it work with kids? The participants with autism in this study were aged between 20 and 68. A diagnostic tool for autism that only works with adults is obviously of limited use. The authors are confident that the MRI strategy will work as well if not better with children, but this needs to be demonstrated. It's also worth bearing in mind that brain scanning young children (especially autistic children) is far from straightforward and often has to be done when the child is asleep or sedated. The quick and easy 15 minute scan suddenly becomes less of a reality.

Can it discriminate between autism and similar disorders? Here is the crux. The 90% accuracy headline figure is in comparison to typically developing brains. I don't think the authors are really suggesting that we should screen the entire population for autism using MRI. Apart from the cost and ethical issues, it wouldn't be at all effective. Given that 10% of non-autistic people are classified as having 'autistic brains' and that only 1% of the population have autism, this means that in a random sample, most of the people identified as having 'autistic' brains wouldn't actually have autism.

More useful, perhaps, would be if people who were already suspected of having autism could be given a brain scan to confirm or deny the diagnosis. People tend to think of autism as being a clear cut condition, but in truth it's a messy affair, with much overlap between different disorders and fuzzy boundaries. A 'brain' test could be really useful, but it would have to pass a much more stringent evaluation process. This would involve comparing two groups of people suspected of having autism - those who go on to have their diagnoses confirmed and those who are not ultimately diagnosed with autism. If these two groups differed consistently in their brain structures then MRI might prove to be a useful diagnostic tool.

Ecker and colleagues make a step in this direction by applying their algorithm to adults with ADHD. It does OK, but still mis-diagnoses 21% of them as having autism. And there's no suggestion that anyone ever thought these people might have had autism.

So why bother looking at brains?

Overall, I have to question whether there really is any value in trying to use brain scans to diagnose autism. Ecker et al.'s paper begins by acknowledging the fact that autism is a highly heterogeneous disorder and should perhaps be referred to as 'the autisms' rather than a single condition. As I've mentioned before, two people can get the same diagnosis by ticking completely different boxes and recent research only goes to strengthen the impression that there are many different causes of autism at the genetic level. So does it then make sense to try and find a brain 'fingerprint' for autism? Probably not.

Where I think this kind of approach might well come in useful in the future is in identifying subgroups within autism. There are already a huge number of autism brain scans that have been collected over the years and it might be possible to run similar analyses and identify clusters in multi-dimensional 'brain space' that correspond to meaningful subgroups.

Perhaps one day, it will be possible for a child newly diagnosed with autism to undergo a brain scan which will indicate what subtype of autism they have. The outcome of this could help determine what intervention strategies and support are likely to be the most beneficial. If and when that day arrives, the media excitement will truly be justified.

Reference:

Ecker C, Marquand A, MourĂ£o-Miranda J, Johnston P, Daly EM, Brammer MJ, Maltezos S, Murphy CM, Robertson D, Williams SC, & Murphy DG (2010). Describing the brain in autism in five dimensions--magnetic resonance imaging-assisted diagnosis of autism spectrum disorder using a multiparameter classification approach. The Journal of neuroscience : the official journal of the Society for Neuroscience, 30 (32), 10612-23 PMID: 20702694

Download a free copy of the original article from the Journal of Neuroscience 

Media reports:
Other critiques of the paper:
    Response from the author to criticism.

    Update (18/12/10):

    Journal of Neuroscience have just published a 'Journal Club' response from Jennifer Stevenson and Kristina Kellett - two members of Morton Gernsbacher's lab. It covers many of the points raised in my post and the other blogs linked to above, but also raises some further salient issues. The authors note that other similar studies of autism have found different patterns of group differences in the regions identified by Ecker et al as discriminating between autistic and non-autistic brains. For example, whereas Ecker et al found reduced cortical thickness in the parahippocampal gyrus, Jiao et al 2010 reported increased cortical thickness in the same region. And to further complicate matters, a cross-sectional study by Raznahan et al (2010) suggests that other regions show developmental changes in the direction of 'abnormality' in autistic brains.

    In another recent development, Nicholas Lange and colleagues claimed that they could differentiate between autistic and non-autistic brains using DTI (another form of MRI). I've only had a very quick look at the paper, but it seems as though many of the concerns raised about the Ecker paper are also relevant here. (22/12/10: There's a write-up of the Lange study at Biology Files, which does indeed raise similar concerns about the prospects for MRI-based diagnoses).

      8 comments:

      1. I haven't read the paper, but CBC tells me that all the subjects were male, too. Would that have an effect on the results?

        ReplyDelete
      2. Yes, all of them were male. I think it would have been difficult to include females within the same analysis. But it does limit the generality of the conclusions.

        By the way, the paper is open access. You can read the whole article here http://www.jneurosci.org/cgi/content/full/30/32/10612

        ReplyDelete
      3. I read what I could understand in the paper and you left a couple of significant facts out. 16 out of 20 (three quarters) of the ASD subjects had Asperger's rather than autism. It might not be good experimental practice to mix the two gruops together. The results may not be applicable to persons on the autism spectrum who had a speech delay anytime in their lives.

        Also, the 90% accuracy refers to sensitivity rather than specificity. The specifity was 80% which might be more valid percent rate in reporting accuracy. Or perhaps you could explain to me the difference between sensitivity and specificity, because i still don't understand that.

        ReplyDelete
      4. @jonathan

        In DSM IV (the current diagnostic 'rulebook'), the main thing differentiating Asperger syndrome from autism is that early language and cognitive development is normal. Given that the subjects were all adults (the oldest was nearly 70), it would have been very difficult to get accurate information to allow the authors to differentiate between those with 'high functioning autism' and Asperger syndrome. In fact, I think I'm right in saying that the new version of the diagnostic rules (DSM V) isn't going to distinguish between autism and Asperger syndrome at all. So I don't think it's a question of bad experimental practice to have lumped them together.

        But you're absolutely right that we don't know how well the test would have done with people at the more severe end of the autism spectrum.

        The way I understand it, sensitivity is how good the test is at identifying autism in people who actually have autism. The sensitivity was 90% because 18 out of 20 people with autism tested positive.

        Specificity relates to the false positives - the number of non-autistic people who tested positive for autism. In their main analysis using all five 'dimensions', 4 out of 20 controls tested positive so the specificity was 80%.

        I think I may have confused matters by focusing on the cortical thickness measure, which actually did a bit better and had a specificity of 90%.

        It's pretty striking that the groups were so different in terms of cortical thickness. It means that there was virtually no overlap between the two groups. And that probably tells us something about the underlying neurobiology of autism.

        ReplyDelete
      5. Exceelent post, Jon.
        Here's another paper that gets results with different patterns of cortical thickening/thinning (and doesn't hype the findings).Wallace, G. L., Dankner, N., Kenworthy, L., Giedd, J. N., & Martin, A. (2010). Age-related temporal and parietal cortical thinning in autism spectrum disorders. Brain, 133(12), 3745-3754.
        Another point I raised in my blog on this topic (http://tinyurl.com/33fwmsx) is that it's about time researchers doing brain imaging studies of autism stopped reporting on N = 20 and started instead cross-replicating findings before publishing. It's the same issue as in genetics: there's far too much scope for false positive findings. In genetics, you now can't publish an association unless you replicate on a new sample. There's now enough studies out there for this to be a reasonable suggestion, and people are starting to move towards making MRI data shareable.

        ReplyDelete
      6. Hello Sir,
        I am developing a website on educational resources for high school students. In this connection, I want to use an image on this page.
        Waiting for your permission ...
        Ramesh Patwardhan
        www.abhyasika.com

        ReplyDelete
      7. Hi Jon, I read your blog with interest. My young son with autism was part of an MRI research study done in the USA. I have written a blog about my experience called 'An MRI scan can get 'messy' at www.kazbrooksblog.wordpress.com

        ReplyDelete
      8. Every week-end I used to pay a fast visit this site, because I’d like enjoyment, because this web site conations certainly fussy material. Naperville 3T MRI Scan

        ReplyDelete