Does smoking weed really result in brain abnormalities? Maybe not.

554px-Cannabis_leaf_2.svgA study purporting to find that marijuana use (even casual marijuana use) may be associated with brain abnormalities has been getting a lot of press lately. You can check out some of the coverage at CNN Health, the Huffington Post, and Fox News. And you can check out the original paper in the Journal of Neuroscience, Cannabis use is quantitatively associated with nucleus accumbens and amygdala abnormalities in young adult recreational users by Gilman et al., here.

Shortly after the study came out, Lior Pachter posted an analysis of some major problems with the study on his blog. I’m posting a link to his post because I think it’s a great example of something science bloggers do very well: they share important information about the quality of recent studies in real time. This is essential stuff you just don’t typically see in media coverage.

I’d also like to note that the statistical issues he points out are very basic ones. Adjusting p-values for multiple testing is something that I think most researchers understand they have to do even after an introductory stats class. So I’m having a difficult time understanding how this manuscript sailed through peer review in its present form. The Journal of Neuroscience is not some fly-by-night journal! I hope that journal editors will see what happened here and realize that if a manuscript contains statistics, it’s probably a good idea to choose at least one reviewer with knowledge of statistics. Failure to control for multiple testing appropriately is something I see over and over again in the articles I review. There is definitely a need for the statistics police in the peer review process.

My take on “Is Breast Truly Best?”

bficon-web
So I’m a little late to this party, but I thought I’d add my two cents about the “Is Breast Truly Best?” study that recently came out in the journal Social Science and Medicine. This study, which analyzed the effects of breastfeeding using 25 years of data from the National Longitudinal Survey of Youth, got a ton of media coverage. You might have seen the Daily Mail headline, for example: “Breast milk is no better for a baby than bottled milk.”

Naturally, in light of the horrendous media coverage, researchers and other folks interested in breastfeeding felt they needed to put their interpretations of the study out there. People used blogs to point out the problems with the media coverage and to help readers better understand how the study was performed and what it actually showed. If you are interested, you can take a look at the posts about this study on Mammals Suck, Biomarkers and MilkResearch The Headlines, Behind the HeadlinesEvolutionary Parenting. There has been a lot of great coverage already, but I think all of this attention has also highlighted some frequent concerns about population health research. I got in touch with Cynthia Colen, the study’s lead author and another Robert Wood Johnson Health & Society Scholar alum to help set the record straight about some of the aspects of the study that have proven more controversial. And, of course, I’d like to provide my take on what this study adds to the field.

The Study: Design and Rationale

I’ll start with just a little background on epidemiological studies of breastfeeding. If you have done much research or reading in this area, then you know population-scale studies of the benefits of breastfeeding are TOUGH. Breastfeeding is so tied to other factors with a strong effect on health and wellbeing outcomes, like socioeconomic status and race, that even when we try to control for those things we usually can’t completely remove their influence upon the data. For example, in the US, roughly 75% of white mothers breastfeed whereas only 60% of black mothers breastfeed. And even in places like Sweden, where there is strong support for new families, women of higher socioeconomic status are more likely to breastfeed. Therefore, when a study shows, for example, that breastfed infants have higher IQs, we can’t be entirely confident that this result isn’t just due to residual confounding associated with the general boost in wellbeing that comes along with belonging to a more affluent family.

In population health studies, one way that researchers try to get around this confounding problem is by performing within-family comparisons. In a normal epidemiological study, we try our hardest to adjust for things like socioeconomic status, hoping that we can make fair comparisons between families–but knowing that we can never completely account for the effects of these factors. By performing within-family comparisons, we are relieved of having to make these tough adjustments. Because we are comparing kids that belong to the same family, if we find that breastfeeding, for example, is associated with some health benefit, we can be more confident that this is not just the result of residual confounding. After all, these kids share exactly the same home environment. In the population health researcher’s bag of tricks, this is an important one!

Given the problems so many existing studies on breastfeeding and health/development studies have encountered, Colen decided to find a dataset she could use to perform this type of within-family comparison. This is what led her to the National Longitudinal Survey of Youth study. This dataset has a number of strengths. First, it’s relatively large. It contains information on 11,504 children and 4,932 mothers. Second, the information on breastfeeding was collected prospectively. That is, the mothers were asked whether they breastfed while their kids were babies, rather than being asked years later (when the outcomes were assessed). This is important, as memory really fades as time goes by!  Third, there was information on 11 different health, behavior, and academic outcomes: (1) BMI; (2) obesity; (3) asthma; (4) hyperactivity; (5) parental attachment; (6) behavioral compliance; (7) reading comprehension; (8) vocabulary recognition; (9) math ability; (10) memory-based intelligence; and (11) scholastic competence. Not bad!

So armed with this data, Colen et al did a two-tiered analysis. After excluding multiple births (which can complicate an analysis like this one), they analyzed the full panel, 8,237 children from 4,071 families, using standard multiple regression models. They controlled for all of the things you would expect (age, race, mom’s marital status, region of the country, maternal education, family income, maternal employment status, insurance status, birth order, preterm birth, maternal smoking during pregnancy, maternal alcohol consumption during pregnancy, prenatal care initiation during first trimester). Their results indicated that breastfed children 4 to 14 did better on every outcome measured except for asthma. So in this part of the study, the benefits of breastfeeding appeared to be impressively wide-ranging. But remember: the potential for confounding was still present in this part of the study.

Next, they moved on to the heart of the study, in which they compared outcomes within families. In 665 of the families studied, siblings had different breastfeeding experiences. That is, one sibling was breastfed while another was not. When the authors investigated the 1,773 children from these families, the results changed. Big time. All of those outcome measures that looked better in breastfed kids in the first part of the study? They lost statistical significance. One interpretation of these results–the authors’ interpretation–is that when we are better able to control for factors like socioeconomic status and race, many of the apparent benefits of breastfeeding in the US disappear. They may not exist at all, or they may be too subtle to detect. If this information is correct, it is important to provide to mothers, as breastfeeding is often a very difficult process, especially for working women. If the benefits aren’t as large as advertised, some mothers may decide not to do it.

Some Potential Limitations

We already talked about the insidious role that residual confounding plays in population health studies. There is another big problem that we face when we study breastfeeding in large populations, though. And that problem is measuring breastfeeding. Getting good measures of breastfeeding for the huge number of kids in a population-based study like this one is next to impossible. These big studies are our primary source of large-scale breastfeeding data, but most of the time whether or not a woman is breastfeeding is just one box of many to be ticked off in a long survey. Breastfeeding is complicated, and these studies don’t do nuance. They’re just not designed that way. What if a mother only breastfeeds a baby in the hospital after delivery? Should we count that baby as breastfed? Is it fair to expect that the benefits associated with breastfeeding for a couple weeks are similar to those associated with breastfeeding for six months or more? Also, many babies are fed a mixture of breastmilk and formula. Typically, surveys don’t ask about this. So measuring the “exposure” being studied, breastfeeding, is really problematic. In this study, breastfeeding was coded as either “yes” (for any length of time) or “no.” However, information on the duration of breastfeeding was also collected (that is, how old the baby was when breastfeeding ceased entirely).

As EA Quinn has pointed out, one key feature of this study that is not described in the article is what “breastfeeding” means in the families in which one child was breastfed and another was not. What are we comparing? It seems as though it would be a little unusual for a woman to be super gung-ho about breastfeeding one kid while not breastfeeding another at all, barring health problems in mother or baby. For that reason, it’s natural to worry that maybe in this particular group, we’re comparing kids who were not breastfed to kids who were breastfed for only a very short time (maybe only days or weeks). If the kids in the intra-family comparison were only breastfed for a very short period relative to the kids in the larger sample, we would expect any beneficial effects of breastfeeding to be much attenuated, which could explain the results. In other words, it’s not that the results from the entire sample were wrong–it’s that the statistical power to detect those beneficial effects in the within-family comparison disappeared. For that reason, I asked Colen about the mean duration of breastfeeding in the within-family comparison. She acknowledged that the duration of breastfeeding WAS significantly shorter than in the full sample (in which the mean duration of breastfeeding was 23 weeks). However, when we talked she wasn’t sure what, exactly, that duration was. If I get that information, I’ll be happy to update the post for any interested readers.

Another important limitation is that these results are primarily relevant to mothers in the US. I think the authors did a fine job of making it clear that these results were relevant to US families. The media, not so much. It’s important to remember that the breastfeeding environment in the US is pretty unique. On the plus side, we have access to clean water, which makes formula-feeding a much safer alternative than it is in many low-income countries. On the minus side, pro-breastfeeding policies in the US are pretty terrible. Our family leave policies are awful, making it very hard for many women who want to breastfeed to do so. Workplace daycares are few and far between. There may be no space/time for women to pump at work. Therefore, this study was geared towards addressing an issue that is particularly problematic for mothers in the US: when breastfeeding is so damn hard, and may even endanger a woman’s livelihood, just how many resources should be devoted to it?

Why Keep Doing These Population Health Studies of Breastfeeding if They Are So Tough to Interpret?

Over at Mammals Suck, understandably frustrated guest poster Melanie Martin asked “What if we stopped looking for small statistical effects of breastfeeding on IQ, and put more of our money and effort into researching the unique and variable aspects of human milk composition and synthesis? What if we conducted clinical studies to determine if different ratios of breast milk intake, or different durations of breastfeeding, result in observable, biologically meaningful differences in metabolic, immune, and neurological function?”

I’m sympathetic. Trying to decipher what’s going on in the tangle of epidemiological studies of breastfeeding in high-income countries is an exercise in frustration. I agree with Martin that we need more information on the proximate effects of breastfeeding. However, I also think it’s incredibly important to keep studying distal outcomes like the ones examined in this study–even if it’s exceedingly difficult. Why? Because the path leading from proximate to distal health outcomes is a winding, confusing one. We need data for every step along the way. In public health, often what we expect to happen, based on what we know about proximal effects, does not actually happen. Consider, for example, the literature on vitamins and cancer. There are all sorts of reasons to suppose that taking vitamins and minerals could help prevent cancer. But big randomized controlled trials have shown us that they actually appear to raise cancer rates in at-risk groups. Now just to be clear, I’m not comparing breastfeeding to cancer! And I’m not predicting that someday, as a result of better studies, we will be shocked to find that breastfeeding is actually bad for kids. Not at all. But given the difficult choices that mothers in this country do have to make, I think it IS important to figure out how the effects of breastfeeding on metabolic, immune, and neurological function translate into actual health outcomes. I think this will be especially important if we want to use breastfeeding research to influence policy.

I truly believe we are making strides. For example, we now have results from randomized controlled trials of breastfeeding. In a trial in Belarus, breastfed babies were less likely to experience gastrointestinal illnesses and atopic eczema, but no reduction in respiratory tract infections was found. That’s valuable information! And George Davey Smith and colleagues have taken another creative approach to the problem of confounding in breastfeeding studies:  they investigated the effects of breastfeeding in two samples in which the structure of confounding differed. One sample was from the UK, where breastfeeding is associated with higher socioeconomic status. The other sample was from Brazil, where there is no association between family income and breastfeeding. After examining effects on blood pressure, body mass index, and IQ, they found that only the beneficial effect of breastfeeding on IQ was present in both samples. This, too, is valuable information! I think we have to be cognizant that no one study is going to give us all of the answers. But studies like these do provide the puzzle pieces we can begin to fit together to learn more about the benefits of breastfeeding.

My Take On The “Is Breast Truly Best?” Study

This study, like any population-based study, certainly has limitations. Based on the available data, it does seem possible that the lack of beneficial effects observed in the intra-family comparison may simply be due to a lack of statistical power. But I think the approach taken by Colen et al., in which siblings were compared to avoid confounding, is a good one. If we can find enough families in which one sibling is breastfed for significant periods of time and another is not, barring extraordinary circumstances like health problems, we can gather some interesting information. I’d be really interested in hearing what the mean duration of breastfeeding was in that intra-family comparison group.

The benefits of breastfeeding as measured in older children may be subtle. And the presence/absence and magnitude of different benefits is bound to be information that struggling new mothers will find valuable, so I believe these findings are worth following up on. As Colen et. al state, “Total commitment to 6 months of exclusive breastfeeding is a very high expectation of mothers, especially in an era when a majority of women work outside the home, often in jobs with little flexibility and limited maternity leave, and in a country that offers few family policies to support newborns or their mothers. The line between providing information about the benefits of breastfeeding and stigmatizing mothers facing structured, valid, and often difficult trade-offs in the care and financial support of their children or in fulfilling their own human potential must be drawn sensitively.”

So let’s push for more breastfeeding research. Let’s let funders know that instead of gathering vague breastfeeding information as an afterthought in big studies, it’s important to gather quality, detailed data. Yes, it will be expensive. But it’s important. This is a subject worth studying, and any good cohort will take years to follow. Let’s get started now!