Open Parachute

Image credit: MSoF – Ooops, [Newsweek] Did It Again

Publication of the Cochrane fluoridation review last year caused quite a flurry. Anti-fluoridation activists launched an avalanche of cherry-picked quotations and claims that completely misrepresented the review’s findings. On the other hand, the Cochrane review was poorly written and laid itself open to this cherry picking and misrepresentation.

Now, an authoritative group of authors has published a new critique which summarises the faults of the Cochrane review and shows how it has come to be misrepresented. The paper is:

Rugg-Gunn, A. J., Spencer, A. J., Whelton, H. P., Jones, C., Beal, J. F., Castle, P., … Zusman, S. P. (2016). Critique of the review of ‘Water fluoridation for the prevention of dental caries’ published by the Cochrane Collaboration in 2015. Bdj, 220(7), 335–340.

The on-line version is full text so readers can check it out for themselves.

The authors say:

“The Cochrane Review’s conclusion that ‘there is very little contemporary evidence. that has evaluated the effectiveness of water fluoridation for the prevention of caries’ is self-fulfiling due to its omission of contemporary studies designed for surveillance of public health programmes.

So they say:

“This, it could be argued, led to what is termed ‘an empty review’.”

And this is because a key problem of the review was “the restrictive inclusion criteria  used to judge adequacy of study design and risk of bias.”

Nature of water fluoridation programmes

A problem with the Cochrane review is that its judgement criteria were more suitable for drug trials than for evaluation of a social health policy. Drug evaluation relies on randomised controlled trials (RCTs), but:

“With public health interventions things are different. There will only sometimes be RCTs demonstrating efficacy and effectiveness. There are frequently no such trials because the highly complex practical, ethical and financial factors involved mean that RCTs are not feasible. Consequently, when determining whether a public health intervention is cost effective, evidence has to be drawn from a wide variety of other scientific methods and research designs including cross-sectional ones and process evaluations. . . . .

“In many cases it is simply impossible to make recommendations for public health interventions and policy if reliance is only placed on RCTs. Further, with public health interventions, the issue is not about individual patient benefit but whether the population as a whole will benefit.”

There are also many confounding effects with fluoridation:

“variations in use of other sources of fluoride, mainly from toothpaste, and diet, particularly sugar consumption. In many societies, these are closely linked to SES, and evaluation should also measure and control for these explanatory factors and interactions.”

Therefore:

“Water fluoridation should be evaluated using contemporary methods which are appropriate for evaluating public health interventions with such complexities, and systematic reviews should take this into account.”

The Cochrane review failed to do this and, although these problems were aired in the review’s discussion, the abstract and main conclusions were, therefore, misleading and open to misrepresentation.

Criteria for study inclusion

This “purist” approach was also demonstrated in the review’s criteria for study selection. The Cochrane authors considered:

“only prospective studies with a concurrent control, comparing at least two populations, one receiving fluoridated water and the other non-fluoridated water, with at least two points in time evaluated. Groups had to be comparable in terms of fluoridated water at baseline.”

But baseline data can be irrelevant when considering long-term fluoridation schemes. Rugg-Gunn et al., (2016) point out:

“Over time, in many countries, coverage of the population with water fluoridation schemes was almost complete, at least to the limits of public health requirements and technical feasibility. In such jurisdictions, the priority for health authorities was to monitor the continued effectiveness of existing schemes. Most recent evaluations of water fluoridation have been of this type, using the most appropriate design, which is a single cross-sectional survey of fluoridated and non-fluoridated groups with control for confounding factors.” One of the critical problems with the 2015 Cochrane Review is that these data have been excluded from the Review.

This, although recent evaluations have usually been of higher quality than older ones because of the availability of computer processing, use of more sophisticated statistical analysis and greater awareness of the need to taker confounding factors into account.

The authors discuss the impracticability of requiring baseline data for such long-term schemes. The Cochrane reviewers themselves stipulated baseline data should be taken within 3 years of implementing water fluoridation:

“an acknowledgement that the communities may, mainly through population change, lose comparability after three years. While this assumption of similarity may be reasonable over a short period, it becomes less tenable as the period between baseline and final examinations increases.”

This requirement looks silly when considering recent data from long-term schemes or considering possible benefits for 50-year-olds because:

“baseline information on the caries experience of people of this age would be required in the community to be fluoridated and in a comparable reference community, as well as information to be collected 50 years later on the caries experience of people from the same age group in the same communities which have continued to remain fluoridated or non-fluoridated for the whole of that very long period. Such requirements are unfeasibly stringent given the potential for community demographic characteristics to change over time, and render 50-year historical comparability of intervention and reference communities meaningless for present-day comparisons.” [My emphasis]

Rugg-Gunn et al., (2016) discuss in-depth other aspects of the criteria used, exclusions if modern research by the Cochrane authors and study design.

Dental fluorosis

The Cochrane review’s inclusion of dental fluorosis seems to be “tacked on” and does not use the criteria outlined for their review and selection of caries studies. In particular, it ignored the influence of different sources of dietary fluoride – leaving the impression that the resulting data related to water fluoridation when it didn’t:

“In the Cochrane Review, the effect of water fluoridation on the prevalence of fluorosis should have been isolated from the confounding effect of other fluorides. The Cochrane Review’s analysis of fluorosis studies is silent on the possible contribution of other fluorides, such as fluoridated toothpaste, which risks leaving readers with the impression that all dental fluorosis arises from fluoride in water supplies. Research since 2000 has indicated that a greater proportion of dental fluorosis risk is due to the use (and therefore swallowing) of fluoride-containing toothpastes than to optimally fluoridated water.

Consequently, an intelligent interpretation of the fluorosis data in the Cochrane review requires some extra calculation – of the sort the reviewers themselves did for the caries studies:

“This comparison between intervention and reference communities was the method used for evaluating caries prevention in the Cochrane Review but, for an unexplained reason, not for the evaluation of dental fluorosis. For communities with lower fluoride concentrations (such as 0.5 mgF/L), their fluorosis levels should be compared with those in the corresponding reference (non-fluoridated) community.”

And

“The highly restrictive approach taken by the Cochrane Review in examining the effect of community water fluoridation on dental caries seems to have been abandoned for dental fluorosis. The reason for this difference is unclear.”

I attempted this “extra calculation” in my article “Cochrane fluoridation review. III: Misleading section on dental fluorosis.”

When differences between “fluoridated” and “non-fluoridated” areas are considered there was no significant contribution of water fluoridation to the “dental fluorosis of aesthetic concern.” However, because  this comparison was not made in the Cochrane review anti-fluoride campaigners are claiming that fluoridation causes a prevalence of 12% “dental fluorosis of aesthetic concern!”

That is very misleading.

Anti-fluoride campaigners love to quote prevalence figures for all forms of dental fluorosis, not just the more serious – implying that even the mildest forms should concern us. Consequently, they cite the Cochrane review to claim a 40% prevalence of dental fluorosis This is for all forms from the most severe to the mildest. But, in fact, when the differences between “fluoridated” and “unfluoridated” areas are calculated the prevalence of all forms of dental fluorosis attributable to community water fluoridation is only 7%. And, remember, these will be only the mildest forms.

Again, very misleading.

Conclusions

The Cochrane fluoridation review agrees with all other authoritative reviews when it states:

“that water fluoridation is effective at reducing caries levels in both deciduous [primary] and permanent dentition in children.”

But its conclusions conflict with the literature on:

“the effectiveness of water fluoridation in respect of: its effectiveness in adults; its effectiveness in reducing social disparities in oral health; and the effect of cessation of water fluoridation. On these, the Cochrane Review said that there was insufficient evidence; it did not say that water fluoridation was ineffective in these regards.”

On these, the review would only say that “there is insufficient evidence.” But that is a self-fulfilling conclusion given their restrictive selection criteria. Some observers may actually say such a conclusion is irresponsible because:

“It is a fundamental premise of interpreting evidence from trials that the absence of evidence, or the existence of poor-quality evidence, should not be confused with, or taken to imply, an absence of effect. There is a risk that the Cochrane Review will be inadvertently, or deliberately, misinterpreted in this way.”

And we know this is, in fact, what has happened. Motivated anti-fluoride campaigners have chosen to present an absence of evidence (because of the restricted selection of studies) as “proof” that fluoridation is not effective.

It just shows how an “empty review” can be used to make the most unwarranted claims.

Similar articles on the Cochrane fluoridation review