Comparison of Application of the ACC/AHA Guidelines, Adult Treatment Panel III Guidelines, and European Society of Cardiology Guidelines for Cardiovascular Disease Prevention in a European Cohort

Importance  The 2013 American College of Cardiology/American Heart Association (ACC/AHA) guidelines introduced a prediction model and lowered the threshold for treatment with statins to a 7.5% 10-year hard atherosclerotic cardiovascular disease (ASCVD) risk. Implications of the new guideline’s threshold and model have not been addressed in non-US populations or compared with previous guidelines.

Objective  To determine population-wide implications of the ACC/AHA, the Adult Treatment Panel III (ATP-III), and the European Society of Cardiology (ESC) guidelines using a cohort of Dutch individuals aged 55 years or older.

Design, Setting, and Participants  We included 4854 Rotterdam Study participants recruited in 1997-2001. We calculated 10-year risks for “hard” ASCVD events (including fatal and nonfatal coronary heart disease [CHD] and stroke) (ACC/AHA), hard CHD events (fatal and nonfatal myocardial infarction, CHD mortality) (ATP-III), and atherosclerotic CVD mortality (ESC).

Main Outcomes and Measures  Events were assessed until January 1, 2012. Per guideline, we calculated proportions of individuals for whom statins would be recommended and determined calibration and discrimination of risk models.

Results  The mean age was 65.5 (SD, 5.2) years. Statins would be recommended for 96.4% (95% CI, 95.4%-97.1%; n = 1825) of men and 65.8% (95% CI, 63.8%-67.7%; n = 1523) of women by the ACC/AHA, 52.0% (95% CI, 49.8%-54.3%; n = 985) of men and 35.5% (95% CI, 33.5%-37.5%; n = 821) of women by the ATP-III, and 66.1% (95% CI, 64.0%-68.3%; n = 1253) of men and 39.1% (95% CI, 37.1%-41.2%; n = 906) of women by ESC guidelines. With the ACC/AHA model, average predicted risk vs observed cumulative incidence of hard ASCVD events was 21.5% (95% CI, 20.9%-22.1%) vs 12.7% (95% CI, 11.1%-14.5%) for men (192 events) and 11.6% (95% CI, 11.2%-12.0%) vs 7.9% (95% CI, 6.7%-9.2%) for women (151 events). Similar overestimation occurred with the ATP-III model (98 events in men and 62 events in women) and ESC model (50 events in men and 37 events in women). The C statistic was 0.67 (95% CI, 0.63-0.71) in men and 0.68 (95% CI, 0.64-0.73) in women for hard ASCVD (ACC/AHA), 0.67 (95% CI, 0.62-0.72) in men and 0.69 (95% CI, 0.63-0.75) in women for hard CHD (ATP-III), and 0.76 (95% CI, 0.70-0.82) in men and 0.77 (95% CI, 0.71-0.83) in women for CVD mortality (ESC).

Conclusions and Relevance  In this European population aged 55 years or older, proportions of individuals eligible for statins differed substantially among the guidelines. The ACC/AHA guideline would recommend statins for nearly all men and two-thirds of women, proportions exceeding those with the ATP-III or ESC guidelines. All 3 risk models provided poor calibration and moderate to good discrimination. Improving risk predictions and setting appropriate population-wide thresholds are necessary to facilitate better clinical decision making.

Prevention of cardiovascular disease (CVD), the leading cause of death worldwide,^1,2 remains feasible³ yet suboptimal. The common approach in CVD primary prevention is to identify individuals at high enough risk for cardiovascular events to justify targeting them for more intensive lifestyle interventions, pharmacological interventions, or both.

The CVD prevention guidelines developed by the National Cholesterol Education Program expert panel,⁴ succeeded by the American College of Cardiology/American Heart Association (ACC/AHA) task force⁵, and the European Society of Cardiology (ESC)⁶ are the major guidelines influencing clinical practice. While the Adult Treatment Panel III (ATP-III) guidelines were based on the 10-year risk of coronary heart disease (CHD) only,⁴ the ACC/AHA guidelines broaden to comprise risk of all hard atherosclerotic CVD (ASCVD), including CHD and stroke,⁵ using the Pooled Cohort equations.⁷ An additional substantial change in the US guideline is a lower risk threshold for statin treatment in asymptomatic individuals from 20% CHD risk in the ATP-III guidelines⁴ to 7.5% ASCVD risk in the new guidelines.⁵ The potential implications of the ACC/AHA guidelines in largely widening the populations endorsed for treatment and the accuracy of the ACC/AHA risk calculator have received much attention.^8-12

To be clinically useful, risk prediction models should provide good discrimination. Because decisions for statin treatment are based on an individual’s absolute risk, calibration of the risk prediction models as well as the risk threshold for treatment are important. Quiz Ref IDVarying approaches to CVD risk estimation and application of different criteria for therapeutic recommendations would translate into substantial differences in proportions of individuals qualifying for treatment at a population level. We therefore aimed to determine implications of the ACC/AHA, the ATP-III, and the ESC guidelines in a prospective cohort of Dutch individuals aged 55 years or older. Our first aim was to determine what proportion of the population would be treated based on each guideline. We then sought to examine discrimination and calibration of the 3 risk prediction models underlying these guidelines.

Analyses were performed within the framework of the Rotterdam Study, a prospective population-based cohort study among persons aged 55 years or older in the Ommoord district of Rotterdam, the Netherlands. The rationale and design of the Rotterdam Study have been described elsewhere.¹³ The baseline examination took place in 1990-1993 (RS-I). In 2000, the cohort was extended to include inhabitants who reached the age of 55 years in 1990-2000 and persons aged 55 years or older who migrated into the research area (RS-II). The Rotterdam Study was approved by the Medical Ethics Committee of the Erasmus Medical Center and all participants provided written informed consent.

The present study used data from the third examination of the original cohort (RS-I, recruited 1997-1999) and the first examination of the extended cohort (RS-II, recruited 2000-2001). Among the participants aged 75 years or younger, there were 2209 men and 2645 women with measurements required for the analyses. Among these individuals, 315 men and 330 women were receiving statin treatment at baseline and therefore were excluded from the population for whom the eligibility for treatment based on each guideline was assessed. For further analyses on examining the performance of each risk scoring model, exclusions were made using the criteria from each guideline.

Main outcomes were hard ASCVD, composed of fatal and nonfatal myocardial infarction (MI), other CHD mortality, and stroke; hard CHD, composed of fatal and nonfatal MI and CHD mortality; and atherosclerotic CVD mortality.^14,15 Prevalent CVD was defined as a history of MI, coronary or other arterial revascularization, stroke or focal transient ischemic attack, or heart failure. Events were assessed until January 1, 2012.

A complete description of the methods for measurement of cardiovascular risk factors, definitions of the outcomes, and details regarding the follow-up time is provided in the eAppendix in the Supplement.

We calculated the 10-year risk of hard ASCVD events for each individual based on age, systolic blood pressure, treatment of hypertension, total and high-density lipoprotein (HDL) cholesterol levels, current smoking, and history of diabetes mellitus, using the sex-specific parameters from the ACC/AHA Pooled Cohort equations.⁷ We used the recommended 5% and 7.5% risk thresholds for categorization of the 2 respective categories of “treatment considered” and “treatment recommended.”⁵ To comply with the ACC/AHA guideline,⁵ the risk estimation for hard ASCVD was calculated among individuals who were not receiving lipid-lowering medication, were free of CVD at baseline, and had low-density lipoprotein (LDL) cholesterol levels below 190 mg/dL.

Using the continuous ATP-III risk prediction model based on age, systolic blood pressure, treatment of hypertension, total and HDL cholesterol levels, and current smoking,¹⁶ we also calculated the 10-year risk of hard CHD for the individuals who were not receiving lipid-lowering medication and were free of CVD and diabetes mellitus, to comply with the ATP-III guideline.⁴ The risk thresholds used for categorization were 10% and 20%, corresponding to the cutoff points for defining the intermediate- and high-risk categories by the ATP-III guideline.⁴

The 10-year risk of CVD mortality for each participant was based on age, systolic blood pressure, total cholesterol levels, and current smoking using the sex-specific intercepts and regression coefficients from the SCORE equation for low-risk European countries.¹⁷ We used the recommended 1%, 5%, and 10% risk thresholds, corresponding to the cutoff points for defining the moderate-risk, high-risk, and very-high-risk groups, respectively, based on the ESC guideline.^6,18 To comply with the ESC guideline, the SCORE risk estimation was performed among the individuals who were not receiving lipid-lowering medication at baseline and were free of CVD, diabetes mellitus, and chronic kidney disease (CKD).⁶Figure 1 describes the inclusion and exclusion criteria for different risk prediction models.

Based on each guideline, we formed 3 categories of treatment: “treatment recommended,” “treatment considered,” and “no treatment.” eTables 1 through 3 in the Supplement describe the criteria used to form these 3 treatment categories by each guideline.

Quiz Ref IDWe assessed the discrimination and calibration of each risk prediction model in our population. Discrimination refers to ability of the model to assign a higher risk to individuals who develop the outcome of interest compared with those who remain free of disease. The discriminative performance of each risk-scoring model was assessed using the C statistic. Calibration is the agreement between the predicted probabilities of disease, based on the risk prediction model, and the actual incidence of events in the population. To assess the calibration of each risk prediction model, the average predicted 10-year risks for each risk function were compared with the average 10-year observed risks (ie, cumulative incidence of the event). Calibration plots were generated to assess the agreement between the predicted and observed risks over the entire range.

Baseline characteristics of the participants are presented in Table 1. The mean age of the participants was 65.5 (SD, 5.2) years and 54.5% were women.

Based on the ACC/AHA guideline,⁵ the “treatment recommended” group included 96.4% (95% CI, 95.4%-97.1%; n = 1825) of men and 65.8% (95% CI, 63.8%-67.7%; n = 1523) of women while the “treatment considered” group included 3.3% (95% CI, 2.6%-4.2%; n = 63) of men and 14.2% (95% CI, 12.8%-15.7%; n = 330) of women. Only 0.3% of men (95% CI, 0.1%-0.7%; n = 6) and 20.0% (95% CI, 18.3%-21.6%; n = 462) of women were categorized in the “no treatment” group (Table 2 and eTable 1 in the Supplement).

Using the ATP-III guideline,⁴ 52.0% (95% CI, 49.8%-54.3%; n = 985) of men and 35.5% (95% CI, 33.5%-37.5%; n = 821) of women were categorized in the “treatment recommended” group, while the “treatment considered” group included 14.2% (95% CI, 12.6%-15.8%; n = 269) of men and 14.1% (95% CI, 12.7%-15.6%; n = 326) of women. The “no treatment” category included the remaining 33.8% (95% CI, 31.7%-35.9%; n = 640) of men and 50.4% (95% CI, 48.4%-52.5%; n = 1168) of women (Table 2 and eTable 2 in the Supplement).

Based on the ESC guideline,⁶ 66.1% (95% CI, 64.0%-68.3%; n = 1253) of men and 39.1% (95% CI, 37.1%-41.2%; n = 906) of women were included in the “treatment recommended” category. The “treatment considered” group comprised 31.6% (95% CI, 29.5%-33.7%; n = 598) of men and 51.4% (95% CI, 49.3%-53.4%; n = 1189) of women. Only 2.3% (95% CI, 1.6%-2.9%; n = 43) of men and 9.5% (95% CI, 8.3%-10.8%; n = 220) of women were assigned to the “no treatment” category (Table 2 and eTable 3 in the Supplement).

eFigure 1 in the Supplement presents the treatment recommendations based on the 3 guidelines for the populations younger than 65 years and aged 65 years or older. The data suggest that almost all men older than 55 years and nearly all women older than 65 years are recommended for statin treatment based on the new ACC/AHA guideline.

eTables 1 through 3 in the Supplement show that while all men and women with prevalent CVD were categorized in the “treatment recommended” group by the ACC/AHA guideline (eTable 1 in the Supplement), 12.9% of men and 4.2% of women with clinical CHD and CHD risk equivalents were categorized in the “treatment considered” or “no treatment” category based on the ATP-III guideline (eTable 2 in the Supplement). Using the ESC guideline, a small group of individuals with clinical CVD and its risk equivalents (0.6% of men and 0.4% of women) were categorized in the “treatment considered” group (eTable 3 in the Supplement).

eTables 4 through 6 in the Supplement provide the description of the proportion of the population to whom each risk estimation model was applied. Among 1513 men and 1920 women included for ASCVD risk prediction (ACC/AHA), 192 men and 151 women developed hard ASCVD over 10-year follow-up. Among 1431 men and 1976 women included for CHD risk prediction (ATP-III), hard CHD occurred in 98 men and 62 women over 10-year follow-up. Among 1366 men and 1816 women included for CVD mortality risk prediction (ESC), 50 men and 37 women died of atherosclerotic CVD over 10-year follow-up. For all outcomes studied, follow-up time was truncated at 10 years for individuals with a longer follow-up time than 10 years.

After calculating the 10-year risk for individuals based on each risk prediction model, we first assessed the discriminative ability of each model. The C statistic for the ACC/AHA model was 0.67 (95% CI, 0.63-0.71) for men and 0.68 (95% CI, 0.64-0.73) for women for hard ASCVD. Use of the ATP-III risk prediction model resulted in a C statistic of 0.67 (95% CI, 0.62-0.72) for men and 0.69 (95% CI, 0.63-0.75) for women for hard CHD. Using the SCORE equation (ESC), the C statistic was 0.76 (95% CI, 0.70-0.82) for men and 0.77 (95% CI, 0.71-0.83) for women for CVD mortality.

We then assessed the calibration of each risk prediction model. Figure 2 compares the average 10-year risks predicted by the ACC/AHA, ATP-III, or SCORE (ESC) risk prediction models with the observed 10-year risks (ie, cumulative incidence of events) in each risk category. Calibration was poor for all 3 models; the ACC/AHA (Figure 2A), the ATP-III (Figure 2B), and the SCORE equation (Figure 2C) overestimated the 10-year risk among men and women across all risk categories. eTable 7 in the Supplement details the percentage of population at different categories of risk using each risk prediction model. The average predicted risks vs observed cumulative incidence of hard ASCVD events were 21.5% (95% CI, 20.9%-22.1%) vs 12.7% (95% CI, 11.1%-14.5%) for men and 11.6% (95% CI, 11.2%-12.0%) vs 7.9% (95% CI, 6.7%-9.2%) for women using the ACC/AHA risk model. The average predicted vs observed cumulative incidences of hard CHD events were 16.1% (95% CI, 15.8%-16.5%) vs 6.8% (95% CI, 5.6%-8.3%) for men and 5.4% (95% CI, 5.2%-5.5%) vs 3.1% (95% CI, 2.4%-4.0%) for women based on the ATP-III. Using the SCORE equation, the average predicted vs observed cumulative incidences of CVD mortality were 6.8% (95% CI, 6.5%-7.1%) vs 3.7% (95% CI, 2.7%-4.8%) for men and 3.8% (95% CI, 3.7%-4.0%) vs 2.0% (95% CI, 1.4%-2.8%) for women. Calibration plots for the ACC/AHA, the ATP-III, and the ESC risk prediction models are presented in eFigures 2 through 4 in the Supplement.

Quiz Ref IDIn this European population-based prospective cohort study of healthy men and women without previous CVD (ie, primary prevention population) aged 55 years or older, we found that nearly all men and more than 65% of women were recommended for drug treatment based on the recent ACC/AHA guideline.⁵

Regarding secondary prevention of CVD, the ACC/AHA guidelines clearly recommend drug treatment for all persons with clinical CVD and its risk equivalents.⁵ Based on the ATP-III and ESC guidelines, however, it is possible that some individuals with clinical CVD are categorized into 2 groups of “treatment considered” or “no treatment” based on their LDL cholesterol levels.^4,6

For primary CVD prevention, based on the evidence from clinical trials of statin drugs,¹⁹ the new ACC/AHA guidelines modified clinical decision making and proposed to recommend statin treatment solely based on a 10-year ASCVD risk greater than 7.5%.⁵ This departure from previous guidelines in the United States and from the current ESC guideline represents a fairly straightforward approach that deviates from risk functions of 10-year hard CHD or CVD mortality combined with blood concentrations of LDL cholesterol.^4,6

The new ACA/AHA guideline recommendations resulted in a larger “treatment recommended” group in our population⁵ in contrast to the larger “treatment considered” group based on the ESC guidelines.⁶ This raises questions about the use of a risk assessment calculator for treatment decisions when so large a proportion of the older population is among the “treatment recommended” group. A decade ago, Wald and Law²⁰ described a radical strategy to prevent CVD by prescribing a daily polypill to everyone aged 55 years or older without requiring risk factors to be measured. Our results suggest that by inclusion of stroke as an outcome and applying the lowered evidence-based risk threshold of 7.5% for treatment,^19,21 the new ACC/AHA guidelines have approached this “age-based” strategy. In our population, almost all men older than 55 years and almost all women older than 65 years qualified for statin treatment based on the ACC/AHA guidelines.⁵

The clinical usefulness of a risk prediction tool is determined by a combination of its discrimination and calibration. In our study, the C statistic for the 3 risk prediction models ranged between 0.67 and 0.77, indicating moderate to good discrimination, with the SCORE equation providing the highest C statistic among the 3 models. Theoretically, if a model has perfect discrimination (ie, the C statistic exceeds 0.98), the cutoff threshold for treatment can be set at any level. However, the modest discrimination ability of the risk prediction models in our study indicates that there is a substantial overlap in the risk distributions of the individuals with and without the events. Therefore, given the current performance of the ACC/AHA risk prediction model, the place of the cutoff threshold for treatment is essential.

When an individual’s absolute risk prediction is used for clinical decision making regarding initiation of treatment, accurate calibration is very important. As also evident from our analyses, concerns regarding model calibration are pertinent to all 3 of the risk prediction models; to the Framingham risk score that formed the basis for the ATP-III,^22-24 to the SCORE equation,²⁵ and recently to the new ACC/AHA risk calculator.¹² Miscalibration of the risk prediction models, once applied in other populations rather than derivation sets, is expected.²⁶Quiz Ref IDImperfect calibration could partly be explained by differences in the characteristics of the new populations, ie, different levels of baseline risk, for which the risk prediction model is applied. Furthermore, if the application cohorts are more contemporary to the cohorts used in the derivation sets, temporal improvements in overall health could partly be responsible for poor calibration. The risk prediction models underlying all 3 guidelines overestimated the risk among men and women in our study. About 17% of men and 16% of women included in the ASCVD risk assessment in our study were eventually prescribed statins over the course of follow-up. Based on the premise that healthy lifestyle and therapeutic measures would reduce the CVD burden, statin prescription together with improvement of high blood pressure treatment, aspirin use, higher smoking quit rates, and other lifestyle modifications over the follow-up period might have contributed to the observed overestimations to some extent.

Related closely to the calibration issue is the threshold for making clinical decisions. The new ACC/AHA guidelines substantially lowered the cutoff for treatment to an evidence-based threshold of 7.5%.^19,21 If the new ACC/AHA risk prediction model led to overestimation among individuals at high levels of actual CVD risk (eg, >20% estimated 10-year risk), it would not necessarily affect the eventual proportion of people recommended for consideration of statin use. However, among individuals with lower actual CVD risks, overestimation by the risk prediction models is of much greater concern. Inaccuracy of the prediction models at the lower levels of risk could indeed result in many more individuals recommended for statins than were intended. While not explicitly stated in the new ACC/AHA guideline, setting of thresholds typically involves both an awareness of clinical benefit of the treatment in the target population combined with a judgment about cost-effectiveness. Different countries and settings may decide on very different thresholds based on cost-effectiveness or resource considerations, which is another reason to look critically at the clinical implications of the risk estimation tool and the risk threshold in other non-US settings. Quiz Ref IDBeyond the need for improving the risk predictions and setting appropriate population-wide thresholds to facilitate better clinical decision making, the large proportion of the population recommended for statin treatment based on new guidelines should be a concerning signal. These large numbers point out the need for (1) preventing risk factor aggregation and (2) conveying information to individuals in ways that effectively lower their risk, in an era when cardiovascular disease remains a worldwide public health challenge.

Strengths of the current study include availability of all risk factors needed for different risk prediction models, which were measured with standardized methods, and detailed follow-up data. However, an important limitation is that our cohort includes white individuals aged 55 years or older. Therefore, the generalizability of our findings to younger and nonwhite populations remains uncertain. Furthermore, this study had relatively small numbers of events for some outcomes.

With application of the recent ACC/AHA guidelines in a healthy European population-based cohort, nearly all men and the majority of women aged 55 years or older were candidates for drug treatment. Application of the ACC/AHA, ATP-III, and ESC risk prediction models led to overestimation of the risk. Given the modest discrimination and poor calibration of the ACC/AHA risk prediction model, the choice of treatment threshold becomes central.

Corresponding Author: Maryam Kavousi, MD, PhD, Department of Epidemiology, Erasmus University Medical Center, Rotterdam, PO Box 2040, 3000 CA Rotterdam, the Netherlands (m.kavousi@erasmusmc.nl).

Published Online: March 29, 2014. doi:10.1001/jama.2014.2632.

Author Contributions: Drs Kavousi and Leening had full access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.

Study concept and design: Kavousi, Leening, Nanchen, Greenland, Ikram, Hofman, Franco.

Acquisition, analysis, or interpretation of data: Kavousi, Leening, Nanchen, Graham, Steyerberg, Ikram, Stricker, Hofman, Franco.

Drafting of the manuscript: Kavousi, Franco.

Critical revision of the manuscript for important intellectual content: All authors.

Statistical analysis: Kavousi, Leening, Steyerberg, Stricker.

Obtained funding: Ikram, Hofman.

Administrative, technical, or material support: Leening, Graham.

Study supervision: Nanchen, Ikram, Stricker, Hofman, Franco.

Conflict of Interest Disclosures: All authors have completed and submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest. Dr Kavousi reports research grants from AXA Research fund (outside the submitted work). Dr Greenland reports consultancy for University of Pennsylvania, Medical College of Wisconsin, the National Heart, Lung, and Blood Institute, Ohio State University, and Rush University. Dr Graham reports speaker fees from Pfizer and MSD. Dr Steyerberg reports royalties from Springer. Dr Franco reports grants from Nestle and Metagenics (outside the submitted work). No other disclosures were reported.

Funding/Support: The Rotterdam Study is funded by Erasmus MC and Erasmus University, Rotterdam, the Netherlands; the Netherlands Organisation for Scientific Research (NWO); the Netherlands Organisation for Health Research and Development (ZonMw); the Research Institute for Diseases in the Elderly (RIDE); the Ministry of Education, Culture and Science; the Ministry for Health, Welfare and Sports; the European Commission (DG XII); and the Municipality of Rotterdam. Dr Kavousi is supported by the AXA Research Fund. Drs Leening and Stricker are supported by a grant from the ZonMw (HTA grant 80.82500.98.10208). Dr Franco works in ErasmusAGE, a center for aging research across the life course funded by Nestle Nutrition (Nestec Ltd), Metagenics Inc, and AXA.

Role of the Sponsors: Erasmus MC and Erasmus University, NWO, ZonMw, RIDE, the Ministry of Education, Culture and Science, the Ministry for Health, Welfare and Sports, the European Commission, the Municipality of Rotterdam; Nestlé Nutrition (Nestec Ltd), Metagenics Inc, and AXA had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; or decision to submit the manuscript for publication.

Disclaimer: Dr Greenland, JAMA senior editor, was not involved in the evaluation of or decision to publish this article.

Additional Contributions: We gratefully acknowledge the dedication, commitment, and contribution of the inhabitants, general practitioners, and pharmacists of the Ommoord district to the Rotterdam Study.