Harmonized Outcome Measures for Use in Depression Patient Registries and Clinical Practice

Major depressive disorder (MDD) is a common mental health condition characterized by heterogeneous presentations in mood, cognitive function, and physical function that persist for 2 weeks or longer. Many questions about depression treatment and outcomes exist. Evidence is lacking about the comparative effectiveness of treatment approaches, how to select the most appropriate initial course of treatment for an individual patient, and when to modify treatment approaches or discontinue pharmacotherapy for patients whose symptoms do not improve. Why depression recurs in some patients and others achieve sustained remission is unclear, and consensus does not exist on how to define or treat treatment-resistant depression (1) or how treatment-resistant depression may be related to antidepressant treatment (2).

Longitudinal observational studies, such as patient registries, already capture a wealth of data on depression treatment patterns and outcomes. Registries are increasing in number (3), and linkage with electronic health records and other sources would provide researchers with the foundation for a harmonized research infrastructure to consistently and efficiently collect high-quality data. Yet, variation in the outcome measures captured in registries and routine clinical practice makes it challenging, if not impossible, to link and compare the data.

To address these issues, the U.S. Department of Health and Human Services, led by the Agency for Healthcare Research and Quality and in collaboration with the Food and Drug Administration and National Library of Medicine, has supported the development of the Outcome Measures Framework, a conceptual model for classifying outcomes that are relevant to patients and providers across most conditions (4). This project had the following 4 objectives: to test the utility of the Outcome Measures Framework for categorizing depression outcomes and supporting harmonization across treatment pathways and care settings, to identify a minimum set of outcome measures that could be captured in depression patient registries and clinical practice, to agree on harmonized definitions for each outcome in the minimum measure set, and to map the harmonized definitions to standardized terminologies to support consistent implementation and collection of the outcome measures within electronic health records.

Methods

This harmonization effort focused on outcome measures that are currently collected in depression registries, other observational studies, and clinical practice. The Appendix describes the harmonization methodology.

Results

Twenty-seven registries and quality improvement efforts were invited to participate. Representatives of 13 of these efforts, representing multiple purposes, patient populations, and care settings, agreed to participate in the registry workgroup (Table 1). Appendix Table 1 describes registries and other efforts that declined to participate. The 15 stakeholders from 12 participating organizations represented patient advocacy organizations, health information technology, professional societies, payers, and federal agencies (Appendix Table 2).

Table 1. Registry Workgroup Participants

Appendix Table 1. Invited Registries That Declined to Participate

Appendix Table 2. Stakeholder Participants

The workgroup identified 27 outcome measures from registries and other sources and categorized them according to the Outcome Measures Framework. Eleven (41%) were categorized as clinical response; 10 (37%) were categorized as patient-reported outcome measures; and the remaining 6 were divided evenly into survival (7%), events of interest (7%), and resource use (7%). No measures of patient experience of care were identified as currently being captured in the participating registries. Patient-reported outcome measures that are used primarily to measure depression severity and response to treatment are included in the clinical response category, whereas measures that capture other domains (such as quality of life) are included in the patient-reported category.

The minimum measure set comprises 10 measures that are intended to apply to all patients with a diagnosis of MDD or its equivalent at any time (5). Table 2 lists measure definitions, and the following sections describe the rationale for selection of the measures and definitions. The workgroup emphasized the importance of documenting enrollment procedures, including criteria for inclusion, exclusion, and MDD diagnosis. The measures could apply to patients with comorbid psychiatric conditions, but information on comorbid conditions and other patient characteristics must be collected to support risk adjustment and to identify factors that influence outcomes in subpopulations (Figure). In addition, although this workgroup focused on MDD, the measures may be relevant for all clinically significant depression; further work is needed to explore their relevance for other types of depression. Last, registries should report clearly on efforts to minimize loss to follow-up and the proportions of patients who are lost to follow-up at the specified intervals.

Table 2. Depression Minimum Measure Set and Harmonized Definitions

• Download figure
• Download PowerPoint

Survival

Two survival measures, all-cause mortality and death from suicide, are included in the minimum measure set. Some research has suggested that depression is associated with higher mortality rates (6), and data from large, diverse patient registries could be helpful to describe any such association. However, information on cause of death may not be readily available, and a different cause of death may be listed for suicides because of perceived stigma. Without systematic ascertainment, survival measures may be inaccurate. The workgroup recognized these limitations and acknowledged that these measures may be difficult to collect in some settings. However, all-cause mortality and death from suicide are highly relevant from both the patient and the clinician perspective, and a minimum measure set would be incomplete without these measures. The workgroup recommended that they be captured when systematic ascertainment is possible, and registries should provide clear guidance about appropriate methods of ascertainment.

Clinical Response

The clinical response measures are grouped into 2 categories: improvement in depressive symptoms and worsening of depressive symptoms. Patients whose symptoms neither improve nor worsen are considered unchanged. The workgroup noted the importance of capturing both remission and response in alignment with widely used quality measures (7, 8). However, it broadened the definitions to allow a wider time frame for measurement and substitution of other instruments.

The clinical response measures are defined on the basis of the Patient Health Questionnaire–9 (PHQ-9). Participating registries use various instruments, including the Hamilton Depression Rating Scale, PHQ-9 (9), and Montgomery–Åsberg Depression Rating Scale (10), that differ in mode of administration, domains covered, length, and time to administer. The use of different instruments among the registries reflects the broader variation seen in research and clinical practice. Clinical trials use such instruments as the Hamilton Depression Rating Scale and Montgomery–Åsberg Depression Rating Scale as primary end points, but these instruments require specialized training and clinician time to administer, making them impractical in many routine clinical care settings.

Because the proposed measure set is intended for wide use in the settings of primary care and mental health care, the workgroup focused on recommending an instrument that is short, easy to score, patient-reported, and publicly available. The PHQ-9 meets these criteria and is an acceptable instrument for the remission and response quality measures. The U.S. Preventive Services Task Force recommendation also discusses the PHQ as an instrument for screening. Recognizing that researchers may wish to use other tools in some settings, the workgroup noted that instruments for which a crosswalk to the PHQ-9 exists are acceptable for these measures. Registries may also include other instruments as supplemental measures.

Measurement at regular intervals is important to capture changes in depressive symptoms. As a patient-administered instrument, the PHQ-9 can be captured directly from patients at regular intervals without a provider encounter.

Although remission and response are widely captured in registries, analogous measures for worsening of depressive symptoms were difficult to identify. The workgroup identified recurrence as an important outcome to collect consistently and defined the concept on the basis of the PHQ-9. This definition requires further validation. In addition, consistent measurement of worsening of depressive symptoms would provide information that helps us understand outcomes in patients who do not achieve remission and for whom treatment efficacy wanes over time. This concept is considered supplemental because it requires additional refinement.

Events of Interest

Suicidal ideation and behavior and adverse events are captured in the minimum measure set. To facilitate consistent measurement across patient populations and care settings, the workgroup recommended a stepped approach to measuring suicidal ideation, in which all patients complete the PHQ-9 and those who score on the suicidal ideation item (question 9) receive additional screening and possibly intervention. Additional screening should be completed using an appropriate, brief, validated instrument, such as the Concise Health Risk Tracking Scale (11). Suicidal behaviors are also important to capture and should be included in this measure when systematic ascertainment from all possible sites of care is possible. Because of the practical challenges of systematic ascertainment, this measure may be infeasible in some settings.

Many questions exist about adverse events related to long-term medication use and discontinuation of medication therapy (12). Clinicians and patient representatives commented that patients may have adverse events (side effects) that they do not recognize as treatment-related or are reluctant to discuss with providers (such as sexual function). However, a single measure to capture all possible side effects across all treatment approaches does not exist. The workgroup recommended that registries consider using the Frequency, Intensity, and Burden of Side Effects Rating Scale, a brief, validated, patient-reported measure designed to capture the side effects of depression medications.

Patient-Reported Outcome Measures

Measurement in depression often focuses on core symptoms, but other factors, such as work and social engagement and quality of life, are important to patients. Participating registries capture a range of patient-reported outcome instruments depending on their patient population and objectives. The workgroup discussed these and other instruments but did not identify an instrument that is patient-centered, widely used in clinical practice, relevant across a range of populations and care settings, validated, and publicly available. The Quality of Life Enjoyment and Satisfaction Questionnaire (13) is provided as an example of a quality-of-life instrument that meets most of the criteria, although it is not commonly used outside research settings.

Resource Use

Depression-related resource use captures payer and patient costs related to treatment or management of depression. The workgroup cautioned that access to care plays an important role in resource use for many patients. Many factors may limit a patient's access to mental health care, including lack of health insurance and shortage of mental health professionals, particularly in rural areas. A patient's ability to access appropriate care, as well as the number and type of visits, should be considered when calculating or interpreting this measure. In addition to direct costs, absenteeism from work and reduced productivity are relevant when considering the overall economic burden of depression. The workgroup recommended the Work Productivity and Activity Impairment Questionnaire for this purpose (14) but noted that few examples exist of the use of this measure in depression registries.

Treatments and Characteristics

The workgroup identified depression-specific characteristics of the participant, disease, and provider for which published evidence shows a correlation with patient outcomes (Figure). It also identified treatments of interest. Collection of the characteristics and detailed treatment data are critical for risk adjustment when measuring depression outcomes and comparing outcomes across care settings and patient populations. Although we do not define a specific risk adjustment approach here, registries and other systems that use the measures must evaluate differences in patient populations and consider the effect on outcome measures.

Standardized Library

The narrative definitions produced by the workgroup were translated into standardized definitions, including data elements, value sets, and the accompanying logic necessary to consistently capture and extract the data from electronic health records (15). Standard codes and value sets for the Quality of Life Enjoyment and Satisfaction Questionnaire and the Work Productivity and Activity Impairment Questionnaire do not exist.

Discussion

We identified patient- and clinician-relevant measures that are feasible to capture in routine clinical practice across care settings and are sufficiently robust to support research and quality improvement efforts related to depression. Of note, this minimum measure set is intended to create the foundation for a learning health care system by bridging the gap between outcomes that are considered relevant to clinicians and patients and those that are relevant in research, so that information from research studies could be tied directly to decisions in clinical practice and measurement of meaningful outcomes in quality initiatives. These measures are also designed to provide a consistent framework for sharing meaningful information across providers to facilitate care coordination for patients with depression.

A major strength of this effort was participation by a wide range of stakeholders who provided different perspectives on the outcomes that are most important to measure and potentially feasible to capture across registries and care settings. A second strength is the translation of the narrative definitions into standardized terminologies. Standardization is intended to reduce duplicate data collection by harmonizing data requirements across the learning health care system.

The minimum measure set is similar to the recommendations for measuring health outcomes in depression and anxiety developed by the International Consortium for Health Outcomes Measurement (ICHOM). Although the objectives of this project differ from those of ICHOM, the proposed case-mix variables, outcomes, and measurement time frames align in most areas. Of note, both proposals recommend the PHQ-9 to measure symptoms and include measures of medication side effects and absenteeism. Significant differences exist in 3 areas. First, the ICHOM proposal includes a measure of functioning, whereas this workgroup prioritized depression-related quality of life. Second, this workgroup included a measure of suicidal ideation and behavior, noting the increasing rate of suicide in the United States. Last, the minimum measure set includes all-cause mortality, which may be challenging to assess but is necessary to provide an understanding of the possible association between depression and mortality. These concepts are not included in the ICHOM proposal.

The minimum measure set has limitations. First, 4 measures rely on the PHQ-9, but workgroup members noted that other instruments may be more appropriate for some purposes, such as efficacy trials. Given the complexity of the instruments used in efficacy trials, such instruments are unlikely to become widely used in routine practice. To bridge this gap, development of new tools to allow comparisons across the PHQ-9 and other instruments is critical to allow researchers to select the most appropriate instrument for their objectives while maintaining the ability to link data from routine clinical practice with research findings. Work is ongoing to develop such crosswalks or common metrics; of note, the depression metric used in the Patient-Reported Outcomes Measurement Information System is a common reporting metric that can be used to link 3 depression scales, including the PHQ-9 (16).

Second, the workgroup did not reach consensus on any patient-reported outcome domains other than depression-specific quality of life. The workgroup believed that the most appropriate domain depended on the patient population of interest. Even within a specific population, individual patients have different treatment goals, and a patient-reported outcome should ideally align with those goals. The workgroup also noted that more information is needed to understand the acceptability from the patient perspective of completing the 3 patient-reported instruments included in the minimum measure set on a regular basis. Clinicians and patient representatives cautioned that some patients may find it difficult to answer these questions repeatedly, especially if their symptoms are not improving.

Third, additional work is needed to operationalize some of the measures. More work is necessary to validate the definition of recurrence, develop a clinician- and patient-relevant definition for worsening symptoms of depression, and facilitate use of an instrument for measuring treatment side effects. Questions remain about how some patient characteristics influence outcomes and how to risk-adjust the measures, and further work is needed to develop a framework for capturing the necessary data to describe and compare psychotherapy or other behavioral health treatments.

Fourth, the scope of this project was limited to registries and other efforts collecting data in the United States. Engagement with registries and other efforts that declined to participate would have provided more perspective on the feasibility of adopting the measures in industry-funded research and the value of the measures for collaborative care.

Last, demonstration of the feasibility of capturing the minimum measure set and the value and validity of these measures for clinical care and research is essential. The Agency for Healthcare Research and Quality recently awarded a pilot project to implement the measures within a health system and 2 patient registries, with the goal of assessing feasibility, burden, and value. That project should provide important information to guide the further adoption of the minimum measure set.

The harmonized measures represent a minimum set of outcomes that are relevant in depression research and multiple clinical practice settings. Consistent collection of these measures in registries and other data collection efforts would create opportunities to link and compare data across sources, potentially enabling new research and assisting in the development of learning health care systems.