Evidence and gap maps (EGMs)
What are EGMs?
EGMs are systematic and visual presentations of the availability of rigorous evidence for a particular policy domain. EGMs consolidate what we know and do not know about "what works" by mapping out existing and ongoing systematic reviews and impact evaluations in this field; and by providing a graphical display of areas with strong, weak or non-existent evidence on the effect of interventions or initiatives. A typical map is a matrix of intervention categories (rows) and outcome domain (columns). There may be additional filters for study design, location and population sub-group.
EGMs show what evidence there is and not what evidence says.
Evidence and gap maps must:
- Have a pre-specified protocol
- Have a systematic search strategy
- Have clear inclusion and exclusion criteria which are systematically applied
- Systematically report all eligible studies
Evidence may be global or for a particular region(s). It may cover different types of evidence (but most examples are of effectiveness studies), and may include primary studies and systematic reviews.
Why make evidence and gap maps?
- Guide users to available high quality evidence to inform strategy and programme development
- Tell users where there is no high quality evidence
- Identify gaps to be filled by evidence synthesis and new studies for researchers and research commissioners – and so more strategic, policy-oriented approach to research agenda
Comparison of EGMs and systematic reviews
|
|
Systematic review |
Evidence and gap map |
Comparison |
| Question setting | Often restricted to a single intervention, and a limited range of outcomes. A PICOS is specified to guide study inclusion criteria. | Broad scope of interventions across a sector or sub-sector, with full range of outcomes across causal chain. A PICOS is specified to guide study inclusion criteria. | EGMs are broader in scope than systematic reviews. |
| Search strategy | A comprehensive and systematic search for primary studies meeting the inclusion criteria (and not exclusion criteria) | A comprehensive and systematic search for systematic reviews and primary studies meeting the inclusion criteria (and not exclusion criteria) | No difference in approach. EGMs search for systematic reviews as well as primary studies. |
| Screening | Identified studies screened against inclusion and exclusion criteria | Identified studies screened against inclusion and exclusion criteria | No difference in approach |
| Coding and data extraction | Coding of study and intervention characteristics, moderators and data extraction of effect sized and related statistics | Coding of a limited number of study and intervention characteristics | EGMs require coding of less data than systematic reviews |
| Critical appraisal | Assessment of quality of included studies using a critical appraisal instrument | Critical appraisal may not be done, but is recommended | Critical appraisal is optional for EGMs |
| Evidence synthesis | Statistical or narrative synthesis of the evidence | Not done | EGMs do not synthesize the evidence |
| Reporting | Systematic reporting of evidence | Graphical representation of map availability of evidence. Descriptive overview of map. | Systematic reviews summarize what the evidence says. EGMs only summarize what evidence is available. |
| Use | To inform policy and practice | To inform research priorities and research funding | Systematic reviews are to inform policy, and EGMS primarily to inform research priorities. |
Method
There are many types of evidence maps. They don't all look the same.
Making an EGM
The map is presented in two dimensions: the rows list interventions and sub-categories, and the columns lists the outcome domains. Each cell shows studies which contain evidence on that combination of intervention and outcome. Included studies are coded for additional characteristics which can be used in filters, such as country, region and population categories.
The evidence is identified by a search following a pre-specified, published search protocol. The map is accompanied by a descriptive report to summarize the evidence for stakeholders such as researchers, research commissioners, policy makers, and practitioners.
EGMs are useful for policymakers and practitioners looking for evidence to inform policies and programs. For donors and researchers, these maps can inform a strategic approach for commissioning and conducting research.
How are Campbell EGMs created?
The essential stage in an EGM is to get the framework right which in turn depends on the scope of the map. Existing typologies of interventions and outcomes are especially useful if they are widely accepted in the relevant community. This involves formation of formal advisory board consisting of experts in the area, policy makers, and international and national research organizations.
The advisory board should be consulted to advice on sources to develop the framework, databases and organizations to include in searches, identification of key stakeholders and review of reports.
EGMs include both completed and on-going systematic reviews and impact evaluations. The online platform is populated with studies identified using a comprehensive and systematic search strategy documented in a pre-specified protocol. It should follow a clear inclusion and exclusion criteria that are systematically applied and all the included and excluded studies are systematically reported.