Formulation of the review question

The first stage consists of formulation of the review question and a review title. The review title should be short yet descriptive enough to reflect research question. It should be possible to clearly define the research question as well as the title in terms of PICO ⁸ principle, i.e., there should be a clearly defined population (P) upon whom the intervention (I) is to be tested along with comparable intervention (C) with a measurable outcome (O).

The research question should be relevant to the researcher as well as the patient either directly or indirectly. E.g., if a parent wants to know whether toothpaste X is better than toothpaste Y for his child′s teeth, the research question becomes "is toothpaste X (I) more effective when compared with toothpaste Y (C) in prevention of dental caries (O) in children (P)?"

The PICO approach holds true for all the questions that are quantitative in nature. In case of qualitative research, as there is no intervention or comparison, our question will consist of only the "P" and "O."

Development of the search protocol

It is essential that the search protocol should be finalized at the very beginning of undertaking a systematic review. The review team should be constituted at this stage comprising of a minimum of two researchers. This enables one to minimize bias and error at all stages. Also, help should be enlisted from other health-care professionals, researchers, reference librarians, clinical and methodological experts etc. ⁹ The search strategy, inclusion and exclusion criteria, data extraction, quality assessment of selected studies, data analysis and plans for dissemination of the review should be addressed while developing the protocol.

Literature search

A systematic search for research is one of the major differences between a traditional literature review and a systematic review. For a systematic review both the published and unpublished literature is carefully searched for the required studies. The aim is to identify as many studies on the topic of interest as is reasonably possible. To aid in this, a comprehensive search strategy is developed and documented in the review protocol prior to commencement. ¹⁰ This should include the number of researchers who will screen titles and abstracts and then full papers and method for resolving disagreements about study′s eligibility. Studies are obtained by using one or more of the following methods.

Searching electronic data bases

Initially a designated number of databases such as MEDLINE, EMBASE and COCHRANE are searched using standardized or customized search filters. ¹¹

Scanning reference lists of relevant studies

The reference list of all retrieved papers is searched to identify any additional studies of interest missed during database searches. ¹¹

Hand searching key journals

This involves scanning the content of journals, conference proceedings and abstracts, page by page. It is an important way of identifying very recent publications that have not yet been included and indexed by electronic databases and also of including articles from journals that are not indexed by electronic databases. ¹²

Citation searching

This involves selecting a number of key papers already identified for inclusion in the review and then searching for articles that have cited these papers. This approach helps to identify a cluster of related and therefore highly relevant, papers. ¹¹

Searching for unpublished literature

This helps minimize publication bias, which is often seen to result from the tendency that research showing a positive outcome is more likely to be accepted and published in journals than research that fails to demonstrate any benefit. Using only published studies may therefore overestimate the effect of the intervention. ⁴ It is difficult to get access to this kind of literature because of lack of public record of their existence. ¹⁰

Searches of specialized search engines, data bases or websites such as those listing conference proceedings, institutional or technical reports and higher degree dissertations or other documents not normally subject to editorial control or peer review may help uncover some of this literature.

Contacting experts and manufacturers

Research groups and other experts as well as manufacturers may be useful sources of research not identified by the electronic searches and may also be able to supply information about unpublished or ongoing research. ¹¹

Searching non-English language literature

Limiting searches to English language papers can introduce language bias in the systematic review. ¹³,¹⁴,¹⁵,¹⁶ Databases, such as MEDLINE, EMBASE, do include a small number of non-English language journals that can be assessed. ¹⁷ Furthermore, using additional databases such as Latin American and Caribbean Health Sciences Literature that contain collections of non-English language research can minimize potential language bias.

Whenever feasible, all relevant studies should be included regardless of language. ¹⁰ However, realistically this is not always possible due to a lack of time, resources and facilities for translation. It is advisable therefore, to identify all non-English language papers, document their existence, but record "language" as the reason for exclusion in cases where they cannot be dealt with. ¹¹

Assessment of studies

Literature searching results in a large number of potentially eligible studies that need to be assessed for inclusion against predetermined criteria. The aim of selection is to ensure that all relevant studies are included in the review. The process for selecting studies should be explicit and conducted in such a way as to minimize the risk of errors and bias. ¹¹,¹⁸,¹⁹,²⁰,²¹,²² It should be documented clearly to ensure reproducibility.

Process for study selection

The selection of studies from electronic databases is usually conducted in two stages:

An initial screening of titles and abstracts against the inclusion criteria to identify potentially relevant papers.

Some searching methods provide access to full papers directly, e.g., hand searching journals and contact with research groups; in such cases, assessment for inclusion is one stage process.

Study quality assessment

After a full text selection stage, the studies are assessed for methodological rigor. Whatever the study design(s) included, it should not be assumed that all studies of the same basic design are equally well-conducted. It is essential to evaluate the methodological quality of a potentially eligible study before it is included as including studies of poor methodology might distort its conclusion. ¹⁰

Parallel independent assessments should be conducted to minimize the risk of errors. If disagreements occur between assessors, they should be resolved according to a predefined strategy using consensus and arbitration as appropriate. ²³ The study selection process should be documented, detailing reasons for exclusion of studies that are "near-misses."

Reporting study selection

Showing the number of studies/papers remaining at each stage by a flow chart is a simple and useful way of documenting the study selection process e.g., PRISMA flow chart ²⁴ Figure 2. It is essentially an update of QUORUM or quality of reporting of meta-analyses (1966). The PRISMA statement consists of a four phase flow diagram Figure 2 and a 27 item checklist Figure 3. It helps authors to improve the reporting of systematic reviews and meta-analyses, minimizes the chances of repetition of reviews addressing the same question and ensures greater transparency in updating a review. It is also useful for critical appraisal of a systematic review which has already been published. ²⁴ The PRISMA statement, checklist and flowchart as well as more information on the same are available freely from its website www.prisma-statement.org.Figure 2

Preferred Reporting Items for Systematic Reviews and Meta-analyses flowchart