This umbrella review was conducted in alignment with PRISMA guidelines to address the following research question: “Which AI models demonstrate the highest diagnostic accuracy and adaptability for detecting OC and OPMD in resource-constrained settings?”

The criteria for eligibility were established based on the following PICOS framework:

(P) Population: Patients undergoing evaluation for OPMD and OC

Research studies adhering to the PICO framework’s “Subjects, Intervention, Control, Outcome” criteria were deemed eligible for inclusion. Systematic reviews of ex vivo cell studies, case reports/series, and clinical trials were specifically excluded.

This umbrella review encompassed systematic reviews of observational studies that analyzed the diagnostic accuracy of various neural networks in identifying OPMD and OC in digital photographs, in contrast to conventional clinical assessments.

Two independent researchers (L C. and R.S.) performed an electronic search for systematic reviews across several databases, including Medline (via PubMed), Web of Science, Scopus, EMBASE, and Google Scholar, covering publications up to October 2024. Detailed search strategies for each database are presented in Table 1. The search was restricted to English-language publications, with no further limitations imposed.

To compile relevant articles, duplicate entries were systematically identified and removed based on title, author, and publication year after consolidating results from various databases. Two independent reviewers (L.C. and R.S.) conducted an initial screening of titles and abstracts, excluding studies that did not meet the predefined PICO framework criteria through a consensus-based approach. Any disagreements regarding article selection were resolved via discussion or, if necessary, by consulting a third and fourth reviewer (M.K. and R.K.). Following this preliminary filtering, each reviewer undertook a rigorous evaluation of the full texts of the remaining articles, with any residual discrepancies resolved through further dialogue [Figure 1].

For studies that satisfied the inclusion criteria, data extraction was performed by two reviewers (L.C. and R.S.), adhering to the Joanna Briggs Institute (JBI) guidelines for umbrella reviews.^[15] The extraction focused on several parameters: (a) author and year of publication, (b) type of study, (c) research setting, (d) range of years covered, (e) number of studies reviewed, (f) dataset size, (g) sources searched, (h) Neural network models used, (i) Comparison (j) tools used for appraisal, (k) outcomes measured, (l) results and findings, (m) significance of the results, and (n) heterogeneity of the studies. These details have been organized and presented in Tables 2 and 3.

The risk of bias (ROB) for each study was independently evaluated by two reviewers (L.C and R.S) using the JBI systematic review critical appraisal tool [Figure 2].^[15] Discrepancies in their assessments were resolved through consultation with a third reviewer (M.K.) to achieve consensus. The ROB findings were visualized using the RoBvis tool.

The study selection process was carried out independently by two authors (L.C and R.S). This involved an initial screening of titles and abstracts, followed by a detailed evaluation of full texts according to set inclusion and exclusion criteria. In the end, data extraction was conducted on six articles [Figure 1]. According to the JBI methodological guidelines, the data were systematically organized and presented in Tables 2 and 3. Table 3 offers a consolidated overview of the outcome measures derived from the included studies. The research predominantly investigated the diagnostic accuracy of various AI models, focusing on metrics such as sensitivity and specificity.

The results of this Umbrella review demonstrate that the six systematic reviews and meta-analyses evaluated provide compelling evidence on the effectiveness of AI in clinical diagnostics across diverse settings and methodologies. The studies span geographical regions including Iran, the UK, India, Mexico, Thailand, and France, covering a wide range of years from 2000 to 2023. They collectively employed various AI models, traditional machine learning techniques, and robust appraisal frameworks to compare diagnostic performance against conventional methods.

Although several reviews reported pooled diagnostic metrics, a quantitative meta–meta-analysis was not performed due to substantial methodological heterogeneity, including differences in study designs, outcome definitions, AI architectures, validation strategies, and pooling methods. Overlap of primary studies and inconsistent reporting of pooled estimates further precluded reliable secondary quantitative synthesis. Accordingly, findings were narratively synthesized in line with JBI guidance, emphasizing comparative trends rather than pooled effect estimates.

Rokhshad et al.^[16] reported high AI diagnostic accuracy (74%–100%), exceeding clinician performance (61%–98%), with pooled DORs of 155 for malignant and 114 for cancerous lesions. However, substantial heterogeneity in sensitivity (I² = 85.75%) and specificity (I² = 97.40%) was observed. Methodological appraisal identified high ROB related to research question clarity, search strategy, appraisal criteria, independent review, and publication bias assessment, limiting the overall reliability despite robust inclusion criteria.

Ferro et al.^[17] compared classical and modern AI models, reporting high sensitivity (90.4%) and area under the curve (AUC) (91.5%) for classical methods, while modern neural networks achieved slightly higher AUC (93.2%) and lower false positive rates (13.9% vs. 15.1%). However, methodological limitations were noted in research question clarity, search strategy, appraisal criteria, independent review, and publication bias assessment, along with moderate heterogeneity in sensitivity (I² = 62%) and specificity (I² = 84%).

Kavyashree et al.^[18] Reported very high accuracy for SVM and logistic regression (100%) and near-comparable performance for convolutional neural networks (CNNs) (99.3%), while random forest (90%) and Multilayer Perceptron (MLP) (94.1%) performed less well. Methodological concerns were limited, primarily involving unclear appraisal criteria (D5), search strategy (D3), publication bias assessment (D9), and future research directives (D11), with strengths in independent review and data extraction supporting overall reliability. Similarly, Beristain-Colorado et al.^[19] demonstrated neural network accuracy exceeding 85% in remote clinical settings, with generally low ROB; limitations were confined to research question clarity (D1), appraisal criteria (D5), and one unclear domain (D9), while robust search and data extraction methods enhanced study reliability.

Warin and Suebnukarn^[20] presented strong evidence for the utility of AI in diagnostics, reporting pooled sensitivity and specificity values of 92%, with an impressive DOR of 2549.08. Precision, recall, and F1 scores were consistent with these metrics, and AUC ranged from 0.74 to 0.91, underscoring the reliability of the models tested. This study has a moderate ROB. Issues exist in research question clarity, study appraisal, and publication bias assessment. In addition, the search strategy (D3) and policy recommendations (D10) are unclear. Despite these concerns, strong methodologies in data extraction, independent review, and future research directives make it relatively stronger than highly biased studies. Finally, de Chauveron et al.^[21] identified MLSO + SVM as the top-performing model in their analysis, while VGG architectures underperformed despite substantial data inclusion. This study mirrors the bias trends of Rokhshad et al.^[16] and Ferro et al.,^[17] with high risks in research question clarity, search strategy, study appraisal, and publication bias. The lack of clarity in policy recommendations and research directives further weakens its reliability. Despite good inclusion criteria and data extraction methods, the study’s overall ROB is high. The only criterion not applicable was the evaluation of publication bias, due to the inclusion of predominantly qualitative evidence synthesis in some studies. Therefore, the overall ROB across the included reviews is considered to be minimal [Figure 2].

Overall, the results affirm the potential of AI, particularly deep learning and hybrid architectures, in enhancing diagnostic accuracy and reliability across clinical domains. However, the variability in study quality, significant heterogeneity, and frequent methodological limitations highlight the need for standardized approaches and more rigorous evaluation in future research.

Substantial heterogeneity was consistently observed across the included reviews, with several meta-analyses reporting I² values exceeding 85% for sensitivity and specificity. Although pooled estimates indicate high overall diagnostic performance, such heterogeneity limits generalizability and suggests that AI model accuracy is highly context dependent. This variability likely arises from differences in patient populations, image acquisition conditions (camera specifications, resolution, lighting, focus, and angulation), AI architectures and training strategies, and validation approaches, with many studies relying solely on internal validation. While formal subgroup or meta-regression analyses were not feasible, future stratification by model type, imaging conditions, and validation strategy may aid interpretation. Importantly, the observed heterogeneity underscores the need for standardized imaging protocols, transparent model reporting, and rigorous external validation to support reliable clinical implementation.

Some results differ from broader findings in the literature. While most studies reviewed here report high accuracy and low false positive rates for AI models, heterogeneity in sensitivity and specificity metrics was a recurring issue. For instance, Rokhshad et al.^[16] reported significant variability with I² values exceeding 85%, suggesting inconsistencies in model performance across included datasets. Other meta-analyses in the field have also noted heterogeneity but often to a lesser degree, potentially reflecting methodological differences such as dataset preprocessing or study inclusion criteria. Furthermore, while classical machine learning models like SVM and logistic regression performed exceptionally well in the review by Kavyashree et al.,^[18] these findings contrast with more recent studies where such methods are increasingly outperformed by ensemble and deep learning methods, especially on larger datasets.

The strengths of this analysis lie in its comprehensive approach, incorporating diverse geographic settings, a wide range of AI models, and robust appraisal tools such as Quality assessment of diagnostic accuracy studies-2. This breadth provides a holistic view of AI’s applicability in clinical diagnostics. Nevertheless, certain limitations must be acknowledged. Many studies exhibited unclear or high risks, as noted by Ferro et al.^[17] and Beristain-Colorado et al.^[19] These issues could bias the pooled results and limit the generalizability of findings. Furthermore, the heterogeneity in metrics such as sensitivity and specificity, particularly in Rokhshad et al.^[16] and Warin and Suebnukarn^[20] underscores the need for standardized reporting and evaluation frameworks to enhance comparability across studies.

Another significant limitation is the lack of detailed subgroup analyses or stratification by factors such as population characteristics, imaging modalities, or model architectures. For instance, while modern AI models consistently outperformed classical approaches, their performance varied significantly across different diagnostic tasks and imaging techniques. Including such stratifications could provide deeper insights into the specific conditions or scenarios where AI excels or underperforms. In addition, certain studies, such as those by Kavyashree et al.^[18] and Beristain-Colorado et al.,^[19] lacked appraisal of methodological rigor, which could affect the reliability of their findings.

Future research should address these limitations by adopting standardized methodologies and reporting guidelines to improve the consistency and reliability of results. Greater emphasis on external validation using independent datasets is also essential to assess the generalizability of AI models. Studies should also focus on exploring hybrid models that combine the strengths of classical and modern techniques, as suggested by the promising performance of hybrid architectures like MLSO + SVM reported by de Chauveron et al.^[21]

In conclusion, while the findings affirm AI’s transformative potential in clinical diagnostics, they also highlight significant gaps in methodology and reporting. Addressing these gaps will be critical in realizing the full potential of AI in delivering accurate, reliable, and scalable diagnostic solutions across diverse clinical settings.

The protocol for this study can be reviewed on the International Prospective Register of Systematic Reviews (PROSPERO) database, referenced under the registration number: CRD420250656539.

This research was conducted, reported, and disseminated without direct engagement or input from patients or the general public.

Data can be obtained through a reasonable request process.

This study is a part of an ICMR project (Project ID IIRP-2023-1049) funded by Small extramural grants – 2023.

The authors of this manuscript declare that they have no conflicts of interest, real or perceived, financial or non-financial in this article.