Artificial intelligence (AI) refers to the use of a machine to simulate human intelligence and perform specific tasks, such as recognizing objects, making decisions, and solving problems. Machine learning is a subcategory of AI that uses algorithms to learn data patterns and predict outcomes. ¹ Deep learning, a category of machine learning models, has recently gained interest due to increasing data, computing power availability, and superior performance compared with conventional machine learning models. ² Deep learning refers to deep (multilayered) neural networks. A neural network consists of several artificial neurons and connections with mathematical operations inspired by human neurons. It can automatically learn data patterns without explicit direction when given a large amount of data. Convolutional neural networks (CNNs) are introduced for processing complex images. They use mathematical convolution functions, which allow them to detect local connectivity patterns such as edges and corners in images. ³ CNNs have been studied in maxillofacial imaging for automated diagnosing and treatment planning. They are used primarily to perform tasks such as semantic segmentation (e.g. segmenting of all teeth as tooth), ⁴ instance segmentation (e.g. segmentation of all teeth as each individual tooth), and object detection. ³ , ⁵ Object detection identifies the locations of objects in an image with rectangular bounding boxes and classifies them into defined groups. ⁶ Several CNN architectures have been proposed for this task, including region-based CNNs (R-CNNs) and You Only Look Once.

Panoramic radiographs are widely used in dental practice due to their advantages, such as low radiation dose for the patient and ease and speed of production. ⁷ They provide a two-dimensional (2D) image that typically includes all the present teeth of both jaws and their supporting structures. They provide valuable information about the patient's dental state that can be used for charting, screening, treatment planning, and forensic investigation. ⁸ , ⁹ To analyze panoramic radiographs, several steps need to be followed, starting with tooth identification. ⁶ Automated tooth identification and enumeration is the first step toward achieving a fully automatic diagnosis and treatment plan. This task can be carried out using CNN architectures. Object detection was applied for automatic tooth enumeration in periapical, ¹⁰ , ¹¹ bitewing, ¹² and panoramic radiographs. ⁶ , ⁹ , ¹³ , ¹⁴ , ¹⁵ , ¹⁶ However, there are two main challenges in training deep learning models on panoramic radiographs: (1) there are up to 32 tooth classes in a single image which makes it difficult for the model to be trained on them and (2) the image contains many structures other than teeth such as cervical vertebrae, nasal spine, maxillary sinuses, and mandibular condyles which results in too much unnecessary data for deep learning models. We propose a two-step method to overcome these issues: first, we trained the model to detect the quadrants, and then, we trained the model to detect the teeth within each quadrant. We hypothesize that this method will enhance the average precision (AP) and recall of the model compared to previous studies.

Study design

This retrospective observational study was conducted and reported according to the Checklist for AI in Medical Imaging guideline. ¹⁷ First, we trained a model for detecting quadrants automatically. Then, we trained models to detect and classify teeth from one to eight in the upper and lower quadrants.

Patient selection

One thousand and seven panoramic radiographs were obtained from various sources in Iran and Brazil, which were as follows:

The Department of Oral and Maxillofacial Radiology at Shahid Beheshti University of Medical Sciences, Tehran, Iran: Samples were taken from the Iranian population. The panoramic device was Promax Dimax 3 Digital Pan/Ceph device (Planmeca, Helsinki, Finland). The images were exported to.jpg format with the size 3252 × 1536. A total of 83 images were annotated from this resource. The device setting was 64–66 kilovoltage peak, 4–7 milliampere, and 15–18 s exposure time

All radiographs were anonymized before including in the study. Radiographs with tilted teeth, implants, retained roots, and crowns or bridges were included. Radiographs with low quality, motion artifacts, deciduous teeth, supernumerary teeth, impacted teeth, and edentulous patients were excluded from the study.

Reference data set

Three independent dentists with at least 3 years of clinical experience provide the ground truth by drawing bounding boxes. They held a calibration session and labeled the first 20 images together. The images were labeled in two separate ways: (1) R. R. and S. S. annotated teeth using LabelImg. ¹⁹ (2) R. R and Z. A. labeled the quadrants. Labeling was double-checked by H. M. R. In case of any conflict or ambiguity in tooth numbers, the sample was excluded. The teeth labeling process involved drawing a rectangular bounding box around the outer edges of each tooth and classifying it according to the two-digit FDI tooth numbering system. For quadrant labeling, in the maxillary and mandibular edentulous quadrants, the midline was taken as the reference line. For the maxilla, the coronoid processes of the mandible are the superior-posterior reference point. At the same time, in the inferior-posterior area, the upper one-third of the retromolar pad serves as the reference point. On the mandible, the superior-posterior reference point was the upper two-thirds of the retromolar pad, and the lower-posterior reference point was the line passing under the inferior mandibular canal. In quadrants with teeth, all teeth with their roots were covered. On the maxilla, the line connecting the palatal roots of the molars and the canine roots was considered the upper reference line. The inferior line was the line under the incisal edges of the central and canine. On the mandible, the superior reference line is the line above the molar's cusp tips, and the inferior reference line is the line under the lowest apex of mandibular roots.

Preprocessing

First, we applied contrast-limited adaptive histogram equalization (CLAHE). CLAHE is a histogram-based image enhancement method that limits amplification based on the clipping done in the histogram to restrict it to a predefined level. Moreover, all the images were resized to 224 × 224 before feeding the model.

Data partitions

First, 1007 panoramic images were allocated to a quadrant detection model, and the outputs of this model were provided to the tooth numbering models. Afterward, the tooth numbering models were provided with the 4028 quadrant images that had been generated. In each step (quadrant detection and tooth enumeration), 60% of the data were used for training, 20% for validation, and 20% for testing. To avoid data leakage, all four quadrants of a patient were included in the same set.

Model

Briefly, the first model, named “Quadrant detection model,” splits the image into four separate quadrants. The second model takes a single quadrant, an output of the first model, as an input and finds the tooth enumeration using object detection. We also trained a one-stage tooth enumeration model as a baseline for the comparison.

Quadrant detection model

The first object detector refines the class score of a region to be a quadrant for the object detection task and generates the final bounding box coordinates using faster RCNN using pretrained weights. The ResNet-50 was used as a base CNN for the quadrant detection model. We have tried to evaluate several approaches for quadrant detection task:

The 4-class method: Our first method divided quadrants into four classes, upper lefts, upper rights, lower lefts, and lower rights. We trained end-to-end faster RCNN

The original images were cropped with an additional margin to ensure that all related teeth are within the cropped images. In the enumeration model, these cropped images become the input. We select the best approach based on the performance of each of the three mentioned methods for the following step.

Enumerating detection model

The enumeration model performs an object detection task, where it identifies and labels individual tooth numbers based on the FDI notation. Each quadrant has four different alignments. To minimize the variance, we used two approaches. First, we flipped the right quadrants from right to left. Then, we divided the tooth enumerating models into two distinct models: one for upper quadrants and the other for lower quadrants. We trained two separate Faster RNNs for each model. Following the enumeration task, quadrants were flipped back into their original alignment.

Training

All model architecture and optimization processes were developed using Python programming language through the Detectron version 2 library. ²⁰ The training procedure was performed on an NVIDIA Tesla K80 with 12GB of GDDR5 VRAM, Intel Xeon Processor with two cores at 2.20 GHz, and 13 GB RAM. The initial learning rate for this model was chosen as 0.001 with further exponential decay. A batch size of 128 was used to train two enumerating networks. Several models with different methods were trained, and both quadrant and tooth detection models were trained for 700 iterations. The Grid search was used for hyperparameter tuning batch size, learning rate, and optimizer. Early stopping allowed the best model weights to be saved based on their performance of AP on a validation dataset, thereby reducing the possibility of overfitting.

Evaluation

Various metrics have been used in this study, such as intersection-over-union (IoU), precision, and recall. The overlap between two boundary boxes is measured by the IoU, widely used in object detection. We evaluated our model's performance on unseen data (test set). We computed IoU scores for all pairs of objects using [INSIDE:1] where A is the ground truth and B is the predicted bounding boxes. The IoU threshold t was selected to identify correctly detected objects. True positive (TP) is defined when IoU is more than t, false positive (FP) is defined when IoU is <t, and false negative (FN) is defined when ground truth is present in the image, but the model fails to detect the object.

We computed precision (P), and recall (R), using a fixed IoU threshold t as follows:

[INLINE:1]

The following formula calculates AP:

[INLINE:2]

where Recall _iis from 11 points interpolated from the precision-recall curve, which is done by segmenting the recalls evenly into 11 parts: (0.5, 0.55, 0.6,…, 0.95, 1) and Precision (Recall _i) is precision in each recall value. Average recall (AR) is formulated as follows:

[INLINE:3]

where t is IoU, which is ∈ {50,100}, and recall (t) is the recall at that point.

Dataset description

The descriptive distribution of each tooth is presented in Figure 1. Mandibular incisors and canines are the most prevalent teeth in images while maxillary third molars were the least frequently observed. Figure 1

Tooth distribution in the total image database according to FDI tooth numbering system. FDI tooth numbering system: 11–18 = Upper right 1–8, 21–28 = Upper left 1–8, 31–38 = Lower left 1–8, 41–48 = Lower right 1–8; 1. Central incisor, 2. Lateral incisor, 3. Canine, 4. First premolar, 5. Second premolar, 6. First molar, 7. Second molar, 8. Third molar.

In every dental procedure, tooth enumeration and charting are the first steps, which is instrumental in giving the clinician the most accurate treatment plan. The charting process is critical in diagnosing, managing, referrals, and treatment. As dental diseases are either directly associated with teeth or closely located to them, the initial charting serves as the foundation for all subsequent dental procedures. ⁵ Consequently, tooth identification and numbering hold significant importance as they form the basis for more intricate AI-based tasks in dental radiographic images. Using digital images to identify teeth automatically is a crucial component of intelligent health care. ⁹ , ¹⁰ , ¹⁶ , ²¹ Studies have been conducted on AI for charting purposes on cone-beam computed tomographies, bitewing radiographs, and periapical radiographs. ¹¹ , ¹² , ²² , ²³ Nevertheless, a panoramic radiograph is the most suitable technique for charting. It provides an overview of the entire dentition in a single image with minimal radiation dose. ⁷

Several studies have examined the automatic numbering of teeth in panoramic images. However, the results must be interpreted with caution. The AP and predefined threshold have not been reported in Prados-Privado et al., ¹⁴ Estai et al., ²⁴ Bilgir et al., ¹⁶ Tuzoff et al., ⁹ and Muramatsu et al. ¹³ studies. AP is a metric commonly used in object detection literature derived from precision and recall. As the data are in 2D space, a predefined threshold of IoUs of bounding boxes is used to define the model's true and false predictions. Therefore, AP is reported in various thresholds, mostly 0.5 and 0.75. Tuzoff et al. relied on expert opinion about the judgment of the model's prediction, which is subject to individual bias. ⁹ Moreover, some studies simplified tooth identification into 4-category (incisor, canine, premolar, and molar) and 3-category (incisor, canine, molar) classifications. ⁶ , ¹³

Our end-to-end single-step approach produced unsatisfactory results. Therefore, we incorporate a two-step approach, similar to Yüksel et al.'s study. ¹⁵ The difference is they used segmentation for quadrant detection and we used object detection task in both steps. Yüksel et al. reported an AP of 89.4% for the enumeration task. ¹⁵ Chung et al. used one-step point-wise localization and distant regularization, instead of the anchor-based method. ²⁵ The 32 tooth boxes were annotated regardless of whether the tooth was present in the image. The detection was achieved with a multitasking, class-agnostic identification neural network that incorporated parallel training for center offsets. They reached an AP of 91%. We obtained a 95% AP score in tooth enumeration, which is an improvement over previous studies.

A significant limitation of our study is the exclusion of radiographs from children with deciduous teeth. Since dentition is rapidly changing, we require large quantities of annotated radiographs at every stage of dental development. Future studies must be conducted in databases of panoramic radiographs in children with mixed dentition, as accurate numbering and charting are vital in mixed dentition, particularly in extraction cases. Furthermore, retained roots, supernumerary teeth, impacted teeth, and implants must be considered in future studies to enhance diversity and generalizability.

We proposed a two-step deep learning-based framework for automatic tooth enumeration on panoramic radiographs. We have obtained promising results with a high level of AP and recall. There is a need for further research on diverse datasets and real-life settings.

Financial support and sponsorship

Nil.

Conflicts of interest

The authors of this manuscript declare that they have no conflicts of interest, real or perceived, financial or non-financial in this article.