One of the observers showed a significant decrease of the confidence level with BSI (from 92 to 90; p=0.01). Two observers significantly increased their confidence level with BSI (from 79 to 84; p<0.001 and from 74 to 76; p<0.001). The remaining three observers showed no significant difference in overall confidence with and without BSI (p=0.6; p=0.2; p=0.6).
Change of confidence was dependent on a change of disease categorization. We found an increase in confidence in those cases in which the observers did not change the reported disease category. A decrease in confidence, however, was observed in cases, in which observers reported a different disease category with the bone suppressed images, regardless, of whether the change was correct or not with respect to the disease reference (Table 3).
Discussion
In this article we showed that bone suppressed images improve detection of focal lesions without negatively affecting the detection of diffuse chest abnormalities and cardiogenic congestion. None of the observers decreased in number of correctly detected diffuse chest abnormalities and cardiogenic congestion with the use of Table 3 Observer confidence.
Number of cases without change in disease category (change in confidence) Number of cases with change in disease category (change in confidence) From false to correct disease category From correct to false disease category From false to false disease category
Observer 3 218 (+0.9) 43 (-6.6) 14 8 21 Observer 4 211 (+4.9) 50 (+3.3) 14 18 18 Observer 5 237 (+0.5) 24 (-29) 7 7 10 Observer 6 218 (+0.4) 43 (-7.8) 9 9 25 Observer 7 244 (+2.3) 17 (+1.2) 3 6 8 Observer 8 255 (+0.1) 6 (-13) 2 0 4
Changes in disease category reported on the score forms by the observers. The number in the parenthesis represent the average change in confidence level (on a scale from 0 – 100).
bone suppressed images. At the same time, we did find a significant increase in detection of focal opacities, especially for the smaller lesions. This beneficial effect of BSI, however, was counterbalanced by the fact that BSI invoked seven false positive results in normal patients, all caused by false observation of focal lesions (Fig. 5). Most interestingly BSI did not result in an overcall of diffuse lung disease in normal patients.
For the use of BSI in clinical practice, it is important to know for which type of pathology the software has beneficial effects, and for which type of disease BSI has
detrimental effects. Previous studies discussed the detection of focal lesions1-3, but
thus far none investigated the effect of BSI on a set of cases with mixed abnormalities, including diffuse lung disease. It is known that only a small portion of the chest
radiographs in clinical practice show incidental solitary lung nodules6, or actionable
lung nodules in screening studies7, 8. Thus, since in a clinical setting a large amount
of chest radiographs will not contain lung nodules, it is important to estimate the effect of BSI on the evaluation of other common chest abnormalities as well as on the evaluation of normal cases.
We were concerned whether increased visualization of the lung architecture would result in an overcall of diffuse lung disease. The results of our observer study revealed that BSI had no detrimental effect on the evaluation of diffuse parenchymal changes: We found considerable variability in the number of reported cases with interstitial lung disease, but these numbers were not affected by BSI.
Figure 5 Example of a normal radiograph of 62 year old female that invoked false positive calls of focal lesions with BSI. Left: original radiograph. Right: bone suppressed image.
We observed a substantial overcall of small focal lesions in normal radiographs (11 on average). With bone suppressed images this number further increased to 18. Overcalling of focal lesions in normal cases will cause unnecessary follow up. One reason for the increased number of reported focal lesions in normal cases could be suboptimal processing resulting in artifacts that mimic focal lesions. Another reason could be the increased visibility of vascular crossings. Bone suppressed images are not developed to replace the chest radiographs, and therefore should always be used as adjunct to the original radiograph; abnormalities seen on BSI have to be verified on the original radiograph to avoid false positive calls. It is very likely that more experience with the software will at least partially overcome this problem: None of the observers had used BSI beyond study conditions. Furthermore, it has to be noted that the case set was enriched with radiographs containing focal lesions, and therefore readers may have reported focal abnormalities more aggressively than they would do in clinical practice.
Categorization of different types of lung disease is somewhat artificial and radiologists are not used to categorize the multiple findings especially seen in radiographs of elderly patients (average age 56) into distinct, non-overlapping categories. In addition, the reader behavior is influenced by study conditions to an unknown extent. The latter probably also contributed to the overcalling of focal lesions in normal radiographs. There was a considerable variability between readers with respect to the use of disease categories. Therefore we focused our analysis on individual changes introduced by BSI and not on absolute performance measures.
For this study, we tried to select a set of clinically representative cases. Although all cases were derived from clinical archives, the case set was not consecutive but enriched with abnormalities; also other abnormalities like bone lesions and pleural lesions were not included in this study. Especially the first aspect has to be considered when transferring the results into a clinical situation. To capture the full range of pathology to be encountered in clinical radiographs at realistic incidences, a very large study group would have been necessary to allow for a meaningful data analysis. We therefore decided for a case selection and disease categorization.
In summary, in this study we showed that bone suppressed images do not negatively influence the interpretation of chest radiographs by radiologists regarding the evaluation of diffuse lung disease and signs of cardiogenic congestion. Radiologists are aided by BSI in cases with focal abnormalities, however, the availability of bone suppressed images causes overcalling of focal opacities in normal cases especially if experience with BSI is still limited.
References
1. Schalekamp S, van Ginneken B, Meiss L, et al. Bone suppressed images improve radiologists’ detection performance for pulmonary nodules in chest radiographs. Eur J Radiol 2013;82:2399–2405.
2. Freedman MT, Lo SCB, Seibel JC, et al. Lung nodules: improved detection with software that suppresses the rib and clavicle on chest radiographs. Radiology 2011;260:265–273.
3. Li F, Engelmann R, Pesce LL, et al. Small lung cancers: improved detection by use of bone suppression imaging–comparison with dual-energy subtraction chest radiography. Radiology 2011;261:937–949. 4. Kiguchi M, Kitagawa Y, Fujioka C, et al. evaluation of improvement of the detection of interstitial lung
diseases by using the dual-energy subtraction radiography method of a flat-panel detector system.
Nihon Hoshasen Gijutsu Gakkai Zasshi 2007;63:1362–1369.
5. MacMahon H, Li F, Engelmann R, et al. Dual energy subtraction and temporal subtraction chest radiography. J Thorac Imaging 2008;23:77–85.
6. Holin SM, Dwork RE, Glaser S, et al. Solitary pulmonary nodules found in a community-wide chest roentgeno graphic survey; a five-year follow-up study 1959;79:427–439.
7. Mazzone PJ, Obuchowski N, Phillips M, et al. Lung cancer screening with computer aided detection chest radiography: design and results of a randomized, controlled trial. PLoS One 2013;8:e59650. 8. Oken MM, Marcus PM, Hu P, et al. Baseline chest radiograph for lung cancer detection in the randomized