Abstract:
Introduction: Coronavirus disease 2019 (COVID-19) is an infectious disease caused by the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) (1). On October 9, 2021 authorities have counted 1.261.102 cases and 25.675 deaths in Belgium (2), illustrating the importance of this disease. Reverse Transcriptase Polymerase Chain Reaction (RT-PCR) is considered the gold standard for the diagnosis. The use of chest CT as a screening tool is controversial. Some studies claim that chest CT has a sensitivity of 94.6% and specificity of 46.0% (1) but other studies reported that 10.6% of proven positive patients had normal chest CTs (1). Nevertheless, in the beginning of the pandemic chest CT was widely used as a screening tool partly due to a lack of evidence based diagnostic protocols and availability of RT-PCR tests.
Purpose: To evaluate the performance of an artificial intelligence (AI) tool, for Covid-19 detection and lung injury quantification, on chest CT during a real-time clinical workflow.
Materials & methods: We retrospectively collected a consecutive dataset of Chest CTs obtained between April 1, 2020 and May 1, 2020. These exams were performed to screen for Covid-19 at hospital admission (e.g., emergency, inpatient, and outpatient). All axial images with a slice thickness of 1.5-3 mm were pseudo-anonymized and sent to the AI tool Quibim Precision platform (QUIBIM S.L) (3) to be analyzed by the Imaging COVID-19 Analyzer (This model was developed under the umbrella of a multi-center European research project during the pandemic). The AI tool then provided a probability score for a COVID-19 infection. Chest CTs were evaluated by 12 radiologists with 10-30 years of experience after intramural training on reporting COVID-19 chest CT findings, based on the RSNA expert consensus (4). RT-PCR was considered the gold standard for COVID-19 diagnosis. The concordance between CT findings and RT-PCR was determined by receiver operating characteristic (ROC) analysis and crosstabs.
Results: The study included 264 Chest CTs ( 146 men; 118 women; Mean age: 67,3) with 84 RT-PCR confirmed COVID-19 positive cases, 33 cases of non-COVID-19 related pneumonia and 147 cases without infectious lung disease. When the COVID-19 probability score cut-off value is set at 0.41 there is a sensitivity of 90.48% (95%CI: 82.09% to 95.80%), specificity of 30.00% (95%CI: 23.42% to 37.26%), PPV of 13.65% (95%CI: 12.32% to 15.11%) and NPV of 96.26 (95%CI: 92.77% to 98.10%) and an AUC of 0.75. Regarding the probabilities we suggest the following ranges with 95% sensitivity to exclude the disease and 95% specificity to include the disease: < 0.38: almost certain negative; 0.39 – 0.62: indeterminate; > 0.63: almost certain positive. 
We chose a relatively low cut-off value in order to have a high sensitivity so it could be used as a screening test. However, this reduced the specificity and diagnostic accuracy. When using the suggested probability ranges, a substantial number of cases (69%) were labeled as indeterminate. False positive cases were partly explained by mislabeling breathing artifacts, hypoventilation in dependent lung areas or linear atelectasis as ground glass opacities. Also, some clear-cut diagnoses for the radiologists (e.g., heart failure, bacterial pneumonia, interstitial lung disease, …) were often given a high probability by the AI tool.
Conclusion: The AI tool can be used as a screening tool with a sensitivity of 90% when the cut-off value is set relatively low. Due to low specificity the AI tool on its own cannot be used as a diagnostic test but has the potential to serve as an adjunct for COVID-19 detection. Training with supplementary artifact datasets should further improve the AI accuracy. 
Currently we are studying the performance of the AI tool compared to the radiologists and the accuracy of the lung injury quantification.
Clinical Relevance/Application: COVID-19 detection and AI-based quantification of lung injury is challenging but may be useful in a pandemic outbreak.

1. Chest CT in COVID-19: What the Radiologist Needs to Know. Thomas C. Kwee and Robert M. Kwee. RadioGraphics 2020 40:7, 1848-1865
2. https://www.coronaviruscovid19.be/coronavirus-cijfers accessed october 9 2021
3. https://quibim.com/about/about-quibim/
4. Radiological Society of North America Expert Consensus Document on Reporting Chest CT Findings Related to COVID-19: Endorsed by the Society of Thoracic Radiology, the American College of Radiology, and RSNA. Scott Simpson, Fernando U. Kay, Suhny Abbara, Sanjeev Bhalla, Jonathan H. Chung, Michael Chung, Travis S. Henry, Jeffrey P. Kanne, Seth Kligerman, Jane P. Ko, and Harold Litt. Radiology: Cardiothoracic Imaging 2020 2:2
Conflict of interest disclaimer:The authors certify that they have NO affiliations with or involvement in the company QUIBIM (3) with any financial interest or non-financial interest in the subject matter or materials discussed in this poster.

**Abstract:**
**Introduction:** Coronavirus disease 2019 (COVID-19) is an infectious disease caused by the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) (1). On October 9, 2021 authorities have counted 1.261.102 cases and 25.675 deaths in Belgium (2), illustrating the importance of this disease. Reverse Transcriptase Polymerase Chain Reaction (RT-PCR) is considered the gold standard for the diagnosis. The use of chest CT as a screening tool is controversial. Some studies claim that chest CT has a sensitivity of 94.6% and specificity of 46.0% (1) but other studies reported that 10.6% of proven positive patients had normal chest CTs (1). Nevertheless, in the beginning of the pandemic chest CT was widely used as a screening tool partly due to a lack of evidence based diagnostic protocols and availability of RT-PCR tests.
**Purpose:** To evaluate the performance of an artificial intelligence (AI) tool, for Covid-19 detection and lung injury quantification, on chest CT during a real-time clinical workflow.
**Materials & methods:** We retrospectively collected a consecutive dataset of Chest CTs obtained between April 1, 2020 and May 1, 2020. These exams were performed to screen for Covid-19 at hospital admission (e.g., emergency, inpatient, and outpatient). All axial images with a slice thickness of 1.5-3 mm were pseudo-anonymized and sent to the AI tool _Quibim Precision platform_ (QUIBIM S.L) (3) to be analyzed by the Imaging COVID-19 Analyzer (This model was developed under the umbrella of a multi-center European research project during the pandemic). The AI tool then provided a probability score for a COVID-19 infection. Chest CTs were evaluated by 12 radiologists with 10-30 years of experience after intramural training on reporting COVID-19 chest CT findings, based on the RSNA expert consensus (4). RT-PCR was considered the gold standard for COVID-19 diagnosis. The concordance between CT findings and RT-PCR was determined by receiver operating characteristic (ROC) analysis and crosstabs.
**Results:** The study included 264 Chest CTs ( 146 men; 118 women; Mean age: 67,3) with 84 RT-PCR confirmed COVID-19 positive cases, 33 cases of non-COVID-19 related pneumonia and 147 cases without infectious lung disease. When the COVID-19 probability score cut-off value is set at 0.41 there is a sensitivity of 90.48% (95%CI: 82.09% to 95.80%), specificity of 30.00% (95%CI: 23.42% to 37.26%), PPV of 13.65% (95%CI: 12.32% to 15.11%) and NPV of 96.26 (95%CI: 92.77% to 98.10%) and an AUC of 0.75. Regarding the probabilities we suggest the following ranges with 95% sensitivity to exclude the disease and 95% specificity to include the disease: < 0.38: almost certain negative; 0.39 – 0.62: indeterminate; > 0.63: almost certain positive. 
We chose a relatively low cut-off value in order to have a high sensitivity so it could be used as a screening test. However, this reduced the specificity and diagnostic accuracy. When using the suggested probability ranges, a substantial number of cases (69%) were labeled as indeterminate. False positive cases were partly explained by mislabeling breathing artifacts, hypoventilation in dependent lung areas or linear atelectasis as ground glass opacities. Also, some clear-cut diagnoses for the radiologists (e.g., heart failure, bacterial pneumonia, interstitial lung disease, …) were often given a high probability by the AI tool.
**Conclusion:** The AI tool can be used as a screening tool with a sensitivity of 90% when the cut-off value is set relatively low. Due to low specificity the AI tool on its own cannot be used as a diagnostic test but has the potential to serve as an adjunct for COVID-19 detection. Training with supplementary artifact datasets should further improve the AI accuracy. 
Currently we are studying the performance of the AI tool compared to the radiologists and the accuracy of the lung injury quantification.
**Clinical Relevance/Application**: COVID-19 detection and AI-based quantification of lung injury is challenging but may be useful in a pandemic outbreak.

~1. Chest CT in COVID-19: What the Radiologist Needs to Know. Thomas C. Kwee and Robert M. Kwee. RadioGraphics 2020 40:7, 1848-1865~
~2. https://www.coronaviruscovid19.be/coronavirus-cijfers accessed october 9 2021~
~3. https://quibim.com/about/about-quibim/~
~4. Radiological Society of North America Expert Consensus Document on Reporting Chest CT Findings Related to COVID-19: Endorsed by the Society of Thoracic Radiology, the American College of Radiology, and RSNA. Scott Simpson, Fernando U. Kay, Suhny Abbara, Sanjeev Bhalla, Jonathan H. Chung, Michael Chung, Travis S. Henry, Jeffrey P. Kanne, Seth Kligerman, Jane P. Ko, and Harold Litt. Radiology: Cardiothoracic Imaging 2020 2:2~
### **~Conflict of interest disclaimer:The authors certify that they have NO affiliations with or involvement in the company QUIBIM (3) with any financial interest or non-financial interest in the subject matter or materials discussed in this poster.~**

The Performance Of AI Covid-19 Detection And Lung Injury Quantification On Chest CT In A Real-time Clinical Setting

[Networking hour](https://app.wonder.me/?spaceId=fe093845-759f-448f-9f55-01d3adccd820)
[COIs](https://drive.google.com/drive/folders/1nbfwAa9lf9Kz-wAZ7iLKhfpBFjpN_gkn?usp=sharing)