Skip to main content

Main menu

  • Home
  • Current Issue
  • Content
    • Current Issue
    • Early Access
    • Multimedia
    • Podcast
    • Collections
    • Past Issues
    • Articles by Subject
    • Articles by Type
    • Supplements
    • Plain Language Summaries
    • Calls for Papers
  • Info for
    • Authors
    • Reviewers
    • Job Seekers
    • Media
  • About
    • Annals of Family Medicine
    • Editorial Staff & Boards
    • Sponsoring Organizations
    • Copyrights & Permissions
    • Announcements
  • Engage
    • Engage
    • e-Letters (Comments)
    • Subscribe
    • Podcast
    • E-mail Alerts
    • Journal Club
    • RSS
    • Annals Forum (Archive)
  • Contact
    • Contact Us
  • Careers

User menu

  • My alerts

Search

  • Advanced search
Annals of Family Medicine
  • My alerts
Annals of Family Medicine

Advanced Search

  • Home
  • Current Issue
  • Content
    • Current Issue
    • Early Access
    • Multimedia
    • Podcast
    • Collections
    • Past Issues
    • Articles by Subject
    • Articles by Type
    • Supplements
    • Plain Language Summaries
    • Calls for Papers
  • Info for
    • Authors
    • Reviewers
    • Job Seekers
    • Media
  • About
    • Annals of Family Medicine
    • Editorial Staff & Boards
    • Sponsoring Organizations
    • Copyrights & Permissions
    • Announcements
  • Engage
    • Engage
    • e-Letters (Comments)
    • Subscribe
    • Podcast
    • E-mail Alerts
    • Journal Club
    • RSS
    • Annals Forum (Archive)
  • Contact
    • Contact Us
  • Careers
  • Follow annalsfm on Twitter
  • Visit annalsfm on Facebook
Research ArticleOriginal Research

Triaging Patients With Artificial Intelligence for Respiratory Symptoms in Primary Care to Improve Patient Outcomes: A Retrospective Diagnostic Accuracy Study

Steindór Ellertsson, Hlynur D. Hlynsson, Hrafn Loftsson and Emil L. Sigur∂sson
The Annals of Family Medicine May 2023, 21 (3) 240-248; DOI: https://doi.org/10.1370/afm.2970
Steindór Ellertsson
1Primary Health Care of the Capital Area, Iceland
MD
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Hlynur D. Hlynsson
2Department of Computer Science, Reykjavik University, Reykjavík, Iceland
PhD
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Hrafn Loftsson
2Department of Computer Science, Reykjavik University, Reykjavík, Iceland
PhD
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Emil L. Sigur∂sson
1Primary Health Care of the Capital Area, Iceland
3Development Center for Primary Health Care in Iceland, Reykjavík, Iceland
4Department of Family Medicine, University of Iceland, Reykjavík, Iceland
MD, PhD
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • For correspondence: emilsig@hi.is
  • Article
  • Figures & Data
  • eLetters
  • Info & Metrics
  • PDF
Loading

Article Figures & Data

Figures

  • Tables
  • Additional Files
  • Figure 1.
    • Download figure
    • Open in new tab
    • Download powerpoint
    Figure 1.

    Study flowchart for clinical text note selection.

    CFEM = clinical feature extraction model; CTN = clinical text note; PC = primary care.

  • Figure 2.
    • Download figure
    • Open in new tab
    • Download powerpoint
    Figure 2.
    • Download figure
    • Open in new tab
    • Download powerpoint
    Figure 2.
    • Download figure
    • Open in new tab
    • Download powerpoint
    Figure 2.
    • Download figure
    • Open in new tab
    • Download powerpoint
    Figure 2.
    • Download figure
    • Open in new tab
    • Download powerpoint
    Figure 2.
    • Download figure
    • Open in new tab
    • Download powerpoint
    Figure 2.
    • Download figure
    • Open in new tab
    • Download powerpoint
    Figure 2.

    The outcome distribution in the cross-validated data set.

    CXR = chest x-ray; CRP = C-reactive protein; ICD-10 = International Classification of Diseases, 10th Revision; J00 = common cold; J15 =bacterial pneumonia; J20 = acute bronchitis; J44 = chronic obstructive pulmonary disease; J45 = asthma.

    Notes: (A-E) bars represent 95% CIs. (F) shaded area represents 95% CIs.

  • Figure 3.
    • Download figure
    • Open in new tab
    • Download powerpoint
    Figure 3.
    • Download figure
    • Open in new tab
    • Download powerpoint
    Figure 3.
    • Download figure
    • Open in new tab
    • Download powerpoint
    Figure 3.
    • Download figure
    • Open in new tab
    • Download powerpoint
    Figure 3.
    • Download figure
    • Open in new tab
    • Download powerpoint
    Figure 3.
    • Download figure
    • Open in new tab
    • Download powerpoint
    Figure 3.
    • Download figure
    • Open in new tab
    • Download powerpoint
    Figure 3.

    The outcome distribution in test set 1.

    CXR = chest x-ray; CRP = C-reactive protein; dL= deciliter; ED = emergency department; ICD-10 = International Classification of Diseases, 10th revision; J00 = common cold; J15 = bacterial pneumonia; J20 = acute bronchitis; J44 = chronic obstructive pulmonary disease; J45 = asthma; mg = milligram; PC = primary care.

    Notes: (B) bars represent 95% CIs. (F) shaded area represents 95% CIs.

  • Figure 4.
    • Download figure
    • Open in new tab
    • Download powerpoint
    Figure 4.
    • Download figure
    • Open in new tab
    • Download powerpoint
    Figure 4.
    • Download figure
    • Open in new tab
    • Download powerpoint
    Figure 4.
    • Download figure
    • Open in new tab
    • Download powerpoint
    Figure 4.
    • Download figure
    • Open in new tab
    • Download powerpoint
    Figure 4.
    • Download figure
    • Open in new tab
    • Download powerpoint
    Figure 4.

    The outcome distribution in test set 2.

    CXR = chest x-ray; CRP = C-reactive protein; dL= deciliter; ICD-10 = International Classification of Diseases, 10th revision; mg = milligram; PC = primary care.

    Notes: (B) bars represent 95% CIs. (F) shaded area represents 95% CIs. ICD-10 code distribution in test set 2 was not examined for these symptomatically similar patients.

Tables

  • Figures
  • Additional Files
    • View popup
    Table 1.

    Demographics, ICD-10 Code Distributions, and Outcomes in the Data Sets

    VariableTraining SetTest Set 1Test Set 2
    Demographics
    Patients, No.1,500500664
    Age, mean (range), y    54
    (18-93)
      55
    (19-91)
      45
    (18-92)
    Sex, female, %  66.1  62.6  61.1
    ICD-10 codes, %
    J00  20.2  19.2    0.0
    J15    0.4    0.2    0.0
    J20  46.5  48.2    0.0
    J44  11.1    9.8    0.0
    J45  21.8  22.6    0.0
    J10    0.0    0.0    0.8
    J11    0.0    0.0  99.2
    Patient outcomes
    CRP value, mean (range), mg /dL  20.7
    (3-183)
      18.8
    (3-84)
      28.2
    (3-178)
    Antibiotics prescribed, %  49.0  50.8  15.5
    CXRs ordered, %    6.6    6.8    3.5
    CXR signs of pneumonia, %  12.6    9.6    9.1
    CXR incidentaloma, %    1.6    3.2    4.3
    PC re-evaluation, %    8.2    6.8  10.0
    ED re-evaluation, %    0.2    0.6    0.8
    • CRP = C-reactive protein; CXR = chest x-ray; ED = emergency department; ICD-10 = International Classification of Diseases, 10th Revision; J00 = common cold; J11 = influenza; J12 = viral pneumonia; J15 = bacterial pneumonia; J20 = acute bronchitis; J44 = chronic obstructive pulmonary disease; ; J45 = asthma; PC = primary care.

    • View popup
    Table 2.

    Comparison of Outcome Rates in the Test Sets Between Low-Risk and High-Risk Groups

    OutcomesaLow-Risk GroupHigh-Risk GroupSensitivitySpecificityPPVNPVP Value
    Test set 1
    Mean CRP values7.923.0…………<.05
    Antibiotics prescribed66 (40%)188 (56.1%)0.740.400.560.60<.005
    PC re-evaluation9 (5.4%)36 (10.7%)0.800.340.110.95.07
    ED re-evaluation1 (0.6%)3 (0.89%)0.500.330.010.98.67
    CXRs ordered6 (3.6%)25 (7.4%)0.810.340.0750.96.07
    Positive CXRsb0 (0.0%)3 (0.89%)1.000.330.011.00.30
    Test set 2
    Mean CRP values29.427.4…………1.00
    Antibiotics prescribed33 (10.9%)68 (18.7%)0.670.480.190.89<.05
    PC re-evaluation30 (9.9%)35 (9.6%)0.540.450.100.90.90
    ED re-evaluation1 (0.3%)4 (1.2%)0.800.460.011.00.38
    CXRs ordered8 (2.7%)15 (4.1%)0.650.460.040.97.40
    Positive CXRsb0 (0.0%)2 (0.6%)1.000.450.011.00.50
    • CRP = C-reactive protein; CXR = chest x-ray; ED = emergency department; NPV = negative predictive value; PC = primary care; PPV = positive predictive value.

    • Note: Test set 1 had 165 patients in low-risk group and 335 in high risk group; Test set 2 had 301 patients in low-risk group and 363 in high-risk group.

    • ↵a Mean CRP values reported for groups in mg/dL. Other outcomes reported for groups as number of patients (percentage).

    • ↵b Positive CXRs included those with signs of pneumonia only, incidentalomas were excluded.

Additional Files

  • Figures
  • Tables
  • SUPPLEMENTAL MATERIALS IN PDF FILE BELOW

    • Sigurdsson_Supp.pdf -

      PDF file

PreviousNext
Back to top

In this issue

The Annals of Family Medicine: 21 (3)
The Annals of Family Medicine: 21 (3)
Vol. 21, Issue 3
May/June 2023
  • Table of Contents
  • Index by author
  • Front Matter (PDF)
  • Plain-Language Summaries
Print
Download PDF
Article Alerts
Sign In to Email Alerts with your Email Address
Email Article

Thank you for your interest in spreading the word on Annals of Family Medicine.

NOTE: We only request your email address so that the person you are recommending the page to knows that you wanted them to see it, and that it is not junk mail. We do not capture any email address.

Enter multiple addresses on separate lines or separate them with commas.
Triaging Patients With Artificial Intelligence for Respiratory Symptoms in Primary Care to Improve Patient Outcomes: A Retrospective Diagnostic Accuracy Study
(Your Name) has sent you a message from Annals of Family Medicine
(Your Name) thought you would like to see the Annals of Family Medicine web site.
CAPTCHA
This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.
5 + 10 =
Solve this simple math problem and enter the result. E.g. for 1+3, enter 4.
Citation Tools
Triaging Patients With Artificial Intelligence for Respiratory Symptoms in Primary Care to Improve Patient Outcomes: A Retrospective Diagnostic Accuracy Study
Steindór Ellertsson, Hlynur D. Hlynsson, Hrafn Loftsson, Emil L. Sigur∂sson
The Annals of Family Medicine May 2023, 21 (3) 240-248; DOI: 10.1370/afm.2970

Citation Manager Formats

  • BibTeX
  • Bookends
  • EasyBib
  • EndNote (tagged)
  • EndNote 8 (xml)
  • Medlars
  • Mendeley
  • Papers
  • RefWorks Tagged
  • Ref Manager
  • RIS
  • Zotero
Get Permissions
Share
Triaging Patients With Artificial Intelligence for Respiratory Symptoms in Primary Care to Improve Patient Outcomes: A Retrospective Diagnostic Accuracy Study
Steindór Ellertsson, Hlynur D. Hlynsson, Hrafn Loftsson, Emil L. Sigur∂sson
The Annals of Family Medicine May 2023, 21 (3) 240-248; DOI: 10.1370/afm.2970
Twitter logo Facebook logo Mendeley logo
  • Tweet Widget
  • Facebook Like
  • Google Plus One

Jump to section

  • Article
    • Abstract
    • INTRODUCTION
    • METHODS
    • RESULTS
    • DISCUSSION
    • Footnotes
    • REFERENCES
  • Figures & Data
  • eLetters
  • Info & Metrics
  • PDF

Related Articles

  • PubMed
  • Google Scholar

Cited By...

  • A comparison of self-triage tools to nurse driven triage in the emergency department
  • Google Scholar

More in this TOC Section

  • Teamwork Among Primary Care Staff to Achieve Regular Follow-Up of Chronic Patients
  • Shared Decision Making Among Racially and/or Ethnically Diverse Populations in Primary Care: A Scoping Review of Barriers and Facilitators
  • Convenience or Continuity: When Are Patients Willing to Wait to See Their Own Doctor?
Show more Original Research

Similar Articles

Subjects

  • Domains of illness & health:
    • Acute illness
  • Methods:
    • Quantitative methods
  • Other research types:
    • Health services
  • Other topics:
    • Health informatics
    • Communication / decision making

Keywords

  • artificial intelligence
  • clinical decision support systems
  • primary care
  • triage
  • respiratory symptoms

Content

  • Current Issue
  • Past Issues
  • Early Access
  • Plain-Language Summaries
  • Multimedia
  • Podcast
  • Articles by Type
  • Articles by Subject
  • Supplements
  • Calls for Papers

Info for

  • Authors
  • Reviewers
  • Job Seekers
  • Media

Engage

  • E-mail Alerts
  • e-Letters (Comments)
  • RSS
  • Journal Club
  • Submit a Manuscript
  • Subscribe
  • Family Medicine Careers

About

  • About Us
  • Editorial Board & Staff
  • Sponsoring Organizations
  • Copyrights & Permissions
  • Contact Us
  • eLetter/Comments Policy

© 2025 Annals of Family Medicine