Abstract
PURPOSE To validate a smartphone-operated, single-lead electrocardiography (1L-ECG) device (AliveCor KardiaMobile) with an integrated algorithm for atrial fibrillation (AF) against 12-lead ECG (12L-ECG) in a primary care population.
METHODS We recruited consecutive patients who underwent 12L-ECG for any nonacute indication. Patients held a smartphone with connected 1L-ECG while local personnel simultaneously performed 12L-ECG. All 1L-ECG recordings were assessed by blinded cardiologists as well as by the smartphone-integrated algorithm. The study cardiologists also assessed all 12L-recordings in random order as the reference standard. We determined the diagnostic accuracy of the 1L-ECG in detecting AF or atrial flutter (AFL) as well as any rhythm abnormality and any conduction abnormality with the simultaneously performed 12L-ECG as the reference standard.
RESULTS We included 214 patients from 10 Dutch general practices. Mean ± SD age was 64.1 ± 14.7 years, and 53.7% of the patients were male. The 12L-ECG diagnosed AF/AFL, any rhythm abnormality, and any conduction abnormality in 23, 44, and 28 patients, respectively. The 1L-ECG as assessed by cardiologists had a sensitivity and specificity for AF/AFL of 100% (95% CI, 85.2%-100%) and 100% (95% CI, 98.1%-100%). The AF detection algorithm had a sensitivity and specificity of 87.0% (95% CI, 66.4%-97.2%) and 97.9% (95% CI, 94.7%-99.4%). The 1L-ECG as assessed by cardiologists had a sensitivity and specificity for any rhythm abnormality of 90.9% (95% CI, 78.3%-97.5%) and 93.5% (95% CI, 88.7%-96.7%) and for any conduction abnormality of 46.4% (95% CI, 27.5%-66.1%) and 100% (95% CI, 98.0%-100%).
CONCLUSIONS In a primary care population, a smartphone-operated, 1L-ECG device showed excellent diagnostic accuracy for AF/AFL and good diagnostic accuracy for other rhythm abnormalities. The 1L-ECG device was less sensitive for conduction abnormalities.
- single-lead
- electrocardiography
- medical device
- atrial fibrillation
- cardiac arrhythmia
- cardiac conduction system disease
- cardiac complexes, premature
INTRODUCTION
Patients frequently visit their primary care physician with symptoms that may be due to cardiac arrhythmias.1 Manifestations include palpitations, light-headedness, and (near) fainting and account for 0.8% to 16% of symptoms that prompt patients to visit their primary care physician.1,2 Some heart rhythm abnormalities, such as ectopic beats, are common electrocardiography (ECG) findings that generally do not require action.3 Others, such as atrial fibrillation (AF) or atrial flutter (AFL), are present in approximately 2% to 3% of the population and warrant further work-up and management to reduce associated risks of stroke and heart failure.4–6 When a cardiac arrhythmia is suspected in a symptomatic patient, resting 12-lead ECG (12L-ECG) should always be performed.7 Unfortunately, in primary care, performing 12L-ECG can be cumbersome, particularly during house visits, and it is not available at every practice. As a result, only in approximately one-third of cases is ECG performed during a symptomatic period.3
The availability of an unobtrusive, handheld ECG device is likely to lower the logistical threshold for performing ECG and may therefore improve detection of relevant arrhythmias in primary care.8 One such device, the KardiaMobile, is a smartphone-connected, single-lead ECG (1L-ECG) device.9,10 Smartphone-operated ECG has been studied for screening purposes and has shown great promise.11 A recent report issued by the United Kingdom’s National Health Service expects the device to be highly cost saving in the context of primary care.12
To our knowledge, the KardiaMobile has not yet been validated against simultaneously performed 12L-ECG in a primary care population. We hypothesized that the information obtained with smartphone-operated 1L-ECG can be used to accurately detect AF/AFL and common ectopic beats. We therefore performed a multicenter validation study in primary care to assess the validity of 1L-ECG as an office/bedside tool for the detection of arrhythmias as well as rhythm and conduction abnormalities compared with simultaneously performed 12L-ECG as assessed by blinded cardiologists as the reference standard.
METHODS
We reported this diagnostic accuracy study in accordance with the Standards for Reporting of Diagnostic Accuracy Studies (STARD) 2015 statement.13 The study protocol was approved by our institution’s Medical Ethical Review Committee. All participants provided written informed consent.
Study Design
We enrolled consecutive patients as part of the Validation of a mobile bedside ECG Screening and diagnostic Tool for Arrhythmias in general practice (VESTA) study. Eligible patients were aged 18 years or older who were assigned to 12L-ECG for any nonacute indication as ordered by the local primary care physician in 1 of 10 participating general practices across the Netherlands. The practices were in possession of a 12L-ECG device and had qualified and skilled personnel to perform 12L-ECG. Exclusion criteria were a clinically acute indication for ECG as defined by the local primary care physician (eg, suspicion of acute coronary syndrome) and presence of a pacemaker rhythm on 12L-ECG. We categorized patients according to indication for 12L-ECG either because of presentation with new symptoms (symptom-driven ECG) or as an integral part of protocolized care for primary or secondary prevention of cardiovascular disease (protocol-driven ECG). For each participant, the study design involved 3 different readings as follows: (1) the 1L-ECG read by the AF detection algorithm of the smartphone application, (2) the 1L-ECG read by cardiologists, and (3) the standard 12L-ECG read by cardiologists.
Index Test
The KardiaMobile (AliveCor, Inc) is a smartphone-connected, 1L-ECG device that displays ECG recordings in real time (30 seconds) via a smartphone application with a built-in AF detection algorithm (Figure 1). The 1L-ECG recordings were assessed in 2 ways as follows:
The AF detection algorithm assessed all 1L-ECG recordings. It classified recordings as either possible AF, normal, or unreadable, or provided no classification. We marked all recordings classified as possible AF as positive for AF. We marked all other algorithm classifications, or when no classification was provided, as negative for AF. The algorithm did not provide a classification for when a 1L-ECG recording was truncated (<30 seconds).
Cardiologists (M.L.H., R.N., J.R.dG.) assessed all 1L-ECG recordings in randomized order. The evaluation consisted of scoring each recording for the presence of arrhythmias, ectopic beats, and conduction abnormalities according to a scoring template designed for this study (see Supplemental Appendix, http://www.AnnFamMed.org/content/17/5/403/suppl/DC1/, for exact definitions).
The KardiaMobile and Kardia smartphone application.
Photograph by Jelle Himmelreich.
Reference Standard
All 12L-ECG recordings were independently evaluated by 2 cardiologists, and in case of disagreement, by a third cardiologist (M.L.H., R.N., J.R.dG.). We presented 12L-ECG recordings to the cardiologists in randomized order. The evaluation consisted of scoring each recording for the presence of arrhythmias, ectopic beats, and conduction abnormalities according to a scoring template designed for this study (see Supplemental Appendix, http://www.AnnFamMed.org/content/17/5/403/suppl/DC1/, for exact definitions).
Rhythm Measurement
Personnel instructed each patient to commence the KardiaMobile recording by holding the device loosely with both hands (corresponding with lead I for 12L-ECG). We advised patients who used hand lotion or had sweaty hands to wash their hands with soap or to use alcohol wipes on the fingertips to optimize electrical conduction quality. When a steady 1L-ECG signal was visible on the smartphone, the local investigator started a 10-second 12L-ECG recording. We thereby obtained 10 seconds of simultaneous recording. We excluded patients for whom 1 or both ECG types were not available or when there was no 10-second overlap between recording types. The 1L-ECG recordings were not used for clinical decision making.
Data Collection
Three investigators (J.C.L.H., E.P.M.K., R.E.H.) visited participating practices to collect the 12L-ECG recordings (as PDF file or photocopy of paper original) as well as patient data at the time of index ECG from the practice’s electronic health records. We collected the corresponding 1L-ECG recordings (PDF files) from the secure online platform that is part of the KardiaMobile software package. Baseline data included sex, age, indication for undergoing 12L-ECG, use of relevant antiarrhythmic drugs, and relevant medical history. We pseudonymized all data before storing it in a secured electronic case report form (Castor EDC).
Statistical Analysis
We expressed diagnostic accuracy for all analyses as sensitivity, specificity, positive and negative likelihood ratios, and positive and negative predictive values, with 95% CI. The primary analyses of this study were (1) the diagnostic accuracy of 1L-ECG as assessed by cardiologists in detecting AF or AFL with 12L-ECG as reference and (2) the diagnostic accuracy of the AF detection algorithm for AF/AFL with 12L-ECG as reference. Secondary analyses were (1) the diagnostic accuracy of 1L-ECG as assessed by cardiologists in detecting any rhythm abnormality, defined as any nonsinus rhythm including AF/AFL and/or presence of any ectopic beat, with 12L-ECG as reference and (2) the diagnostic accuracy of 1L-ECG as assessed by cardiologists in detecting any conduction abnormality, defined as presence of atrioventricular (AV) block, bundle branch block (BBB), and/or left axis deviation and/or left anterior fascicular block, with 12L-ECG as reference. We counted the cardiologists’ generic assessment of BBB on 1L-ECG as true positive even if specification of subtype of BBB (left BBB or right BBB) was provided by the corresponding 12L-ECG.
We performed an exploratory analysis of the primary and secondary outcomes stratified by whether ECG was performed based on symptoms or as part of protocol-driven care. We performed a sensitivity analysis on the comparison of the AF detection algorithm vs 12L-ECG for the outcome AF/AFL, in which we excluded all patients with a truncated 1L-ECG recording.
We presented discrete variables as number and percentage and continuous variables as mean ± standard deviation. We compared continuous variables using the Student t test and proportions using the Fisher exact test or Pearson χ2 test and used 2-tailed tests. We evaluated statistical significance in all analyses at the .05 level and performed analyses using IBM SPSS Statistics version 24.0 (IBM Corp) and MedCalc version 18.10.2 (MedCalc Software).
RESULTS
We included 219 patients during the period April 2017 to July 2018. After excluding 2 patients for missing 1L- and/or 12L-ECG recordings and 3 patients for nonoverlapping recordings, the remaining 214 patients comprised the study population. No adverse device effects were reported. Baseline characteristics of the included patients are listed in Table 1. Mean age was 64.1 ± 14.7 years, and 53.7% of the patients were male. As shown in Figure 2, the indication for performing 12L-ECG was symptom driven for one-half of the patients (n = 108). Among those presenting with new symptoms, most (44.4%) reported palpitations as the primary symptom (Table 2). The 12L-ECG recordings revealed that AF/AFL, any rhythm abnormality, and any conduction abnormality were present in 23, 44, and 28 patients, respectively (Table 3).
Baseline Characteristics of the Study Population
Study flow diagram.
1L = single-lead; 12L = 12-lead; ACA = any conduction abnormality; AF = atrial fibrillation; AFL = atrial flutter; ARA = any rhythm abnormality; bpm = beats per minute; ECG = electrocardiography; PCP = primary care physician.
Indications for Undergoing 12L-ECG (n = 214)
Outcomes of 12L-ECG (n = 214)
Diagnostic Accuracy of the 1L-ECG
Data on diagnostic accuracy with calculated 95% CIs are summarized in Table 4. The 2×2 contingency tables with detailed information, including the rhythm diagnoses of all true positives, false negatives, and false positives, can be found in Supplemental Figure 1, http://www.AnnFamMed.org/content/17/5/403/suppl/DC1/.
Diagnostic Accuracy Measures of the Interpretation of 1L-ECG by Cardiologists or the Smartphone Algorithm Using 12L-ECG as Reference Standard
For the primary outcome of AF/AFL, we found that cardiologists were able to correctly classify all 23 cases using 1L-ECG, resulting in a sensitivity and a specificity of 100%. The smartphone-integrated algorithm correctly identified 20 of 23 AF cases and incorrectly classified 4 cases of sinus rhythm as possible AF (sensitivity: 87%; specificity: 97.9%). Interpretation of 1L-ECG was less robust for the secondary endpoints of any rhythm abnormality (sensitivity: 90.9%; specificity: 93.5%) and any conduction abnormality (sensitivity: 46.4%; specificity: 100%). Explicitly for ectopic beats, 1L-ECG correctly classified 20 of 23 cases of known ectopic beats. The false positives for the outcome any rhythm abnormality could all be attributed to misclassified ectopic beats (n = 11).
Additional Analyses
The stratified analysis according to indication for ECG (symptom or protocol driven) and the sensitivity analysis in which we excluded truncated 1L-ECG recordings (n = 6) rendered similar results (see Supplemental Figure 2, Supplemental Table 1, and Supplemental Table 2, respectively, http://www.AnnFamMed.org/content/17/5/403/suppl/DC1/).
DISCUSSION
The diagnostic properties of the KardiaMobile 1L-ECG device as assessed by cardiologists against simultaneously performed 12L-ECG in a primary care population were excellent for AF/AFL. The AF detection algorithm showed high sensitivity and specificity for AF/AFL. Visual assessment of the 1L-ECG recordings by cardiologists resulted in high sensitivity and specificity for rhythm abnormalities and high specificity but low sensitivity for conduction abnormalities. To our knowledge, this is the first study to validate the KardiaMobile device for both AF and common non-AF ECG abnormalities against simultaneously performed 12L-ECG in a primary care population.
Clinical Relevance
Patients who present to their primary care physician with palpitations often no longer have symptoms at the time of consultation or when ECG is performed.3 When ECG is performed during palpitations, an abnor mal heart rhythm is found in approximately one-half of patients, whereas this drops to approximately one-fifth when symptoms are no longer present at the time of ECG.3 The findings from the present study are therefore highly relevant for primary care physicians because the smartphone-operated ECG device operates as a point- of-care test and allows for immediate rhythm assessment during a symptomatic episode. Moreover, our findings support patients’ use of the device at home as a 1L event recorder, provided that the ECG readings are assessed by a cardiologist. We note that the device is already available on the consumer market for this purpose.
Our stratified analysis by indication for ECG showed that in older patients for whom ECG was not indicated primarily for cardiac symptoms, a negative reading excluded AF with a similarly high degree of certainty as that for symptomatic patients, despite differences in pretest likelihood within our sample. These results may be relevant for primary care physicians because they are encouraged to perform proactive case identification in asymptomatic patients with elevated risk of developing AF (eg, via pulse palpation followed by ECG).16 Here, the 1L-ECG device could be a valuable point-of-care tool for at-risk patients for whom traveling to the practice for standard 12L-ECG is too cumbersome or for primary care physicians who do not possess a 12L-ECG device.
We added the comparison on any rhythm abnormality because for primary care patients, cardiac symptoms may often be explained by ectopy.1 We found that the 1L-ECG device can correctly classify instances of ectopic beats, suggesting that it may be useful as a point-of-care diagnostic instrument for this rhythm anomaly.
The 1L-ECG device was less sensitive for conduction abnormalities, which in the present study particularly involved the detection of first-degree AV blocks. For primary care physicians, however, the detection of conduction abnormalities is generally less clinically relevant than the detection of arrhythmias, with the notable exception of decisions regarding the prescribing of QT-prolonging medication.17 Whereas the QT interval was not scored in the present study, others have reported the KardiaMobile’s ability to accurately assess QT intervals.18
Strengths and Limitations
Our study had a number of strengths. First, we included consecutive patients who underwent 12L-ECG as part of routine medical practice, resulting in a cohort generalizable to general practice. Second, the study design ensured simultaneous rather than consecutive 1L- and 12L-ECG recordings, as done in prior studies.10,19,20 This allowed for a comparison on the detection of ectopic beats, which may be a frequent cause for palpitations in primary care.21 Third, by providing a stratified analysis according to indication for ECG, we were able to show that the 1L-ECG device performed similarly in patients with symptoms vs those who present as part of protocol-driven (secondary) preventive care. Fourth, we ensured standardized interpretation of all recordings by blinded assessment of 1L-and 12L-ECG recordings in random order.
Several limitations should be mentioned. First, this study was not designed to determine to what extent primary care physicians are able to assess the 1L-ECG signal, but rather to describe the test characteristics of the 1L-ECG device in a representative primary care patient sample when analyzed by experts (cardiologists/electrophysiologists). Second, the use of recordings of different durations (10-second 12L-ECG vs 30-second 1L-ECG) may have led us to underestimate the specificity of 1L-ECG in the analysis of any rhythm abnormality, given that ectopic beats might have occurred during the nonoverlapping 20 seconds of the 1L-ECG recording. Third, we presented cardiologists with the PDF file of the 1L-ECG recording instead of having them assess the recording from a smartphone or tablet screen, which is how physicians will often use the device.20 Fourth, the KardiaMobile application did not provide an automated assessment of conduction intervals in milliseconds, as is done for most 12L-ECGs. This might have affected sensitivity in the analysis on any conduction abnormality. Fifth, the 95% CIs were relatively wide, owing to sample size and prevalence of the studied outcomes among the cohort. Finally, the present study was not designed to study whether the availability of a smartphone 1L-ECG would change ECG use, diagnosis, or patient management.
Previous Work
The good diagnostic properties that we found for the 1L-ECG device for AF/AFL, when assessed by cardiologists or by the smartphone application algorithm, coincide with a number of prior studies10,11,19,20,22–27 (see Supplemental Table 3, and Supplemental Table 4, http://www.AnnFamMed.org/content/17/5/403/suppl/DC1/for an overview of prior studies that validated the KardiaMobile 1L-ECG device for rhythm and/or conduction abnormalities). Notable exceptions are 2 studies by Chan et al8,28 and 1 study by Desteghe et al29 that reported sensitivities of 71.4%, 66.7%, and 65.9%, respectively, for the KardiaMobile algorithm to detect AF. The authors provide no clear explanation for the AF-detection algorithm’s low sensitivity in their respective studies, which were all performed with selected elderly patients.
Although a number of studies have assessed the presence of ectopic beats on 1L-ECG recordings, none have validated 1L-ECG for ectopy alone or as part of a composite outcome.8,10,11,22–24,28,29 One study validated 1L-ECG against 12L-ECG for conduction abnormalities. That study, by Haberman et al,19 found high specificity but sensitivities of 77.3% and 72.4%, respectively, for AV block and BBB. The results for AV block contrast with those from our present study, in which none of the AV blocks were detected using the 1L-ECG device (Figure 2). We note that Haberman et al19 determined automated conduction intervals for 1L-ECG before assessment by electrophysiologists, whereas in our present study, automated intervals for 1L-ECG were absent.
Our present work adds to the literature by validating 1L-ECG against 12L-ECG in a primary care setting of consecutive patients and by validating 1L-ECG for a broad spectrum of cardiac arrhythmias and conduction disturbances including ectopic beats.
Future Work
Further study is required to evaluate the safety and efficacy of the 1L-ECG device in the hands of primary care physicians instead of cardiologists, particularly for detecting AF/AFL. Moreover, future studies are warranted to determine whether the availability of 1L-ECG changes the use of 12L-ECG, diagnosis, and/or patient management. Data should be obtained to study the net benefit as well as impact on cost-effectiveness of adding the KardiaMobile algorithm’s or cardiologists’ assessment to that of primary care physicians for arrhythmia detection. Findings from such studies might determine whether and how the KardiaMobile 1L-ECG device can be safely and effectively implemented in clinical practice as well as used in future screening programs for detecting AF in at-risk general populations.
CONCLUSION
A smartphone-operated, 1L-ECG device is a reliable instrument for detecting AF when assessed by the internal detection algorithm, and even more so when assessed by cardiologists. Moreover, the 1L-ECG recording can display atrial and ventricular ectopy with high sensitivity. The 1L-ECG recording was less robust for detecting conduction delays. Our primary care–based study provides important insights for physicians who are in need of a point-of-care ECG device that can lower the logistical threshold for performing ECG to improve diagnostic gain.
Acknowledgments
We thank all participating primary care practices for their cooperation in performing this study. We thank Mrs Lucinda Bertels for her assistance with Figure 1.
Footnotes
↵* These authors contributed equally to this work.
Conflicts of interest: authors report none.
To read or post commentaries in response to this article, see it online at http://www.AnnFamMed.org/content/17/5/403.
Funding support: This work was supported by the Netherlands Organisation for Health Research and Development (ZonMw) (80-83910-98-13046). Salary support for Dr Harskamp was provided by a Rubicon fellowship of the Netherlands Organization for Scientific Research (NWO). Dr de Groot is supported by a personal VIDI grant from NWO/ZonMW (016.146.310), reports research grants through his institution from Abbott, Atricure, Boston Scientific, and Medtronic, and received consultancy/speakers fees from Atricure, Bayer, Daiichi Sankyo, Johnson & Johnson, Medtronic, Novartis, and Servier; all outside the scope of this study. All devices and research efforts were paid from university funds. The authors received no funding from the device’s producer or local distributor. The authors report no ties to the manufacturer of the investigated device and had full autonomy in the design, conduct, and reporting of this manuscript.
Previous presentations: A scientific poster presentation (hard copy) with preliminary results was presented by Jelle Himmelreich at the HartVaatHAG conference for Dutch primary care practitioners, October 5, 2018, Utrecht, The Netherlands, and at the Amsterdam Public Health Annual Meeting, November 22, 2018, Amsterdam; The Netherlands. Abstracts for these local conferences have not been published either in print or electronically.
Supplemental materials: Available at http://www.AnnFamMed.org/content/17/5/403/suppl/DC1/.
- Received for publication December 18, 2018.
- Revision received April 3, 2019.
- Accepted for publication April 12, 2019.
- © 2019 Annals of Family Medicine, Inc.