Point-of-Care Ultrasound in General Practice: A Systematic Review

Dear Tyler Johnson et al.

Thank you for your interest in our study and for appreciating our methodological approach. Below you will find a point-by-point answer to your comments on our study.

You call for a more clear definition of POC-US. In the introduction, we write: "POC-US is implemented across the medical field by clinicians for bedside examination of patients" and provide several references for the definition. You would have preferred that we had compared differences between rural and urban areas in terms of need. We agree that there may be differences in the needs. However, we were not aiming to describe ultrasound-use across settings and in many studies setting was not clearly defined.

You also highlight that including indication for performing ultrasound would have been helpful. We agree that indication for performing an ultrasound examination is important information, if the aim is to understand why ultrasound was used. Our aim however, was to describe what ultrasound was used for. We described whether ultrasound was used for diagnostic, screening or procedure-related purposes. Besides, only few studies provided information about the indication in detail. You call for a more precise definition on supervision. There was a large variation in the training programs described in the included studies. Due to the word limit, we were not able to report details on the described training programs. Recently, we have conducted a new systematic literature review focusing on training programs for general practitioners in the use of ultrasound with an updated literature search. (PROSPERO registration number CRD42016038302). We defined supervision as a real-time guidance and oversight of the performed scan by a more qualified person e.g. physician or sonographer. You also call for a more clear definition on overlooked conditions. In the article we write: Misdiagnoses (false positives) and overlooked conditions (false negative).

You question our finding that the extend and anatomical location of the ultrasound on the patient was associated with increased quality of scans and you ask if it was instead the training. In the table in web appendix 5 we listed the results of the quality assessment in the different studies next to hours of training. The table revels that there is no clear relationship between hours of training and quality of the performed scan. Combining the quality of the performed scans reported in each study with the extend of the examination (focused vs detailed examination) and the anatomical area being scanned revels that some anatomical areas e.g. aorta are easier to master even after few hours of training, while others e.g. the heart seem to cause more trouble even after extensive training. Additionally, articles describing focused examinations report better quality than article with full explorative or detailed examinations.

Finally, you question the generalizability of our results as we included both older studies and poor-quality studies. Following the Cochrane guidelines, we included the totality of published papers, which entailed including both older studies and low quality studies. The results of this systematic literature review must be assessed in light of the poor reporting in most of the included studies. Several interesting perspectives on the use of ultrasound in general practice remain unanswered due to missing information. The aim of our review was to provide an overview of the existing literature and to point to areas where future research should focus.

Competing interests: None declared

The purpose of this study was to evaluate literature on point of care ultrasound (POCUS) used by general practitioners. In this systematic review, the authors specifically looked at use of POCUS, quality of scans, training of general practitioners, frequency of ultrasound, financial cost for the practice, potential harm to the patient, and patient satisfaction with POCUS.

We thought this was a well designed study using the PRISMA guidelines to evaluate each article equally and the Cochrane data extraction form to determine which articles should be included in the review. Any disagreements between the two reviewers were settled by a third reviewer. The use of standardized forms between multiple reviewers is useful in eliminating bias any single reviewer may have. However, we thought a few areas of clarification would have been helpful to readers. For example, point of care (POC) generally means you are able to do it right there with the patient in front of you. However, this was not defined anywhere within the article. This incorrectly assumes that everyone is utilizing the same definition when reading the article.

The study had to describe the POC exam for the ultrasound performed by a general practitioner (GP). The definition for GP was very clear, i.e. an MD working in hospital or outpatient setting, family medicine doctor, primary care doctor, or residents working in the field of FM. However, there was no patient population/setting restriction for the study. Hence it is difficult to compare differences between rural and urban areas in terms of need. Although this may not be considered a limitation, it would be interesting to compare the data in the articles conducted in a rural vs. urban setting to evaluate the differences in use in the two settings.

This group discussed some of the limitations of the study. It would help to know the indications for why the ultrasounds were ordered not just the organ system involved. We also believe including the indications for why an ultrasound was obtained in the articles would have been helpful. It was unclear as to what symptoms or complaints warranted an ultrasound of a specific anatomical space. We also discussed the gap in the literature in looking at the frequency of use with POCUS. Only 5/51 accepted articles reported frequency of use for broad application. It's therefore, more difficult to generalize the finding to your own practice. Many articles (33/51) addressed training in ultrasound. However, training time varied from 4-320 hours. The potential lack of training in ultrasound indicates a need for a guideline or training program that can be implemented into existing programs. There is no uniformity in training programs, so clinicians need to take that into consideration. Standardizing clinician use of these would add further value to results of the study. The authors of the article noted this limitation and explained that training programs have since then been developed. In addition, some of the articles in the study included the quality of the ultrasound images. There were 12 studies where supervision was used to determine quality; however, this group was unsure of what supervision meant and would have preferred that to be better defined. Being able to pass an exam or sit in a class for certification does not necessarily correlate to ultrasound proficiency and only 9/51 studies actually looked at diagnostic accuracy of the ultrasounds. This study also included potential harms found in the systematic review. They noted incidental findings on ultrasounds that would not normally be identified which made this group wonder about indications for the ultrasound and thus, what incidental finding was found. False negatives also identified; however, they did not identify what the "overlooked conditions" meant. False positives really speaks to training issue previously addressed and quality of the stands. Again, what was the ultrasound being utilized for and what were they hoping to find.

Few articles (5/51) even looked at patient perspective but feedback from the patient's was good (found the US pleasant, non-invasive, non-stressful, and sense of security about their health). 66% of patients thought US should be a part of routine screening, but this group could not determine why those patients felt that way. Even fewer articles (3/51) looked into financials and even then only performed "limited economic evaluations." Two studies looked at eliminating the need for further testing, but understanding what was being screened for/indicated would have been nice to understand why further testing was not necessary. Only one study indicated that patients were willing to pay more for their general practitioner, but did not address what setting or environment that applied to. Looking into differences in cost between ultrasounds performed in general practitioner's office compared to at a larger facility such as a hospital would be interesting. This group was also interested in the financial implications of having the ultrasound done in the office by the GP vs. an ultrasound technician conducting the study and a radiologist reading it.

In summary, ultrasounds were used for number of different applications, accuracy in general was pretty good (false positive not as high), and training varied significantly. However, this group found there was no data supporting the claim that the extent and anatomical location of the ultrasound on the patient was associated with increased quality of scans, and not the number of hours of training.

We also found low generalizability of data due to few numbers of practitioners reported. Some of the articles go back to the 80s and 90s when ultrasound technology was not as advanced and refined as it is now so these studies would potentially have lower quality scans and less accurate findings. Excluding these studies would provide more accurate results that can be generalized to POCUS today. We applaud the authors for including this limitation to their study in the paper. On the other hand, we felt the inclusion of low quality studies in this study greatly skews the validity of conclusions drawn from the results.

The group also discussed how we would conduct this study differently if we had the chance. Given the general accessibility to healthcare is more difficult in rural communities, we felt that it would be interesting to differentiate between rural and urban populations. Having an ultrasound machine and trained general practitioners in smaller clinics would prevent the need to send patients elsewhere for the study. Other benefits and limitations would be interesting to know and would be directly applicable to many physicians.

Competing interests: None declared

Dear Dr. Mumoli

Thank you for your interest in our manuscript and for pointing to relevant papers. One of these papers is actually included in our upcoming review focusing on training of general practitioners (Education of general practitioners in the use of ultrasonography, PROSPERO registration number CRD42018102738).

We agree with you, that point-of-care ultrasound is relevant for bedside examination of patients in many clinical settings and not exclusively in general practice. We also agree that ultrasound skills improve with training and experience, however, some ultrasound examinations are easier to master than others [1,2,3]. Hence, it is important to focus on which training is needed to achieve skills within different types of point-of-care ultrasound examinations.

Equally important is to explore how ultrasound competences are maintained over time in the relevant clinical setting of the ultrasound-performing healthcare provider. In our review [4], we found a considerably variation in the frequency of ultrasound use (between 43-601 ultrasound examinations per GP per year). To maintain ultrasound competences, providers must perform examinations regularly and more research is needed to define sufficiently trained in different ultrasound applications.

1. Bailey RP, Ault M, Greengold NL, Rosendahl T, Cossman D. Ultrasonography performed by primary care residents for abdominal aortic aneurysm screening. J Gen Intern Med. 2001;16(12):845-849.
2. Ellington LE, Gilman RH, Chavez MA, et al. Lung ultrasound as a diagnostic tool for radiographically-confirmed pneumonia in low resource settings. Respir Med. 2017;128:57-64.
3. Bornemann P, Johnson J, Tiglao S, et al. Assessment of primary care physicians' use of a pocket ultrasound deviceTM to measure left ventricular mass in patients with hypertension. 2015:706-712
4. Andersen CA, Holden S, Vela J, Rathleff MS, and Jensen MB. Point-of-Care Ultrasound in General Practice: A Systematic Review. Ann. Fam. Med 2019 17:61-69; doi:10.1370/afm.2330

Competing interests: None