A Core Outcome Set for Multimorbidity Research (COSmm)

Comparison With Other Core Outcome Sets

Our core outcome set was developed specifically for studies of interventions for patients with multimorbidity, but other studies have used consensus approaches to develop outcomes to measure the quality of care for people with multimorbidity using the electronic health record.²² A range of consensus process methods have been used to develop various core outcome sets.¹ We chose to use a Delphi panel approach on the basis of our previous experience and a review of core outcome set literature.^19,21,23 Other potential approaches include expert panel groups, nominal group techniques, semi-structured group discussions, and questionnaires.²¹ Face-to-face approaches allow greater discussion but are restricted by limited accessibility for international stakeholders. Some elements of other approaches were used by conducting workshops to identify the draft core outcome set before the consensus process. This is an evolving area of research, and there is no clear evidence suggesting benefit of one consensus approach over another.¹⁹ We chose an online Delphi method as it allowed for involvement of clinicians, researchers, and patients in multiple sites across continents. This strategy also allowed for purposive selection of a panel to ensure representation of a range of stakeholders from different countries and health care systems with both professional and patient representation.²⁴

Panelists achieved a high level of consensus after the first round for 11 outcomes. The level seen is higher than might have been expected. There are clearly many important outcomes for evaluating care delivered to complex populations. After the first round, we consulted the COMET group regarding the number of outcomes, and they provided helpful feedback, which suggested that our core outcome set is not unusual given the broad scope of multimorbidity.

We used a 5-point Likert scale for outcome scoring. Other core outcome sets have been developed using alternative outcome-scoring methods, which may have been more appropriate.¹ These alternatives include use of a higher number of Likert scale options, scoring based on average agreement scores, and a ranking system. The methodology for developing core outcome sets has been evolving rapidly over the last few years, and there is no clear evidence as yet on which is the best scoring method. The COS Star Guidance¹ was produced after we had published our protocol and secured ethical approval. We considered changing our approach to outcome scoring after round 1, but decided not to deviate from our protocol at that point.

Strengths and Weaknesses

Our new core outcome set for multimorbidity studies was developed over a period of several years by an international group of researchers working in this area. It built on a number of workshops held with a wide range of participants at different international primary care meetings. We formed a Delphi panel consisting of a broad range of experts with experience in multimorbidity research from 13 countries and with patient and public representation. Identifying appropriate patient and public representatives was a challenge given the broad nature of multimorbidity, but we worked with experienced research groups to identify appropriate panelists from 3 countries. A further strength was the very high response rate for the first round of the consensus process and 100% participation for the second round.

One potential limitation of our COSmm is the high number of outcomes identified by the panel. We already had a high number of included outcomes after round 1, and we provided a cover letter for the round 2 survey stressing to panelists that there was not necessarily a need to identify additional outcomes. Despite this note, a further 6 outcomes were added. This result may relate to our consensus methods, but a range of approaches have been described in the literature with no clear agreement on the best approach to date. Another limitation may relate to our decision not to provide panelists with feedback on scores from other panelists for round 2. We used this approach to ensure that panelists could make independent judgments and avoid moving toward a group average score.²⁴ Guidance on the use of Delphi techniques recommends that a measure of distribution around final scores should be reported, which we have done. Reporting of the final scores alone can mask major disagreement within the group, which might be the case if there was a wide distribution around scores.²³ Our results do not suggest that this was the case.

Panel size and selection are always key considerations and potential limitations for any Delphi process. We chose to identify a purposive sample of panelists with prior exposure to the concept of multimorbidity given the complexity of this topic. This strategy could be seen as a limitation in that it did not open up the COSmm to a wider group. The broad nature of multimorbidity and the frequent debate about its definition and construct,¹¹ however, were repeated themes at all the workshops that led to the development of the preliminary COSmm, and these workshops were open to all attendees at the 2 largest primary care research meetings in North America and Europe. We therefore felt it was appropriate to limit the Delphi panelists to those with prior exposure to multimorbidity as they had already considered issues pertaining to whether multimorbidity itself represents a construct.

Other core outcome set studies have used larger panel sizes to increase representation; however, there tends to be a lower response rate for larger panels.²⁵ Some studies use snowballing techniques allowing open access to an online survey, which will lead to larger numbers of participants. This approach does not necessarily ensure broad and balanced representation of key stakeholders, however. Another potential limitation of our study is that although we included panelists from 13 countries, all were based in high-income countries. In addition, although we included patient representatives as Delphi panel members, as for many previous core outcome set studies,²⁴ it is a limitation that they were not directly involved in the development of the initial COSmm.

Conclusions

In conclusion, this consensus study resulting in development of a core set of outcomes specifically for multimorbidity research, the 17-outcome COSmm, involved a wide range of international experts who identified a large number of core outcomes for intervention studies among populations having multiple medical conditions. Such studies are likely to vary depending on target populations and intervention types, so given the scope of multimorbidity, the large number of outcomes identified is not surprising. Our results suggest that quality of life, mental health outcomes, and mortality should be regarded as essential core outcomes. Investigators should, however, also consider the full range of outcomes in the COSmm depending on their individual study aims and interventions.