Skip to main content

Main menu

  • Home
  • Content
    • Current Issue
    • Online First
    • Multimedia
    • Collections
    • Past Issues
    • Articles by Subject
    • Articles by Type
    • Supplements
    • Plain Language Summaries
    • Call for Papers
  • Info for
    • Authors
    • Reviewers
    • Media
    • Job Seekers
  • About
    • Annals of Family Medicine
    • Editorial Staff & Boards
    • Sponsoring Organizations
    • Copyrights & Permissions
    • Announcements
  • Engage
    • Engage
    • e-Letters (Comments)
    • Subscribe
    • RSS
    • Email Alerts
    • Journal Club
    • Annals Forum (Archive)
  • Contact
    • Feedback
    • Contact Us
  • Careers

User menu

  • My alerts

Search

  • Advanced search
Annals of Family Medicine
  • My alerts
Annals of Family Medicine

Advanced Search

  • Home
  • Content
    • Current Issue
    • Online First
    • Multimedia
    • Collections
    • Past Issues
    • Articles by Subject
    • Articles by Type
    • Supplements
    • Plain Language Summaries
    • Call for Papers
  • Info for
    • Authors
    • Reviewers
    • Media
    • Job Seekers
  • About
    • Annals of Family Medicine
    • Editorial Staff & Boards
    • Sponsoring Organizations
    • Copyrights & Permissions
    • Announcements
  • Engage
    • Engage
    • e-Letters (Comments)
    • Subscribe
    • RSS
    • Email Alerts
    • Journal Club
    • Annals Forum (Archive)
  • Contact
    • Feedback
    • Contact Us
  • Careers
  • Follow annalsfm on Twitter
  • Visit annalsfm on Facebook
EditorialEditorials

The Implications of Measuring Complexity

Lee A. Green
The Annals of Family Medicine July 2010, 8 (4) 291-292; DOI: https://doi.org/10.1370/afm.1147
Lee A. Green
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • Article
  • Info & Metrics
  • eLetters
  • PDF
Loading
  • Systems theory
  • nonlinear dynamics
  • ambulatory care
  • process assessment (health care)

In this issue of the Annals, Katerndahl, Wood, and Jaén wade boldly into deep waters indeed: attempting to quantitatively evaluate the complexity of outpatient medical care. This work was first presented 7 years ago, and the lengthy delay in publication illustrates the conceptual and methodological difficulty of this subject. The article will be beaten up, as such pioneering work typically is, but it is not important whether this particular measure of relative complexity is right—it is important that the authors have offered a rigorous quantitative measure to debate. As Box has pointed out, “All models are wrong, but some are useful.”1

Is this model of complexity of outpatient care useful? Since their approach is explicitly based on the assumption that they are measuring a complex nonlinear system, there are 2 issues that must be distinguished in evaluating it. One is the measurement of complexity; the other, and more important, is whether what is being measured is in fact a complex nonlinear system. The authors provide a sound grounding in information theory for their measurement, and they find also that the same rank-ordering of findings emerges when other approaches are applied to their sample. Hence, the measurement itself is reasonable.

It is one thing to calculate a statistic, however, and quite another to ask whether it means anything. Is the system being measured in fact a complex nonlinear system, one whose underlying dynamics are described by complexity science? And if so, what does that imply for measurement?

Leading thinkers in family medicine have long believed that complex systems theory can help us understand the dynamics of primary care and of primary care practices.2 The footprints of complexity seem to be present: systems with rich patterns of behavior, sometimes surprising behavior, self-similarity across scales, interconnectedness, and positive-feedback interactions among agents. But while we see complexity’s footprints, we have yet to capture the beast—to prove that the mathematics of complex nonlinear dynamics describe primary care and those of traditional linear modeling approaches do not.

Not all that is complicated is complex. A system with many inputs and outputs and many functions, each of which behaves in an essentially linear fashion, can produce rich and sometimes surprising patterns of behavior and contain a great deal of information. Such a complicated system is best described by a large number of linear equations, and its behavior is theoretically fully predictable even if hard to predict in practice. A complex nonlinear system, on the other hand, is very simple in the sense that a very small number of equations model it. Yet it produces extremely intricate patterns of results, often impossible to fully describe and not predictable even in theory. The crucial difference is that a complicated system produces rich behavior by having many elements behaving linearly, while a complex system produces rich behavior from a few elements that interact in highly nonlinear fashion.

A priori, we should consider it more likely that primary care is a complex nonlinear than a complicated linear dynamic system. Experience in other sciences, from physics and meteorology to biology, has shown that in nature complex nonlinear dynamics are more common than linear dynamics, though the bulk of research has been done on the much more tractable linear systems. That experience offers us a likelihood, however, not a conclusion. More solid evidence is needed, because this is no mere academic question.

For if primary care is indeed a complex nonlinear dynamic system, measures such as the Ambulatory Diagnostic Groups3 and Ambulatory Severity Index,4 based as they are on linear models, are not only inaccurate—they are fundamentally wrong as measures of primary care. If the models are wrong, the results, no matter how carefully calculated or statistically significant, will be specious. Policies based on such measures will have an unintended, unexpected, and likely adverse, impact on the care of patients. What gets measured is what gets managed, and bad measures will lead to bad management—to incentive structures and other policies that overvalue individual performance metrics, or composite metrics that are simple linear combinations of individual measures. Such simple measures will improve care if the linear model is correct, but may markedly worsen overall outcomes if primary care really is a complex system.

Policies based on such linear models will also tend to disadvantage family medicine as a specialty. Linear model–derived measures will make “partialist” care appear better and completely fail to capture the overall outcomes improvement of complex nonlinear generalist care.

To avoid such inappropriate measurement, it is not enough that we assert our claim that what we do should be measured with approaches based on complex nonlinear dynamics. We must show that our use of complexity science is not merely “the emperor’s new toolkit,”5 that complex systems models describe important phenomena in primary care not adequately described by complicated linear models, and that measures based on complexity science deliver answers that differ from those of linear models in meaningful ways.

Demonstrating that complexity science delivers better models for primary care poses considerable methodological challenges. Inference, critically distinguishing between linear and nonlinear dynamics, is especially challenging in our setting. Definitive demonstrations of complex nonlinear dynamics in other disciplines rely on extremely large data sets, with thousands of observations of the same individual system elements over time. Work in biology comes closest to our needs in primary care systems, but much development remains before small systems can be analyzed effectively over relatively short time spans. Such approaches as agent-based modeling simulations and genetic-algorithm modeling of postulated fitness landscapes are feasible with our current tools, and they can provide at least supportive results while critical inference tools are developed.

In the meantime, measurement efforts based on the assumption of complexity are at least as reasonable as traditional linear-based measures. It is important to put them forward for consideration for 2 reasons. One reason is that we must make linear-methods measurements of primary care put up or shut up. We must put out quantified, rigorous, complexity-based measures, so that simplistic linear methods no longer get a free pass, but have to be defended—or abandoned. The other reason is that struggling to produce quantified measures based on complexity is a major step toward putting complex systems investigation in primary care on a sounder footing. The information scientist Michael Cohen has said that “to be useful, complexity research must move beyond the festival of bad metaphors.” Katerndahl, Wood, and Jaén have taken an important step in helping us make that move.

Footnotes

  • Conflict of interest: none reported

  • Received for publication April 29, 2010.
  • Revision received May 20, 2010.
  • Accepted for publication May 24, 2010.
  • © 2010 Annals of Family Medicine, Inc.

REFERENCES

  1. ↵
    Box GEP, Draper NR. Empirical Model-Building and Response Surfaces. New York, NY: John Wiley & Sons, 1987.
  2. ↵
    Miller WL, Crabtree BF, McDaniel R, Stange KC. Understanding change in primary care practice using complexity theory. J Fam Pract. 1998;46(5):369–376.
    OpenUrlPubMed
  3. ↵
    Weiner JP, Starfield BH, Lieberman RN. Johns Hopkins Ambulatory Care Groups (ACGs). A case-mix system for UR, QA and capitation adjustment. HMO Pract. 1992;6(1):13–19.
    OpenUrlPubMed
  4. ↵
    Horn SD, Buckle JM, Carver CM. Ambulatory severity index: development of an ambulatory case mix system. J Ambul Care Manage. 1988;11(4):53–62.
    OpenUrlPubMed
  5. ↵
    Reid I. Complexity science. Let them eat complexity: the emperor’s new toolkit. BMJ. 2002;324(7330):171.
    OpenUrlFREE Full Text
PreviousNext
Back to top

In this issue

The Annals of Family Medicine: 8 (4)
The Annals of Family Medicine: 8 (4)
Vol. 8, Issue 4
1 Jul 2010
  • Table of Contents
  • Index by author
  • In Brief
Print
Download PDF
Article Alerts
Sign In to Email Alerts with your Email Address
Email Article

Thank you for your interest in spreading the word on Annals of Family Medicine.

NOTE: We only request your email address so that the person you are recommending the page to knows that you wanted them to see it, and that it is not junk mail. We do not capture any email address.

Enter multiple addresses on separate lines or separate them with commas.
The Implications of Measuring Complexity
(Your Name) has sent you a message from Annals of Family Medicine
(Your Name) thought you would like to see the Annals of Family Medicine web site.
CAPTCHA
This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.
12 + 2 =
Solve this simple math problem and enter the result. E.g. for 1+3, enter 4.
Citation Tools
The Implications of Measuring Complexity
Lee A. Green
The Annals of Family Medicine Jul 2010, 8 (4) 291-292; DOI: 10.1370/afm.1147

Citation Manager Formats

  • BibTeX
  • Bookends
  • EasyBib
  • EndNote (tagged)
  • EndNote 8 (xml)
  • Medlars
  • Mendeley
  • Papers
  • RefWorks Tagged
  • Ref Manager
  • RIS
  • Zotero
Get Permissions
Share
The Implications of Measuring Complexity
Lee A. Green
The Annals of Family Medicine Jul 2010, 8 (4) 291-292; DOI: 10.1370/afm.1147
Reddit logo Twitter logo Facebook logo Mendeley logo
  • Tweet Widget
  • Facebook Like
  • Google Plus One

Jump to section

  • Article
    • Footnotes
    • REFERENCES
  • Info & Metrics
  • eLetters
  • PDF

Related Articles

  • No related articles found.
  • PubMed
  • Google Scholar

Cited By...

  • Capturing complexity in clinician case-mix: classification system development using GP and physician associate data
  • The Challenges of Measuring, Improving, and Reporting Quality in Primary Care
  • Ethics, Support for Care, Prevention, and What's Important
  • Google Scholar

More in this TOC Section

  • Recruiting, Educating, and Taking Primary Care to Rural Communities
  • Returning to a Patient-Centered Approach in the Management of Hypothyroidism
  • An Opportunity to Emphasize Equity, Social Determinants, and Prevention in Primary Care
Show more Editorials

Similar Articles

Subjects

  • Methods:
    • Quantitative methods
  • Other research types:
    • Health policy
    • Health services
    • Professional practice

Content

  • Current Issue
  • Past Issues
  • Past Issues in Brief
  • Multimedia
  • Articles by Type
  • Articles by Subject
  • Multimedia
  • Supplements
  • Online First
  • Calls for Papers

Info for

  • Authors
  • Reviewers
  • Media
  • Job Seekers

Engage

  • E-mail Alerts
  • e-Letters (Comments)
  • RSS
  • Journal Club
  • Submit a Manuscript
  • Subscribe
  • Family Medicine Careers

About

  • About Us
  • Editorial Board & Staff
  • Sponsoring Organizations
  • Copyrights & Permissions
  • Contact Us
  • eLetter/Comments Policy

© 2023 Annals of Family Medicine