Abstract
PURPOSE Researchers often struggle to integrate quantitative and qualitative data. Joint displays of data collected using mixed methods provide a framework for supporting integration, yet the literature lacks methodologic articles illustrating in detail the iterative nature of constructing such displays. We demonstrate the process for creating a joint display for integrating the collection of data obtained by qualitative and quantitative methods.
METHODS Within a convergent mixed methods cohort study, the Early Discharge of Febrile Neutropenic Children with Cancer Study, we constructed a joint display to inform integrated collection of 2 forms of data (quantitative and qualitative) from 2 sources (a patient-caregiver mixed methods survey and a manual abstraction of medical records).
RESULTS In a first step, we used a data sources table to align related quantitative and qualitative data. The resulting table consisted of 2 side-by-side columns based on the mixed survey data. After several additional iterative steps, we constructed a final 6-column joint display. This final display delineated the separate data sources, linked constructs to the quantitative and qualitative variables within each source, and integrated the constructs across the separate data sources.
CONCLUSIONS Challenges of integration, though not unique to prospective mixed methods cohort studies, stem from the sheer volume of qualitative and quantitative information and the need to logically organize the data in preparation for integrated data analysis. Tailoring joint displays to specific studies is challenging, but mixed methods researchers who embrace the methodologic malleability can produce effective joint displays to illustrate the mixed data collection linkages and create a preliminary structure ultimately for organizing mixed data findings.
- febrile neutropenia
- mixed methods integration
- qualitative research
- quantitative research
- methodological studies
- data display
- pediatric cancer
- research design
- study planning
- supportive care
INTRODUCTION
Pivotal to mixed methods research is integrating the findings and conveying meta-inferences, that is, interpreting the mixed data findings.1 In less than a decade, the creation of joint displays has emerged as a powerful tool to plan integrated data collection, conduct mixed qualitative and quantitative data analyses,2 and illuminate, in final published form, the complex interactions, relationships, and merging of the qualitative and quantitative data.3 Miles and Huberman4 first described joint displays, alternatively termed joint matrix or integration displays,5 as overall visual presentations of merged quantitative and qualitative data. These tables or figures serve as powerful tools that enable investigators to convey the complex relationships between qualitative and quantitative data. The rigor of mixed methods research can be compromised, however, if researchers inadequately plan for integrated data collection, a situation that may result from knowledge gaps surrounding contemporary methodologic literature on mixed methods.6 Across disciplines, joint displays have proven to be an effective tool for various objectives in mixed methods integration.7-10
Joint Displays for Presenting Findings
Joint displays are largely known for their use in presenting findings, and since the emergence of joint displays in the mixed methods literature, several researchers have made important additions to the process.7,8,11-14 The joint display can give depth and magnitude of related findings and provide the key context for explaining the why and how of the quantitative or qualitative data.1,5 Joint displays of findings show quantitative and qualitative data alongside each other, merging, connecting, or building together these results.1,5 The most common joint display format is a side-by-side display of quantitative and qualitative data.9
Joint Displays for Analyzing Data
More recently, joint displays have been identified as tools for analyzing mixed data.15-17 When used for this objective, they facilitate discovery of previously unforeseen linkages between the quantitative and qualitative data.7,13 Plano Clark18 highlighted that an overlooked value of joint displays is the analytic and integrative thinking critical to creating the visual displays. In an example of the use of joint display analysis from empirical work to add technology in the classroom, Haynes-Brown and Fetters19 provided details on their process of mixed data analysis in an explanatory sequential mixed methods study design, showing how a novel joint display was built and interpreted. Fetters and Guetterman15 provided a detailed accounting of how they created a series of joint displays to optimize both understanding of the mixed data results and presentation of the mixed data findings.
Joint Displays for Integrating Data Collection
A third application of joint displays, use for integrated data collection, ironically originated chronologically earlier than the aforementioned applications that debuted first in the published literature.7 Joint displays of integrated data collection depict in a table or figure the linkage of qualitative and quantitative data collection across common constructs or domains. By linking in this way, investigators can prepare for an integrated mixed data analysis.
In a recent review, Guetterman et al9 systematically examined joint display features in 33 articles across disciplines. They reported health sciences were most likely to use joint displays, although other disciplines (eg, education, social sciences, psychology, nursing) also used them in their studies. They delineated the important attributes of successful joint displays of results; however, the recommended attributes for joint displays of data collection were only a minor focus. Although some researchers have alluded to joint displays of integrated data collection,7,10,17 Fetters and Tajima3 published the first article dedicated to describing these displays. Although recommending joint displays of integrated data collection as a highly effective, yet underused tool that researchers could use to organize, plan, and represent their mixed methods studies, the authors did not provide an example of how to build these displays.
With the growing use of joint displays for data collection and analysis among mixed methods researchers,10 detailed examples of constructing joint displays of integrated data collection are needed. In this article, we illustrate the iterative process of creating a joint display for integrated collection of mixed methods data using the example of a prospective cohort study of febrile neutropenia among pediatric oncology patients. We detail the methodologic approach taken through iterative steps, describe the construction of the joint display of integrated data collection for the study, and identify its features and their utility.
METHODS
Primary Study
Our primary study was the Early Discharge of Febrile Neutropenic Children With Cancer Study. Febrile neutropenia, defined as a fever exceeding 38°C for more than an hour or a single fever exceeding 38.3°C in the setting of severe neutropenia with a neutrophil count of less than 500/μL,20 is an emergency in the oncology population associated with increased risk for infection and associated complications.21 Although most pediatric oncology patients are seen frequently by their primary oncologist, many maintain continuity of care with their primary care physician, allowing for the opportunity for collaborative management and anticipatory guidance. The study was a prospective quality improvement study, aimed to implement early inpatient discharge among pediatric oncology patients with low-risk febrile neutropenia episodes using a clinical decision risk tool21 in combination with serial procalcitonin levels.22
The study followed a convergent mixed methods design, which features synchronous quantitative and qualitative data collection and analysis.23,24 During the planning phase, we developed a mixed methods data sources table, which identified 2 separate data sources in the study: a patient-caregiver mixed methods survey and the clinical course obtained by manual abstraction of the medical record (Supplemental Table 1). Ethical approval from the University of Michigan Institutional Research Board was received in November 2021.
Date Sources
Mixed Methods Survey
The primary aim of the patient-caregiver mixed methods survey was to engage the patient-caregiver dyads in the quality improvement initiative by assessing quality of life and patient-reported outcomes associated with febrile neutropenia. The questionnaire used was a modified version of an original instrument,25 initially used to compare the quality of life measures of patient-caregiver dyads receiving inpatient or outpatient therapy for febrile neutropenia.26 We modified the quantitative questions with a Likert scale of “far below average” to “far above average” and expanded the abbreviated question items to the full questionnaire item. Qualitative questions were added to elucidate data that would expand on the constructs synergistically with the quantitative questions, and numerically organized within the instrument in close proximity to the corresponding quantitative question. The Cronbach α reliability coefficient was 0.782, confirming adequate internal reliability.27
Medical Record Abstraction
In addition to collecting information on febrile neutropenia episodes with the mixed methods survey, we performed a manual medical record abstraction of all episodes that met inclusion criteria for analysis. The aim of collecting these clinical data was to compare febrile neutropenia episodes by their risk assignment, assessing overall safety of early hospital discharge in the cases of low-risk episodes. This abstraction similarly yielded both quantitative and qualitative data.
Planning for Integration of Study Data
Considerable volumes of quantitative and qualitative data would be collected from both the survey and the medical record abstraction. We thus identified the need to better organize these data sources to link the data collection procedures. This approach would enable us to merge the 2 forms of data (quantitative and qualitative) based on matching the constructs held in common. We chose to develop a joint display of data collection based on existing literature3,17 but soon recognized construction of this table would require multiple iterations to fully organize integrated data collection. We ultimately used a methodologic process with 4 iterations to develop the final joint display of integrated data collection. During this process, the named descriptions of each data source (the survey and medical record abstraction) evolved and became more refined. For the purpose of clarity, however, we maintain the original names here.
RESULTS
Methodologic Approach
Our study team identified quantitative and qualitative variables within both the patient-caregiver survey and the clinical data from the manual abstraction of medical records. Following procedures illustrated by Fetters,17 we constructed a mixed methods data sources table (Supplemental Table 1), the imperative primary step that stimulated the subsequent iterative process. We then developed, reformatted, and refined our joint display through 4 iterations, ultimately arriving at the display used in the study.
Iteration 1: Developing a 2-Column Joint Display
Process
Building on the mixed methods data sources table, we constructed a side-by-side structured joint display table with 2 columns, separating the quantitative survey questions with a Likert scale from the qualitative open-ended survey questions (Table 1). Abbreviated question items from the survey were added for clarity and categorized by construct of best fit. A theoretical framework for the study, informed by the patient-caregiver survey, identified 6 constructs for the study: role functioning, emotional functioning, cognitive functioning, social functioning, symptoms, and global functioning.28
Iteration 1: Developing a 2-Column Joint Display
Impressions
This first iteration clearly communicated the appropriate constructs for the quantitative and qualitative variables found within the patient-caregiver survey. At this critical junction, we compared this first iteration (Table 1) with the mixed methods data sources table (Supplemental Table 1) and noted the need for greater comprehensiveness; data from the second source, the medical record abstraction, were not included. We subsequently began to iteratively reorganize the display to comprehensively integrate the quantitative and qualitative variables from both sources. We recognized the value of this joint display; in the mixed methods analysis, it could assist us in eventually linking related findings across related constructs and identifying meta-inferences from the 2 data sources.
On a reflective note, one author with limited experience in mixed methods research (C.N.N.) found it challenging to learn the basic development of a joint display table and felt bound by implied restrictive descriptions of developing such displays. Similarly, another author (L.V.G.), a clinician (nurse) and researcher, although trained in mixed methods designs during her dissertation work, felt uncertain about the process because of the lack of literature depicting illustrations of joint displays using multiple data sources.
Iteration 2: Incorporating Variables From Record Abstraction
Process
The second iteration (Table 2) represented our first attempt to incorporate the quantitative and qualitative clinical course variables from the medical records in the established 2-column, joint display table. This display proved difficult to organize clearly because of the large volume of information and complexity. It was not obvious how to integrate the clinical course variables. We shaded the constructs and itemized the variables to improve organization, but the resulting display still fell short of conveying the fact that the data would originate from distinct sources.
Iteration 2: Incorporating Variables From Record Abstraction
Impressions
Although the overall structure and organization were consistent with those of the prior iteration, this second iteration (Table 2) now included the quantitative and qualitative variables from both the patient-caregiver survey and the medical record abstraction. Yet, this iteration did not succeed in depicting how mixed methods data in the study would arise from 2 separate sources. The constructs and linkage of each construct to both the quantitative and qualitative variables were not clear. Additionally, this format felt somewhat chaotic, and the patient-caregiver survey items were not apparent in the new depictions of the list of variables.
After this second iteration of the joint display table, we realized the established 2-column format of the joint display of data collection for the study would not sufficiently portray that the data came from separate sources. We reflected on a critical question at this juncture: were joint display table structures malleable to be different, or would it be more effective to have separate tables for each data source? The junior investigator for the febrile neutropenia study (C.N.N.) was uncertain about the iterative process in joint display table development and was concerned about achieving a joint display that could be used for the study.
Iteration 3: Expanding the Joint Display by Data Sources
Process
Building on the second iteration, which incorporated clinical course variables abstracted from medical records, the third iteration (Table 3) focused on separating the data sources: the patient-caregiver survey and the medical record abstraction. To improve the organization while retaining the construct rows, we added 2 columns to include the quantitative and qualitative variables from each source; for example, quantitative and qualitative survey questions were now in unique columns. We also added a clear title to the table to appropriately describe the integration of variables from different sources, changing the shading with formatting edits.
Iteration 3: Expanding the Joint Display by Data Sources
Although the quantitative and qualitative variables from each source were now in unique columns, the separate data sources between the patient-caregiver survey and the medical record abstraction were not well delineated. Labeling the columns with respective names helped indicate different data sources in the fully integrated table. As the table now was wider, with 2 additional columns, the linking of constructs was again lost across the table rows; therefore, we added another column on the left of the table to specify the construct found in each row. This change highlighted the integrated organization of variables within the individual constructs.
Impressions
The revised 5-column third iteration (Table 3) showed the various constructs as they related to the unique quantitative and qualitative variables from each source and seemed to better delineate between the data sources. Although the theoretical framework for the study was informed by the survey instrument, a focus of this iteration was to show the relationship of a construct to both types of data within both sources. This iteration also demarcated the integration of the mixed methods variables across each source. Despite the improved organization with the additional columns, the abbreviated survey question items now did not appear to provide enough information to justify the organizational rationale, and the clinical course variables from the medical records were not organized clearly within each construct row.
Although this iteration further improved on prior versions, with additional structuring and labels, additional organizational restructuring was needed. The desired message of the joint display table—how mixed methods data arising from separate data sources would be related—seemed to be clearer with this iteration than with previous iterations. There were, however, no comparable examples in the literature to affirm the adequacy of this structure. This version was shared with senior authors (S.W.C. and M.D.F.) for advice and clarity around a question that arose: would it be acceptable and effective to add more columns? Although the team was aided by senior mentorship, they had a persistent feeling that the expanded iteration of the joint display table was still incomplete.
Iteration 4: Refining Rationale and Organization in the Joint Display
Process
The focus of the fourth and final iteration of the joint display table of integrated data collection (Table 4) was to enhance the organizational rationale of the mixed methods variables and respective fit within each construct. This focus led to renaming and simplifying the column headers to communicate that the variables were clearly arising from separate sources: the patient-caregiver survey and the medical record abstraction. We removed abbreviations and short-hand terms from the clinical course variables to eliminate excessive detail from the abstraction and thereby better balance conciseness and text clarity.
Iteration 4: Refining Rationale and Organization in the Joint Display
We incorporated a sixth column on the right of the table that included example questions for each construct to explain the organizational structure and potential questions that could be answered with the mixed methods analysis. An example question within the role functioning construct was, “Do patient-caregiver dyads with shorter hospital length of stay report improved role functioning?” We expanded the abbreviated question items to the full, numbered items to further improve transparency for the rationale of the variables from separate data sources within each construct. This iteration would readily allow linking of sources after completing data collection, in anticipation of an integrated mixed methods analysis and result reporting. Finally, we italicized survey questions focusing on patients to distinguish them from those focusing on caregivers.
Impressions
This final iteration (Table 4) clearly communicated 2 central aspects of the study by conveying the primary linkage of the mixed data collection—how and why mixed methods variables from separate sources were related. It clarified that the study collected 2 types of data (quantitative and qualitative) from 2 separate sources by arranging the type from each source in a unique column. Additionally, this joint display table emphasized the organizational rationale of the variables across each construct by using the full survey items and example questions. Although it was somewhat arbitrary, we agreed this final iteration concisely and comprehensively described the data to be collected within the study. The end point for desisting from additional iterations was consensus that further organizational restructuring would not improve the depiction of integrated data collection.
DISCUSSION
As illustrated by our example, creating a joint display table of integrated data collection can entail an iterative process. Drawing from our experience, we created a series of steps for researchers to consider when creating a joint display of integrated data collection (Table 5). Through several iterations, we conveyed how quantitative and qualitative data collection from 2 separate data sources would be linked and provided a simple rationale for its organization. The primary step in developing the joint display of integrated data collection was to construct a comprehensive mixed methods data sources table (Supplemental Table 1). This process allowed the study team to clearly distinguish between the data sources and identify the qualitative and quantitative variables within each source, and eventually stimulated the iterative process to merge both data sources in the joint display. To clearly delineate that the variables arose from 2 separate sources, the sources and their respective mixed variables were placed in separate, unique columns in the joint display table, as this parallel arrangement was clearest for our group.
Steps for Creating a Joint Display of Integrated Data Collection
The next step in organizing the integrated joint display table incorporated the constructs from the theoretical framework used in the study analysis and assigned each variable to the most appropriate construct row. After the quantitative and qualitative variables were assigned to the construct of best fit, the variables could be arranged by construct within each data source column (ie, survey or record abstraction) in a manner that also effectively maintained a clear delineation between sources. The final step featured examples of questions to explain the organizational rationale of the constructs, mixed variables, and separate sources (ie, the variables within each source and the variables within specific constructs). This final column of Organizational Rationale allowed us to anticipate key linkages within each construct of the theoretical framework to consider at data analysis, after collection.
Methodologic Insight
This article expands the current mixed methods research literature in several ways. First, we provide a novel iteration of a joint display of integrated data collection that depicts how collection of data was planned from 2 related but separate sources (a patient-caregiver mixed methods survey and a manual abstraction of medical records) for a prospective cohort study. Second, this illustration highlights how a joint display of data collection lays the groundwork for a joint display analysis, and ultimately, a joint display of integrated findings.17 The theoretical framework and creation of the joint display of integrated data collection was a powerful process for anticipating and organizing the mixed methods analysis before data collection. For example, the joint display and the rationale provided within the role functioning construct informed the coding process, identified the mixed variables from each data source, and aided in identifying a major theme of “treatment location.” Third, although joint displays of data collection are used in other study designs,3,10 this example is the first illustration of use in a prospective cohort study, a common design in the health sciences. Fourth, our experience illustrates the malleability of an approach that began with a basic mixed methods data sources table and subsequently evolved into a sophisticated structure for conceptualizing many diverse sources of data. Future research can explore how our display arising from a convergent mixed methods design may be further adapted.
Finally, as a reflective insight of the iterative process, the junior investigators of the team were novices in the process, whereas the senior team members remained confident of an eventual outcome (ie, the development of a final joint display of integrated data collection both comprehensive and clear enough to be used in the study). Each step was incremental and reactive to the iterative changes, much as was the process reported by Haynes-Brown and Fetters19 in their creation of a joint display of integrated data findings. We hope this methodologic description will stimulate investigators to expand on the approach and to adapt or develop additional visual joint display structures that can be applied to other designs, such as the sequential exploratory and explanatory mixed methods designs, further described by others.3,9,16
Conclusions
Joint displays of integrated mixed methods data collection can convey complex relationships between quantitative and qualitative data. The joint display fully links the organization of the mixed data collection and lays the groundwork for subsequently conducting a mixed methods analysis conducive to producing meta-inferences. We hope our illustration will stimulate mixed methods researchers conducting prospective cohort studies with multiple mixed data sources to use similar joint displays of integrated data collection. We highlight the malleability of the process and encourage other researchers to creatively expand joint display structures to maximize their utility, not only for mixed methods cohort studies, but for other study designs as well.
Footnotes
Conflicts of interest: authors report none.
Funding support: Drs Nessle and Ghazal are supported by grant T32-CA236621, Interdisciplinary Research Training Center in Cancer Care Delivery, from the National Cancer Institute (NCI) of the National Institutes of Health (NIH). Dr Nessle is also supported by the Fogarty International Center and the NCI of the NIH under grant 3D43TW009345-11S4 awarded to the Northern Pacific Global Health Fellows Program. Dr Choi is supported by grant K24HL156896, which supports mentoring in patient-oriented research. Dr Fetters’ participation was made possible through support of the Mixed Methods Program, Department of Family Medicine, University of Michigan.
Disclaimer: The views expressed are solely those of the authors and do not necessarily represent official views of the authors’ affiliated institutions or funders.
- Received for publication August 29, 2022.
- Revision received March 3, 2023.
- Accepted for publication March 28, 2023.
- © 2023 Annals of Family Medicine, Inc.
