Real world data:
Best practices for clinical applications and evidence generation
Introduction
The wealth of real world data and evidence holds huge potential for hospitals and health systems to engage in clinical research. When exploring these new opportunities, many questions emerge about how to establish and execute projects involving RWD. Based on our expertise and experience with RWD, both internally and in collaboration with our 1,200 hospital partners, Q-Centrix has identified several best practices for getting started with the RWD captured in the clinic.
The massive opportunity presented by RWD may initially spark excitement for health care systems to participate, but the lack of a measured strategy outlining where to leverage data against which use cases can create false starts and skepticism. A system- or site-wide review is recommended as a first step to determine whether the appetite for the opportunity is sufficient. Then, create a framework for success by defining the project scope and workflow to ensure that the essential components of an RWD model are addressed. Once established, the teams with the appropriate resources to execute the workflow should be identified and engaged. Finally, the recommended best practice for the manual curation of unstructured data is that it be handled exclusively by individuals with the expertise and experience to safeguard data integrity to ensure the desired outcomes are achieved. Use cases for the RWD generated within the hospital can come from many departments, and organizational structure and preferences can vary significantly, so flexibility is key.
Setting the framework
The design of the overall workflow should live with the leaders of the organization. It takes leadership to set up and facilitate the workflow, provide access to technology platforms, and establish consistent procedures and standards across the organization to foster the curation of RWD. The workflow should be informed by the research hypothesis or desired process improvement that drives the effort, with the elements of the workflow defined by the use case.
To build out a workflow as efficiently as possible, many hospitals and health systems bring in third-party partners with the agility and expertise to get a project off the ground quickly. While use cases and research questions across projects may be different, partnering with an organization that has experience in managing these collaborations can help keep multiple cycles running smoothly while providing consistent expertise between departments on data best practices and cross-team organizational goals. Q-Centrix has found that the work driven by quality departments to support registry submissions often aligns with broader organizational objectives on using the data for other purposes.
Components of an ideal RWD model
Four essential components should be integrated into an RWD research project: stakeholders, data sources, technology, and outcomes.
Stakeholders
Stakeholders in an RWD model can include clinicians, researchers, operations, and other personnel within the hospital system. Before embarking on an RWD project, identify who is already involved in the partnership and who else might need to be engaged. Encourage stakeholders to coordinate their efforts in order to fully leverage their existing expertise. Patients should also be included as stakeholders in research projects; in many cases, patient advisory committees comprising patients from diverse backgrounds and geographical areas serve this purpose. Transparency and security are crucial to facilitate patient acceptance of the use of RWD in research initiatives. Health care systems should strive to ensure patients are aware of the project goals, the types of data and collection methods that will be used and their purpose, how their privacy will be protected, and how they will be informed about the usage of their data and the project outcomes. Finally, stakeholders can also include external partners that can offer access to necessary data.
Data sources
Potential data sources are numerous and include clinical, electronic health, or medical records (including various silos of unstructured data like pathology reports, prior clinical records, imaging reports, etc.), case report forms, study or clinical trial protocols, retrospective chart reviews, observational data, registries (which have up to 80% data overlap with primary data sets used for clinical trials), patient-generated data, patient-reported outcomes, health care cost and utilization data, and public health data. At the start of an RWD project, identify the sources that are relevant, ensuring they are trusted and prioritize data integrity, and determine how the data will be accessed and enriched, if needed. Clinical data, as it exists in queryable form within a hospital, is not ready for most use cases without further cleansing and expert-driven abstraction.
Technology
Where to house the data is a key question for RWD-driven research projects. When possible, use centralized platforms designed to support the aggregation of data, to eliminate manual work and duplication of effort, and to provide consistency among data elements. Such platforms support streamlined reporting practices that lead to actionable data. Centralized platforms that are easy to use and access can act as a consistent baseline for each project, reducing manual labor and bringing continuity across projects.
Outcomes
Identify the outcomes that are relevant to the effort. These can include internal metrics, compliance with guidelines or standards of accreditation or regulatory bodies, quality measures, health status, and study endpoints. Enhanced delivery of care to patients, streamlined internal workflows, and more publications are often among the desired outcomes.
Executing the workflow
Implementing a successful RWD workflow requires the right expertise and resources. Operationalizing the workflow can involve either a quality department or a specialty department within the system, depending on the research project and the organization’s structure. Whichever department ultimately executes this process should have full access to and an enterprise view of the data to achieve the goal of using RWD in experiments, trials, and other applications.
In centralized systems with demonstrable data integrity, quality departments may also be engaged in the workflow. These teams, or individuals, typically have access to the appropriate processes and resources to curate the required data. Quality is a broad term in the hospital setting. Clinicians who are tasked with ensuring data quality can be single individuals or teams; often, they are specifically focused ,on quality as it relates to infection prevention, emergency readiness, or accreditation and are embedded in the respective departments. Not all specialty areas of a hospital have the capacity or operating procedures to support RWD efforts, and some may be motivated by the need to publish, which could bias the data.
Curating the data
Many different types of experts may be part of the curation process including clinicians, research professionals, quality managers, and clinical data analysts. These individuals must safeguard and maintain data integrity and therefore require a high level of training and great deal of expertise. Analysts and biostatisticians may review data globally and serve as ‘checks and balances’ to ensure data fidelity and reliability. This layer of review can act as a fail-safe to prevent bias in the data, particularly when it is ultimately used for outcomes reporting or publication.
Generally, hospitals struggle to curate data at scale for research initiatives due to several different factors: lack of dedicated clinical talent, burden on existing clinical patient-focused, staff, inability to hire dedicated staff, absence of dedicated research project management, and lack of data management software (including the associated quality assurance and inter-rater reliability necessary). Leveraging registry data, which is already abstracted and can constitute up to 80% of the required information for clinical data sets, can significantly reduce these resource burdens. Outsourcing this work to expert third parties can further mitigate many of these challenges.
Data integrity
Data integrity is fundamental to the use of RWD in research and refers to data accuracy, completeness, timeliness, and consistency. Many accrediting bodies require a data accuracy of 90–95%. Of course, institutions desire to be as close to 100% as possible but might not always have sufficient control or influence over recruitment, training, quality reviews, and other aspects that impact their ability to reach this number. Regulations change rapidly, and staff members should keep up to date on current guidelines, standards, and requirements. Frequent training sessions can keep their knowledge current and bring new team members up to speed. Inter-rater reliability programs should be in place as well. Third-party partners with expertise in data collection, validation, and management can help to identify and correct any inconsistencies before they are found during audits and to ensure the data is free of bias.
Hospital system data collection programs were not historically designed to leverage data for clinical purposes, as it was not always linked to reimbursement. Engaging a third-party partner with data management expertise is a good option for systems without structured internal data integrity programs.
Summary of best practices
To summarize, when launching a RWD project, executive leaders, together with their research partners, should create a framework that fully addresses the stakeholders, data sources, technology, and outcomes of the research. Quality departments may already have portions of the workflow covered from the hospital’s registry submission procedures, bringing the expertise and confidence to ensure data integrity. Third-party partners with exceptional agility, experience in data management, and analysis skills can facilitate both the engagement between the hospital and research entity and the curation of data to make the most of RWD.
RWD in action
A project involving a national database exploring the effects of a common surgical procedure faced a major challenge, in that the data was distributed across multiple repositories and could not be readily analyzed or reported. Asking individual hospitals to report out was infeasible, as it would place a heavy burden on their quality departments. To solve this challenge, Q-Centrix developed a large cross-functional data collection methodology to allow for easy analysis and reporting, ensure removal of bias, and provide a data integrity model that instilled confidence.
Conclusion
There is huge potential in RWD, and its role will continue to grow and evolve as systems explore its applications in planning strategy and improving patient outcomes. Robust management and analysis as well as data integrity will be key to unlocking the full value of RWD. Using RWD in clinical research is new territory for many hospitals and health systems, and the nature of research projects requires both expertise and agility. Third-party partners can provide both, helping hospitals and health systems to get the most out of their RWD.