Outcomes and measures

Due to the intervention’s multiple components, ranging from aspects of clinical effectiveness (management of HTN in CKD patients) to usability principles and strategies (PCP adherence to CDS guidelines), outcomes and measures will be guided by the RE-AIM framework.

Reach will refer to the overall use of the CKD CDS, including the number of PCPs and patients for whom it fires. To assess the reach of the CDS, statistics on the quantity and types of firings will be collected through enterprise data warehouse and Epic queries. Concurrent manual review of Epic reports and chart review on CDS firing statistics will be conducted by team members to verify the automated monthly summaries. Analytic variables will include the percentage and types of clinicians in primary care who use the software, descriptions of excluded clinicians, PCP review and/or response to pledge email, PCP interaction with the CDS, signing of orders or accountable justification documentation within the CDS and whether the BPA fired appropriately during encounter.

Effectiveness will refer to the clinical efficacy (process and outcome measures), usability of the software in the primary care environment, process measures and both positive and negative unanticipated consequences. In evaluating effectiveness, the primary endpoint is the change in mean SBP between baseline and 6 months compared across arms. This outcome was chosen as the primary outcome because of the growing need in primary care to monitor patients’ HTN in order to help mitigate negative long-term outcomes of CKD such as kidney failure, cardiovascular events and death. A meta-analysis of three large cohorts of CKD patients without diabetes concluded that maintaining blood pressure below 140/90 mm Hg decreases risk of these outcomes significantly.²¹ Several guidelines have been issued to emphasise the importance of HTN control in CKD.^22–24 Additional secondary outcomes are listed in tables 1 and 2.

Outcome variables and measures for both arms

BPA, Best Practices Advisories; eGFR, estimated glomerular filtration rate; SBP, systolic blood pressure.

Outcome variables and measures for intervention arm only

PCP, Primary Care Practice.

Adoption will refer to the percentage and types of settings and staff that embrace the innovation. We will analyse the various ways in which the intervention PCPs accept, reject and generally interact with the BPA. Using a BPA report extracted through Epic, we will collect which CDS fired, the date and time of the firing, the medical records number (MRN) and demographic information of the patient on whom it fired, and the user (PCP). We will also extract the user follow-up action (whether the PCP indicated that they would order a medication or basic metabolic panel through the BPA) or which acknowledge reason the PCP chose. If ‘other’ is indicated as an acknowledge reason, the PCP will be asked to elaborate with a comment. Finally, in conducting our manual chart review, we will record whether the BPA firing for the patient was appropriate and whether the course of action that the PCP indicated they would take through the BPA differed from the course that they actually took.

Implementation will refer to the consistency of CDS use, any support resources used, any barriers and/or enabling factors that are identified, any workarounds to barriers that develop, any changes from preintervention to intervention period and any unintended consequences to patient safety or workflows. Prior to the clinical trial, a pilot study was conducted in live clinical settings. The BPAs were turned on for approximately 2 weeks for all intervention clinicians, and the patients for whom the BPAs fire during this period were classified as ‘Pilot Patients’. Interaction with the BPAs was monitored. The first time a BPA fired for a PCP, the research team contacted the PCP by email to gather feedback through a survey or an interview. The experiences of these PCPs were noted, but no changes were made to the BPA. However, some early workarounds that were discovered included selecting the accountable justification ‘other’ without a valid reason (eg, ‘x’) in order to circumvent the BPA, as well as ordering a medication or panel without completing the order by signing off on it.

Maintenance will refer to how well the innovation components and their effects are sustained, as well as any strategies that are used to uphold the intervention over time. This will be recorded by qualitative descriptions of system performance longitudinally, emerging workflow changes and long-term unintended consequences, and how the BPA fit into the existing PCP workflow. The methods used will include contextual inquiry sessions, interviews and surveys.