Measures

Charts were reviewed both 1 week and 4 weeks post appointment. Charts were reviewed by a member of the team who had received training and quality control checks on ACP chart review protocol. Chart review findings were spot checked by a second member of the team (one every 20 records) to ensure accuracy. The protocol outlined that training and education interventions would be used to respond to discrepancies in chart review rates and quality assessments. However, interventions were not needed because spot checking did not yield discrepancies. The participant’s demographics, presence of ACP (including before and after the visit), quality of ACP if present, and number of MyChart messages sent in the last year were recorded. The intervention charts were reviewed to see if the patient had read the intervention on their portal and responded to any of the questions.

Quality of ACP was measured using a 20-point scoring criteria entitled “Criteria for Scoring Quality of ACP Documentation” (Multimedia Appendix 1) that has been used in previous studies [11,27]. The rubric was created by our team after reviewing best practice reports on how to measure ACP quality, such as those from the United Kingdom National Health Service, the Australian Quality Advance Care Planning Board, and the National Hospice and Palliative Care Organization’s “Caring Connections” program [29-31]. This method of scoring the quality of ACP has not been validated, but assigned points for the presence of items in the patient’s ACP. The ability to quantify the quality of our patient’s ACP was a crucial component of the study and gave us more thorough feedback on the utility of the novel framework.