You are here

Article Text

Article menu
Renal medicine
Original research
Global use of electronic patient-reported outcome systems in nephrology: a mixed methods study
  1. Nicola Elizabeth Anderson1,2,3,
  2. Derek Kyte1,4,
  3. Christel McMullan1,5,6,
  4. Paul Cockwell1,7,
  5. Olalekan Lee Aiyegbusi1,3,6,8,9,
  6. Rav Verdi10,
  7. Melanie Calvert1,3,5,6,8,9
  1. 1Institute of Applied Heath Research, Centre for Patient Reported Outcomes Research, University of Birmingham, Birmingham, UK
  2. 2Research, Development and Innovation, University Hospitals Birmingham NHS Foundation Trust, Birmingham, UK
  3. 3NIHR Applied Research Collaboration, West Midlands, University of Birmingham, Birmingham, UK
  4. 4School of Allied Health and Community, University of Worcester, Worcester, UK
  5. 5NIHR SRMRC, University Hospitals Birmingham NHS Foundation Trust and University of Birmingham, Birmingham, UK
  6. 6NIHR Blood and Transplant Research Unit (BTRU) in Precision Transplant and Cellular Therapeutics, University of Birmingham, Birmingham, UK
  7. 7Department of Renal Medicine, University Hospitals Birmingham NHS Foundation Trust, Birmingham, UK
  8. 8NIHR Birmingham Biomedical Research Centre, University of Birmingham and University Hospitals Birmingham NHS Foundation Trust, University of Birmingham, Birmingham, UK
  9. 9Birmingham Health Partners Centre for Regulatory Science and Innovation, University of Birmingham, Birmingham, UK
  10. 10Patient Partner, Institute of Applied Health Research,Centre for Patient-Reported Outcomes Research (CPROR), University of Birmingham, Birmingham, UK
  1. Correspondence to Dr Nicola Elizabeth Anderson; n.e.anderson.1{at}bham.ac.uk

Abstract

Objectives The use of electronic patient-reported outcome (ePRO) systems to support the management of patients with chronic kidney disease is increasing. This mixed-methods study aimed to comprehensively identify existing and developing ePRO systems, used in nephrology settings globally, ascertaining key characteristics and factors for successful implementation.

Study design ePRO systems and developers were identified through a scoping review of the literature and contact with field experts. Developers were invited to participate in a structured survey, to summarise key system characteristics including: (1) system objectives, (2) population, (3) PRO measures used, (4) level of automation, (5) reporting, (6) integration into workflow and (7) links to electronic health records/national registries. Subsequent semistructured interviews were conducted to explore responses.

Setting and participants Eligible systems included those being developed or used in nephrology settings to assess ePROs and summarise results to care providers. System developers included those with a key responsibility for aspects of the design, development or implementation of an eligible system.

Analytical approach Structured survey data were summarised using descriptive statistics. Interview transcripts were analysed using Codebook Thematic Analysis using domains from the Consolidated Framework for Implementation Research.

Results Fifteen unique ePRO systems were identified across seven countries; 10 system developers completed the structured survey and 7 participated in semistructured interviews. Despite system heterogeneity, reported features required for effective implementation included early and sustained patient involvement, clinician champions and expanding existing electronic platforms to integrate ePROs. Systems demonstrated several common features, with the majority being implemented within research settings, thereby affecting system implementation readiness for real-world application.

Conclusions There has been considerable research investment in ePRO systems. The findings of this study outline key system features and factors to support the successful implementation of ePROs in routine kidney care.Cite Now

  • telemedicine
  • dialysis
  • quality of life
  • registries
  • patient reported outcome measures
  • qualitative research

Data availability statement

Data are available upon reasonable request. De identified data may be available upon reasonable request via the corresponding author (ORCID 0000-0002-0614-3198).

https://creativecommons.org/licenses/by/4.0/

This is an open access article distributed in accordance with the Creative Commons Attribution 4.0 Unported (CC BY 4.0) license, which permits others to copy, redistribute, remix, transform and build upon this work for any purpose, provided the original work is properly cited, a link to the licence is given, and indication of whether changes were made. See: https://creativecommons.org/licenses/by/4.0/.

https://doi.org/10.1136/bmjopen-2022-070927

Statistics from Altmetric.com

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

STRENGTHS AND LIMITATIONS OF THIS STUDY

  • The mixed-methods approach offered a greater understanding of the barriers and facilitators of electronic patient-reported outcome (ePROs) implementation, supplementing survey findings on key system characteristics with in-depth interview data from ePROs system developers.

  • The electronic survey and interview topic guide were developed with input from stakeholders, including clinicians, PROs methodologists and patients.

  • The review was restricted to English-language publications of ePRO systems. Therefore, it is possible that potentially relevant publications published in other languages were excluded.

  • Purposive sampling methods were used due to the limited numbers of people who could act as a primary data source, that is, survey and interview participants could have a clinical or non-clinical role but must have carried key responsibility for aspects of design, development or implementation of an eligible ePRO system.

  • It is possible that other ePRO systems that could be used in nephrology settings were not identified by this study.

Introduction

Patients at all stages of chronic kidney disease (CKD), and particularly those undertaking kidney replacement therapy, experience a high symptom burden and often report diminished health-related quality of life (HRQOL).1–3 However, some symptoms remain under-recognised and unalleviated, leading to increased patient burden.4 HRQOL is not only an important outcome in itself but also associated with clinical outcomes such as healthcare utilisation and mortality.5 Symptom burden and impact on HRQOL can be assessed using patient-reported outcomes (PROs), defined as ‘any report of the status of a patient’s health condition that comes directly from the patient, without interpretation of the patient’s response by a clinician or anyone’.6 PROs assessing the impact of CKD and associated treatments can be collected electronically (electronic PROs; ePROs).

With the increase in telemedicine, accelerated by the COVID-19 pandemic,7 8 the collection of ePROs has increasing significance. Web-based systems and ‘bring your own device’ schemes offer patients new ways to report symptoms, function and HRQOL; they can report PROs in ‘real-time’ from home or in clinic prior to their consultation. Such use could assist in the management of symptoms, while ensuring that healthcare resources are used effectively to maximise impact.9

There is increasing evidence, particularly from oncology, that ePROs are cost-effective,10 can facilitate shared decision-making, promote self-management, improve symptoms and HRQOL, reduce unplanned hospitalisation and enhance long-term outcomes including survival.11–14 In contrast to paper-based collection, ePRO systems offer enhanced ease of use and reduced burden, increased user satisfaction (patient and clinician) and lower rates of missing dataf.15–18

Evidence around implementation of ePROs in nephrology is growing. Studies from North America demonstrate the feasibility of ePRO collection in haemodialysis (HD) settings,16 19 20 while the Australian Symptom monitoring With Feedback Trial21 and Canadian EMPATHY studies22 are cluster randomised controlled trials exploring the usefulness of integrating PRO assessments in the clinical management of patients receiving HD. In advanced CKD populations, the use of an electronic Patient-Reported Outcome Measure in the Management of Patients with Advanced CKD (RePROM) study in the UK piloted the use, and established feasibility, of an ePRO for remote symptom monitoring in real time.9 23 The AmbuFlex telepro system is used in practice to manage renal outpatient follow-up in Denmark and as a complimentary tool in consultations24 25 and is currently being evaluated through the PRO-KID trial: a non-inferiority pragmatic randomised controlled trial investigating the effectiveness of the quality of care, use of resources and patient outcomes associated with PRO-based follow-up in patients with CKD.24

With this growth of ePRO system research and implementation in CKD globally, there is an opportunity to explore the key characteristics associated with successful implementation to support widespread adoption. The objectives of this study were to1 comprehensively identify ePRO systems designed specifically for use with CKD populations, including those under development,2 to explore system characteristics, including methods of administration, levels of integration into existing workflow and electronic health records (EHR) and the reporting of ePROs to manage patient safety.

The aim is being to provide a comprehensive classification of core factors, which contribute to successful implementation of ePROs in nephrology, including common facilitators and barriers.

Study design

Methods

Predicated on the research paradigm of pragmatism, with a focus on analysis of study data through the lens of its practical consequences and actionable knowledge,26 this three-phased study used mixed methods:

Phase 1 comprised a scoping review identifying ePRO systems and corresponding developers.27 Eligible systems were those being developed, used or under study that were designed for or reconfigured specifically for use in adult nephrology settings but excluding those developed for the management of acute kidney injury and paediatric CKD populations. Systems were also excluded if PROs were not assessed electronically and did not provide a summary of the patients’ responses to their care provider for use in routine care, that is, were being used solely to collect research or population-level data or to assess the effectiveness of an intervention.

Eligible systems from any country were included. A system developer could have a clinical or non-clinical role but must have carried key responsibility for aspects of design, development or implementation of an eligible ePRO system.

Systems were identified through publications in English, including conference abstracts and grey literature; adapting a previously applied search strategy is used to investigate nephrology PRO measures.28 Databases (OVID Medline, EMBASE, APA Psychinfo and CINAHL) were searched from dates of inception to 15 December 2021 (see online supplemental file 1, for example, search strategy) followed by ongoing ‘pearl’ and ‘snowballing’ methodology, that is, searching from known key references and checking reference lists.29–31 Field experts were also consulted. Data from identified publications were used to create a structured survey exploring key system characteristics (see online supplemental file 2), and to identify system developers.

Phase 2 involved survey administration via an online survey platform Smart Survey (www.smartsurvey.co.uk), which was piloted prior to use. System developers were invited to participate via email. The survey focused on system design and software features, integration of e-PRO collection and reporting in clinical care.

Phase 3: optional follow-up semistructured interviews were undertaken online via videoconferencing software with survey participants, or a suitable team member nominated by the original survey respondent. The purpose of these interviews was to expand on survey responses and allow more detailed system/organisational enquiry. All follow-up interviews used a piloted study-specific topic guide (see online supplemental file 3). Participants were given the opportunity to check and review transcripts, to comment and avoid disclosure of any proprietary information. Interviews were conducted by NEA, a renal research nurse with previous experience of conducting qualitative research.32 This study was undertaken as part of a Doctoral Research Fellowship, which was disclosed in the participant information sheet and during the consent process. Some interview participants were previously known to the interviewer, as experts in the field of study. Maintenance of a reflective diary, memos and field notes, along with discussions with the study management team, were used to minimise the influence of any prior relationships on data analysis.

Analysis

Data derived from the phase 1 review were abstracted and charted (see table 1), and descriptive statistics from the phase 2 structured survey were tabulated (see tables 2–4). Analysis of phase 3 interview data was undertaken using Codebook Thematic Analysis33 34 and the domains from the Consolidated Framework for Implementation Research (CFIR) (figure 1). Primary data analysis was conducted by lead author (NEA) with a second investigator reviewing coding (CM) for consistency and appropriateness. The CFIR is a determinant framework that can be used to identify and delineate contextual factors (ie, barriers or facilitators) that influence the outcome of implementation efforts.35 36 Computer-Assisted Qualitative Data Analysis Software CASDAQ (QSR NVIVO V.1237) was used to facilitate qualitative data analysis. Thematic analysis using a framework approach was chosen to systematically identify and analyse patterns of meaning within data, with the aim of highlighting the most salient features.33 The CFIR was used to gain insight into the overall effectiveness of the ePRO systems and associated implementation strategies. It is possible to rate constructs using the CFIR to undertake organisational comparison via Qualitative Comparative Analysis.38 Due to the heterogeneity of systems, their context and stage of development, there was a danger of oversimplifying complex, dynamic descriptions of implementation processes and contexts, so this aggregated data approach was not taken. Due to time constraints, participant checking of findings was undertaken by one participant only.

View this table:
Table 1

Overview of ePRO systems (see online supplemental file 4 for additional system data)

View this table:
Table 2

System characteristics

View this table:
Table 3

Data collection and assessment

View this table:
Table 4

System reporting

Figure 1
Figure 1

Conceptual framework and key findings.

Patient and public involvement

Patient partners were involved in the design of this study and coauthored the final manuscript; where they specifically highlighted the importance of ensuring that ePROs do not exacerbate existing health inequalities due to digital inequalities (lack of digital access or skills).

Results

The scoping review identified 14 ePRO systems across seven countries from 43 papers and one developing system identified by field experts, totalling 15 systems: Canada n=4, USA n=3, United Kingdom n=4, Denmark n=1, France n=1, The Netherlands n=1, Australia n=1 (table 1). Online supplemental file 1 includes the Preferred Reporting Items for Systematic Reviews and Meta Analyses Extension for Scoping Reviews (PRISMA ScR) diagram.

Ten system developers responded to the survey (66% response rate). System developers held varying academic and clinical academic research posts, all demonstrating outcomes methodology expertise. Key responsibilities ranged from funding acquisition and stakeholder engagement, to overseeing system development, piloting and evaluation. Seven developers agreed to a follow-up interview: one expert delegated this task to a suitable member of their study team; three declined. Reasons for not taking part in the follow-up interview were not collected. Interviews ranged from 42 to 82 min in length (mean 63 min).

Key system characteristics

Table 2 outlines characteristics of surveyed systems and system objectives.

The most common primary objective was symptom management associated with CKD and/or renal replacement therapy by HD. However, systems were implemented across kidney disease stages and treatment modalities (see table 1 and online supplemental file 4 for additional system data). Aggregate uses of ePROs for benchmarking or commissioning of services were not common; one system collected satisfaction with care data39 and two others assessed quality of care.21 40 There was variation in funding source; academic (40%), government (30%) and charitable funding (30%), with 70% of systems being developed in the last 5 years. Half of the systems were developed in-house by local informatics teams, with the rest reporting collaborations across sectors with registries and technology companies. Systems allowed ePROs to be completed in different settings; the most common setting for completion was hospital clinic (80%). Only one system could be accessed by primary care.40

One surveyed system, Ambuflex, had been implemented into routine practice but was undergoing additional evaluation;25 all other systems were at varying stages of study with most having conducted pilot/feasibility testing. Two identified systems were being evaluated within a cluster randomised controlled trial,21 22 while another was being evaluated in a codesigned demonstration trial.40 Tablet was the most popular mode of administration. Two systems offered a clinic-based kiosk facility.25 41 Training was available or planned for clinical staff in all systems but focused largely on system access and functionality. Seven systems had developed training materials for patients and four for administrative staff. Four systems had patient education material administered within the system, only one of these linked self-management guidelines to scores.9 Only 60% of systems were currently linked with the EHR.

All the surveyed systems involved patients during design and implementation, from measure choice to evaluation. The ePRO Collaborate system used a codesign process to refine the dashboard and the constructs being assessed in their demonstration trial, combining the use of healthcare coproduction frameworks with stakeholder engagement.40 Developers of the OPTimising engagement in routine collection of Electronic Patient-Reported Outcomes (OPT-ePRO) system used participatory codesign guided by normalisation process theory to guide design and planning with clinicians, patients and caregivers.42

Patients with CKD at varying stages were involved in the decision to create a study-specific questionnaire for the RePROM study, in order to capture outcomes they felt were most relevant.9

Data collection and assessment

Table 3 summarises key data collection functions.

Similarly, half the developers reported using a newly developed PRO measure, either completely new or a mix of existing tools. Only three surveyed systems had ePRO availability in languages other than English,20 21 41 43 although several developers reported plans for future translations, dependent on funding. There were a mixture of mechanisms to remind patients to complete their ePROs, with email (50%) and verbal reminders (50%) being most popular. Reminders were seen as the best way to avoid missing data (55.6%), but some systems offered a save and return function.

Frequency of reporting was varied, with most common timing of assessment being prior to a clinical assessment (40%). Some systems did have set time points ranging from 1 to 6 monthly. Two systems allowed patients to complete as required.9 41

System reporting

Table 4 outlines system-reporting features.

All systems allowed patients and clinicians direct access to ePRO results or summary reports. Doctors (44.4%) and nurses (55.6%) were responsible for initial review and action of results. Follow-up (face to face, virtual, telephone) was largely dependent on scores. Electronic PRO reports contained current scores (summary and individual scores) in 70% of systems, 80% presented longitudinal changes and 40% offered cut scores (low, medium and high). One system was part of an overall dashboard with the option for patients to combine ePRO scores with lab values in a graph,40 and the Ambuflex PRO questionnaire was also viewed alongside other available clinical data, such as biochemistry, blood pressure, body weight, as a decision aid together to determine whether the patient needs a phone call or a face-to-face visit.24

In terms of data visualisation, graphical presentation was most popular (80%) with 50% of ePROs using colour coding to highlight significance of scores. Less frequently used approaches of score presentation were tables (30%), numerical scores (30%) and emoticons (10%). Assistance with interpretation within the report was uncommon, but 30% of ePROs had developed a safety alert system dependent on respondent scores.9 20 25 The RePROM system produced real-time email alerts directly to the kidney nursing team,9 while the SMaRRT HD system generated email alerts, for which the severity alert thresholds required adaption, so that they were frequent enough to ensure symptom capture without overburdening the workflow with emails (range 2–22/week).20 The AmbuFlex system keeps track by giving patients a red, yellow and green status with non-responders presented to clinicians on an alert list.25 The eNephro study by Thilly et al44 was investigating the cost-effectiveness of home telemonitoring in patients with CKD. This system included a clinical decision support algorithm to detect situations of risk; relying largely on biological parameters, it could alert in cases of a complication associated with predefined symptoms.44

Follow-up interviews

Semistructured interviews were undertaken with seven ePRO developers, allowing clarification of survey responses: to explore the development process and identify contextual factors working for and against implementation. All interviews took place online using videoconferencing software (Zoom) with no-one else present. Analysis generated rich data which are presented in line with the CFIR key domains.35 Figure 1 outlines the conceptual framework (Illustrative quotations to support the key findings are provided in online supplemental file 5).

Intervention characteristics

Analysis highlighted the heterogeneity of the systems. However, developers agreed that systems need to be flexible, responsive, and follow an iterative development strategy. All interview participants felt that their systems would need future refinement, whether simple adaption to increase language availability or more complex programming to allow real-time symptom alerting or provision of self-management tools. Knowledge of context (population and setting) and purpose of ePRO data collection should be clearly defined and communicated to all stakeholders including patients. Developers acknowledged that the evidence of ePROs benefit is less mature in nephrology than other clinical specialities. However, participants indicated ePROs could deliver a relative advantage over current care delivery: improving communication, supporting symptom management, enhancing self-care and shared decision making, with potential to reduce healthcare utilisation. However, all participants recognised the challenges, particularly of clinician engagement. They cited clinician anxieties around potential increases in workload, attendance to symptoms and problems unrelated to kidney disease. Consequently, several developers chose not to include real-time safety alerts associated with severe symptom reports.

Several systems used currently available infrastructure, utilising existing platforms, registries, and validated measures. While this offered the advantage of existing expertise and reduced cost, this was offset by restrictions on local adaption and global comparability due to lack of harmonisation.

Outer setting

All ePROs system programmes had been impacted and delayed by the COVID-19 pandemic. Several developers reported that key staff including informatics teams had reprioritised activity and had less availability to support ePRO implementation. However, they stated that the rapid shift to virtual consultations and recognition of the potential impact of ePROs in managing care, particularly outpatient planning, invigorated interest in ePROs.

Developers highlighted the importance of partnership and collaboration – across clinical, academic, and commercial organisations. Registries were seen as an ideal collection platform. However, developers indicated multi-organisational approaches could lead to competing interests and governance issues that can cause delays.

Developers acknowledged the value of integration of ePROs directly into the EHR, but not all systems had this functionality. Some still required manual entry, which was reported as time consuming, costly and would not support real-time care. Many systems were standalone in nephrology secondary care, meaning data would not ‘follow’ patients across care settings and could not be used to manage transition of care needs.

Inner setting

Developers suggested potential unintended consequences of ePRO implementation included further widening of health inequities, due to inadequate accessibility to digital devices, variable digital competence and lack of inclusivity caused by poor availability of systems in multiple languages or failure to assess cultural compatibility. Some developers were investigating such outcomes in particularly vulnerable groups, for example, elderly, ethnic minorities.

Developers cited the continued prevalence of biomedical models of care, despite general acceptance of the need to move to person-centred care, as a barrier to ePRO implementation. They indicated approaches for shifting ‘power’ to patients by focusing on patient understanding of ePROs and involvement at all stages of design and implementation, along with patient peer support to counteract cultural barriers.

Doctors were the professional group most often involved in review and action of ePRO responses, but several systems directed ePRO scores to nursing staff or physician assistants as the initial handler. Resources needed to be available to allow the engagement of the wider multidisciplinary team, including supportive and rehabilitation services.

While clinicians were largely receptive to ePROs, workflow compatibility was seen as crucial to uptake, and this was reflected in the preference for a focus on symptom management as primary objective of capture. Several systems included free text options to report symptoms or problems and clinicians needed reassurance that this would not significantly impact workload. Participants reported that pilot study data suggested free text responses did not adversely increase workload but provided another mechanism for patient/provider communication, likewise with initial data on use of symptom report alerts.

Characteristics of individuals

Participants discussed the importance of measuring outcomes, which matter most to patients but emphasised the hypothetical danger of over digitising the patient experience, meaning that clinicians risk responding to scores alone rather than listening and connecting with the patient to further interpret ePRO data. Several participants looked forward to a future when ePROs might be given similar status to other forms of medical data. They noted clinicians frequently discuss burden associated with the collection and use of ePROs but did not consider similar factors associated with non-essential blood or invasive tests.

Developers reflected on the varied ability to deal effectively with responses, both at an organisational and individual level. They recognised the heavy and complex workload within the specialty, and while some systems offered management guides or further resources, participants were not all in favour of prescriptive treatment decision algorithms, some preferring to allow the exercise of clinical judgement.

The process of implementation

Systems were in different stages of development and use, with some being part of wider research or quality improvement programmes. A pragmatic approach to implementation was often described: for example, measures were chosen according to what was already being used and what patients would be prepared to complete. Existing platforms that had been previously assessed for governance and regulatory compliance were exploited. Implementation readiness for use in routine care was affected by most systems being investigated outside a real-world context. Sustainability was an issue: future planning would need to include activities undertaken by research personnel, including administration tasks, completion support and training. Long-term IT and informatics support needs to be factored into costing for maintenance and adaption.

All developers stated the importance of patient involvement at all stages and outlined different ways patients had offered input and insight into codesign/production to research participation in usability testing. Staff dedicated to engagement were useful, ensuring extended community groups were involved.

Education and training associated with the knowledge needed to support ePROs appeared to be underdeveloped when compared with other aspects of system design. Training resources were often in development and current iterations focused predominantly on system use and did not extend to interpreting or actioning responses. It was recognised that training was time-consuming and needed to be ongoing.

Overarching themes: barriers and facilitators to effective implementation

Across all CFIR constructs, analysis identified potential barriers and facilitators. Barriers included the culture associated with biomedical models, variable clinical engagement particularly from doctors and governance issues. Participants suggested facilitators included the development of patient-driven systems, utilisation of clinical champions, who need both time and belief to promote the system, and adapting existing resources.

Discussion

This review summarises and integrates survey and interview data on the features and implementation factors of 15 nephrology ePRO systems used across seven countries with table 5, outlining key priorities for successful implementation.

View this table:
Table 5

Key priorities for successful implementation

The majority of surveyed systems prioritised symptom management over other uses of ePROs such as benchmarking and commissioning (see table 2). While some systems were designed to collect ePROs across CKD stages and treatment modalities, patients undergoing HD were the most common population for ePROs use. According to recent reviews, patients with CKD report between 56 and 68 common and/or severe signs and symptoms, with the exact symptom burden often determined by stage of disease and treatment.3 45 Participants in this study reflected that symptom management was a good primary objective on which to concentrate early implementation efforts, with capability to add additional functionality and use.

None of the surveyed systems was reported as being developed to align with national quality measures, such as two times yearly In-Center Hemodialysis Consumer Assessment of Healthcare Providers and Systems46 and annual assessment of HRQOL using the KDQOL-36 included in the U.S. Medicare End Stage Renal Disease Quality Incentive Program.47 However, increased utilisation of national registries to collect ePROs for routine care and population level use48 highlights the potential to assess HRQoL and symptoms through a quality registry.

Stakeholder involvement

This study highlights the increased development of nephrology ePROs in the last 5 years, with all developers reporting positive and negative effects of the COVID-19 pandemic. While there were delays to implementation due to reprioritisation of activity, the increased drive to deliver patient centred care via telemedicine has enhanced attention on ePROs.

Patient and public involvement (PPI) is a prerequisite to inclusive and equitable PROs’ research49 and the meaningful involvement of patients in ePROs implementation programmes is crucial to effective uptake.50 Well-established PPI groups, reflective of the total population, can offer advice on measure choice, language requirements and support resources. Patient input in design of software features including reporting and data visualisation can ensure data is interpreted correctly51 using various devices such as graphs, emoticons and heat maps.52 53 Patient ‘navigators’ can offer health literacy and communication peer support for successful digital completion.54 All the surveyed systems had involved patients in aspects of development from measure choice to evaluation.

While other important contributors to ePROs implementation are quality assurance, governance, informatics and information technology departments, any successful ePRO system will need effective clinician ‘buy in’. This study identified this as a key challenge, particularly from those clinicians not involved in ePROs research. Clinical staff state they are anxious about disruptions to already tight workflows and while ePROs are seen as a useful conversation starter, clinicians fear ePROs lack clinical utility for decision-making, citing the current lack of empirical evidence on the benefits of using this data in nephrology.32 55 Indicating a need for research to improve the evidence base of ePROs use at individual and aggregated levels,56 including studies which employ cluster designs and use techniques to maintain allocation concealment.57 Studies included in this review are going some way to provide this evidence.

Developers described utilising the assistance of ‘clinician champions’; individuals with both the time and influence to drive ePRO implementation. These champions would encourage clinicians to view ePRO scores as key health data, in the same way, as a biomarker or a lab test, to manage care.

Studies have highlighted tension from clinicians in their ability to respond to ePROs, whether due to workload or belief that their scope of practice should be confined to matters related to nephrology.32 58 59 Some clinicians indicate that they are more comfortable assessing, rather than actively managing, psychosocial symptoms.41 While holistic care is advocated within kidney care, it is not necessarily supported within the system, and multidisciplinary approaches and further training in managing mental health problems are needed.60 Although research from other specialities suggests patient encounters using ePROs do not take longer,61 studies on workflow changes and impact are required. Training in all systems was predominantly linked to ePROs system functionality; formal interpretation guidance beyond provision of cut scores or basic colour coding was rare or ‘to be developed’. Electronic questionnaires which use and calculate scores or colour codes specifically for decision aids for treatment are considered medical devices and must comply with relevant legislation.62 Most systems offered or were developing self-management guidance for patients. Effective training need analysis at the planning stage is crucial to assess the support needed by the multidisciplinary team to respond to ePROs. In general, systems were being developed iteratively, with training and support materials being developed or expanded after launch and this may be counterproductive to early and sustained engagement from clinical teams.

Further research is needed to investigate how we interpret PROs for CKD management and event prediction, such as dialysis start time, the impact of changing dialysis modality or the decision to undertake a conservative care pathway.

Real-world implementation

A shift to implementation science for guidance on how to promote adoption, enhance readiness and optimise use of ePROs in real-world settings will also identify any unintended consequences of use, such as exacerbation of health inequalities and potential over digitisation of the patient experience.63 64

Many groups who are already subjected to disadvantage and worse health outcomes are also subjected to digital exclusion.65 Flexible systems enhance accessibility, offering patients’ choice on time, frequency, mode of administration and place of completion. Electronic PROs are commonly completed in a healthcare facility via tablet, where completion support is available. However, onsite completion may impact workflow and support assistance traditionally delivered by research staff will need to be factored into sustainability plans. Home completion may need to be incentivised and alternative means of completion should be offered as required, that is, voice activation response systems, hardcopy by post, following assessment of local population needs. Combinations of digital inclusion approaches are likely needed, to support patients with the digital and health literacy skills required to negotiate both their EHR and any corresponding ePRO data66 and offer access to hardware.49 65

Early findings from the EMPATHY study stress the difficulty in implementing ePROs in HD settings, indicating that routine PROs use failed to demonstrate a significant improvement in patient–clinician communication.67 While these findings suggested relatively good communication pretrial, the qualitative data offered potential reasons for no effect: insufficient patient and clinician understanding of the purpose of PROs, challenges with administration, inconsistencies with PROs as communication tools and limited perceived value. Highlighting that evaluation of current ePROs systems should be investigating implementation outcomes (ie, fidelity, appropriateness, acceptability, feasibility, reach, adoption and sustainability)20 40 as well as whether ePROs are excluding already disadvantaged groups.68 Evaluation using implementation science methodology, while formally investigating the effectiveness of ePROs interventions, offers a potential avenue to build the evidence base on benefit while supporting adoption, adaption and sustainability.69

While effort was made to comprehensively identify nephrology ePRO systems for inclusion, a key methodological limitation was the possibility that systems which were not referenced in the literature, available in English, would be missed. This method placed reliance on appropriate and reliable indexing systems. However, consultation with experts in the field, including study participants, to identify suitable systems aimed to minimise these effects. It is suggested that as ePROs systems proliferate in healthcare settings, an international prospective register of clinical ePROs systems could offer one solution for knowledge transfer and collaboration, but issues of ownership, resource and upkeep would need to be addressed. Additionally, while the sample sizes were small, sampling was purposive, with cases being selected by virtue of their capacity to provide richly textured information, relevant to the phenomenon under investigation.70 The prevailing concept for sample size in qualitative research is ‘saturation’; however, this study has been guided by the concept of ‘information power’, that is, the larger information power the sample holds, the lower N is needed.71 The specificity of experiences and knowledge offered by the participants for an in-depth analysis meant the actual sample held adequate information power to develop new knowledge, referring to the aim of this study.

Areas for future research

Data linkage is central to multiple usages and harmonisation of collection platforms and associated measures will support this. Integration of ePROs within the EHR can offer ‘one stop’ access for clinicians and patients, if given access. Extended accessibility across healthcare providers would allow long-term monitoring across the kidney care continuum. A study of daily ePROs collection reported specific events such as fistula formation and modality changes led to PRO changes, demonstrating PROs can capture differential patient experience in CKD.72 This study highlights the need to both implement measures that are sensitive and responsive enough to detect clinically relevant change, such as disease progression, over both short and long term, while considering how best to minimise respondent burden.

New technologies offer a potential solution, computerised adaptive tests (CATs) offer sophisticated measure delivery using algorithms based on item response theory to personalise ePROs for patients. CAT-based measures are shorter, more accurate and efficient.73 None of the reviewed systems was currently using CAT administration; however, several developers reported that this was where they saw the future of ePROs and were actively researching this area; a renal-specific item bank and associated CAT is currently being developed for use in the UK.74

To conclude, while there has been considerable research investment in the development of ePROs, to measure CKD symptoms and HRQOL, the next step is to accelerate the implementation gap between research and practice; this study supports this objective by outlining key system features and exploring factors to optimise the delivery ePROs in routine care settings within nephrology.

Data availability statement

Data are available upon reasonable request. De identified data may be available upon reasonable request via the corresponding author (ORCID 0000-0002-0614-3198).

Ethics statements

Patient consent for publication

Ethics approval

This study involves human participants and was approved by Interviews with professional experts—Ethics approval was issued on 06/07/2021 by University of Birmingham Science, Technology, Engineering and Mathematics Ethical Review Committee: ERN_21-0495.

References

  1. ↵
    1. Almutary H,
    2. Bonner A,
    3. Douglas C
    . Symptom burden in chronic kidney disease: a review of recent literature. J Ren Care 2013;39:140–50. doi:10.1111/j.1755-6686.2013.12022.x
    OpenUrlPubMed
  2. ↵
    1. Lowney AC,
    2. Myles HT,
    3. Bristowe K, et al
    . Understanding what influences the health-related quality of life of Hemodialysis patients: A collaborative study in England and Ireland. J Pain Symptom Manage 2015;50:778–85. doi:10.1016/j.jpainsymman.2015.07.010
    OpenUrlPubMed
  3. ↵
    1. Fletcher BR,
    2. Damery S,
    3. Aiyegbusi OL, et al
    . Symptom burden and health-related quality of life in chronic kidney disease: A global systematic review and meta-analysis. PLoS Med 2022;19:e1003954. doi:10.1371/journal.pmed.1003954
  4. ↵
    1. Anderson NE,
    2. Calvert M,
    3. Cockwell P, et al
    . The use of patient-reported outcomes in patients treated with maintenance Hemodialysis: A perspective. Am J Kidney Dis 2019;74:399–406. doi:10.1053/j.ajkd.2019.01.035
    OpenUrl
  5. ↵
    1. Tang E,
    2. Bansal A,
    3. Novak M, et al
    . n.d. Patient-reported outcomes in patients with chronic kidney disease and kidney transplant-part 1. Front Med;4. doi:10.3389/fmed.2017.00254
  6. ↵
    1. United States of America Department of Health and Human Services Food and Drug Administration
    . Guidance for industry. patient reported outcomes Measaures: use in medical product development to support labeling claims. 2009. Available: http://www.fda.gov/Drugs/GuidanceComplianceRegulatoryInformation/Guidances/default.htm
  7. ↵
    1. Hull SA,
    2. Rajabzadeh V,
    3. Thomas N, et al
    . Do virtual renal clinics improve access to kidney care? A preliminary impact evaluation of a virtual clinic in East London. BMC Nephrol 2020;21:10. doi:10.1186/s12882-020-1682-6
  8. ↵
    1. Young A,
    2. Orchanian-Cheff A,
    3. Chan CT, et al
    . Video-based Telemedicine for kidney disease care. Clin J Am Soc Nephrol 2021;16:1813–23. doi:10.2215/CJN.06660521
    OpenUrlAbstract/FREE Full Text
  9. ↵
    1. Kyte D,
    2. Anderson N,
    3. Auti R, et al
    . Development of an electronic patient-reported outcome measure (ePROM) system to aid the management of patients with advanced chronic kidney disease. J Patient Rep Outcomes 2020;4:55. doi:10.1186/s41687-020-00223-8
  10. ↵
    1. Lizée T,
    2. Basch E,
    3. Trémolières P, et al
    . Cost-effectiveness of web-based patient-reported outcome surveillance in patients with lung cancer. Journal of Thoracic Oncology 2019;14:1012–20. doi:10.1016/j.jtho.2019.02.005
    OpenUrl
  11. ↵
    1. Velikova G,
    2. Booth L,
    3. Smith AB, et al
    . Measuring quality of life in routine oncology practice improves communication and patient well-being: a randomized controlled trial. J Clin Oncol 2004;22:714–24. doi:10.1200/JCO.2004.06.078
    OpenUrlAbstract/FREE Full Text
  12. ↵
    1. McCann L,
    2. Maguire R,
    3. Miller M, et al
    . Patients' perceptions and experiences of using a mobile phone-based advanced symptom management system (Asyms) to monitor and manage chemotherapy related toxicity. Eur J Cancer Care (Engl) 2009;18:156–64. doi:10.1111/j.1365-2354.2008.00938.x Available: http://blackwell-synergy.com/doi/abs/10.1111/ecc.2009.18.issue-2
    OpenUrlCrossRefPubMedWeb of Science
  13. ↵
    1. Basch E,
    2. Deal AM,
    3. Dueck AC, et al
    . Overall survival results of a trial assessing patient-reported outcomes for symptom monitoring during routine cancer treatment. JAMA 2017;318:197. doi:10.1001/jama.2017.7156
    OpenUrlCrossRefPubMed
  14. ↵
    1. Basch E,
    2. Stover AM,
    3. Schrag D, et al
    . Clinical utility and user perceptions of a Digital system for electronic patient-reported symptom monitoring during routine cancer care: findings from the PRO-TECT trial. JCO Clinical Cancer Informatics 2020;4:947–57. doi:10.1200/CCI.20.00081
    OpenUrl
  15. ↵
    1. Meirte J,
    2. Hellemans N,
    3. Anthonissen M, et al
    . Benefits and disadvantages of electronic patient-reported outcome measures: systematic review. JMIR Perioper Med 2020;3:e15588. doi:10.2196/15588
  16. ↵
    1. Schick-Makaroff K,
    2. Molzahn A
    . Brief communication: patient satisfaction wit the use of tablet computers: a pilot study in two outpatient home dialysis clinics. Can J Kidney Health Dis 2014;1:22. doi:10.1186/s40697-014-0022-9
  17. ↵
    1. Schick-Makaroff K,
    2. Molzahn AE
    . Evaluation of real-time use of electronic patient-reported outcome data by nurses with patients in home dialysis clinics. BMC Health Serv Res 2017;17:439. doi:10.1186/s12913-017-2377-y
  18. ↵
    1. Aiyegbusi OL,
    2. Nair D,
    3. Peipert JD, et al
    . A narrative review of current evidence supporting the implementation of electronic patient-reported outcome measures in the management of chronic diseases. Ther Adv Chronic Dis 2021;12:20406223211015958. doi:10.1177/20406223211015958
  19. ↵
    1. Gabbard J,
    2. McLouth CJ,
    3. Brenes G, et al
    . Rapid electronic capturing of patient-reported outcome measures in older adults with end-stage renal disease: A feasibility study. Am J Hosp Palliat Care 2021;38:432–40. doi:10.1177/1049909120954805
    OpenUrl
  20. ↵
    1. Flythe JE,
    2. Tugman MJ,
    3. Narendra JH, et al
    . Feasibility of tablet-based patient-reported symptom data collection among Hemodialysis patients. Kidney Int Rep 2020;5:1026–39. doi:10.1016/j.ekir.2020.04.021
    OpenUrl
  21. ↵
    1. Greenham L,
    2. Bennett PN,
    3. Dansie K, et al
    . The symptom monitoring with feedback trial (SWIFT): protocol for a Registry-based cluster randomised controlled trial in Haemodialysis. Trials 2022;23:419. doi:10.1186/s13063-022-06355-0
  22. ↵
    1. Johnson JA,
    2. Al Sayah F,
    3. Buzinski R, et al
    . A cluster randomized controlled trial for the evaluation of routinely measured patient reported outcomes in Hemodialysis care (EMPATHY): a study protocol. BMC Health Serv Res 2020;20:731.:731. doi:10.1186/s12913-020-05557-z
    OpenUrlPubMed
  23. ↵
    1. Kyte D,
    2. Anderson N,
    3. Bishop J, et al
    . Results of a pilot feasibility randomised controlled trial exploring the use of an electronic patient-reported outcome measure in the management of UK patients with advanced chronic kidney disease. BMJ Open 2022;12:e050610. doi:10.1136/bmjopen-2021-050610
  24. ↵
    1. Grove BE,
    2. Ivarsen P,
    3. de Thurah A, et al
    . Remote follow-up using patient-reported outcome measures in patients with chronic kidney disease: the PROKID study - study protocol for a non-inferiority pragmatic randomised controlled trial. BMC Health Serv Res 2019;19:631. doi:10.1186/s12913-019-4461-y
  25. ↵
    1. Schougaard LMV,
    2. Larsen LP,
    3. Jessen A, et al
    . Ambuflex: TELE-patient-reported outcomes (telePRO) as the basis for follow-up in chronic and malignant diseases. Qual Life Res 2016;25:525–34. doi:10.1007/s11136-015-1207-0
    OpenUrlPubMed
  26. ↵
    1. Kelly LM,
    2. Cordeiro M
    . Three principles of pragmatism for research on organizational processes. Methodological Innovations 2020;13:205979912093724. doi:10.1177/2059799120937242
    OpenUrl
  27. ↵
    1. Tricco AC,
    2. Antony J,
    3. Zarin W, et al
    . A Scoping review of rapid review methods. BMC Med 2015;13:224. doi:10.1186/s12916-015-0465-6
  28. ↵
    1. Aiyegbusi OL,
    2. Kyte D,
    3. Cockwell P, et al
    . Measurement properties of patient-reported outcome measures (Proms) used in adult patients with chronic kidney disease: A systematic review. PLoS One 2017;12:e0179733. doi:10.1371/journal.pone.0179733
  29. ↵
    1. Greenhalgh T,
    2. Peacock R
    . Effectiveness and efficiency of search methods in systematic reviews of complex evidence: audit of primary sources. BMJ 2005;331:1064–5. doi:10.1136/bmj.38636.593461.68
    OpenUrlAbstract/FREE Full Text
  30. ↵
    1. Horsley T,
    2. Dingwall O,
    3. Sampson M
    . Checking reference lists to find additional studies for systematic reviews. Cochrane Database Syst Rev 2011;2011:MR000026. doi:10.1002/14651858.MR000026.pub2
  31. ↵
    1. Lefebvre C,
    2. Glanville J,
    3. Wieland LS, et al
    . Methodological developments in searching for studies for systematic reviews: past, present and future Syst Rev 2013;2:78. doi:10.1186/2046-4053-2-78
  32. ↵
    1. Anderson NE,
    2. McMullan C,
    3. Calvert M, et al
    . Using patient-reported outcome measures during the management of patients with end-stage kidney disease requiring treatment with Haemodialysis (PROM-HD): a qualitative study. BMJ Open 2021;11:e052629. doi:10.1136/bmjopen-2021-052629
  33. ↵
    1. Braun V,
    2. Clarke V
    . Can I use TA? should I use TA? should I not use TA? comparing Reflexive thematic analysis and other pattern-based qualitative analytic approaches. Couns Psychother Res 2021;21:37–47. doi:10.1002/capr.12360 Available: https://onlinelibrary.wiley.com/toc/17461405/21/1
    OpenUrl
  34. ↵
    1. Braun V,
    2. Clarke V
    . One size fits all? What counts as quality practice in (Reflexive) thematic analysis Qualitative Research in Psychology 2021;18:328–52. doi:10.1080/14780887.2020.1769238
    OpenUrl
  35. ↵
    1. Damschroder LJ,
    2. Aron DC,
    3. Keith RE, et al
    . Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci 2009;4:50. doi:10.1186/1748-5908-4-50
  36. ↵
    1. Damschroder LJ,
    2. Reardon CM,
    3. Opra Widerquist MA, et al
    . Conceptualizing outcomes for use with the Consolidated framework for implementation research (CFIR): the CFIR outcomes Addendum. Implement Sci 2022;17:7. doi:10.1186/s13012-021-01181-5
  37. ↵
    1. QSR International Pty Ltd
    . Nvivo (version 12). 2018. Available: https://www.qsrinternational.com/nvivo-qualitative-data-analysis-software/home
  38. ↵
    1. Damschroder LJ,
    2. Lowery JC
    . Evaluation of a large-scale weight management program using the Consolidated framework for implementation research (CFIR). Implementation Sci 2013;8:51. doi:10.1186/1748-5908-8-51
  39. ↵
    1. Schick-Makaroff K,
    2. Tate K,
    3. Molzahn A
    . Use of electronic patient reported outcomes in clinical Nephrology practice: A qualitative pilot study. Can J Kidney Health Dis 2019;6:205435811987945. doi:10.1177/2054358119879451
  40. ↵
    1. Perry LM,
    2. Morken V,
    3. Peipert JD, et al
    . Patient-reported outcome Dashboards within the electronic health record to support shared decision-making: protocol for Co-design and clinical evaluation with patients with advanced cancer and chronic kidney disease. JMIR Res Protoc 2022;11:e38461. doi:10.2196/38461
  41. ↵
    1. Evans JM,
    2. Glazer A,
    3. Lum R, et al
    . Implementing a patient-reported outcome measure for Hemodialysis patients in routine clinical care perspectives of patients and providers on Esas-R:renal. Clin J Am Soc Nephrol 2020;15:1299–309. doi:10.2215/CJN.01840220
    OpenUrlAbstract/FREE Full Text
  42. ↵
    1. Schick-Makaroff K,
    2. Klarenbach S,
    3. Kwon J-Y, et al
    . A process evaluation in home dialysis: electronic patient reported outcomes in clinical kidney practice (ePRO kidney). Ther Adv Chronic Dis 2023;14:20406223231173624. doi:10.1177/20406223231173624
  43. ↵
    1. Flythe JE,
    2. Dorough A,
    3. Narendra JH, et al
    . Development and content validity of a Hemodialysis symptom patient-reported outcome measure. Qual Life Res 2019;28:253–65. doi:10.1007/s11136-018-2000-7
    OpenUrl
  44. ↵
    1. Thilly N,
    2. Chanliau J,
    3. Frimat L, et al
    . Cost-effectiveness of home Telemonitoring in chronic kidney disease patients at different stages by a pragmatic randomized controlled trial (eNephro): rationale and study design. BMC Nephrol 2017;18:126. doi:10.1186/s12882-017-0529-2
  45. ↵
    1. Flythe JE,
    2. Karlsson N,
    3. Sundgren A, et al
    . Development of a preliminary conceptual model of the patient experience of chronic kidney disease: a targeted literature review and analysis. BMC Nephrol 2021;22:233. doi:10.1186/s12882-021-02440-9
  46. ↵
    1. Dad T,
    2. Tighiouart H,
    3. Fenton JJ, et al
    . Evaluation of non-response to the in-center hemodialysis consumer assessment of healthcare providers and systems (ich cahps) survey. BMC Health Serv Res 2018;18. doi:10.1186/s12913-018-3618-4
  47. ↵
    1. Cohen DE,
    2. Lee A,
    3. Sibbel S, et al
    . Correction to: use of the KDQOL-36 for assessment of health-related quality of life among dialysis patients in the United States. BMC Nephrol 2019;20:461. doi:10.1186/s12882-019-1630-5
  48. ↵
    1. Nimmo A,
    2. Bell S,
    3. Brunton C, et al
    . Collection and determinants of patient reported outcome measures in Haemodialysis patients in Scotland. QJM 2018;111:15–21. doi:10.1093/qjmed/hcx180
    OpenUrl
  49. ↵
    1. Calvert MJ,
    2. Cruz Rivera S,
    3. Retzer A, et al
    . Patient reported outcome assessment must be inclusive and equitable. Nat Med 2022;28:1120–4. doi:10.1038/s41591-022-01781-8
    OpenUrlPubMed
  50. ↵
    1. Absolom K,
    2. Holch P,
    3. Woroncow B, et al
    . Beyond lip service and box ticking: how effective patient engagement is integral to the development and delivery of patient-reported outcomes. Qual Life Res 2015;24:1077–85. doi:10.1007/s11136-014-0909-z
    OpenUrl
  51. ↵
    1. Albers EAC,
    2. Fraterman I,
    3. Walraven I, et al
    . Visualization formats of patient-reported outcome measures in clinical practice: a systematic review about preferences and interpretation accuracy. J Patient Rep Outcomes 2022;6:18. doi:10.1186/s41687-022-00424-3
  52. ↵
    1. Snyder C,
    2. Smith K,
    3. Holzner B, et al
    . Making a picture worth a thousand numbers: recommendations for graphically displaying patient-reported outcomes data. Qual Life Res 2019;28:345–56. doi:10.1007/s11136-018-2020-3
    OpenUrlCrossRefPubMed
  53. ↵
    1. Grossman LV,
    2. Feiner SK,
    3. Mitchell EG, et al
    . Leveraging patient-reported outcomes using data visualization. Appl Clin Inform 2018;9:565–75. doi:10.1055/s-0038-1667041
    OpenUrl
  54. ↵
    1. Guha C,
    2. Lopez-Vargas P,
    3. Ju A, et al
    . Patient needs and priorities for patient navigator programmes in chronic kidney disease: a workshop report. BMJ Open 2020;10:e040617. doi:10.1136/bmjopen-2020-040617
  55. ↵
    1. Zhang R,
    2. Burgess ER,
    3. Reddy MC, et al
    . Provider perspectives on the integration of patient-reported outcomes in an electronic health record. JAMIA Open 2019;2:73–80. doi:10.1093/jamiaopen/ooz001
    OpenUrl
  56. ↵
    1. Schick-Makaroff K,
    2. Levay A,
    3. Thompson S, et al
    . An evidence-based theory about PRO use in kidney care: A realist synthesis. Patient 2022;15:21–38. doi:10.1007/s40271-021-00530-2
    OpenUrlPubMed
  57. ↵
    1. Cochrane Effective Practice and Organisation of Care Group,
    2. Gibbons C,
    3. Porter I, et al
    . Routine provision of feedback from patient-reported outcome measurements to Healthcare providers and patients in clinical practice. Cochrane Database Syst Rev 2021;2021. doi:10.1002/14651858.CD011589.pub2
  58. ↵
    1. Viecelli AK,
    2. Duncanson E,
    3. Bennett PN, et al
    . Perspectives of patients, nurses, and Nephrologists about electronic symptom monitoring with feedback in Hemodialysis care. Am J Kidney Dis 2022;80:215–26. doi:10.1053/j.ajkd.2021.12.007
    OpenUrl
  59. ↵
    1. Tong A,
    2. Winkelmayer WC,
    3. Wheeler DC, et al
    . Nephrologists' perspectives on defining and applying patient-centered outcomes in Hemodialysis. CJASN 2017;12:454–66. doi:10.2215/CJN.08370816
    OpenUrl
  60. ↵
    1. Schick-Makaroff K,
    2. Wozniak LA,
    3. Short H, et al
    . Burden of mental health symptoms and perceptions of their management in in-centre Hemodialysis care: a mixed methods study. J Patient Rep Outcomes 2021;5:111. doi:10.1186/s41687-021-00385-z
  61. ↵
    1. Basch E,
    2. Barbera L,
    3. Kerrigan CL, et al
    . Implementation of patient reported outcomes in routine medical care. Am Soc Clin Oncol Educ Book 2018;38:122–34. doi:10.1200/EDBK_200383
    OpenUrl
  62. ↵
    1. Hjollund NHI,
    2. Larsen LP,
    3. de Thurah AL, et al
    . Patient-reported outcome (PRO) measurements in chronic and malignant diseases: ten years' experience with PRO-algorithm-based patient-clinician interaction (telePRO) in Ambuflex. Qual Life Res 2023;32:1053–67. doi:10.1007/s11136-022-03322-9
    OpenUrl
  63. ↵
    1. Rabin BA,
    2. Brownson RC,
    3. Haire-Joshu D, et al
    . A glossary for dissemination and implementation research in health. J Public Health Manag Pract 2008;14:117–23. doi:10.1097/01.PHH.0000311888.06252.bb
    OpenUrlCrossRefPubMedWeb of Science
  64. ↵
    1. Stover AM,
    2. Haverman L,
    3. van Oers HA, et al
    . Using an implementation science approach to implement and evaluate patient-reported outcome measures (PROM) initiatives in routine care settings. Qual Life Res 2021;30:3015–33. doi:10.1007/s11136-020-02564-9
    OpenUrl
  65. ↵
    1. Honeyman MMD,
    2. Evans H,
    3. Davies A
    . Digital technology and health inequalities: a scoping review (2020). Cardiff: Public Health Wales NHS Trust, 2020. Available: https://phw.nhs.wales/publications/publications1/digital-technology-and-health-inequalities-a-scoping-review/?dm_i=21A8,75FES,2H1914,SZM5X,1
  66. ↵
    1. Chapman R,
    2. Haroon S,
    3. Simms-Williams N, et al
    . Socioeconomic deprivation, age and language are barriers to Accessing personal health records: a cross-sectional study of a large hospital-based personal health record system. BMJ Open 2022;12:e054655. doi:10.1136/bmjopen-2021-054655
  67. ↵
    1. Schick-Makaroff K,
    2. Wozniak LA,
    3. Short H, et al
    . How the routine use of patient-reported outcome measures for Hemodialysis care influences patient-clinician communication: A mixed-methods study. Clin J Am Soc Nephrol 2022;17:1631–45. doi:10.2215/CJN.05940522
    OpenUrlAbstract/FREE Full Text
  68. ↵
    1. Van Der Veer SN,
    2. Ercia A,
    3. Caskey FJ, et al
    . Developing an intervention to implement electronic patient-reported outcomes in renal services in the UK. Stud Health Technol Inform 2020;270:936–40.
    OpenUrlPubMed
  69. ↵
    1. Flythe JE
    . Integrating Proms in routine dialysis care: the devil is in the (implementation) details. Clin J Am Soc Nephrol 2022;17:1580–2. doi:10.2215/CJN.10840922
    OpenUrlFREE Full Text
  70. ↵
    1. Vasileiou K,
    2. Barnett J,
    3. Thorpe S, et al
    . Characterising and justifying sample size sufficiency in interview-based studies: systematic analysis of qualitative health research over a 15-year period. BMC Med Res Methodol 2018;18:148. doi:10.1186/s12874-018-0594-7
  71. ↵
    1. Malterud K,
    2. Siersma VD,
    3. Guassora AD
    . Sample size in qualitative interview studies: guided by information power. Qual Health Res 2016;26:1753–60. doi:10.1177/1049732315617444
    OpenUrlCrossRefPubMed
  72. ↵
    1. Pittman ZCL,
    2. John SG,
    3. McIntyre CW
    . Collection of daily patient reported outcomes is feasible and demonstrates differential patient experience in chronic kidney disease. Hemodial Int 2017;21:265–73. doi:10.1111/hdi.12470
    OpenUrlPubMed
  73. ↵
    1. Bass M,
    2. Morris S,
    3. Neapolitan R
    . Utilizing multidimensional computer adaptive testing to mitigate burden with patient reported outcomes. AMIA Annu Symp Proc 2015;2015:320–8.
    OpenUrl
  74. ↵
    1. Institute of Applied Health Research UoB
    . RCAT-renal computerised adaptive test. Birmingham: University of Birmingham, 2022. Available: https://tinyurl.com/13vdwh0m
    1. Duncanson E,
    2. Bennett PN,
    3. Viecelli A, et al
    . Feasibility and acceptability of E-Proms data capture and feedback among patients receiving Haemodialysis in the symptom monitoring with feedback trial (SWIFT) pilot: protocol for a qualitative study in Australia. BMJ Open 2020;10:e039014. doi:10.1136/bmjopen-2020-039014
    1. Morton RL,
    2. Lioufas N,
    3. Dansie K, et al
    . Use of patient-reported outcome measures and patient-reported experience measures in renal units in Australia and New Zealand: A cross-sectional survey study. Nephrology (Carlton) 2020;25:14–21. doi:10.1111/nep.13577
    OpenUrlPubMed
    1. Duncanson EB,
    2. Viecelli PN,
    3. Dansie AK, et al
    . Feasibility and acceptability of electronic patient reported outcome measures (E-Proms) data capture in the symptom monitoring with feedback trial (SWIFT) pilot: perspectives of Nephrologists, nurses and patients. Nephrology 2020;25:35.
    1. Schick-Makaroff K,
    2. Mharapara P
    . Use of electronic patient-reported outcomes in the care of patients with kidney failure. Nephrol Nurs J 2020;47:465–72.
    OpenUrlPubMed
    1. Schick-Makaroff K,
    2. Molzahn A
    . Strategies to use tablet computers for collection of electronic patient-reported outcomes. Health Qual Life Outcomes 2015;13:2. doi:10.1186/s12955-014-0205-1
    1. Schick-Makaroff K,
    2. Tate K,
    3. Molzahn A
    . Use of electronic patient reported outcomes in clinical Nephrology practice: A qualitative pilot study. Can J Kidney Health Dis 2019;6:2054358119879451. doi:10.1177/2054358119879451
    1. Silbernagel P,
    2. Sawatzky R,
    3. Klarenbach S, et al
    . Patient-reported outcomes in clinical kidney practice: development of education sessions for Clinicians. J Pain Sympt Manag 2018;56:e83. doi:10.1016/j.jpainsymman.2018.10.299
    1. Knowles SE,
    2. Ercia A,
    3. Caskey F, et al
    . Participatory Co-design and normalisation process theory with staff and patients to implement Digital ways of working into routine care: the example of electronic patient-reported outcomes in UK renal services. BMC Health Serv Res 2021;21:706. doi:10.1186/s12913-021-06702-y
    1. van der Veer SN,
    2. Couchoud C,
    3. Morton RL
    . The role of kidney registries in expediting large-scale collection of patient-reported outcome measures for people with chronic kidney disease. Clin Kidney J 2021;14:1495–503. doi:10.1093/ckj/sfab061
    OpenUrl
    1. van der Veer SN,
    2. Aresi G,
    3. Gair R
    . Incorporating patient-reported symptom assessments into routine care for people with chronic kidney disease. Clin Kidney J 2017;10:783–7. doi:10.1093/ckj/sfx106
    OpenUrl
    1. Aiyegbusi OL,
    2. Kyte D,
    3. Cockwell P, et al
    . Using patient-reported outcome measures (Proms) to promote quality of care and safety in the management of patients with advanced chronic kidney disease. BMJ Open 2017;7:e016687. doi:10.1136/bmjopen-2017-016687
    1. Kyte D,
    2. Bishop J,
    3. Brettell E, et al
    . Use of an electronic patient-reported outcome measure in the management of patients with advanced chronic kidney disease: the Reprom pilot trial protocol. BMJ Open 2018;8:e026080. doi:10.1136/bmjopen-2018-026080
    1. Aiyegbusi OL,
    2. Kyte D,
    3. Cockwell P, et al
    . Development and usability testing of an electronic patient-reported outcome measure (ePROM) system for patients with advanced chronic kidney disease. Comput Biol Med 2018;101:120–7. doi:10.1016/j.compbiomed.2018.08.012
    OpenUrlPubMed
    1. Wong D,
    2. Cao S,
    3. Ford H, et al
    . Exploring the use of tablet computer-based electronic data capture system to assess patient reported measures among patients with chronic kidney disease: a pilot study. BMC Nephrol 2017;18:1–10. doi:10.1186/s12882-017-0771-7
    OpenUrlCrossRefPubMed
    1. Schell JO,
    2. Lupu DE
    . A step in the right direction the promise of Proms in routine Hemodialysis care. Clin J Am Soc Nephrol 2020;15:1228–30. doi:10.2215/CJN.12350720
    OpenUrlFREE Full Text
    1. Grove BE,
    2. Schougaard LMV,
    3. Ivarsen PR, et al
    . Patient-reported outcome measures for clinical decision-making in outpatient follow-up: validity and reliability of a renal disease questionnaire. J Patient Rep Outcomes 2021;5:107. doi:10.1186/s41687-021-00384-0
    1. Schougaard LMV,
    2. Larsen LP,
    3. Jessen A, et al
    . Ambuflex: TELE-patient-reported outcomes (telePRO) as the basis for follow up in chronic and malignant diseases. Qual Life Res 2016;25:525–34. doi:10.1007/s11136-015-1207-0
    OpenUrlPubMed
    1. Grove BE,
    2. Valen Schougaard LM,
    3. Ivarsen P, et al
    . Remote follow-up based on patient-reported outcomes in patients with chronic kidney disease: A qualitative study of patient perspectives. PLoS ONE 2023;18:e0281393. doi:10.1371/journal.pone.0281393
    1. Grove B,
    2. Ivarsen P,
    3. De Thurah A, et al
    . n.d. Tele follow-up using patient-reported outcomes (PRO) measures in patients with chronic kidney disease-the PRO-KID study: A study protocol for a non-inferiority randomised controlled trial in Denmark. Qual Life Res;27:S143–4.
    1. Davison SN,
    2. Klarenbach S,
    3. Manns B, et al
    . Correction to: patient-reported outcome measures in the care of in-centre Hemodialysis patients. J Patient Rep Outcomes 2021;5:115. doi:10.1186/s41687-021-00388-w
    1. van der Willik EM,
    2. Hemmelder MH,
    3. Bart HAJ, et al
    . Routinely measuring symptom burden and health-related quality of life in dialysis patients: first results from the Dutch Registry of patient-reported outcome measures. Clin Kidney J 2021;14:1535–44. doi:10.1093/ckj/sfz192
    OpenUrlPubMed
    1. van der Willik EM,
    2. Meuleman Y,
    3. Prantl K, et al
    . Patient-reported outcome measures: selection of a valid questionnaire for routine symptom assessment in patients with advanced chronic kidney disease - a four-phase mixed methods study. BMC Nephrol 2019;20:344. doi:10.1186/s12882-019-1521-9
    1. van der Willik EM,
    2. Milders J,
    3. Bart JAJ, et al
    . Discussing results of patient-reported outcome measures (Proms) between patients and healthcare professionals in routine dialysis care: a qualitative study. BMJ Open 2022;12:e067044. doi:10.1136/bmjopen-2022-067044
    1. Snyder CW
    . Users’ guide to integrating patient-reported outcomes in electronic health records. Baltimore, MD: John Hopkins University, 2017. Available: https://www.pcori.org/document/users-guide-integrating-patient-reported-outcomes-electronic-health-records
    1. Aaronsen NET,
    2. Greenhalgh J,
    3. Halyard M, et al
    . User’s guide to implementing patient-reported outcomes assessment in clinical practice. Int Soc Qual Life Res 2015. Available: https://www.isoqol.org/wp-content/uploads/2019/09/2015UsersGuide-Version2.pdf
    1. Zbrozek A,
    2. Hebert J,
    3. Gogates G, et al
    . Validation of electronic systems to collect patient-reported outcome (PRO) data-recommendations for clinical trial teams: report of the ISPOR ePRO systems validation good research practices task force. Value Health 2013;16:480–9. doi:10.1016/j.jval.2013.04.002
    OpenUrlCrossRefPubMed

Supplementary materials

Footnotes

  • Twitter @neanderson100, @christel_uob

  • Contributors NEA, MC, CM, PC, OLA and DK were involved in the conception of the study, and methodological design. NEA conducted data acquisition. NEA and CM undertook analysis and interpretation. NEA wrote the first and revised drafts of the manuscript. RV is a Patient Partner, involved in study design and review of key study documents including protocol, participant information sheet, survey and topic guide. All authors were involved in the final approval of the version to be submitted and published, and agree to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work were appropriately investigated and resolved. NEA is responsible for the overall content as the guarantor.

  • Funding This manuscript refers to independent research funded by the National Institute for Health and Care Research (NIHR) under its Clinical Doctoral Research Fellowship Programme (Grant Reference ICA-CDRF-2018-04-ST2-027).

  • Competing interests NEA receives funding from the National Institute of Health and Care Research (NIHR) under its Clinical Doctoral Research Fellowship Programme (Grant Reference ICA-CDRF-2018-04-ST2-027), NIHR Applied Research Collaboration (ARC), West Midlands and declares personal fees from GlaxoSmithKline (GSK) outside the submitted work. MC is Director of the Birmingham Health Partners Centre for Regulatory Science and Innovation, Director of the Centre for the Centre for Patient Reported Outcomes Research and is a National Institute for Health and Care Research (NIHR) Senior Investigator. MC receives funding from the NIHR Birmingham Biomedical Research Centre, NIHR Surgical Reconstruction and Microbiology Research Centre, NIHR Blood and Transplant Research Unit (BTRU) in Precision Transplant and Cellular Therapeutics, and NIHR ARC West Midlands at the University of Birmingham and University Hospitals Birmingham NHS Foundation Trust, Health Data Research UK, Innovate UK (part of UK Research and Innovation), Macmillan Cancer Support, SPINE UK, UKRI, UCB Pharma, Janssen, GSK and Gilead. MC has received personal fees from Astellas, Aparito Ltd, CIS Oncology, Takeda, Merck, Daiichi Sankyo, Glaukos, GSK and the Patient-Centered Outcomes Research Institute (PCORI) outside the submitted work. DK reports grants from Macmillan Cancer Support, Innovate UK, the NIHR, NIHR Birmingham Biomedical Research Centre, and NIHR SRMRC at the University of Birmingham and University Hospitals Birmingham NHS Foundation Trust, and personal fees from Merck and GSK outside the submitted work. DK has received funding from the NIHR and Kidney Research UK and is Chief Investigator for the RePROM and RCAT studies. CM receives funding from the National Institute for Health Research (NIHR) Surgical Reconstruction and Microbiology Research Centre, the NIHR Birmingham-Oxford Blood and Transplant Research Unit (BTRU) in Precision Transplant and Cellular Therapeutics, Innovate UK, and has received personal fees from Aparito Ltd outside the submitted work. OLA receives funding from the NIHR Birmingham Biomedical Research Centre (BRC), NIHR Applied Research Collaboration (ARC), West Midlands, NIHR Birmingham-Oxford Blood and Transplant Research Unit (BTRU) in Precision Transplant and Cellular Therapeutics at the University of Birmingham and University Hospitals Birmingham NHS Foundation, Innovate UK (part of UK Research and Innovation), Gilead Sciences Ltd, Janssen Pharmaceuticals, Inc., and Sarcoma UK. OLA declares personal fees from Gilead Sciences Ltd, GlaxoSmithKline (GSK) and Merck outside the submitted work. The views expressed are those of the author(s) and not necessarily those of the NHS, the NIHR or the Department of Health and Social Care. The study sponsor and funders have no role in study design, including collection, management, analysis, and interpretation of data; writing of the report and the decision to submit the report for publication.

  • Patient and public involvement Patients and/or the public were involved in the design, or conduct, or reporting, or dissemination plans of this research. Refer to the Study design section for further details.

  • Provenance and peer review Not commissioned; externally peer reviewed.

  • Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.