User Acceptance and Adoption of a Clinical Reminder System in Ambulatory Care: A Developmental Trajectory Approach

Context: Evaluation studies of clinical decision support systems (CDSS) have tended to focus on assessment of system quality and clinical performance in a laboratory setting. Relatively few studies have used field trials to determine if CDSSs are likely to be used in routine clinical settings and whether reminders generated are likely to be evaluated by end-users. We argue that such beneficial outcomes are not likely to occur if use of the system results in side-effects such as decreased end-user efficiency and unanticipated changes in normal workflows. Objective: To assess medical residents’ acceptance and adoption of a clinical reminder system for two chronic diseases and five preventive care procedures. Design: Longitudinal, correlational study using a novel developmental trajectory analysis. This group-based, semi-parametric, statistical modeling method identifies distinct groups, following distinct usage trajectories, among those who recorded use of the reminder system. Usage data were collected from computer logs for an evaluation period of 10 months, and qualitative analysis was conducted based on user satisfaction surveys. Setting: An ambulatory primary care clinic of an urban teaching hospital offering comprehensive healthcare services. This clinic serves as a rotation site for the hospital’s residents. The reminder system has been integrated into the clinic’s routine operation since February 2002 to assist in treating a patient population of about 4,600. Participants: 51 users (41 residents and 10 nursing and clerical staff) used the system in their daily practice. Use of the system was strongly recommended, but not made mandatory. Results: Residents can be clustered into three distinct groups: light user group (41.46%) who used the system steadily over time for about 35% of all patient encounters, moderate user group (36.59%) who had the highest initial usage rate (70%) but this rate consistently decreased to a level comparable with that of the light users, and heavy user group (21.95%) who started moderately (50%) while ended with nearly 100% compliance. This clustering is related to user characteristics: a female, U.S. citizen, and third-year resident3 is most likely to be a heavy user. Previous computer use and optimism, rather than computer knowledge, are also found to be positively correlated with level of usage. Compliance with reminders varies across usage groups: heavy users tend to respond more favorably while light users skipped a large proportion of them. Conclusion: Resistance exists to use of the clinical reminder system. Although some residents rigorously evaluate the reminders and take necessary actions, others tend to avoid them. Level of the resistance and its developmental trends are related to user characteristics: gender, citizenship, year of residency, previous computer use, and optimism about use of computers in clinical medicine all play a significant role. Tailored training programs and novel incentive strategies need to be developed for groups of people who demonstrate distinct adoption behaviors. Actual usage may be a key variable to measure clinical impact after a system moves from lab to field. Despite the acknowledged benefits, the actual usage of this clinical reminder system is relatively low due to various contextual factors, including decreased efficiency, increased data entry effort, and diminished quality of physician-patient communication. We also show that this new methodology has considerable promise for providing insights into system usability and adoption issues that may benefit evaluation studies of CDSSs, as well as information systems more generally.