Quality improvement : implications for health care professionals and managers 9781845441838, 9781845440190

Citation preview

ijhcqa_cover_(i).qxd

13/04/2005

14:41

Page 1

Volume 17 Number 6 2004

ISBN 1-84544-019-6

ISSN 0952-6862

The International Journal of

Health Care Quality Assurance Addressing the Issues of Management and Quality Quality improvements: implications for health-care professionals and managers Guest Editor: Kristina L. Guo

www.emeraldinsight.com

International Journal of Health Care Quality Assurance Volume 17, Number 6, 2004

ISSN 0952-6862

Quality improvement: implications for health-care professionals and managers Guest Editor: Kristina L. Guo

Contents 286 Access this journal online 287 Abstracts & keywords 289 French abstracts 291 Guest editorial 294 Analysis of the forms, functions and facilitation of social support in one English county: a way for professionals to improve the quality of health care Derek Milne, Andrea McAnaney, Ben Pollinger, Katie Bateman and Emma Fewster 302 A new tool for measurement of process-based performance of multispecialty tertiary care hospitals Seetharaman Hariharan, Prasanta K. Dey, Harley S.L. Moseley, Areti Y. Kumar and Jagathi Gora

313 Nursing work environment and quality of care: differences between units at the same hospital Jane McCusker, Nandini Dendukuri, Linda Cardinal, Johanne Laplante and Linda Bambonye 323 Quality management in health care: a 20-year journey Ulises Ruiz and Jose Simon 334 Quality improvement techniques to improve patient satisfaction E. Joseph Torres and Kristina L. Guo 339 Identification of seniors at risk: process evaluation of a screening and referral program for patients aged 75 in a community hospital emergency department Rebecca N. Warburton, Belinda Parke, Wynona Church and Jane McCusker 349 Completing the circle: from PD to PDSA Paul Walley and Ben Gowland 359 Call for papers

Access this journal electronically The current and past volumes of this journal are available at:

www.emeraldinsight.com/0952-6862.htm You can also search over 100 additional Emerald journals in Emerald Fulltext:

www.emeraldinsight.com/ft See page following contents for full details of what your access includes.

www.emeraldinsight.com/ijhcqa.htm As a subscriber to this journal, you can benefit from instant, electronic access to this title via Emerald Fulltext. Your access includes a variety of features that increase the value of your journal subscription.

How to access this journal electronically To benefit from electronic access to this journal you first need to register via the Internet. Registration is simple and full instructions are available online at www.emeraldinsight.com/ rpsv/librariantoolkit/emeraldadmin Once registration is completed, your institution will have instant access to all articles through the journal’s Table of Contents page at www.emeraldinsight.com/0952-6862.htm More information about the journal is also available at www.emeraldinsight.com/ijhcqa.htm Our liberal institution-wide licence allows everyone within your institution to access your journal electronically, making your subscription more cost effective. Our Web site has been designed to provide you with a comprehensive, simple system that needs only minimum administration. Access is available via IP authentication or username and password.

E-mail alert services These services allow you to be kept up to date with the latest additions to the journal via e-mail, as soon as new material enters the database. Further information about the services available can be found at www.emeraldinsight.com/ usertoolkit/emailalerts Emerald WIRE (World Independent Reviews) A fully searchable subject specific database, brought to you by Emerald Management Reviews, providing article reviews from the world’s top management journals. Research register A web-based research forum that provides insider information on research activity world-wide located at www.emeraldinsight.com/researchregister You can also register your research activity here. User services Comprehensive librarian and user toolkits have been created to help you get the most from your journal subscription. For further information about what is available visit www.emeraldinsight.com/usagetoolkit

Key features of Emerald electronic journals

Choice of access

Automatic permission to make up to 25 copies of individual articles This facility can be used for training purposes, course notes, seminars etc. This only applies to articles of which Emerald owns copyright. For further details visit www.emeraldinsight.com/copyright

Electronic access to this journal is available via a number of channels. Our Web site www.emeraldinsight.com is the recommended means of electronic access, as it provides fully searchable and value added access to the complete content of the journal. However, you can also access and search the article content of this journal through the following journal delivery services: EBSCOHost Electronic Journals Service ejournals.ebsco.com Huber E-Journals e-journals.hanshuber.com/english/index.htm Informatics J-Gate www.j-gate.informindia.co.in Ingenta www.ingenta.com Minerva Electronic Online Services www.minerva.at OCLC FirstSearch www.oclc.org/firstsearch SilverLinker www.ovid.com SwetsWise www.swetswise.com TDnet www.tdnet.com

Online publishing and archiving As well as current volumes of the journal, you can also gain access to past volumes on the internet via Emerald Fulltext. Archives go back to 1994 and abstracts back to 1989. You can browse or search the database for relevant articles. Non-article content Material in our journals such as product information, industry trends, company news, conferences, etc. is available online and can be accessed by users. Key readings This feature provides abstracts of related articles chosen by the journal editor, selected to provide readers with current awareness of interesting articles from other publications in the field. Reference linking Direct links from the journal article references to abstracts of the most influential articles cited. Where possible, this link is to the full text of the article. E-mail an article Allows users to e-mail links to relevant and interesting articles to another computer for later use, reference or printing purposes.

Additional complementary services available Your access includes a variety of features that add to the functionality and value of your journal subscription:

Emerald Customer Support For customer support and technical help contact: E-mail [email protected] Web www.emeraldinsight.com/customercharter Tel +44 (0) 1274 785278 Fax +44 (0) 1274 785204

A new tool for measurement of process-based performance of multispecialty tertiary care hospitals

Abstracts & keywords

Seetharaman Hariharan, Prasanta K. Dey, Harley S.L. Moseley, Areti Y. Kumar and Jagathi Gora Keywords Hospitals, Performance measurement (quality), Customer services quality, Analytical hierarchy process, Barbados, India There is an increasing need of a model for the process-based performance measurement of multispecialty tertiary care hospitals for quality improvement. Analytic hierarchy process (AHP) is utilized in this study to evolve such a model. Each step in the model was derived by group-discussions and brainstorming sessions among experienced clinicians and managers. This tool was applied to two tertiary care teaching hospitals in Barbados and India. The model enabled identification of specific areas where neither hospital performed very well, and helped to suggest recommendations to improve those areas. AHP is recommended as a valuable tool to measure the process-based performance of multispecialty tertiary care hospitals. Analysis of the forms, functions and facilitation of social support in one English county: a way for professionals to improve the quality of health care Derek Milne, Andrea McAnaney, Ben Pollinger, Katie Bateman and Emma Fewster

Nursing work environment and quality of care: differences between units at the same hospital

Keywords Social care, Community care, Health services, Quality improvement, England

Jane McCusker, Nandini Dendukuri, Linda Cardinal, Johanne Laplante and Linda Bambonye

Voluntary organisations are an integral part of community care, and the available research indicates the value of their social support role. However, surprisingly little is known about the forms and functions of this support, or the links to the formal support provided by the National Health Service (NHS), so hampering quality improvements. Therefore, a small sample of voluntary service organisations in one English county participated in a pilot study. This involved the staff and users of these organisations, and a geographically linked sample of NHS mental health professionals. Interview data indicated that the voluntary sector users and staff held similarly positive views of the appropriately varied forms and functions of the provided social support, and all participants held unusually similar and positive views of their links, although areas for improvement were suggested by both groups (e.g. links to GPs).

International Journal of Health Care Quality Assurance Volume 17 · Number 6 · 2004 · Abstracts & keywords q Emerald Group Publishing Limited · ISSN 0952-6862

Keywords Nursing, Hospitals, Institutional care, Quality, Canada The literature suggests that improvements in nurses’ work environments may improve the quality of patient care. Furthermore, monitoring the work environment through staff surveys may be a feasible method of identifying opportunities for quality improvement. This study aimed to confirm five proposed sub-scales from the Nursing Work Index – Revised (NWI-R) to assess the nursing work environment and the performance of these subscales across different units in a hospital. Data were derived from a cross-sectional survey of 243 nurses from 13 units of a 300-bed university-affiliated hospital in Quebec, Canada, during 2001. Using confirmatory factor analysis, the five sub-scales were confirmed; three of the sub-scales had greater ability to discriminate between units. Using hierarchical regression models, “resource adequacy” was the sub-scale most strongly associated with the perceived quality of care at the last shift. The NWI-R sub-scales are potentially useful for comparison of work environments of different nursing units at the same hospital.

287

Abstracts & keywords

International Journal of Health Care Quality Assurance Volume 17 · Number 6 · 2004 · 287-288

Quality management in health care: a 20-year journey Ulises Ruiz Keywords Quality management, Patient care, Safety, Spain In this article, the total quality programme in the Spanish health-care system (1986-1992) and the subsequent quality improvement steps that have led to definition and implementation of such an integrated framework, seeking a quality management system and patient safety, are discussed.

Quality improvement techniques to improve patient satisfaction E. Joseph Torres and Kristina L. Guo Keywords Patients, Customer satisfaction, Quality improvement This paper describes several approaches for implementing quality improvement initiatives to improve patient satisfaction, which enables healthcare organizations to position themselves for success in today’s global and increasingly competitive environment. Specifically, measuring the views of patients, improving patient satisfaction through a community-wide effort, and using a Six Sigma program are discussed. Each of these programs can be an effective mechanism for quality improvement. A key component to quality improvement techniques involves collaborative efforts by all health-care professionals and managers as they seek to increase patient satisfaction.

Identification of seniors at risk: process evaluation of a screening and referral program for patients aged $ 75 in a community hospital emergency department Rebecca N. Warburton, Belinda Parke, Wynona Church and Jane McCusker Keywords Elderly people, Emergency services, Hospitals, Patient care, Quality improvement, Canada Reports on the authors’ experience with a patient safety quality improvement program, intended to reduce the incidence and severity of adverse

outcomes for emergency department (ED) patients aged $ 75. The Identification of Seniors at Risk scale was used for screening, and those at high risk were referred for appropriate intervention. The plan-dostudy-act improvement cycle was followed, conducting process evaluation to diagnose and correct implementation difficulties. Reports that: implementing an ED screening and referral program is deceptively difficult; process evaluation and multidisciplinary working group meetings are an essential improvement tool; screening inclusion criteria had to be adapted to the subject population in order to make efficient use of staff time; the screening questions and process required ongoing assessment, revision, and local adaptation in order to be useful; and high-risk screening in the ED is critical to a hospital system’s ability to anticipate clinical problems; the plan-do-study-act improvement cycle is a practical and useful tool for improving quality and systems in a real care setting.

Completing the circle: from PD to PDSA Paul Walley and Ben Gowland Keywords Continuous improvement, Team working, Employees, Health services, United Kingdom Problem-solving teams, involving front-line staff, are widely used to achieve continuous process improvement. Approaches such as “plan-do-studyact” (PDSA) cycles, are now a core element of many health-care improvement initiatives. This paper evaluates the use of PDSA improvement cycles within the UK National Health Service, using emergency care improvement activity as a source of research evidence. It was found that, despite an abundance of information on how to implement this type of change, many senior professionals still misinterpret how this should work. This has implications for how such methodologies are implemented. There is a long way to go in allowing greater employee involvement, moving much further away from the “management committee” style of change. Care has to be taken to ensure that empowered employees are working to consistent and appropriate objectives. It is important that senior personnel develop process understanding alongside the workforce, rather than simply providing distant support.

288

Abstracts & keywords French

Analyse des formes, fonctions et de la facilitation du soutien social dans un comte´ anglais: un moyen pour les professionnels d’ame´liorer la qualite´ des soins de sante´ Derek Milne, Andrea McAnaney, Ben Pollinger, Katie Bateman et Emma Fewster Mots-cle´s Soins sociaux, Soins communautaires, Services de sante´, Ame´lioration de la qualite´, Angleterre Les organismes volontaires font partie inte´grante des soins communautaires et les recherches disponibles soulignent la valeur que reveˆt leur roˆle dans le soutien social. Cependant, ce qui est surprenant c’est le manque d’informations concernant les formes et les fonctions de ce soutien, ou concernant les liens qui le relient au soutien formel apporte´ par le Service de Soins de la Se´curite´ Sociale (National Health Service – NHS), et cela pose un obstacle aux ame´liorations a` la qualite´. En conse´quence, un petit e´chantillon (n = 10) d’organismes prestant des services volontaires dans un comte´ anglais participa a` une e´tude-pilote. Cette e´tude impliquait le personnel (n = 33) et les utlilisateurs (n = 37) de ces organismes (p.ex. la Socie´te´ pour la Maladie d’Alzheimer’), ainsi qu’un e´chantillon ayant un rapport ge´ographique, et compose´ de (n = 17) professionnels de la sante´ mentale du NHS. Les donne´es recueillies au moyen d’interviews indiquaient que les utilisateurs et le personnel du secteur volontaire avaient des opinions tout aussi positives concernant les diverses formes et fonctions du soutien social offert, et que tous les participants avaient des opinions inhabituellement semblables et positives concernant leurs liens, bien que certains domaines d’ame´lioration furent sugge´re´s par les deux groupes (p.ex. liens avec les ge´ne´ralistes). Un nouvel outil permettant de mesurer la performance fonde´e sur les processus dans les hoˆpitaux de soins tertiaires a` spe´cialite´s multiples Seetharaman Hariharan, Prasanta K. Dey, Harley S.L. Moseley, Areti Y. Kumar et Jagathi Gora Mots-cle´s Hoˆpitaux, Mesurage de la performance (qualite´), Qualite´ des services au client, Processus de hie´rarchie analytique, Barbades, Inde Il est devient plus en plus ne´cessaire de disposer d’un mode`le permettant de mesurer la performance fonde´e sur les processus dans les hoˆpitaux de soins International Journal of Health Care Quality Assurance Volume 17 . Number 6 . 2004 . French abstracts # Emerald Group Publishing Limited . ISSN 0952-6862

tertiaires a` spe´cialite´s multiples, si l’on veut ame´liorer la qualite´. Le processus de hie´rarchie analytique (analytic hierarchy process – AHP), c’est-a`-dire une technique de prise de de´cisions a` attributs multiples, est utilise´e dans l’e´tude que voici pour mettre au point ce genre de mode`le. Chacune des e´tapes comprises dans le mode`le fut de´rive´e a` l’issue de discussions en groupe et de sessions de brainstorming avec des cliniciens et directeurs expe´rimente´s. L’outil fut applique´ a` deux hoˆpitaux d’enseignement pour soins tertiaires, e´tablis aux Barbades et en Inde. L’e´valuation cumulative de la performance de l’hoˆpital des Barbades e´tait de 0,38 par rapport a` l’e´valuation cumulative de la performance de l’hoˆpital indien, qui e´tait de 0,27; ceci indique que l’hoˆpital indien performait a` 71 pour cent par rapport a` l’hoˆpital des Barbades. Le mode`le permit d’identifier les domaines spe´cifiques dans lesquels les deux hoˆpitaux ne performaient pas tre`s bien; il permit aussi de faire quelques suggestions visant a` ame´liorer ces domaines. L’article recommande le processus de hie´rarchie analytique, car il s’agit d’un outil pre´cieux qui permet de mesurer la performance fonde´e sur les processus dans les hoˆpitaux pour soins tertiaires a` spe´cialite´s multiples. L’environnement de travail du personnel infirmier et la qualite´ des soins: diffe´rences entre les divers services d’un meˆme hoˆpital Jane McCusker, Nandini Dendukuri, Linda Cardinal, Johanne Laplante et Linda Bambonye Mots-cle´s Soins infirmiers, Hoˆpitaux, Soins institutionnels, Qualite´, Canada Les publications existantes sugge`rent que des ame´liorations aux environnements de travail du personnel infirmier permettraient d’ame´liorer la qualite´ des soins aux patients. De plus, la surveillance de l’environnement de travail entreprise au moyen d’enqueˆtes mene´es aupre`s du personnel repre´sente peut-eˆtre un moyen possible d’identifier les possibilite´s d’ame´lioration de la qualite´. L’e´tude que voici cherchait a` confirmer cinq sous-e´chelles propose´es, emprunte´es a` l’Index du Travail du Personnel Infirmier – version re´vise´e (Nursing Work Index – Revised – NWI-R) pour e´valuer l’environnement de travail du personnel infirmier et la performance de ces sous-e´chelles dans divers services d’un hoˆpital. Les donne´es furent de´rive´es d’une enqueˆte transversale entreprise en 2001 aupre`s de 243 membres du personnel infirmier, provenant de 13 services d’un hoˆpital de 300 lits, affilie´ a` une universite´ au Que´bec, au Canada. Les cinq souse´chelles furent confirme´es au moyen de l’analyse des facteurs confirmative; trois des sous-e´chelles permettaient de mieux discriminer entre les services. Au moyen de mode`les de re´gression hie´rarchiques, il fut possible de de´terminer que ‘‘le caracte`re ade´quat des ressources’’ constituait la sous-e´chelle la plus ´ ue lors fortement associe´e a` la qualite´ des soins perO de la dernie`re e´quipe de travail. Les sous-e´chelles de l’Index NWI-R peuvent servir pour comparer les

289

French abstracts

International Journal of Health Care Quality Assurance Volume 17 . Number 6 . 2004 . 289-290

environnements de travail de diffe´rents services infirmiers dans un meˆme hoˆpital. Gestion de la qualite´ dans les soins de sante´: une e´pope´e qui dura 20 ans Ulises Ruiz Mots-cle´s Gestion de la qualite´, Soins aux patients, Se´curite´, Espagne L’article que voici traite du programme de qualite´ totale que renferme le syste`me espagnol des soins de sante´ (1986-1992), ainsi que les e´tapes d’ame´lioration de la qualite´ qui ont suivi et qui ont permis de de´finir et de mettre en oeuvre une structure inte´gre´e de ce genre, visant a` un syste`me de gestion de la qualite´ et a` la se´curite´ des patients. Techniques d’ame´lioration de la qualite´ visant a` ame´liorer la satisfaction des patients E. Joseph Torres et Kristina L. Guo Mots-cle´s Patients, Satisfaction des clients, Ame´lioration de la qualite´ L’article que voici de´crit plusieurs approches facilitant la mise en oeuvre des initiatives d’ame´lioration de la qualite´ visant a` ame´liorer la satisfaction des patients, et qui permettent aux organismes prestant des soins de sante´ d’avoir des chances de re´ussir dans l’environnement mondial d’aujourd’hui, qui est de plus en plus compe´titif. Plus particulie`rement, il discute du mesurage de l’opinion des patients, de l’ame´lioration de la satisfaction des patients au moyen d’efforts communautaires et de l’utilisation d’un programme Six Sigma. Chacun de ces programmes peut repre´senter un me´canisme d’ame´lioration de la qualite´ efficace. L’un des principaux composants des techniques d’ame´lioration de la qualite´ est repre´sente´ par les efforts de collaboration de´ploye´s par tous les professionnels et directeurs des soins de sante´ qui cherchent a` accroıˆtre la satisfaction des patients. Identification des personnes aˆge´es a` risque: e´valuation des processus d’un programme de de´pistage et de renvoi pour les patients de = 75 ans dans le service des urgences d’un hoˆpital communautaire Rebecca N. Warburton, Belinda Parke, Wynona Church et Jane McCusker Mots-cle´s Personnes aˆge´es, Services des urgences, Hoˆpitaux, Soins aux patients, Ame´lioration de la qualite´, Canada L’article rend compte de l’expe´rience retire´e par les auteurs d’un programme d’ame´lioration de la qualite´, qui avait trait a` la se´curite´ des patients et qui avait pour but de re´duire le nombre de conse´quences graves ou adverses dans un service des urgences (emergency department – ED), pour les patients aˆge´s de = 75 ans. L’e´chelle d’Identification des Personnes oˆge´es a` Risque (Identification of Seniors at Risk – ISAR) fut

utilise´e pour la se´lection, et les personnes pre´sentant un risque e´leve´ furent renvoye´es a` l’hoˆpital pour recevoir une intervention approprie´e. Le cycle PDSA (plan-do-study-act) fut suivi, une e´valuation du processus eut lieu afin de de´terminer et de corriger les difficulte´s rencontre´es au cours de la mise en pratique. L’article arrive aux conclusions suivantes: la mise en oeuvre d’un programme de se´lection et de renvoi vers le service des urgences est plus difficile qu’il ne paraıˆt; l’e´valuation des processus est un outil d’ame´lioration essentiel; les re´unions en groupes de travail multidisplinaires repre´sentent un outil d’ame´lioration essentiel; les crite`res d’inclusion pour la se´lection doivent eˆtre adapte´s a` la population en question, afin d’utiliser le temps du personnel de manie`re efficace; les questions et le processus de se´lection doivent eˆtre constamment e´value´s, re´vise´s et adapte´s aux circonstances locales pour pouvoir eˆtre utiles; le de´pistage des personnes a` risque e´leve´ dans le service des urgences est essentiel pour que le syste`me de l’hoˆpital puisse pre´voir les proble`mes cliniques; le cycle d’ame´lioration ‘‘Plan-Do-Study-Act est un outil pratique et utile qui permet d’ame´liorer la qualite´ et les syte`mes dans un environnement de soins re´el. Comple´ter le cycle – du de´veloppement des processus (PD) au cycle PECA Paul Walley et Ben Gowland Mots-cle´s Ame´lioration continue, Travail en e´quipe, Employe´s, Services de sante´, Royaume-Uni Les e´quipes charge´es de re´soudre les proble`mes, qui impliquent le personnel de front, sont commune´ment mises sur pied pour obtenir une ame´lioration continue des processus. Les me´thodes d’ame´lioration, comme par exemple les cycles PECA (pre´paration-exe´cutioncontroˆle-ame´lioration – ‘‘plan-do-study-act’’ (PDSA)) constituent a` pre´sent un e´le´ment principal dans bon nombre d’initiatives d’ame´lioration des soins de sante´. L’article que voici e´value l’utilisation des cycles d’ame´lioration PECA dans le Service de la Sante´ au Royaume-Uni, en utilisant l’activite´ d’ame´lioration des soins d’urgence comme source de preuves pour sa recherche. Il trouva que, malgre´ une abondance d’informations concernant la manie`re dont ce type de changement est mis en pratique, bon nombre de professionnels supe´rieurs interpre`tent toujours incorredctement son mode de fonctionnement. Ceci pre´sente des implications pour la manie`re dont les me´thodologies de ce genre sont mises en oeuvre. Il reste encore beaucoup de progre`s a` re´aliser pour permettre une implication plus pousse´e de la part des employe´s, en s’e´loignant davantage du style de changement fonde´ sur le ‘‘comite´ de gestion’’. Des mesures doivent eˆtre prises pour s’assurer que les employe´s a` qui les pouvoirs ont e´te´ de´le´gue´s recherchent des objectifs cohe´rents et approprie´s. Il importe que le personnel supe´rieur de´veloppe une meilleure compre´hension des processus, en meˆme temps que leurs employe´s, plutoˆt que de leur offrir tout simplement un soutien a` distance.

290

Guest editorial Kristina L. Guo

The author Information about the Guest Editor can be found at the end of the article.

Electronic access The Emerald Research Register for this journal is available at www.emeraldinsight.com/researchregister The current issue and full text archive of this journal is available at www.emeraldinsight.com/0952-6862.htm

International Journal of Health Care Quality Assurance Volume 17 · Number 6 · 2004 · pp. 291-293 q Emerald Group Publishing Limited · ISSN 0952-6862

Quality improvement processes, techniques and strategies are critical in health care to achieving success and attaining the competitive edge. Quality improvement is process and outcome oriented, customer driven and involves organizational commitment not only to empower employees, but it also requires a team-based approach to effect change. More importantly, commitment to quality improvement by the governing body and senior-level management fosters quality initiatives and thereby transforms the organization. The purpose of this special issue is to demonstrate the importance of professionals (managers in health care organizations and systems) and their roles in quality improvement. This reflects the Journal’s commitment towards presenting valuable information, comment and debate on quality issues impacting on today’s health care industry. Having worked as a practice administrator for several surgical departments for a number of years at a major academic medical center in the USA, I gained first-hand knowledge into the numerous opportunities and challenges faced by health care professionals and managers. I found that one of my primary challenges as a health care manager was dealing with internal and external customers. The most difficult task encountered was being able to satisfy the needs of our customers. Many times, making decisions that may not have been in the best interest of our departments from the point of view of eyeing the bottom line was difficult. Yet, such sacrifices often meant pleasing patients who will repeatedly market our services by telling their friends of our high quality performance. As I gathered more experience in the field, I was able to apply this knowledge to academia. My dissertation relied on my work experience and focused on exploring managerial work roles in the changing managed care environment of US academic medical centers. I completed my doctorate degree in public administration (although my core concentration was in health services administration) in 1999 and left the practice setting to pursue research. My practitioner’s background and knowledge have been very valuable since they allowed me to integrate real world experience into my research, teaching, and career advisement for my students. In my current role as an Assistant Professor of Health Services Administration at Florida International University in Miami, Florida, my primary research interests are in the areas of health care management and policy. My publications have addressed political, economic, organizational and management implications in the managed care environment. In particular, I have developed new

291

Guest editorial

International Journal of Health Care Quality Assurance Volume 17 · Number 6 · 2004 · 291-293

models for describing the roles of managers in health care organizations. My research now focuses on the three major challenges facing health care systems in the USA. These are to increase health care access, decrease cost and improve quality. To investigate their impact on changing policies at the state level, my most current research project involves writing of a book to explore innovations in health care policies among all states. I have found wide disparities in quality assurance practices from the 50 states. As a result of this project, it has led me to examine further the need for health care quality improvements from both practical and research perspectives. This special issue is a product of my search to learn more about quality improvement. It is also a collaborative exploration of advancing research on this topic. The input of many notable researchers in this area helps us to gain a better understanding of what we know and what we seek to learn. In this issue, several papers have been selected for their significant contribution to the dissemination of knowledge in the area of health care quality improvement. They are summarized as follows. Warburton et al. utilize the plan-do-study-act (PDSA) quality improvement cycle in the implementation of a community hospital emergency department’s screening and referral program to identify seniors at risk. They found that process evaluation and multidisciplinary working group meetings were essential improvement tools. They conclude that the PDSA is a practical and useful tool for improving quality and systems in a real care setting. Ruiz discusses the total quality program that was established in the Spanish health care system from 1986-1992 and subsequent quality improvement steps that have been undertaken in Spain to implement an integrated quality management system aimed to improve patient safety. A stepwise integration of various industrial approaches, such as ISO 9000 and Excellence Models, and specific health care models, such as the Canadian Council on Health Facilities Accreditation, the Australian Council on Healthcare Standards and the Joint Commission on Accreditation of Healthcare Organizations, were presented in order to focus on the safety of the patient as the core consideration. Hariharan et al. applied the Analytic Hierarchy Process model, a multiple attribute decisionmaking technique, to improve the process-based performance of multispecialty tertiary care hospitals. This model was applied to two tertiary care teaching hospitals in Barbados and India. The cumulative performance rating of the Indian hospital was higher than that of the Barbados

hospital. The model enabled identification of specific areas where the hospitals did not perform very well, and helped to suggest recommendations to improvise those areas. Torres and Guo described several approaches for implementing quality improvement initiatives that involve collaborative efforts by all health care professionals and managers to improve patient satisfaction. Approaches include measuring the views of patients, improving patient outcomes with a community-wide effort, and using a Six Sigma method. Each of these programs can be an effective mechanism for quality improvement. Milne et al. conducted a pilot study of ten voluntary service organisations in one English county to determine the value of their social support role. Results indicate that users and providers held similar views of the nature and value of voluntary sector services. Their study suggests that examining voluntary service organisations may help health care professionals to improve the quality of social support in their locality. Walley and Gowland evaluated the use of the PDSA improvement cycle within the UK National Health Service using emergency care improvement activity as a source of research evidence. They found that despite an abundance of information on how to implement this type of change, many senior professionals still misinterpret how this should work. Their study suggests that care must to be taken to ensure that empowered employees work toward consistent and appropriate objectives. McCusker et al. investigated improvements in the quality of patient care through an examination of nurses’ work environments using staff surveys. Data were derived from a cross-sectional survey of 243 nurses from 13 units of a 300-bed universityaffiliated hospital in Quebec, Canada. The results confirmed five sub-scales from the Nursing Work Index – Revised (NWI-R), which is a potentially useful tool for comparing the work environments of different nursing units in the same hospital. About the Guest Editor Kristina L. Guo is an Assistant Professor of Health Services Administration in the Stempel School of Public Health at Florida International University. Professor Guo graduated from Florida International University with a PhD in Public Administration in 1999, and she holds a Master’s degree in Public Health from the University of Miami. Her dissertation investigated the impact of the managed health-care environment on managers in academic medical centers. Prior to joining the FIU faculty, Dr Guo accumulated extensive experience in the health-care field. She was the Assistant Director of the University of

292

Guest editorial

International Journal of Health Care Quality Assurance Volume 17 · Number 6 · 2004 · 291-293

Miami’s intellectual property division, where she conducted market and patent research and analyses. As the manager of several physician practices, she was responsible for formulating and implementing strategies to increase the organization’s viability and growth potential. She was Administrator of a surgical division at the University of Miami, where she successfully led its residency program through its accreditation process. Dr Guo’s areas of research include healthcare management, organizational behavior, entrepreneurship, and health and public policy, especially those pertaining to policies addressing the various needs of and services for the elderly. She has also developed new models for describing the roles of managers in the current managed care environment. She has published numerous scholarly articles. Professor Guo has recently completed the first draft of her book: Health Care Policy in the 50 States for M.E. Sharpe. She received funding for two chapters analyzing case studies of two of the top ten states: Minnesota and Oregon. Dr Guo has served as Guest Editor for the Journal of Health and Human Services Administration, International Journal of Public Administration and the International Journal of Health Care Quality Assurance,where she developed three symposia focusing on entrepreneurship in health and human services organizations, and quality improvement

models, techniques and strategies for health-care managers. Dr Guo has presented her work at various local, regional, national and international conferences. She was the program chair for the Economics and International Business Research Conference and the IVth International Seminar on Hospital Administration sponsored by FIU’s Latin-American and Caribbean Center. In addition to research, Dr Guo teaches several graduate and undergraduate courses in health policy and management, including Health Services Organizations and Management, Ambulatory Care Management, Health-Care Organizational Behavior and People, and Power and Politics in Health Affairs. She has developed innovative teaching techniques and utilizes WebCT, an online teaching tool, to supplement and reinforce student learning. Dr Guo is active in professional and community service. She serves on numerous committees in her program, school and college, including the undergraduate curriculum, scholarship and residency committees in Health Services Administration. She also serves on the School of Public Health committee to study the feasibility of an undergraduate program in public health. Professor Guo is a member of Academy Health, The American College of Health-care Executives and the South Florida Health-care Executive Forum.

293

Analysis of the forms, functions and facilitation of social support in one English county: a way for professionals to improve the quality of health care Derek Milne Andrea McAnaney Ben Pollinger Katie Bateman and Emma Fewster The authors Derek Milne, Andrea McAnaney, Ben Pollinger, Katie Bateman and Emma Fewster are all based at the University of Newcastle upon Tyne, Newcastle upon Tyne, UK.

Keywords Social care, Community care, Health services, Quality improvement, England

Abstract Voluntary organisations are an integral part of community care, and the available research indicates the value of their social support role. However, surprisingly little is known about the forms and functions of this support, or the links to the formal support provided by the National Health Service (NHS), so hampering quality improvements. Therefore, a small sample of voluntary service organisations in one English county participated in a pilot study. This involved the staff and users of these organisations, and a geographically linked sample of NHS mental health professionals. Interview data indicated that the voluntary sector users and staff held similarly positive views of the appropriately varied forms and functions of the provided social support, and all participants held unusually similar and positive views of their links, although areas for improvement were suggested by both groups (e.g. links to GPs).

Electronic access The Emerald Research Register for this journal is available at www.emeraldinsight.com/researchregister The current issue and full text archive of this journal is available at www.emeraldinsight.com/0952-6862.htm International Journal of Health Care Quality Assurance Volume 17 · Number 6 · 2004 · pp. 294-301 q Emerald Group Publishing Limited · ISSN 0952-6862 DOI 10.1108/09526860410557543

Introduction The quality of health care stands to gain from more collaboration between the statutory and voluntary sectors, as emphasised by the community care agenda. Voluntary organisations are typically separate from their formal sector neighbours, such as the National Health Service (NHS) (Hatzidimitriadou, 2002), but may complement them by responding to local needs informally and by filling gaps in services. They can also provide a channel of communication for consumers’ views to the NHS (National Association of Health Authorities/National Council for Voluntary Organisations Joint Working Party, 1987), influencing policy making and its implementation (Crombie and Coid, 2000). The UK Government’s policy of “Community Care” depends heavily on voluntary organisations. The Community Care Act (Department of Health, 1990) gave more opportunity for the involvement of these services, and latterly the National sERVICE Frameworks (Department of Health, 1999) have been founded on partnership reaching out to the community, to individual groups and organisations, including the voluntary sector. The National Service Frameworks also sets out that primary care groups should enable patients and their families to make contact with such local selfhelp groups and voluntary organisations. For these reasons, collaboration between the statutory services and the voluntary sector should increase. However, in most cases collaboration between statutory and voluntary organisations is weak (Simpson, 1996) and problematic (Adams, 1990), despite all of the above advantages. This may contribute to a “faltering” community care programme (Wistow, 1995, p. 236). Why should this partnership be weak? First of all, according to Black (1988), there is a lack of understanding of one another’s organisations and objectives, and significant differences in the respective structures, resources (Wilson, 1994), and value systems (Mosher and Burti, 1994). There may also be a related lack of confidence in self-help groups amongst professionals. For instance, Graham (1995) surveyed 55 general practitioners in his catchment area and found that only 18 per cent of them had ever referred a family to the Alzheimer’s Disease Society. Also, there are a number of costs and consequences linked to such a partnership for mental health professionals, ones that might work against changes in their practices. To illustrate, an early analysis by Froland et al. (1981) noted negative attitudes towards sharing skills and conducting The authors are indebted to the participants for their time and support, and to Remy Marckus and Helen Taylor for preparing the manuscript.

294

A way for professionals to improve the quality of health care

International Journal of Health Care Quality Assurance

Derek Milne et al.

Volume 17 · Number 6 · 2004 · 294-301

projects with non-professionals. Some even perceive such helpers as a threat to professionals (Gussow and Tracy, 1976). Another problem is that mental health professionals are trained primarily for a oneto-one role as a therapist, which can lead to frustration when they are encouraged to participate in community interventions, such as in supporting the social supporters (Mosher and Burti, 1994). More recent analyses have confirmed these points (e.g. Murray and Shepherd, 1996; Rachman, 1995; Simpson, 1996). Joint working is not necessarily any more straightforward when considered from the perspective of the voluntary sector. To illustrate, Bhugra and La Grenade (1997) found that half of the voluntary organisations surveyed in one innercity catchment area in London had generally negative experiences of statutory services. The formal services provided were perceived as being rigid and inflexible. A large majority of the organisations wanted services to be more flexible, as well as to provide more training for voluntary organisations. This lack of mutual awareness and collaboration is a cause for concern, as it fundamentally serves to limit the “social support” that is available in the community, and to limit the efficiency of professionals’ efforts, including their therapy (Milne, 1999). Social support is a construct referring to the informal help provided to people with emotional distress by their friends, relatives and others, such as the members of voluntary sector “self-help” organisations (Cowen, 1982). Various forms of social support have been identified by researchers, most commonly, those of emotional, informational, practical and companionship support (Milne, 1999). Research findings indicate that informal help of this kind ameliorates the need for formal support, and plays a primary role in fostering wellbeing and adaptive behaviour (Grant et al., 2000; Orford, 1992). For example, a clear finding of several surveys is that the majority of those who have mental health needs are far more likely to be receiving informal than professional help (Barker et al., 1990), and that this help is likely to buffer individuals against stress and bolster their coping efforts (Cowen, 1982;Thoits, 1986). In a recent illustration, Green et al. (2002) interviewed 27 people with severe mental health problems. Qualitative analysis consistently indicated that social support had a clear and beneficial effect, especially at times of difficulty (e.g. discharge from hospital). They concluded that assessment of social networks was important as people “cannot be treated in a social vacuum” (Green et al., 2002, p. 576). Six such benefits or “functions” of social support have been noted by Caplan (1974), as summarised in Table I. These complement the four above “forms” of support and are reflected in more

recent accounts of the functions of social support (e.g. Barnes and Duck, 1994). Despite this favourable theoretical, research and policy background, there has been relatively little research on voluntary sector services in the UK. We know little about how the voluntary or informal sector provides social support, whether this help is considered to be beneficial, nor how it links to the help provided by the formal or statutory services (NHS and Social Services), particularly in terms of quantitative research in the mental health sector. Other important issues for research are to ascertain users’ views of their health care (Department of Health, 1998), and to acknowledge the inevitability of variability in services. The voluntary services provide care at multiple levels to different groups of clients. In order to demonstrate the wide range of social support provided by different voluntary organisations, between-organisation comparisons of users’ and workers’ perceptions of the different forms and functions of social support need to be carried out. On this reasoning, Birrell et al. (1992) surveyed users’ views of six voluntary agencies in Edinburgh, and found that they varied in their perceived functions, arising from the kind of facilities that they offered. Two centres offered opportunities for “self development”, while the other centres offered places to visit, things to do and sources of information and advice. Hatzidimitriadou (2002) studied 14 self-help groups in London and south-east England and reported similarly variable support. In keeping with this diversity, Wilson (1994) found variability in the ways that professionals were involved across 49 self-help groups. Further important tasks for research are to establish how much support is provided to voluntary organisations by the mental health professionals working in the co-terminous statutory sector (the support from such professionals might enable voluntary services to provide better care to their users). Therefore, the objectives of this pilot study were: . to compare the perceptions of voluntary sector staff and users regarding the form and function of the social support provided within the sampled organisations; . to compare the different organisations with one another, in terms of their respective social support provision; and . to assess the perceived adequacy of the links between the local voluntary and formal sector staff. Based on prior comparable research, we hypothesised that: H1. The staff and users would hold significantly different views of how social support is provided (the “form” aspect) and whether it

295

A way for professionals to improve the quality of health care

International Journal of Health Care Quality Assurance

Derek Milne et al.

Volume 17 · Number 6 · 2004 · 294-301

Table I A summary of the forms of social support and their associated functions Forms of social supporta

Functions of social support: the psychological needs met by these forms of supportb

Informative support

Guidance (e.g. feeling helped through advice, information, problem solving and goal-setting) Personal attachment (e.g. feeling nurtured, supported and safe) Acceptance (e.g. feeling welcome and accepted; opposite of criticism or rejection) Practical assistance (e.g. borrowing equipment or receiving a helping hand) Social belonging (e.g. feeling part of a group; integrated; mutual trust and ease) Recognition (e.g. feeling socially validated; having a sense of worth, based on personal qualities)

Emotional support

Practical support Social companionship

Sources: a Based on Cowen (1982); b Based on Caplan (1974)

H2.

H3.

is beneficial or not (the “function” aspect) (Antonucci and Israel, 1986) The participating organisations would be perceived to be providing differential social support, in keeping with their respective objectives (Birrell et al., 1992). Mutual dissatisfaction would be expressed, concerning the weak links or collaboration between the staff of the neighbouring voluntary and formal sector organisations (Bhugra and La Grenade, 1997).

Their users had been assessing these voluntary services for an average of 34.9 months (range: half an hour – 15 years). All formal sector staff were full-time employees of a Mental Health NHS Trust in the same county. They were recruited as a “representative sample” by their managers. Of the 17 interviewed, 12 (70.6 per cent) were female, nine (52.9 per cent) were nurses, four (23.5 per cent) were occupational therapists, two (11.8 per cent) were community psychiatric nurses (s) and two (11.8 per cent) worked in outreach services. They worked in a range of service “tiers” (see Paxton et al., 2000), mostly at tiers 2 and 3. The median age range for staff was between 36 and 45.

Method Design The study adopted a cross-sectional design, with the data collected by means of an ad hoc, structured interview. Between-group comparisons were made to assess the respective users’ and providers’ perceptions of social support in the voluntary services, and to compare the forms and functions of social support across the different voluntary services. Participants Ten mental health voluntary sector services from the County of Northumberland in England participated in the study, comprising 70 participants (37 users and 33 providers). The age range of the voluntary service workers and users participating in this research was 18 to 65+. The median age range for the workers was 36-45, and 46-55 for the users. The services had a much higher proportion of females than males (workers: 81.8 per cent female, 18.2 per cent male; users: 75.7 per cent female, 24.3 per cent male). The average number of months the informal sector staff participants had been working for a particular voluntary agency was 35.8 (range: 8-144 months). On average, these staff worked 19.3 hours per week (range: 30 minutes - 40 hours per week).

Measures As no suitable instruments could be located in the literature, parallel semi-structured interviews were developed to satisfy a number of important practical criteria, as in being brief, simple to administer, and straightforward to score and interpret (Barkham et al., 1998). The interviews were capable of assessing negative as well as the positive social support impacts. The average time taken to complete the interview was about 30 minutes[1]. The parallel interviews for voluntary sector users, providers and for statutory sector staff were designed in such a way as to try to keep the schedules as similar as possible, for purposes of statistical comparisons. To assess the validity, the interview was next piloted with four users and 11 providers (formal and informal services), and several practical weaknesses were identified and rectified. As set out in Table I, the final version of the interviews for users and providers assessed four main types of social support (i.e. informational, emotional, practical and companionship support) by means of five-point Likert ratings of the support frequency (from 0 ¼ “not at all” to 4 ¼ “a lot”), followed by qualitative information (e.g. “what

296

A way for professionals to improve the quality of health care

International Journal of Health Care Quality Assurance

Derek Milne et al.

Volume 17 · Number 6 · 2004 · 294-301

form did this take?”, and “who provided this help?”). Both versions also contained questions on the functions that these types of support might serve (e.g. “personal attachment” and “social belonging”), which were also rated on a five-point frequency scale. The users’ interview then asked for information on other sources of support, on how adequately supported they felt overall, and for any other comments on the support received. The providers version asked about their links to NHS/ Social Services staff or informal services (as appropriate), and for ideas on how these might be developed. It ended with the same two concluding questions as above (suitably rephrased to express the providers’ perceptions). To provide the interview with content validity as a measure of social support, these items were pooled from the relevant literature (i.e. Cowen, 1982; Weiss, 1972; Caplan, 1974; Atkinson and Coia, 1995; Barker et al., 1990). Also, two experts in the field provided comments on the interview content. The reliability of the interviews was also assessed, using the inter-rater agreement and testretest methods. These assessments are detailed in the following section.

voluntary sector staff, who described the project and what would be required of them if they decided to participate. A service-user information leaflet was also distributed. Those users interested in participating indicated their wish to the staff members. A total of 50 percent of the staff and volunteers (up to a maximum of five people), one of whom was the most experienced member of staff, were interviewed at each agency. Informal services participants were next interviewed, either over the telephone (n ¼ 15, 21.4 per cent) or face-to-face (n ¼ 55, 78.6 per cent), depending on the preference of the interviewee, given the extensive travel implications. The face-toface interviews were conducted in the respective voluntary services’ buildings. Two interviews were conducted in the home of the interviewee. Interviews were conducted either by the second or fourth author. Prior to each interview, a pack was provided (containing a covering letter, a consent letter, a consent form, a service user information leaflet, the interview schedule and a stamped addressed envelope) and before the interview proper, the research was briefly described to the interviewee. Questions were answered as appropriate, with debriefing undertaken later. Every participant was assured of strict confidentiality and anonymity, and informed of the voluntary nature of the study. If agreeable, the user signed the consent form. All users received a five pound fee at the end of their interview. During the training phase the two interviewers observed each other interviewing and discussed the problems that arose during the piloting of the interviews. Inter-rater reliability between the interviewers was then checked with interviews with two users and two volunteers. To assess the relative reliability of face-to-face versus telephone-based interviewing, one of these users and one of the volunteers were interviewed over the telephone, the others in person. The volunteers and the users were recruited from two non-participating voluntary organisations, based in a nearby city. Telephone interviews were found to be as reliable as the face-to-face interviews (83.5 per cent and 87.1 per cent, respectively). The formula used to calculate reliability was:

Procedure Following ethical approval for the study, a comprehensive list of 26 voluntary service organisations within Northumberland was established and a letter sent to each of the agencies, describing the aims and objectives of the study. Services were asked to return a reply slip, indicating whether or not they were interested in participating in the research and all returned the reply slip. Of these organisations, 18 were interested in participating in the research and eight not. They were next subjected to a number of inclusion criteria: they had to deal with mental health problems among adults; to represent a range of rural and urban agencies; and to deal with carers and sufferers. As a result of applying these criteria, we defined a sample of 13 agencies. Subsequently, one agency pulled out of the research, as it was unable to recruit participants, and two organisations failed to resume contact and as a result they were excluded from the study. Ten voluntary organisations therefore participated in the study. Eight of these agencies were closely affiliated to UK-wide voluntary organisations and two centres were local organisations. The details of the study were next discussed with the manager of each organisation, who was asked to inform the staff and volunteers of the project and to assess interest in participation. In turn, able service users were approached by these

Number of agreement
number of disagreements Total number of codings 100 : £ 1 Ten days separated these reliability checks, to balance between the memory effect and the possibility of changes occurring in the interviewee’s experience. When all interviews were finished, a further “drift” inter-reliability check was conducted, to ascertain whether the interviewers

297

A way for professionals to improve the quality of health care

International Journal of Health Care Quality Assurance

Derek Milne et al.

Volume 17 · Number 6 · 2004 · 294-301

had remained consistent with one another. This involved the same steps as per the initial assessment. Two voluntary service users and two volunteers were involved. The result of the check was 82.8 per cent, indicating that the reliability of the interviews had remained satisfactorily high. With regard to the formal service participants, the manager of each service division within the local NHS Trust selected up to ten of their staff to participate in this project. These staff were to be selected so as to represent the different kinds of service users and service locations within the Trust. The 17 staff thus selected did appear to be broadly representative, as set out in the “participants” section above. All participants had a copy of the proforma to refer to during the interview, and were interviewed either in person or over the telephone, by one of the authors.

between voluntary organisations (based on the reports of both providers and users) indicated that there were indeed clear and appropriate differences between the voluntary services in the types of social support that they provided. For example, while “Northern Causeway” provided all four forms of help, “Victim Support” did not provide any social activities. The different functions of social support were also compared across the participating voluntary services. On “criticism” there was a significant difference between the perceptions of voluntary service providers and users: at the “Alzheimer’s Disease Society” the participants and users thought that the users never felt criticised, whereas at “Northumberland Women’s Aid” all of the providers and 67 per cent of the users thought that the users felt criticised at times. These data indicate that there were differences between the voluntary organisations on the forms and functions of social support, and therefore H2 was accepted. Our final prediction was that there would be mutual dissatisfaction on the part of informal and formal services with their links. However, it was found that almost half of voluntary sector workers were “quite a bit” of “very” satisfied with the support that they had received from statutory services. Only 15 percent of the voluntary service workers felt that statutory services were “not at all” supportive. Similarly, the formal service providers thought that the links were fairly good (mean overall rating of 2.85 out of a possible 4; SD 0.69). The responses of the participants in terms of detailing useful statutory service links were next analysed using content analysis. The most common themes related to the belief that it would be helpful if mutual awareness was increased, as in general practitioners (GPs) and CPNs knowing more about their organisations, especially about the services that they provide, which might help to guide referrals from them. It was also felt that GPs and CPNs could give information about new advances in areas relevant to their work. Furthermore, there was the feeling that greater recognition by health professionals of the importance of the services provided by voluntary agencies could promote effective collaboration with statutory services. Table II summarises these and the suggested links, from both the voluntary and formal service providers’ perspectives.

Results In order to assess whether there were the expected differences between users’ and providers’ perceptions of the voluntary services, both qualitative and quantitative analyses (comprising both descriptive and inferential statistics) were undertaken. Using the interview data, which were coded and processed quantitatively using SPSS 8. The data were initially pooled from all the participating voluntary services, in order to analyse whether there was a difference between the users’ and providers’ perceptions of the four forms of social support received from the voluntary sector services. We obtained small but non-significant differences between their perceptions of the four forms of social support provided within the ten sampled organisations (Fisher’s Exact Test: all p . 0.10). However, when analysed in relation to the six observed functions of the assessed social support, a significant difference was obtained for the category “criticism” (Chi squared ¼ 4.7 p , 0.05). This indicated that the providers perceived there to be significantly more criticism in the assessed organisations than did the users. There were no other significant differences between the users and the providers on the functions of social support and, given the earlier non-significant findings concerning the four “forms” of the provided social support, we rejected our first hypothesis: users and providers do not appear to hold different views of the nature or effectiveness of the assessed social support. Our second hypothesis was that the forms and functions of social support would differ significantly across the ten voluntary organisations, in keeping with their stated aims. We found that the comparisons of the types of social support

Discussion The main findings to emerge from this pilot study were that, on most dimensions of social support, there were no differences between the consistently

298

A way for professionals to improve the quality of health care

International Journal of Health Care Quality Assurance

Derek Milne et al.

Volume 17 · Number 6 · 2004 · 294-301

Table II Staff’s suggestions on ways to strengthen links and to enhance social support Suggested development themes

Examples from formal services staff (n 5 10)

Examples from informal services staff (n 5 33)

1.

Increasing mutual awareness

a) Better or new links to GPs/CPNs/social services b) More recognition of the contribution they make

2.

Reducing barriers to the use of voluntary agencies

3.

Creating additional resources

a) Networking, promotion and information sharing – especially a directory of what is available in the local voluntary sector b) Interaction and dialogue (e.g. good practices) c) More communication between GPs and sectors d) Educating the public a) Merging some services and/or activities to reduce insularity and isolation (“mental health ghettos”) and to increase welcome/ acceptance/collaboration b) Advocacy c) User-involvement a) More staff (including resource investigator or “champion” role/post) b) More places for users to go, especially for young people and men c) Opportunities for moving on to places that increase users’ self-worth d) More funding (e.g. holidays; individual needs) e) More and better buildings/venues/equipment (e.g. mini-bus) f) Staff training g) Developing activities and interventions h) Involve voluntary sector volunteers in NHS groups/befriending

positive perceptions of the convenience sample of service providers and users. “ Criticism”, one of the negative functions of social support, was the only exception: we found that a significantly higher percentage of providers than users thought that service users would sometimes feel criticised. Second, the investigation of the four forms and six functions of social support confirmed that voluntary organisations provided the type of support that was appropriate to their aims. For example, Victim Support provided emotional and informational help to its users, whereas the Alzheimer’s Disease Society and ME Association mostly provided practical help for carers and sufferers, respectively. Other variations between the ten agencies were attributable to their differential emphasis on mutual help – i.e. helping their users partly by providing them with an opportunity to share experiences. This provided a forum for empathy and peer problem solving, as took place, for instance, in the Alzheimer’s Disease Society. These findings support those obtained by Birrell et al. (1992), in that there were also differences between their sample of voluntary organisations, in terms of both the forms and functions of social support. The findings are encouraging, as research indicates that the assessed dimensions (especially emotional, informational and practical support) are important to social integration (Gracia and Herrero, 2004). Third, and contrary to our expectation, it was found that almost 50 per cent of the providers who were interviewed found the overall support from statutory services to be “quite” or “very” adequate. Only 15 per cent of the group thought

a) GPs and others giving out more information b) More referrals from formal sector staff c) More recognition of volunteers a) Training from professionals b) More and better information on clients from formal sector staff c) More information also on parallel voluntary sector provision

this support was “not at all” adequate, although there was a high percentage of voluntary service providers who were dissatisfied with the support from certain health professionals and services. Of these, the most commonly held views were that GPs and CPNs should be more aware of and recognise more fully the services provided by the voluntary services. Overall, though, the data on satisfaction with the link between the voluntary and statutory sector services indicate that our third hypothesis should be rejected, as most of the respondents were satisfied with the supported received from statutory services. However, a range of suggestions were offered by both the voluntary and formal service providers (Table II). The main themes were to increase mutual awareness, to reduce barriers and to enhance resources. A number of practical ideas were noted. In summary, our main findings are that users and providers held similar views of the nature and value of voluntary sector services, contrary to the findings of Antonucci and Israel (1986). We also found that there was no significant dissatisfaction on the part of the voluntary sector providers with their links with (and support from) their statutory service colleagues, contrary to the findings presented by Bhugra and La Grenade (1997). We did, though, find differences across voluntary sector organisations, in keeping with Birrell et al. (1992). How might these discrepancies be explained? Clearly, ours was an exploratory, pilot study and so a number of methodological differences exist, such as sampling. Indeed one likely explanation is simply to do with differential sampling: our data

299

A way for professionals to improve the quality of health care

International Journal of Health Care Quality Assurance

Derek Milne et al.

Volume 17 · Number 6 · 2004 · 294-301

agree with those from the relevant study of a UK sample (i.e. Birrell et al., 1992), and differ from the two that are based on samples from other countries. This is a plausible explanation, as the voluntary sectors are likely to differ between countries. Also, it is possible that statutory services have to rely more on voluntary services in the delivery of community care in the rural county studied here. It might be that, as a result of this reliance, the links between statutory and voluntary services are much stronger in the present case than urban contexts. Another possible sampling explanation is that those agencies that were dissatisfied with the support from statutory services might not have agreed to participate in the present study, which would have resulted in a more positive, biased picture from the obtained subsample of agencies. This is clearly a risk with convenience samples such as the present one. A further possible sampling explanation is that, because Bhugra and La Grenade’s (1997) study was conducted some years ago, it is possible that the findings of the present study reflect a shift during recent years towards better co-operation between statutory and voluntary services. Finally, the service users in the present study were selected by the voluntary services, and so may have been atypical. It is possible that those users who were not fully satisfied with the services of the agency were not asked to participate in the study. Another possible explanation is that “dissatisfied users” might not attend the services as regularly as those who were satisfied, and for this reason they were not available for interview. Therefore, the data on the different dimensions and functions of social support might reflect a more positive, biased perception drawn from a selected subsample. A second methodological confound may have been our reliance on ad hoc structured interviews. For these reasons, future research should attempt to standardise the approach to sampling and measurement, and to use one or more assessment methods to address the forms and function of social support (Roth and Fonagy, 1996). In the present study a direct observation of the client’s interactions could have been carried out, for example by using the Support Observation System (Milne and Netherwood, 1997) to collect complementary, objective data on social support. Local studies such as the present one, comparing the perceptions of users and different service providers, should also be conducted in order to promote users’ involvement in services (Department of Health, 1999) so as to promote changes in the roles of professionals. Although many health professionals want to know what users think “few want their actions constrained by knowledge of the findings” (Jones, 1989, p. 262). The results of local research should therefore, be

examined jointly by service planners, users and providers so that the views of consumers are duly incorporated into practice. The important point is that no single organisation can meet all the needs of individuals with mental health problems, and so a multi-agency model should be used by service planners to ensure that the care provided is the care that is needed, for both psychological and policy reasons (Thornicroft et al., 1992). The role of professionals can then be extended to include a long overdue emphasis on social support (Caplan, 1974; Milne, 1999). This could also include supporting and liaising with existing voluntary services and self-help groups, and assisting in the development of other resources. Specifically, professionals could select, train and supervise helpers. As per the present study, mental health professionals could also help voluntary groups by providing service evaluations and the resulting corrective feedback, so as to assist development (see also Milne and Gibson, 1994). Offering support to voluntary services and co-operating with them could help these services to develop, so enhancing the quality of community care. This can, therefore, represent an efficient deployment of scarce professional time, as the voluntary services may be enabling users to avoid and reduce their demands on formal services. Milne et al. (1989) offer an illustration. By facilitating social support in this way, mental health professionals can strengthen the “so vital and yet so problematic” partnership with the voluntary sector (Simpson, 1996, p. 361) and prevent this aspect of the community care programme from faltering (Wistow, 1995).

Conclusions This pilot study of the forms, functions and facilitation of social support affords a methodology and comparative data, enabling mental health professionals to assess and improve the quality of the social support provided by voluntary services in their locality or to conduct generalisable research. As a needs assessment, this may help health care professionals to improve the quality of social support in their locality, as in facilitating the successful resettlement of people with mental health problems (Green et al., 2002). Voluntary agencies should also be aware of what neighbouring agencies provide, so that interagency referrals and collaboration can take place successfully. The voluntary sector could also usefully input to NHS policy development and implementation (Crombie and Coid, 2000). It would be appropriate in future research to consider afresh the old, but still intriguing,

300

A way for professionals to improve the quality of health care

International Journal of Health Care Quality Assurance

Derek Milne et al.

Volume 17 · Number 6 · 2004 · 294-301

question of how the variety of social support forms, functions and providers help different people in their different ways (Cowen, 1982). This may yield valuable insights into community mental health and help develop the ways in which professionals facilitate social support (Milne, 1999). It appears that the users of mental health services move seamlessly between the formal and informal services (Green et al., 2002), so perhaps professionals should follow their lead.

Gracia, E. and Herrero, J. (2004), “Determinants of social integration in the community”, Journal of Community and Applied Social Psychology, Vol. 14, pp. 1-15. Graham, N. (1995), “GPs and voluntary organisations”, British Journal of General Practice, Vol. 45, p. 273. Grant, C., Goodenough, T., Harvey, I. and Hine, C. (2000), “A randomised controlled trial and economic evaluation of a referrals facilitator between primary care and the voluntary sector”, British Medical Journal, Vol. 320, pp. 419-23. Green, G., Hayes, C., Dickinson, D., Whittaker, A. and Gilheany, B. (2002), “The role and impact of social relationships upon wellbeing reported by mental health service-users: a qualitative study”, Journal of Mental Health, Vol. 11, pp. 565-79. Gussow, Z. and Tracy, G.S. (1976), “The role of self-help clubs in adaptation to chronic illness and disability”, Social Science and Medicine, Vol. 10, pp. 407-14. Hatzidimitriadou, E. (2002), “Political ideology, helping mechanisms and empowerment of mental health self-help/mutual aid groups”, Journal of Community and Applied Social Psychology, Vol. 12, pp. 271-85. Jones, I.M. (1989), “The consumer’s voice: learning how to listen”, Health Bulletin (Edinburgh), Vol. 47, pp. 258-63. Milne, D.L. (1999), Social Therapy: A Guide to Social Support Interventions for Mental Health Practitioners, John Wiley & Sons, Chichester. Milne, D.L. and Gibson, L. (1994), “Quality assurance in the voluntary sector”, International Journal of Health Care Quality Assurance, Vol. 7 No. 6, pp. 16-19. Milne, D.L. and Netherwood, P. (1997), “Seeking social support: an observational instrument and illustrative analysis”, Behavioural & Cognitive Psychotherapy, Vol. 25, pp. 173-85. Milne, D.L., Jones, R.Q. and Walters, P. (1989), “Anxiety management in the community: a social support model and preliminary evaluation”, Behavioural Psychotherapy, Vol. 17, pp. 221-36. Mosher, L.R. and Burti, L. (1994), Community Mental Health: Principles and Practice, W.W. Norton & Co., New York, NY. Murray, A. and Shepherd, G. (1996), “Suspicious minds”, Health Services Journal, Vol. 106, p. 27. National Association of Health Authorities/National Council for Voluntary Organisations Joint Working Party (1987), Partnerships for Health, National Association of Health Authorities, Birmingham. Orford, J. (1992), Community Psychology: Theory and Practice, Wiley, Chichester. Rachman, R. (1995), “Community care; changing the role of the hospital social worker”, Health and Social Care in the Community, Vol. 3, pp. 163-72. Roth, A. and Fonagy, P. (1996), What Works for Whom? A Critical Review of Psychotherapy Research, Guilford Press, New York, NY. Simpson, R.G. (1996), “Relationships between self-help organizations and professional health-care providers”, Health and Social Care in the Community, Vol. 4, pp. 359-70. Thoits, P.A. (1986), “Social support as coping assistance”, Journal of Consulting and Clinical Psychology, Vol. 54, pp. 416-23. Thornicroft, G., Brewin, C.R. and Wing, J. (Eds) (1992), Measuring Mental Health Needs, Gaskell, London. Weiss, R.S. (1972), “Helping relationships: relationships of clients with physicians, social workers, priests and others”, Social Problems, Vol. 20, pp. 319-28. Wilson, J. (1994), Two Worlds: Self-help Groups and Professionals, Venture Press, Birmingham. Wistow, G. (1995), “Aspirations and realities: community care at the crossroads”, Health and Social Care in the Community, Vol. 3, pp. 227-40.

Note 1 Copies of the interviews are available on request to the first author.

References Adams, R. (1990), Self-help, Social Work and Empowerment, Macmillan, London. Antonucci, T.C. and Israel, B.A. (1986), “Veridicality of social support: a comparison of principal and network members’ responses”, Journal of Consulting & Clinical Psychology, Vol. 54 No. 4, pp. 432-7. Atkinson, J.M. and Coia, D.A. (1995), Families Coping with Schizophrenia, Wiley, Chichester. Barker, C., Pistrang, N., Shapiro, D.A. and Shaw, I. (1990), “Coping and help seeking in the UK adult population”, British Journal of Clinical Psychology, Vol. 29, pp. 271-85. Barkham, M., Evans, C., Margison, F., McGrath, G., Mellor-Clark, J., Milne, D. and Connell, J. (1998), “The rationale for developing and implementing care batteries for routine use in service settings and psychotherapy outcome research”, Journal of Mental Health, Vol. 7 No. 1, pp. 35-47. Barnes, M.K. and Duck, S. (1994), “Everyday communicative contexts for social support”, in Burleson, B.R., Albrecht, T.L. and Sarason, I.G. (Eds), Communication of Social Support, Sage, London. Bhugra, D. and La Grenade, J. (1997), “Community organisations’ expectations of mental health statutory services”, Irish Journal of Psychological Medicine, Vol. 14, pp. 57-9. Birrell, G.M., Frazer, N.A. and Philip, A.E. (1992), “The Chief Scientist reports . . . voluntary agency support for people with mental health problems”, Health Bulletin, Vol. 50, pp. 270-7. Black, N. (1988), “Partnership for health: voluntary organisations and the NHS”, British Medical Journal, Vol. 296, p. 82. Caplan, G. (1974), Support Systems and Community Mental Health, Behavioral Publications, New York, NY. Cowen, E.L. (1982), “Help is where you find it”, American Psychologist, Vol. 37, pp. 385-95. Crombie, I.K. and Coid, D.R. (2000), “Voluntary organisations: from Cinderella to white knight?”, British Medical Journal, Vol. 320, pp. 392-3. Department of Health (1990), National Health Service and Community Care Act, HMSO, London. Department of Health (1998), Modernising Mental Health Services, Department of Health, London. Department of Health (1999), Mental Health National Service Frameworks, Department of Health, London. Froland, C., Pancoast, D.L., Chapman, N.J. and Kimboko, P.J. (1981), Helping Networks and Human Services, Sage, Beverly Hills, CA.

301

Introduction

A new tool for measurement of process-based performance of multispecialty tertiary care hospitals Seetharaman Hariharan Prasanta K. Dey Harley S.L. Moseley Areti Y. Kumar and Jagathi Gora The authors Information about the authors can be found at the end of the article.

Keywords Hospitals, Performance measurement (quality), Customer services quality, Analytical hierarchy process, Barbados, India

Abstract There is an increasing need of a model for the process-based performance measurement of multispecialty tertiary care hospitals for quality improvement. Analytic hierarchy process (AHP) is utilized in this study to evolve such a model. Each step in the model was derived by group-discussions and brainstorming sessions among experienced clinicians and managers. This tool was applied to two tertiary care teaching hospitals in Barbados and India. The model enabled identification of specific areas where neither hospital performed very well, and helped to suggest recommendations to improve those areas. AHP is recommended as a valuable tool to measure the process-based performance of multispecialty tertiary care hospitals.

Electronic access The Emerald Research Register for this journal is available at www.emeraldinsight.com/researchregister The current issue and full text archive of this journal is available at www.emeraldinsight.com/0952-6862.htm

International Journal of Health Care Quality Assurance Volume 17 · Number 6 · 2004 · pp. 302-312 q Emerald Group Publishing Limited · ISSN 0952-6862 DOI 10.1108/09526860410557552

Health services sector is a complex area that is unique in all its characteristics. It has too many dimensions to be fitted into a simple singular unit and it is therefore essentially very difficult to approach the measurement of the performance of healthcare services by using one method or another. Traditionally hospital performance measurement is done by measuring certain specific metrics that are thought to be the important indicators of the overall performance. Broadly healthcare delivery is evaluated by three categories of measurement namely, structure, process and outcome (Donabedian, 1980). The progress report “America’s best hospitals” released annually by the US News & World Report since 1990, incorporates all the three quality measures in attempting to rate the best hospitals nationwide in USA (US News & World Report, 1990). The structure of the hospitals is assessed by the human and material resources available in each hospital. Outcomes are usually evaluated by the standardized mortality ratio which is the ratio of the observed to expected mortality rate in each hospital. Process of hospitals has been difficult to measure by specific metrics. The authors of the annual progress report have acknowledged that there is inherent difficulty in measuring the process of care in hospitals (US News & World Report, 1996). Hence they relied on a survey involving physicians for the evaluation of the process of healthcare delivery in hospitals. They selected a cross-section of physicians from the American Medical Association’s master-file and asked them to name five “best” hospitals in their respective field. Obviously this is a highly subjective assessment because the physicians were not given any criteria to rate the hospitals. Although process was considered as the primary aspect of assessment among all the three measures of quality, there have been few methods developed so far to measure the process of care in a healthcare institution reliably. Therefore, in an attempt to measure the performance of a multi-specialty tertiary care hospital with regards to its processes of healthcare delivery, we devised a model using the analytic hierarchy process (AHP), a multiple criteria decision-making technique developed initially by Saaty (1980). (A brief expalanation of AHP can be found in the Appendix.) We applied this model to evaluate the performance of two hospitals in developing countries namely Barbados and India and compared the results. Both the hospitals are university teaching hospitals with public sector role, with similar bed-numbers and are referral centres catering to populations of about 250,000. While there have been many applications of AHP in the medical field (Weingarten, 1997; Carter, 1999; Hummel, 2000; Sloane et al., 2003),

302

A new tool for measurement of process-based performance

International Journal of Health Care Quality Assurance

Seetharaman Hariharan et al.

Volume 17 · Number 6 · 2004 · 302-312

we have been unable find an instance where it has been applied so far as performance measurement tool for a hospital.

active involvement and discussions among the clinicians and managers in the hospital environment. Patient care sector was further classified into acute-care areas and chronic-care areas. The acute-care areas comprised of Accident & Emergency (A&E), operating rooms, intensive care units (ICUs), labor and dialysis units. Chronic-care areas included out-patient clinics, general wards and the physical therapy unit. The establishment sector was subdivided into the following: . management of pharmacy, laboratory sciences including microbiology; . pathology and hematology and blood bank; . radiology including radiotherapy; . central sterilization and infection control management; . patient nutrition; . ospital ethics committee; . communication systems; and . library/academic activities.

Methodology In developing the performance measurement model of hospital services, a consensus opinion method was adopted following an initial questionnaire survey with active involvement of the managers and clinicians of the hospitals in both India and Barbados. An AHP-based approach to measure the performance of a tertiary-care hospital involved the following steps: (1) identification of critical success factors; (2) identification of sub-factors, their ratings and constructing the hierarchical model; (3) comparison of critical success factors and sub-factors in a pair-wise fashion to derive their importance and assigning weights for the individual ratings; (4) derivation of the weights of ratings for the two hospitals individually; and (5) gap analysis by comparing the ratings individually as well as by cumulative ratings.

Step 1: identifying the critical success factors The first step was to identify the critical success factors for the “process” of healthcare delivery of a tertiary care multispecialty hospital. Critical success factors, by definition, are the characteristics, conditions, or variables that when properly sustained, maintained, or managed can have a significant impact on the successful management of an organization (Leidecker and Bruno, 1984). In the present study, an initial questionnaire survey was conducted to find what clinicians and managers think of as the most important factors for the optimal functioning of a hospital. Later, extensive brainstorming sessions were undertaken within clinicians and managers all of whom have more than 20 years’ service experience in the healthcare system and the critical success factors were identified. Accordingly, three main areas were identified as the most important sections whose processes could be measured and used as indicators of the performance of healthcare delivery process in a hospital: (1) patient care sector; (2) establishment sector; and (3) administrative sector.

Step 2: identifying sub-factors, ratings and constructing the hierarchical model All the critical success factors were sub-divided into sub-critical success factors again with the

The administrative sector was classified into the following subdivisions: . human relations and personal management of staff including medical, nursing, paramedical and support staff; . overall supply-chain management; . financial management; . clinical engineering and house keeping management; and . medical records management. For all the sub-factors in the patient care sector three common ratings (attributes) were identified, namely patient turnover, patient comfort and adverse patient occurrences (APO). These three ratings were again qualified by three characteristics namely, high/good, average and low/ poor. For the sub-factors in the establishment sector the ratings were: good/ state-of-art technology, average/ semiautomated technology, poor/outdated technology. For the sub-factors in the administrative sector, the ratings were whether the management of each subdivision was good, average or poor. The next step was to assign the weights for the ratings of each subfactor. For the “best” rating in each category the weight was 0.6, for the “average” rating the weight was 0.3 and for the “worst” rating in each category a weight of 0.1 was assigned. For example a high incidence of APO in a unit will get a weight of 0.1, a moderate incidence will receive a weight of 0.3 and a low incidence of APO will get a weight of 0.6. The definitions of the ratings and the weight they should receive were decided following extensive brainstorming sessions and discussions. Figure 1 shows the entire hierarchical model in AHP framework consisting of the critical success factors and sub-factors identified as the most important for the optimal performance of a

303

A new tool for measurement of process-based performance

International Journal of Health Care Quality Assurance

Seetharaman Hariharan et al.

Volume 17 · Number 6 · 2004 · 302-312

Figure 1 Entire hospital processes in AHP framework

hospital. The goal for the analysis is located on the highest level of the hierarchy, the critical success factors on the second level and the critical sub-factors on the third level and their ratings on the fourth level. .

Step 3: pair-wise comparison of critical success factors and sub-factors The next step was determining the importance of each critical success factor by pair-wise comparison. Multiple brainstorming sessions were held among the clinicians and the comparisons were derived for each critical success factor and then the sub-factors. The priorities were derived by comparing each set of elements in a pair-wise fashion with respect to each of the element in higher stratum. A nine-point numerical scale was used for the comparison. The intensity and the definitions of the pair-wise comparison used for prioritization is given in Table I. In a common objectives context where all members of the group have the same objectives, there are four ways that could be used for setting the priorities:

Table I Nine-point scale for pair-wise comparison Intensity of pair-wise comparison 1 3 5 7 9 2, 4, 6, 8

Importance Equal importance, two activities contribute equally to the object Moderate importance, slightly favors one over another Essential or strong importance, strongly favors one over another Demonstrated importance, dominance of the demonstrated importance in practice Extreme importance, evidence favoring one over another of highest possible order of affirmation Intermediate values, when compromise is needed

Source: Brook and McGlynn (1996)

(1) consensus; (2) vote or compromise; (3) geometric mean of the individuals’ judgments; and (4) separate models or players (Dyer and Forman, 1992). Although initial brainstorming sessions had many conflicting opinions, these were later resolved by way of discussions and ultimately all the clinicians

304

A new tool for measurement of process-based performance

International Journal of Health Care Quality Assurance

Seetharaman Hariharan et al.

Volume 17 · Number 6 · 2004 · 302-312

and managers arrived at a consensus after being adequately satisfied regarding the priority of each level in the hierarchy. The prioritization of each critical success factor was then constructed into a normalized matrix, which gave the overall weight of each critical success factor (Table II). Similarly the normalized matrices for the importance levels of each sub-factor gave the weights of each sub-factor, which is known as the local percentage (LP).

both the hospitals. The sum of performance ratings of all the critical success factors will give the overall performance rating of the given hospital. The comparison of performance ratings at the sub-factor level as well as the overall cumulative performance of each hospital and enabled the gap analysis.

Summary of derivation of cumulative performance

Step 4: deriving the weights of ratings for the hospitals The next step was to derive the ratings for each sub-factor for a hospital in Barbados and also for an hospital in India. The hospital clinicians in each unit as well as the managers were interviewed regarding the ratings of each sub-factor. Ratings of each sub-factor were allocated to each unit in the given hospital according to the clinician or managers’ view of where the particular unit stands according to the attribute. For example, in a given hospital, if the A&E department had a high patient turnover, a weight of 0.6 was assigned, if the mean waiting time for patients in the A&E was long (e.g. three to six hours), the patient comfort was considered to be average with a weight of 0.3, (if longer than six hours the rating would have been poor with a weight of 0.1); if the incidence of adverse patient occurrences were low, again the unit was assigned a weight of 0.6. Weights for each and every unit and section of the hospitals were derived in a similar fashion by explaining the criteria and characteristics of each attribute to the clinicians and managers. The weights of all the factors and the sub-factors along with the weights assigned to each attribute in the AHP framework are given in Figure 2.

(1) Pair-wise comparison of critical success factors and normalization: overall weight of critical success factor . . . (2) Pair-wise comparison of sub-factors and normalization: weight of individual sub-factor (LP) . . . (3) Product of 1 and 2: GP of each sub-factor . . . (4) Derivation of weight of each sub-factor according to its rating for either hospital . . . (5) Product of 3 and 4: performance rating of each sub-factor for either hospital . . . (6) Sum of all the sub-factor performance ratings: performance rating for each critical success factor . . . (7) Sum of performance ratings of all the three critical success factors: cumulative performance rating of either hospital . . . Although dedicated software for AHP is available (ExpertChoicee), we did our calculations using Microsoft Excele software.

Results By pair-wise comparison of the critical success factors, “Patient care” factor received the highest weight followed by the “Establishment sector”, and “Administrative sector” (Table II). The consistency ratios, which were calculated for the normalized matrices of all the pair-wise comparisons both in the critical factor and subfactor levels, were found to be less than 0.1 which is in the acceptable limits. There were many differences between the two hospitals in many areas of patient care. Most of the attributes in many patient care areas received only “Average” and “Poor” ratings in both the

Step 5: calculating and comparing cumulative performance between the hospitals By multiplying the overall weight of each critical success factor with those of the sub-factors (LP), the global percentage (GP) of individual subfactor was derived. The product of the weight of the rating of each sub-factor with the GP is the factor level performance of each sub-factor and the sum of these gave the overall factor level performance of the sub-factors of each critical success factor (Tables III-VI). This method was employed to calculate the performance ratings of Table II Pair-wise comparison in factor level Factor Patient care Establishment Administration

Patient care

Establishment

Administration

Overall weight

1 1/2 1/3

2 1 1

3 1 1

0.55 0.25 0.20

305

A new tool for measurement of process-based performance

International Journal of Health Care Quality Assurance

Seetharaman Hariharan et al.

Volume 17 · Number 6 · 2004 · 302-312

Figure 2 AHP framework within prioritizations

Table III Weights and performance ratings for acute care areas in “patient care” Sub-factors

Ratings

Barbados hospital Weights Performancea

Indian hospital Weights Performancea

A&E

Patient turnover Patient comfort APO Patient turnover Patient comfort APO Patient turnover Patient comfort APO Patient turnover Patient Comfort APO Patient turnover Patient Comfort APO

Average Poor Average Average Poor Low Average Average Average Average Average Low Average Good Low

High Poor Average Poor Poor Poor Average Poor Average Average Average Average High Average Low

OR

ICU

Dialysis unit

Labor ward

Cumulative performance

0.005 0.0011 0.0083 0.0087 0.0029 0.0149 0.005 0.0058 0.0058 0.0041 0.0017 0.005 0.0083 0.01 0.007 0.093

0.01 0.001 0.0083 0.0029 0.0029 0.0025 0.005 0.002 0.006 0.004 0.0017 0.0025 0.017 0.005 0.007 0.076

Notes: a Performance as calculated by the AHP steps; A&E: Accident & Emergency; ICU: intensive care units; OR: operating rooms; APO: adverse patient occurrences

hospitals. The Indian hospital received “Poor” rating for more number of attributes for different units, however the patient turnover was always high in India, which increased the overall

performance rating (Tables III and IV) The Indian hospital performed at 85 percent of the performance of Barbados hospital with respect to the patient care sector.

306

A new tool for measurement of process-based performance

International Journal of Health Care Quality Assurance

Seetharaman Hariharan et al.

Volume 17 · Number 6 · 2004 · 302-312

Table IV Weights and performance ratings for chronic care areas in “patient care” Sub-factors

Ratings

Barbados hospital Weights Performancea

Indian hospital Weights Performancea

General wards

Patient Patient APO Patient Patient APO Patient Patient APO

Average Average Average Average Average Average Average Average Average

High Average High High Poor High Average Poor Average

Out-patient clinics

Physical therapy unit

turnover comfort turnover comfort turnover comfort

0.0083 0.017 0.005 0.0083 0.017 0.0083 0.007 0.005 0.0083 0.086

Cumulative performance

0.02 0.017 0.003 0.017 0.006 0.0028 0.005 0.001 0.0083 0.075

Notes: a Performance as calculated by the AHP steps; APO: adverse patient occurrences

Table V Weights and performance ratings for “establishment” factor and its sub-factors Barbados hospital Performancea

Sub-factors

Weights

Laboratories Pharmacy Radiology/radiotherapy Central sterilization/infection control Dietetics and nutrition Ethical committee Communication systems Library/academic activities Cumulative performance

State-of-the-art State-of-the-art State-of-the-art Good Good Average Good Average

0.045 0.045 0.03 0.008 0.008 0.0019 0.008 0.0019 0.146

Indian hospital Weights Performancea Average Average Average Average Average Average Poor Average

0.023 0.023 0.015 0.0038 0.0014 0.0019 0.0013 0.0019 0.073

Note: a Performance as calculated by the AHP steps

Table VI Weights and performance ratings for “administration” factor and its sub-factors Sub-factors Staff management Medical Nursing Paramedical Support Supply chain management Clinical engineering Maintenance Housekeeping Financial management Medical records management Cumulative performance

Barbados hospital Weights Performancea

Weights

Indian hospital Performancea

Average Average Poor Average Average

0.0053 0.0038 0.0013 0.0023 0.015

Average Average Average Average Average

0.0053 0.0038 0.0038 0.0023 0.015

Average Average Average Average

0.008 0.0036 0.012 0.006 0.0575

Poor Poor Average Average

0.0028 0.0012 0.012 0.006 0.052

Note: a Performance as calculated by the AHP steps

In the establishment section, the Barbados hospital surpassed the Indian hospital thanks to the state-of-art technology being employed in many areas. Many areas in Barbados hospital received the highest rating compared to Indian hospital which received only “Average” and “Poor” ratings (Table V). The gap analysis for this sector showed that the Indian hospital performed at only 50 percent of that of the Barbados hospital.

In the administrative section, there were not many areas which grossly differed between the two hospitals. Both hospitals received “Average to “Poor” ratings for most of the sub-factors (Table VI) and the Indian hospital performed almost equal (90 percent) to the hospital in Barbados with respect to the “Administrative sector”. When all the performance ratings were summed, the cumulative rating of performance of

307

A new tool for measurement of process-based performance

International Journal of Health Care Quality Assurance

Seetharaman Hariharan et al.

Volume 17 · Number 6 · 2004 · 302-312

Barbados hospital was 0.38 and that of the Indian hospital was 0.27. The overall percentage of performance of Indian hospital with respect to the Barbados hospital was 71 percent. The comparison between the Barbados hospital and the Indian hospital with respect to the critical success factors is illustrated in Figure 3.

hospitals such as board-certified physicians to number of beds, although there were many ambiguities in the method of coding. The expertise of healthcare providers is also a difficult area to measure. Process is defined as what is actually done in giving and receiving care while outcome is defined as a change in the patient’s current health status that can be attributed to antecedent healthcare. However a quality measure based on process may not be able to predict the outcome of a patient and an outcome-based quality measure may not adequately consider the factors which are not under the control of the physicians (Brook and McGlynn, 1996). Mortality rates are the usual endpoints of outcome-based models of quality measurement. Risk adjusted mortality rate as a measure of quality is highly controversial, because of its inability to appropriately adapt to different case-mixes of different hospitals. Many factors such as socio-economic and cultural factors may influence outcomes in a given setting. Many patients may not seek immediate medical attention due to their cultural beliefs and also due to inaccessibility to modern medical care in many developing countries. Thus, despite adequate delivery of healthcare, there can still be poor outcomes in certain hospitals depending on the case-mix of the hospitals. Hence there have been many dissident opinions for the use of mortality rates to evaluate quality of healthcare delivery (Sherck and Shatney, 1996; Normand et al., 1996). Our earlier research for evaluating the outcome of a surgical ICU in Barbados used a mortality-based model and assessed the ICU’s performance to be excellent because our observed mortality rate was lower than the predicted rate (Hariharan et al., 2002). However, when we later evaluated the performance of the same ICU with a model based on the AHP, we could exactly delineate the areas of weakness in our ICU (Hariharan et al., 2003). Hence in a situation where majority of the healthcare organizations qualify with honors when evaluated according to the mortality-based models, it is difficult to distinguish between genuine quality and grade inflation (Popovich, 2002). The Health Care Financing Administration (HCFA) in the USA even discontinued the public release of the riskadjusted mortality rates of hospitals because of the skeptical opinions regarding the value of mortalitybased models as a reliable quality measure (New York Times, 1993). Thus there can be a no single measure that can assess the overall quality of a hospital and this is why the US News & World Report incorporates all the three factors to rate America’s best hospitals. By combining all three factors, an Index of

Discussion Quality of healthcare delivery is a very difficult paradigm to measure quantitatively and it was thought it might be impossible to measure it at all. Donabedian (1988) outlined the classical three categories of measurement of quality of healthcare delivery namely structure of the healthcare institution, process of care and outcome of the patient. The structure-based quality measure should ideally evaluate the human, physical and financial resources that are needed to provide care. The clinical and technological expertise with respect to all the specialties in medical field forms the skeleton of this assessment, although the exact evaluation methodology is known to have many disadvantages. For this type of measurement, the progress report “America’s best hospitals” used American Hospital Association’s coding of Figure 3 Overall comparisons between Barbados and Indian hospitals

308

A new tool for measurement of process-based performance

International Journal of Health Care Quality Assurance

Seetharaman Hariharan et al.

Volume 17 · Number 6 · 2004 · 302-312

Hospital Quality (IHQ) was devised with scores distributed along a scale ranging from 0 through 100 (Hill and Winfrey, 1995). Although process of healthcare delivery is considered as the most important factor of quality measure, this has been identified as the “weakest link” in the IHQ chain by the authors of “America’s best hospitals” (US News & World Report, 1996). This is because of the non-availability of a method to effectively measure the process of healthcare delivery. The current method of physician survey is completely subjective, because the survey asked the physicians to choose the “best” hospitals without even defining the factors that they have to consider before arriving at such a conclusion. The reputation of the hospital was the predominant factor used in deciding this attribute as “best”. The reputation of the hospitals further went up since these hospitals quoted their ratings as part of public relation exercises, although the method of rating was known to be highly controversial (Mod. Health Care, 1995). Thus proxies such as reputation of a hospital are not sufficient as a measure of the process and it is only through a clear understanding of the complete process of healthcare that the other two quality measures namely the outcome and structure can be understood and measures for quality improvement and remedies for deficiencies can be implemented (Green et al., 1997). Palmer (1997) has suggested ways of constructing process-based methods to measure quality of health care. The ratio of the number of patients who are treated according to evidencebased guidelines to those who are eligible to receive such a management, has been suggested by this author as a process-based measure. The biggest disadvantage of this method is that it is difficult to apply this as a measure of the process of healthcare delivery in a tertiary care hospital. A patient who attends a tertiary care hospital will be most often managed by evidence-based guidelines and hence it may not be easy to find those patients who have not been treated according to such guidelines. The variability in the management of patients who have had similar diagnoses may be attributed to either different schools of thoughts within the standard guidelines or individual variations in the clinical presentation of each patient. A general practitioner who has not updated his or her knowledge for a longer period of time may be unaware of the recently established guidelines and hence may treat patients by an outdated therapeutic guideline. Therefore this method might be applicable in a situation when performance of an individual is being assessed rather than assessing the process of a system at the organizational level.

Nowadays, with the explosive growth of information technology (IT), patients are frequently well informed from the World Wide Web regarding the standard guidelines for the management of their illnesses. Therefore it is difficult to exactly pinpoint a situation where a patient will be managed not in accordance with the standard guidelines. This is especially true in this era of managed care and litigations. Therefore non-evidence-based treatment situations will always be the rare and exceptional cases in a tertiary care hospital and hence the ratio of “inappropriately” treated patients to those who were eligible to receive such treatment (if this denominator is accurately quantifiable) will be always close to “one”. The consequence of this again is that most of the hospitals will qualify with honors if this method is used to rate their performance. In a tertiary level hospital, assessment of the processes in every sector of the hospital which sequentially leads to overall better patient care, is a better option than assessing the quality of individual healthcare provider. The Accreditation Manual for Hospitals (AMH) released by the Joint Commission on Accreditation of Healthcare Organizations (JCAHO, 1993) in USA describes the need for not only knowing whether things are done the right way, but also whether things done the right way are being done well. Therefore, it has emphasized for a shift of focus in evaluating the quality of process of healthcare delivery from the individual level performance to performance of the systems and processes in an organizational level (JCAHO, 1993). Measurements such as infection surveillance, review of resource utilization are suggested as methods for measuring the quality at an organizational level (Slovensky, 1996). Although these methods assess certain specific processes of a hospital, there is no method currently available to evaluate all the important processes of a healthcare institution in a global way. Hence we developed this model using AHP so that multiple criteria could be used to evaluate the overall performance of the hospital as an organization. Although at the first sight, our method may appear to be one that evaluates the structure and resources of a hospital, it is the attribute we have assigned for each sector that exactly defines the process of healthcare rather than the availability of resources. For example, the patient turnover in a unit will reveal whether the scheduling and allocation of resources are done in the proper way, patient comfort will reveal if a patient-focused care is delivered properly and the incidence of adverse patient occurrences will indirectly reveal if the patient received management according to the

309

A new tool for measurement of process-based performance

International Journal of Health Care Quality Assurance

Seetharaman Hariharan et al.

Volume 17 · Number 6 · 2004 · 302-312

established standards and without negligence. The performance of the different sections in the establishment sector such as radiology and laboratory with a state-of-art technology will identify if the care given is faster and conforms to an updated good quality method, rather than just evaluating if the service is available or not. As another example, a blood bank which has state-of-the-art facilities to separate components, will invariably supply only components, and this will serve to decide if the quality of the process of blood bank operation conforms to high standards. Similarly, a hospital pharmacy, which is not developed by an IT-based system management, will have time-consuming and inferior quality methods of dispensing medications which will eventually lead to more patient discomfort and poor performance. Thus the present method clearly defines all the processes of healthcare delivery in a given hospital and by asking the hospital managers to assign their weights for the ratings of the critical success factors and sub-factors, it will be possible to evaluate the performance of the hospital which in turn enables comparison between different hospitals. This can be easily done relatively, because attributes such as patient turnover can be accessed from records, patient comfort can be assessed by data such as waiting time in areas such as A&E and out-patient clinics, cancellations and postponements in the OR etc., and APOs can be assessed by the audit in each patient-care unit and data such as successful lawsuits against those patient care sectors. Another advantage of the method is that it clearly demarcates the areas where one hospital is doing well and where it is deficient. Thus applying this methodology, it is possible for a hospital manager to identify all the rectifiable areas where the hospital is weak which helps to enable improvisation of that specific area and in turn improve the overall function of the hospital. A sensitivity analysis may also done by which it is possible to know the impact of improvisation of that area and prioritizations can be done. The method we have used is not without pitfalls. Although the weights and ratings for each subfactor were allocated by detailed discussions and brainstorming sessions, it may be argued that these are still subjective. However, the consistency ratios for the pair-wise comparison were in the acceptable limits and additionally the mathematical basis of AHP itself has allowances for these subjective components in the hierarchy. When compared to the absolutely subjective nature of asking the physicians to select “best” hospitals, the present way of asking the hospital managers to allocate specific weights uniformly to the different areas of their hospitals will minimize

the subjective component. Certain attributes may not exactly reflect the quality accurately in some circumstances. For example, patient turnover is considered to be one of the important criteria in the all the patient-care areas of the hospital. However, this may be influenced by the geographical location of the hospital, demographics of the area and the pattern of hospital outpatient appointments. For example, a hospital in a developing country like India, where there are frequently no regulated appointment systems for patients to attend outpatient clinics, the patient turnover may be very relatively high in many hospitals. This does not necessarily mean that the quality of care given is good in the hospital; on the contrary it may be even low. This is the main drawback of considering patient turnover alone as a measure of the quality of performance of a hospital. However, if it is either known or assumed that a hospital is adequately equipped to handle a high patient turnover, it will be useful to consider this factor as a metric of hospital performance. Hence we included patient turnover as an attribute to the sub-factors in areas of patient care. We suggest that the model may be applied with minimal modifications for every hospital tailoring to the local needs. Managers may also identify their own critical success factors, sub-factors by questionnaire evaluation and/or brainstorming sessions and identify their respective weights to the various factors and sub-factors. This is one of the benefits of applying AHP as a tool of healthcare quality measurement. Healthcare delivery is highly complex and the patterns are different in every region in accordance with the social, cultural, economic and political setting of the particular region. When different factors take priority in different settings, and there are trade-offs between decision criteria, AHP is one of the most appropriate tools for the successful application in multiple criteria decision-making situations (Sloane and Liberatore, 2002). It allows application of the different perspectives of managers according to their own context. In summary, there are many advantages of using AHP in the performance measurement of a hospital: . Healthcare service is multifactorial and the factors are both objective and subjective in nature and measurement of the performance of such a system can be easily modeled using AHP. . Performance measurement is also a group decision-making process, and AHP allows the same. . AHP has a sound mathematical basis, and its application is user-friendly.

310

.

.

.

A new tool for measurement of process-based performance

International Journal of Health Care Quality Assurance

Seetharaman Hariharan et al.

Volume 17 · Number 6 · 2004 · 302-312

AHP enables to identify the deficiencies in the specific areas of the hospital. AHP allows carrying out a sensitivity analysis that may help hospital managers understanding the effect of their decisions, and prioritize the areas requiring improvement. Software is available and can be utilized for easy measurement.

Conclusions With the focus of performance measurement in healthcare shifting from individual practitioner’s level to the organizational level, there is an increasing need for an easily applicable tool to evaluate the existing level of performance of all the systems and processes of a hospital for improvement in quality. The present study has established AHP as a very useful tool for a process-based performance measurement of a tertiary care hospital and comparing its performance with another.

References Brook, R.H. and McGlynn, E.A. (1996), “Measuring quality of care”, N. Engl. J. Med., Vol. 335, pp. 966-9. Carter, K.J. (1999), “Analysis of three decision-making methods: a breast cancer patient as a model”, Med. Decis. Making, Vol. 19, pp. 49-57. Donabedian, A. (1980), “Basic approaches to assessment: structure, process and outcome”, The Definition of Quality and Approaches to Its Assessment, Health Administration Press, Ann Arbor, MI, pp. 77-128. Donabedian, A. (1988), “The quality of care – how can it be assessed?”, JAMA, Vol. 260, pp. 1743-8. Dyer, R.F. and Forman, E.H. (1992), “Group decision support with the analytic hierarchy process”, Decision Support Systems, Vol. 8, pp. 99-124. Green, J., Wintfield, N., Krasner, M. and Wells, C. (1997), “In search of America’s best hospitals – the promise and reality of quality assessment”, JAMA, Vol. 277, pp. 1152-5. Hariharan, S., Moseley, H.S.L. and Kumar, A.Y. (2002), “Outcome evaluation in a surgical intensive care unit in Barbados”, Anaesthesia, Vol. 57, pp. 434-41. Hariharan, S., Dey, P.K., Kumar, A.Y. and Moseley, H.S.L. (2003), “Service performance measurement of intensive care units in hospitals – a case of Barbados”, Proceedings, Portland International Conference for Management of Engineering and Technology (PICMET), July. Hill, C.A. and Winfrey, K.L. (1995), Index of Hospital Quality, National Opinion Research Center, University of Chicago, Chicago, IL. Hummel, J.M. (2000), “Medical technology assessment: the use of analytic hierarchy process as a tool for multidisciplinary evaluation of medical devices”, Int. J. Artif. Organs, Vol. 23, pp. 782-7.

Joint Commission on Accreditation of Health-care Organizations (JCAHO) (1993), Accreditation Manual for Hospitals, Vol 1: Standards, JCAHO, Chicago, IL. Leidecker, J.K. and Bruno, A.V. (1984), “Identifying and using critical success factors”, Long Range Planning, Vol. 17, pp. 23-32. Mod. Health Care (1995), “Hospitals have PR field day as ‘honor roll’ is published”, No. 25, p. 92. (The) New York Times (1993), “Ratings of hospitals is delayed in an effort for stronger data”, June, section A, p. 19. Normand, S.T., Glickman, M.E., Sharma, R.G.V.R.K. and McNeil, B.J. (1996), “Using admission characteristics to predict short-term mortality from myocardial infarction in elderly patients: results from Cooperative Cardiovascular Project”, JAMA, Vol. 275, pp. 1322-8. Palmer, R.H. (1997), “Process-based measures of quality: the need for detailed clinical data in large health-care databases”, Ann. Intern. Med., Vol. 127, pp. 733-8. Popovich, M.J. (2002), “If most intensive care units are graduating with honors, is it genuine quality or grade inflation?”, Crit. Care Med., Vol. 30, pp. 2145-6. Saaty, T.L. (1980), The Analytic Hierarchy Process, McGraw-Hill, New York, NY. Sherck, J.P. and Shatney, C.H. (1996), “ICU scoring systems do not allow prediction of patient outcomes or comparison of ICU performance”, Crit. Care Clin., Vol. 12, pp. 515-23. Sloane, E.B. and Liberatore, M.J. (2002), “Medical decision support using the analytic hierarchy process”, J. Health-care Inf. Management, Vol. 16, pp. 38-43. Sloane, E.B., Liberatore, M.J., Nydick, R.L., Luo, W. and Chung, Q.B. (2003), “Using the analytic hierarchy process as a clinical engineering tool to facilitate an iterative, multidisciplinary, microeconomic health technology assessment”, Computers & Operations Research, Vol. 30, pp. 1447-65. Slovensky, D.J. (1996), “Quality assessment and improvement”, in Abdelhak, M., Grostick, S., Hanken, M.A. and Jacobs, E. (Eds), Health Information: Management of a Strategic Resource, W.B. Saunders, Philadelphia, PA, pp. 320-57. US News & World Report (1990), “America’s best hospitals”, April 30, pp. 51-85. US News & World Report (1996), “America’s best hospitals”, John Wiley & Sons, New York, NY. Weingarten, M.S. (1997), “A pilot study of the use of analytic hierarchy process for the selection of surgery residents”, Acad. Med., Vol. 72, pp. 400-2.

Appendix. A brief explanation of AHP The AHP developed by Saaty provides a flexible and easily understood way of analyzing complicated problems. It is a multiple criteria decision-making technique that allows subjective as well as objective factors to be considered in decision-making process. The AHP allows the active participation of decision-makers in reaching agreement, and gives managers a rational basis on which to make decisions. The AHP is a theory of measurement for dealing with quantifiable and intangible criteria that has been applied to numerous areas, such as decision theory and conflict resolution. AHP is a problem-solving framework and a systematic

311

A new tool for measurement of process-based performance

International Journal of Health Care Quality Assurance

Seetharaman Hariharan et al.

Volume 17 · Number 6 · 2004 · 302-312

procedure for representing the elements of any problem. AHP is based on the following three principles: decomposition, comparative judgments, and synthesis of priorities. Formulating the decision problem in the form of a hierarchical structure is the foundation of AHP. In a typical hierarchy, the top level reflects the overall objective (focus) of the decision problem. The elements affecting the decision are represented in intermediate levels. The lowest level comprises the decision options. Once a hierarchy is constructed, the decision-maker begins a prioritization procedure to determine the relative importance of the elements in each level of the hierarchy. The elements in each level are compared as pairs with respect to their importance in making the decision under consideration. A verbal scale is

used that enables the decision-maker to incorporate subjectivity, experience, and knowledge in an intuitive and natural way. After comparison matrices are created, relative weights are derived for the various elements. The relative weights of the elements of each level with respect to an element in the adjacent upper level are computed as the components of a normalized matrix. Composite weights are then determined by aggregating the weights through the hierarchy. This is done by following a path from the top of the hierarchy to each alternative at the lowest level, and multiplying the weights along each segment of the path. The outcome of this aggregation is a normalized vector of the overall weights of the options. The mathematical basis for determining the weights was established by Saaty.

About the authors Seetharaman Hariharan is Senior Registrar, School of Clinical Medicine and Research, University of West Indies, Queen Elizabeth Hospital, Bridgetown, Barbados. Prasanta K. Dey is Senior Lecturer, Aston Business School, Aston University, Birmingham, UK. Harley S.L. Moseley is Senior Lecturer, Areti Y. Kumar is Associate Lecturer and Jagathi Gora is Research Fellow, all at the School of Clinical Medicine and Research, University of West Indies, Queen Elizabeth Hospital, Bridgetown, Barbados.

312

Nursing work environment and quality of care: differences between units at the same hospital Jane McCusker Nandini Dendukuri Linda Cardinal Johanne Laplante and Linda Bambonye The authors Information about the authors can be found at the end of the article.

Keywords Nursing, Hospitals, Institutional care, Quality, Canada

Abstract The literature suggests that improvements in nurses’ work environments may improve the quality of patient care. Furthermore, monitoring the work environment through staff surveys may be a feasible method of identifying opportunities for quality improvement. This study aimed to confirm five proposed sub-scales from the Nursing Work Index – Revised (NWI-R) to assess the nursing work environment and the performance of these sub-scales across different units in a hospital. Data were derived from a cross-sectional survey of 243 nurses from 13 units of a 300-bed university-affiliated hospital in Quebec, Canada, during 2001. Using confirmatory factor analysis, the five subscales were confirmed; three of the sub-scales had greater ability to discriminate between units. Using hierarchical regression models, “resource adequacy” was the sub-scale most strongly associated with the perceived quality of care at the last shift. The NWI-R sub-scales are potentially useful for comparison of work environments of different nursing units at the same hospital.

Electronic access The Emerald Research Register for this journal is available at www.emeraldinsight.com/researchregister The current issue and full text archive of this journal is available at www.emeraldinsight.com/0952-6862.htm

International Journal of Health Care Quality Assurance Volume 17 · Number 6 · 2004 · pp. 313-322 q Emerald Group Publishing Limited · ISSN 0952-6862 DOI 10.1108/09526860410557561

Introduction Quality management programs in hospitals need information on the quality of patient care and services, assessed at the level of the structure, process, and outcomes of care. A critical aspect of the structure of care concerns nursing staff, for example: qualifications of nursing staff; the ratio of nurses to patients; satisfaction of nurses; turnover and absenteeism rates; and the organizational model of nursing (Steinbrook, 2002; Aiken et al., 2001; Needleman et al., 2002; Shamian and Lightstone, 1997; Sovie and Jawad, 2001; McGillis Hall et al., 2003). Nurses’ perceptions of their work environment reflect many of these structural issues. There is mounting evidence that the adverse working conditions and low satisfaction of nurses are associated with increased nursing turnover (Irvine and Evans, 1995; Leveck and Jones, 1996; Davidson et al., 1997) and with the process and outcomes of care (Mitchell and Shortell, 1997), including patient satisfaction (Weisman and Nathanson, 1985), mortality and readmission rates (Knaus et al., 1986; Baggs et al., 1992; Shortell et al., 1994), and needlestick injuries (Clarke et al., 2002). If hospital and nursing managers are to evaluate and improve the work environment of nurses at their institution, tools are needed to measure both objective and subjective aspects of the environment, both institution-wide and within specific nursing units. In this study we investigate a tool to measure nurses perceptions of the nursing work environment, and evaluate its performance in an acute-care hospital. A Nursing Work Index Revised (NWI-R) has been developed to measure the organizational attributes that characterize professional nursing practice environments, and which may affect quality of patient care and outcomes (Aiken and Patrician, 2000). Higher NWI-R scores have been found among nurses working in “magnet” or exemplary hospitals in the USA (Aiken et al., 1994), and dedicated AIDS units (Aiken et al., 1997). Characteristics of these organizations included: low nurse turnover rates; flexible scheduling; strong, The authors are grateful to the following: Rosa Sourial, N, MSc(A), for her help in developing and carrying out the survey; Serge Garneau, MSc, for assistance with interpretation of the results and obtaining absenteeism data; Eric Belzile, MSc, for assistance with the data analysis; Bruce Brown, MD, for helpful suggestions on a draft of the manuscript; and members of the Council of Nurses for distributing the questionnaires. This work was supported by the Quality Management Program, St Mary’s Hospital Center, Montreal, Que´bec, Canada.

313

Nursing work environment and quality of care

International Journal of Health Care Quality Assurance

Jane McCusker et al.

Volume 17 · Number 6 · 2004 · 313-322

supportive, and visible nursing leadership; good relationships with physicians; and career advancement opportunities. In a US AIDS care study of 40 units in 20 hospitals, the NWI-R differentiated nurses working in dedicated vs scattered bed units and explained between-unit differences in nurse burnout (Aiken and Patrician, 2000). The NWI-R was also used in a recent international study of working conditions of nurses (Aiken et al., 2001). Five sub-scales have been empirically derived, two of which were hypothesized would measure the hospital-wide environment (“nurse participation in hospital affairs” and “nursing foundations for quality of care”), and three would assess the environment on specific units “resource adequacy”, “nurse manager ability”, and “nurse-physician relations” (Lake, 2002). However, the ability of the subscales to detect differences in work environments across nursing units within the same hospital has not been evaluated. In contrast, Estabrooks et al. (2002), using data from a survey of 17,965 registered nurses working in hospitals in three Canadian provinces, proposed a one-factor solution to the NWI-R. The conceptual model for this research builds on the structure-process-outcome model for evaluating the quality of care proposed by Donabedian (1966). The structure of health care (including characteristics of the health care providers and the organization of care) affects the process of care (interaction between providers and patients), which in turn, affects patient outcomes (health outcomes and satisfaction) (Brook et al., 1996). The structural variables in our study, measured by the NWI-R, assess aspects of the organization of nursing practice at a hospital, both overall and on specific units. The nursing practice environment reflects the hospital managers’ approaches to organizing nursing care (Lake, 2002). A professional model of nursing care (characterized by a greater involvement of registered nurses with patients, greater decisionmaking authority and flexibility) is considered preferable to a bureaucratic model (characterized by hierarchical authority structure) (Flood and Scott, 1987). An environment that reflects a professional model of nursing care is hypothesized to improve nurse-patient interactions and the perceived quality of patient care which, in turn, results in improved patient health outcomes and greater satisfaction with care (Flood, 1994). This framework suggests that improvements in nurses’ work environment have the potential to improve the quality of patient care and reduce adverse outcomes (e.g. nosocomial infections, medication errors, complaints by patients and families). The study was conducted in a 300-bed, university-affiliated, government-funded general acute-care hospital in Montreal, in February 2001.

This hospital has undergone organizational change during the past decade, with the introduction of a program management model although nurses have been affected less than other hospital staff (Association des Hoˆpitaux du Que´bec, 2000). Other province-wide influences during the same period include a significant reduction in the nursing workforce, resulting from multiple factors including: a number of trained nurses leaving the profession in the 1990s due to the decrease in employment opportunities; a reduction in the number of openings in nursing training programs that have been reopened in the last two to three years; in the mid-1990s, an early retirement incentive program resulting in many senior and experienced nurses leaving the institutions; and an increasing difficulty in retaining nurses, particularly during the first few years after graduation. At the same time, hospitals have experienced a series of cuts to global operating budgets (Ministe`re de la Sante´ et des Services Sociaux, 2001). The Hospital’s Council of Nurses (legally mandated in Quebec hospitals for an advisory role for quality assurance) decided to survey its members, using questions from the NWI-R and from a recent five-country study of hospital nurses (Aiken et al., 2001) to assess perceptions of the work environment, quality of care, and changes over time in these factors. The intent was to use the survey instrument as a tool to identify specific aspects of the work environment among the Hospital’s nursing units that might be amenable to improvement, and subsequently to monitor changes in the work environment. The objectives of the study were to confirm the five work environment sub-scales proposed by Lake, and to assess their performance in comparing the work environment across different units in the same hospital. We hypothesized that, for the sub-scales measuring unit environment, between-unit differences would be greater than within-unit differences. In contrast, we expected greater homogeneity of responses for the subscales that measure hospital characteristics. We also hypothesized that those sub-scales that were sensitive to the environment on specific units would be associated with measures of the quality of care, frequency of adverse incidents, and absenteeism rate among the units.

Methods Study population and survey methods Questionnaires, which contained no names or identifying numbers, were distributed by members of the Hospital’s Council of Nurses to all active staff members of the Council. Envelopes were

314

Nursing work environment and quality of care

International Journal of Health Care Quality Assurance

Jane McCusker et al.

Volume 17 · Number 6 · 2004 · 313-322

provided to return the questionnaires to the Quality Assessment office. Staff were also asked to mail back, separately, a numbered postcard on which they checked either that they had returned the questionnaire or that they did not wish to participate. A second distribution of questionnaires was made to those staff who did not return the postcard. The study was approved by the Hospital’s Research Ethics Committee.

were included in the subscales proposed by Lake (2002). Because the latter had not been published at the time we began our survey, we included all items from the original, theoretically-based subscales on autonomy, control over work environment, relationships with physicians, and organizational support for staff (Aiken and Patrician, 2000), plus other items judged to be relevant to our hospital. Nurses were asked to rate each statement on the extent to which it was present in their current job using a four-point scale (strongly agree, somewhat agree, somewhat disagree, strongly disagree).

Survey questionnaire The survey questionnaire was based in part on several questions contained in the survey of Pennsylvania nurses (Aiken et al., 2001). Employment information Work status questions included: full- vs part-time (including availability), temporary vs permanent employment; length of employment at the Hospital (less than one year, one to less than three years, three or more years); job title (nurse, nursing assistant, nurse manager); unit(s) assigned to; and unit of last shift. Work environment The nursing survey included 28 items selected from the NWI-R, of which 21 (listed in Table I)

Quality of care Nurses were asked two questions about how they would describe the quality of patient care: care in the Hospital overall and nursing care at their last shift (excellent, good, fair, poor). Adverse events Nurses were asked how often during the past year they had observed each of the following incidents: patient received wrong medication or dose, nosocomial infections, complaints from patients or their families, work-related injuries to employees,

Table I Factor loadings (n ¼ 160) Standardized factor loadings

R2

Sub-scale 1: nurse participation in hospital affairs 1. Opportunity for nurses to participate in policy decisions 2. Administration that listens and responds to employee concerns 3. Nurse managers consult with staff on daily problems and procedures 4. Staff nurses have the opportunity to serve on hospital and nursing committees

0.67 0.67 0.66 0.45

0.45 0.45 0.44 0.20

Sub-scale 2: nursing foundations for quality of care 1. A clear philosophy of nursing care that pervades the patient care environment 2. An active quality improvement program 3. Active inservice/continuing education programs for nurses 4. Working with nurses who are clinically competent 5. A mentorship program for newly hired nurses 6. Patient care assignments that foster continuity of care

0.61 0.73 0.63 0.48 0.44 0.40

0.38 0.53 0.40 0.23 0.19 0.16

0.70

0.49

0.73 0.66 0.70

0.54 0.44 0.49

Sub-scale 4: staffing and resource adequacy 1. Enough staff to get the work done 2. Enough nurses on staff to provide quality patient care 3. Enough time and opportunity to discuss patient care problems with other nurses 4. Adequate support services allow me to spend time with my patients

0.93 0.87 0.70 0.59

0.86 0.76 0.49 0.35

Sub-scale 5: nurse-physician relations 1. Much teamwork between nurses and physicians 2. Collaboration (joint practice) between nurses and physicians 3. Physician and nurses have good working relationships

0.85 0.78 0.73

0.72 0.61 0.53

Sub-scale and component items

Sub-scale 3: nurse manager ability, leadership and support of nurses 1. A nurse manager is a good manager and leader 2. A nurse manager backs up the nursing staff in decision making, even if the conflict is with a physician 3. A supervisory staff that is supportive of nurses 4. Praise and recognition for a job well done

315

Nursing work environment and quality of care

International Journal of Health Care Quality Assurance

Jane McCusker et al.

Volume 17 · Number 6 · 2004 · 313-322

incidents of verbal abuse directed toward employees (never, rarely, occasionally, frequently). Translation The survey questionnaire was translated into French and independently back-translated to validate the accuracy of the translation.

.

.

Absenteeism Previous research on nurses’ job satisfaction and work environment has used turnover rates as an outcome, although one study found an association between nosocomial urinary tract infection rates and absenteeism (Taunton et al., 1994). In our study, because of the small size of the nursing units, estimates of turnover rates would be expected to be unstable; instead we used absenteeism rates due to illness and work-related injuries. The absenteeism rate in each nursing unit was estimated from the Hospital’s administrative databases as the total number of hours missed due to illness divided by the total available working hours in that unit. Statistical methods Responses of strongly disagree, somewhat disagree, somewhat agree and strongly agree to each question of the NWI questionnaire were coded as 1, 2, 3 and 4, respectively. Confirmatory factor analysis A confirmatory factor analysis (Stevens, 1996) of the five sub-scales was performed in order to validate the sub-scale structures described by Lake (2002) and Estabrooks et al. (2002). We observed a high correlation between sub-scales 1 and 2 of Lake in our sample, so we also evaluated a model where these two sub-scales (i.e. nurse participation in hospital affairs and nurse foundations for quality of care) were combined into a single sub-scale. Questions were forced to load on to a single sub-scale (or factor), but correlation between the sub-scales was allowed. A maximum likelihood solution for the factor loadings was obtained using the PROC CALIS procedure in the SAS software program (SAS Institute Inc., 1999). For each model, parameter estimates (factor loadings, variances of individual questions, covariances between sub-scales) were examined to ensure the models were well defined and there were no anomalies due to improper specification of the model. T-tests were used to evaluate whether individual parameters were significantly different from zero. Several different measures of goodnessof-fit were used to compare the different models, as no single method is considered superior: . the chi-square statistic, lower values of which indicate better fit;

.

the adjusted goodness of fit (AGFI), parsimonious goodness of fit index (PGFI) and the comparative fit index (CFI) all of which range between 0 and 1, with values closer to 1 indicating a good fit; the Bayesian Information Criterion (BIC), lower values of which are indicative of better fit; and the expected cross validation index (ECVI), lower values of which indicate that the model would cross-validate in another sample of the same size.

When comparing models with a similar fit, the BIC tends to prefer more parsimonious models. A difference of six to ten points on the BIC is considered strong evidence in favour of a model (Raftery A E 1995). The ECVI is useful for comparing models when the sample size is not large. For each question we report the standardized factor loading, which gives an indication of the importance of the question to the sub-scale it is part of, and the R2 statistic, which is a measure of the variance in the sub-scale explained by the question. Reliability of sub-scales The internal consistency of each sub-scale was assessed using Cronbach’s alpha coefficient. An individual nurse’s score on each sub-scale was estimated as the average score across questions on that sub-scale; a maximum of one missing item on each sub-scale was allowed. For each sub-scale, an intra-class correlation co-efficient (ICC) was calculated to compare between-unit to within-unit variation. Higher values of the ICC indicate the sub-scale reflects unit-level characteristics, while lower values indicate the sub-scale reflects hospital-wide characteristics. Association between unit-level sub-scale scores and outcomes A unit-level score on each sub-scale was calculated by estimating the mean sub-scale score in a unit. Univariate and multivariate logistic regression models were used to study if the unit-level subscale scores were associated with perceived quality of care and adverse events. These models were fit using generalized estimating equations to allow for dependence between nurses in the same unit (Liang and Zeger, 1986). Multivariate regression models included all sub-scales, nurses’ work status (full-time (yes/no), permanent (yes/no)), the number of years worked in the Hospital (more than three years (yes/no)) and the language of the questionnaire (English/French). The strength of the association between absenteeism rate and unitlevel sub-scale score was estimated using Spearman’s rank correlation coefficient.

316

Nursing work environment and quality of care

International Journal of Health Care Quality Assurance

Jane McCusker et al.

Volume 17 · Number 6 · 2004 · 313-322

Statistical analyses were implemented using SAS (SAS Institute Inc., 1999) software packages. In the interpretation of the results, odds ratios of 2 or more (or 0.5 or less) were considered to be clinically significant. Results were considered statistically significant if the p-value of a two-sided test was less than 0.05 or the 95 percent confidence interval (CI) did not include the null value (1 for odds ratios, 0 for Spearman’s correlation coefficient).

Confirmation of sub-scales The confirmatory factor analysis was conducted using the 160 questionnaires with complete data on the 21 items comprising the five sub-scales. The five-factor model proposed by Lake (2002) was superior on all measures of goodness of fit, particularly in comparison to the one-factor model proposed by Estabrooks (Estabrooks et al., 2002) (Table III). The AGFI, PGFI and CFI were all relatively high indicating a good fit. The fourfactor model (combining sub-scales 4 and 5 of Lake) performed similarly to the five-factor model on most indices. However, the BIC for the fivefactor model was 7.33 points less compared to the four-factor model providing strong evidence of its superiority (Raftery, 1995). Thus the five-factor model was used for the remainder of our analyses. There were no anomalies in any of the individual parameter values (factor loadings, error variances and covariances), all of which were found to be significantly different from zero. Table I lists the standardized factor loadings together with the R2 values. The standardized factor loadings ranged between 0.59-0.93 for questions on sub-scales 3, 4 and 5 and correspondingly high R2 values suggesting that these questions contributed to explaining the variance on these sub-scales. However, on sub-scales 1 and 2 some standardized factor loadings ranged between 0.40-0.45 suggesting they were less important in explaining the variance on those sub-scales. There was a high correlation (0.96) between subscales 1 and 2. Correlation between the remaining sub-scales ranged from 0.58 to 0.80.

Results A total of 512 nurses were eligible for the study and were given a questionnaire. Completed questionnaires were received from 336 nurses (participation rate of 65.6 percent). In order to compare our survey results with those in the literature, we excluded 16 nurse managers, 35 nurse assistants, and two with missing information on job title, giving a sample size of 283 nurses. Because we were interested in comparing well-defined nursing units, we excluded 37 nurses who did not work on a specific unit or who worked on multiple units. The characteristics of the 246 nurses in the final sample are shown in Table II. There were no significant differences in any of these characteristics between the nurses included and those excluded from the final sample (data not shown). Table II Characteristics of 246 nurses in 13 units

N

n

%

Full-time Permanent Worked at hospital more than three years Language of questionnaire: English

246 245 245 246

141 224 217 217

57.3 91.4 88.6 88.2

Quality of care Overall quality of care: excellent or good Quality of care (at last shift): excellent or good

241 245

151 195

62.7 79.6

Incidents observed occasionally or frequently in the past year Patient received wrong medication Nosocomial infections Complaints from patients Work-related injuries to employees Incidents of verbal abuse

235 219 239 239 242

85 130 175 157 184

36.2 59.4 65.7 65.7 76.0

244 234

Mean 2.43 2.69

SD 0.63 0.56

240 246 241

2.49 2.03 2.79

0.74 0.73 0.70

Sub-scale scores Nurse participation in hospital affairs Nursing foundations for quality of care Nurse manager ability, leadership and support of nurses Staffing and resource adequacy Nurse-physician relations Note: N less than 246 when data were missing

Reliability of sub-scales All sub-scales had adequate internal consistency (Table IV). The ICCs for each sub-scale are also shown in Table IV. Two of the three sub-scales that had been hypothesized to measure primarily the unit-specific work environment had higher ICC values (“nurse-physician relations” and “nurse manager ability”).

Work environment, quality of care, and absenteeism Table V shows the relationships between the unitlevel sub-scale scores and the quality of care and adverse event measures. The odds ratios, adjusted for nurse characteristics, were the strongest for the two quality of care outcomes; they were weaker and not statistically significant for the five measures of adverse events. “Resource adequacy” was significantly associated with both overall quality of care at the hospital and with the quality of nursing care on the last shift. “Nurse manager ability” and “nurse-physician relations” were

317

Nursing work environment and quality of care

International Journal of Health Care Quality Assurance

Jane McCusker et al.

Volume 17 · Number 6 · 2004 · 313-322

Table III Results of confirmatory factor analysis: indices of goodness-of-fit (n ¼ 160) No. of factors in the model 1 4 5

Chi-square statistic

dfs

AGFI

PGFI

CFI

BIC

ECVI

90 percent confidence interval

568.26 379.31 346.61

190 184 179

0.68 0.77 0.78

0.67 0.71 0.71

0.76 0.88 0.89

2 396.02 2 554.52 2 561.85

4.15 3.05 2.92

3.70, 4.65 2.71, 3.45 2.60, 3.29

Notes: dfs: degrees of freedom; AGFI: adjusted goodness of fit index; PGFI: parsimonious goodness of fit index; CFI: comparative fit index; BIC: Bayesian information criterion; ECVI: expected cross-validation index

Table IV Internal consistency and intra-class correlation coefficients of sub-scales among nursing staff Sub-scale 1 2 3 4 5

Nurse participation in hospital affairs Nursing foundations for quality of care Nurse manager ability, leadership, and support of nurses Staffing and resource adequacy Nurse-physician relations

associated with overall quality, but not with quality at the last shift. Lower rates of absenteeism were associated with higher mean unit score on the “resource adequacy” sub-scale (Figure 1). The Spearman’s rank correlation coefficient was 2 0.39 (95 percent CI 20.77, 0.21). There was no evidence of a relation in the hypothesized direction between absenteeism and the four other sub-scales.

Discussion This study aimed to evaluate five sub-scales to assess perceptions of the nursing work environment among nurses working on 13 units at a university-affiliated Quebec hospital. These subscales have been used to compare nurses working in different hospitals but have not previously been used to compare units within the same hospital. The results suggest that several of these sub-scales are significantly associated with perceived quality of care at the hospital and are potentially useful tools for management to use to assess and monitor the work environment of nurses. Our confirmatory factor analysis confirmed the five sub-scales proposed by Lake (2002) in our sample of nurses, using a sub-set of 21 items from the original sub-scales. Two of the six items comprising the “nursing foundations for quality of care” performed less well in this analysis, and might be excluded. The sub-scales had adequate levels of internal consistency reliability (coefficients between 0.69 and 0.84). Lake (2001) hypothesized that the “nurse manager ability”, “resource adequacy”, and “nurse-physician relations” sub-scales would reflect unit-specific characteristics whereas the sub-scales on “nurse

n

Cronbach’s coefficient alpha (standardized)

Intra-class correlation coefficient

213 184 203 214 225

0.71 0.71 0.77 0.83 0.84

0.16 0.16 0.24 0.17 0.24

participation in hospital affairs” and “nursing foundations for quality of care” would reflect hospital-wide characteristics. These hypotheses were at least partly confirmed. The values of the intra-class correlation coefficients (0.11 to 0.24) indicate only a small to moderate level of clustering of scores within units. Thus, the scales appear to reflect primarily the hospital-wide work environment. The “nurse manager ability” and “nurse-physician relations” sub-scales had the highest intra-class correlation coefficients, suggesting a greater clustering within units on these sub-scales. However, the “resource adequacy” sub-scale was the most sensitive of the five sub-scales to differences in self-reported quality of care among the units, particularly in the quality of nursing care at the last shift. The “nurse manager ability” and “nurse-physician relations” sub-scales were also associated with overall quality of care but not with quality of nursing care at the last shift. The “resource adequacy” sub-scale was also the scale most strongly associated with absenteeism rates, although this association was not statistically significant. The study was conducted in the context of a recent organizational change to a program management model, nation-wide reductions in the nurse:population ratio (Canadian Nurses Association, 1999), and provincial budget cuts (Association des Hoˆpitaux du Que´bec, 2000; Ministe`re de la Sante´ et des Services Sociaux, 2001). At the same time, nurses have been asked to assume new roles and responsibilities in relation to patients, families, and other health care providers. Nurses must be able to respond to more complex health care situations, technological innovations and shorter hospital stays which place greater

318

Nursing work environment and quality of care

International Journal of Health Care Quality Assurance

Jane McCusker et al.

Volume 17 · Number 6 · 2004 · 313-322

Table V Associations between unit-level sub-scale scores, quality of care and adverse outcomes Outcome and sub-scale

Univariate models Odds ratiod 95% CI

Multivariate modelsc Odds ratiod 95% CI

Overall quality of carea (n 5220) Nurse participation Nursing foundations Nurse manager ability Resource adequacy Nurse-physician relations

3.0 3.1 2.8 3.4 2.5

0.8; 0.8; 1.4; 1.6; 1.3;

10.3 12.2 5.7 7.4 4.9

3.1 3.1 2.8 3.6 2.5

0.9; 0.8; 1.4; 1.5; 1.3;

10.5 12.7 5.6 8.7 4.7

Quality of care in the last shifta (n 5224) Nurse participation Nursing foundations Nurse manager ability Resource adequacy Nurse-physician relations

2.0 2.3 1.7 4.2 1.2

0.5; 0.5; 0.5; 1.7; 0.4;

8.2 10.9 6.1 10.7 3.7

1.9 2.1 1.5 4.5 1.1

0.4; 0.4; 0.4; 1.7; 0.4;

8.2 10.2 5.8 11.9 3.4

Wrong medicationb (n 5 213) Nurse participation Nursing foundations Nurse manager ability Resource adequacy Nurse-physician relations

1.6 0.7 1.3 1.7 0.7

0.3; 0.1; 0.5; 0.8; 0.3;

7.2 3.8 3.4 3.4 1.6

1.4 0.7 1.1 2.1 0.7

0.3; 0.1; 0.4; 0.8; 0.3;

8.4 4.9 3.2 5.3 1.7

Nosocomial infectionsb (n 5 193) Nurse participation Nursing foundations Nurse manager ability Resource adequacy Nurse-physician relations

1.4 0.8 2.9 1.3 1.0

0.2; 0.1; 0.5; 0.2; 0.1;

8.4 6.1 15.8 10.0 7.0

1.9 1.0 3.1 1.8 0.9

0.3; 0.1; 0.5; 0.2; 0.1;

12.0 8.2 19.1 13.5 7.1

Complaints from patients or their familiesb (n 5 218) Nurse participation Nursing foundations Nurse manager ability Resource adequacy Nurse-physician relations

0.3 0.2 1.4 1.7 0.7

0.1; 0.0; 0.4; 0.6; 0.2;

2.0 2.4 5.1 4.8 2.0

0.3 0.2 1.3 1.7 0.6

0.0; 1.8 0.0; 2.2 0.4; 4.3 0.6;4.7 0.2; 1.7

Verbal abuseb (n 5222) Nurse participation Nursing foundations Nurse manager ability Resource adequacy Nurse-physician relations

2.1 4.6 0.9 2.1 1.6

0.5; 0.9; 0.4; 0.5; 0.6;

8.1 23.9 2.0 9.2 4.8

2.2 5.0 0.8 2.7 1.4

0.4; 0.7; 0.3; 0.5; 0.4;

10.4 34.2 1.9 14.5 5.6

Work-related injuriesb (n 5 219) Nurse participation Nursing foundations Nurse manager ability Resource adequacy Nurse-physician relations

1.0 0.9 0.8 1.9 1.1

0.1; 0.1; 0.1; 0.5; 0.3;

7.3 13.3 4.9 7.7 4.5

1.1 0.9 0.7 2.0 0.9

0.1; 0.1; 0.1; 0.6; 0.2;

9.4 14.1 4.0 6.9 3.5

Notes: a Responses of excellent or good (vs fair or poor); b Responses of never of rarely (vs occasionally or frequently); c Adjusted for the following nurse characteristics: full-time, permanent, more than three years, language of questionnaire; d Odds ratios greater than 1 indicate that greater satisfaction with work environment is associated with better quality and lower frequency of adverse events

emphasis on discharge planning to prepare patients and families for discharge and provide appropriate linkages with community providers (Association des Hoˆpitaux du Que´bec, 1999; Canadian Nurses Association, 1999). Nurses are also expected to play a more autonomous role within multidisciplinary teams. These changes require the development and mastering of critical

thinking skills (Ministe`re de la Sante´ et des Services Sociaux, 2001). It is not possible to attribute the results of this descriptive study to any one of the above changes. Nurses in our survey scored the lowest on the “resource adequacy” sub-scale and highest on the “nurse-physician relations” sub-scale. Results from the five-country study also indicate that

319

International Journal of Health Care Quality Assurance

Jane McCusker et al.

Volume 17 · Number 6 · 2004 · 313-322

Figure 1 Relation between average score on work environment sub-scale and absenteeism rate on 13 nursing units

Nursing work environment and quality of care

320

Nursing work environment and quality of care

International Journal of Health Care Quality Assurance

Jane McCusker et al.

Volume 17 · Number 6 · 2004 · 313-322

nurse-physician relationships are less problematic than other aspects of the work environment (Aiken et al., 2001). A longitudinal study with two-year follow-up in an Ontario hospital found no effect of organizational changes during a two-year followup on the perceived staff competence as caregivers (Woodward et al., 1999). Some potential limitations of this study should be noted. First, we used a cross-sectional design, which did not allow us to assess causality in the relationships examined. Second, our indicators of quality of care and frequency of adverse events are subjective and may be biased (for example, by recall and other response biases). Third, the lack of statistical significance of some of our results may be due to the small number of nurses in some units. Finally, the study was conducted at a single hospital, and the results may not be generalizable to other hospitals or settings. Future research should be conducted in different settings, should use longitudinal designs to assess the predictive validity and responsiveness of the sub-scales, and should use independent and, ideally, objective measures of quality of care. In conclusion, the results of this study suggest that several work environment sub-scales, particularly “resource adequacy”, but also “nurse manager ability” and “nurse-physician relations” may be useful tools for management to monitor the environment for nursing care within a hospital and to assist in the development of quality improvement strategies. The sub-scales evaluated in this study may be useful as outcome measures to assess the effects of quality improvement initiatives that aim to improve specific aspects of the work environment of hospital nurses.

Hospitaliers du Que´bec, Publications de l’Association des Hoˆpitaux du Que´bec, Montre´al. Baggs, J.G., Ryan, S.A., Phelps, C.E., Richeson, F. and Johnson, J.E. (1992), “The association between interdisciplinary collaboration and patients’ outcomes in a medical intensive care unit”, Heart and Lung, Vol. 21, pp. 18-24. Brook, R.H., McGlynn, E.A. and Cleary, P.D. (1996), “Quality of health care – part 2: measuring quality of care”, New England Journal of Medicine, Vol. 335 No. 13, pp. 966-70. Canadian Nurses Association (1999), Repair, Realign, and Resource Health Care, Canadian Nurses Association, Montreal. Clarke, S.P., Sloane, D.M. and Aiken, L.H. (2002), “Effects of hospital staffing and organizational climate on needlestick injuries to nurses”, American Journal of Public Health, Vol. 92 No. 7, pp. 1115-19. Davidson, H., Folcarelli, P.H., Crawford, S., Duprat, L.J. and Clifford, J.C. (1997), “The effects of health-care reforms on job satisfaction and voluntary turnover among hospitalbased nurses”, Medical Care, Vol. 35 No. 6, pp. 634-45. Donabedian, A. (1966), “Evaluating the quality of medical care”, Milbank Memorial Fund Quarterly, Vol. 44 No. 3, pp. 166-203. Estabrooks, C.A., Tourangeau, A.E., Humphrey, C.K., Hesketh, K.K., Giovanetti, P., Thomson, D., Wong, J., Acorn, S., Clark, H. and Shamian, J. (2002), “Measuring the hospital practice environment: a Canadian context”, Research in Nursing and Health, Vol. 25, pp. 256-68. Flood, A.B. (1994), “The impact of organizational and managerial factors on the quality of care in health-care organizations”, Medical Care Research and Review, Vol. 51, pp. 381-428. Flood, A.B. and Scott, W.R. (1987), Hospital Structure and Performance, Johns Hopkins University Press, Baltimore, MD. Irvine, D.M. and Evans, M.G. (1995), “Job satisfaction and turnover among nurses: integrating research findings across studies”, Nursing Research, Vol. 44 No. 4, pp. 246-53. Knaus, W.A., Draper, E.A., Wagner, D.P. and Zimmerman, J.E. (1986), “An evaluation of outcome from intensive care in major medical centers”, Annals of Internal Medicine, Vol. 104 No. 3, pp. 410-18. Lake, E.T. (2002), “Development of the practice environment scale of the nursing work index”, Research in Nursing and Health, Vol. 25, pp. 176-88. Leveck, M.L. and Jones, C.B. (1996), “The nursing practice environment, staff retention, and quality of care”, Research in Nursing and Health, Vol. 19, pp. 331-43. Liang, K.Y. and Zeger, S.L. (1986), “Longitudinal data analysis using generalized linear models”, Biometrika, Vol. 73, pp. 13-22. McGillis Hall, L., Baker, G.R., Pink, G.H., Sidani, S., O’Brien-Pallas, L. and Donner, G.J. (2003), “Nurse staffing models as predictors of patient outcomes”, Medical Care, Vol. 41 No. 9, pp. 1096-109. Ministe`re de la Sante´ et des Services Sociaux (2001), Rapport du Forum National sur la Planification de la Main d’Oeuvre Infirmie`re, Gouvernement du Que´bec, Montre´al. Mitchell, P.H. and Shortell, S.M. (1997), “Adverse outcomes and variations in organization of care delivery”, Medical Care, Vol. 35 No. 11, pp. NS19-NS32. Needleman, J., Buerhaus, P., Mattke, S., Stewart, M. and Zelevinsky, K. (2002), “Nurse-staffing levels and the quality of care in hospitals”, New England Journal of Medicine, Vol. 346 No. 22, pp. 1715-22.

References Aiken, L.H. and Patrician, P.A. (2000), “Measuring organizational traits of hospitals: the revised nursing work index”, Nursing Research, Vol. 49 No. 3, pp. 146-53. Aiken, L.H., Sloane, D.M. and Lake, E.T. (1997), “Satisfaction with in-patient acquired immunodeficiency syndrome care: a national comparison of dedicated and scattered-bed units”, Medical Care, Vol. 35 No. 9, pp. 948-62. Aiken, L.H., Smith, H.L. and Lake, E.T. (1994), “Lower medicare mortality among a set of hospitals known for good nursing care”, Medical Care, Vol. 32 No. 8, pp. 771-87. Aiken, L.H., Clarke, S.P., Sloane, D.M., Sochalski, J.A., Busse, R., Clarke, H., Giovannetti, P., Hunt, J., Rafferty, A.M. and Shamian, J. (2001), “Nurses’ reports on hospital care in five countries”, Health Affairs, Vol. 20 No. 3, pp. 43-53. Association des Hoˆpitaux du Que´bec (1999), Identification des Compe´tences Requises par les Infirmie`res et le Personnel d’Assistance en Fonction des Besoins des Cliente`les du Re´seau de la Sante´ et des Services Sociaux, Publications de l’Association des Hoˆpitaux du Que´bec, Montre´al. Association des Hoˆpitaux du Que´bec (2000), Organisation par Programme-cliente`le: l’Expe´rience des Centres

321

Nursing work environment and quality of care

International Journal of Health Care Quality Assurance

Jane McCusker et al.

Volume 17 · Number 6 · 2004 · 313-322

Raftery, A.E. (1995), “Bayesian model selection in social research”, in Marsden, P. (Ed.), Sociological Methodology, Blackwells, Cambridge, MA, pp. 111-96. SAS Institute Inc. (1999), SAS Online Doc, Version 8, SAS Institute Inc., Cary, NC, available at: www.sas.com Shamian, J. and Lightstone, E.Y. (1997), “Hospital restructuring initiatives in Canada”, Medical Care, Vol. 35 No. 10, pp. OS62-0S69. Shortell, S.M., Zimmerman, J.E., Rousseau, D.M., Gillies, R.R., Wagner, D.P., Draper, E.A., Knaus, W.A. and Duffy, J. (1994), “The performance of intensive care units: does good management make a difference?”, Medical Care, Vol. 32 No. 5, pp. 508-25. Sovie, M.D. and Jawad, A.F. (2001), “Hospital restructuring and its impact on outcomes”, Journal of Nursing Administration, Vol. 31 No. 12, pp. 588-600.

Steinbrook, R. (2002), “Nursing in the crossfire”, New England Journal of Medicine, Vol. 346 No. 22, pp. 1757-66. Stevens, J. (1996), Applied Multivariate Statistics for the Social Sciences, Lawrence Erlbaum Associates, Mahwah, NJ. Taunton, R.L., Kleinbeck, S.V.M., Stafford, R., Woods, C.Q. and Bott, M.J. (1994), “Patient outcomes: are they linked to registered nurse absenteeism, separation, or work load?”, Journal of Nursing Administration, Vol. 24 No. 4S, pp. 48-55. Weisman, C.S. and Nathanson, C.A. (1985), “Professional satisfaction and client outcomes”, Medical Care, Vol. 23 No. 10, pp. 1179-92. Woodward, C.A., Shannon, H.S., Cunningham, C., McIntosh, J., Lendrum, B., Rosenbloom, D. and Brown, J. (1999), “The impact of re-engineering and other cost reduction strategies on the staff of a large teaching hospital: a longitudinal study”, Medical Care, Vol. 37, pp. 556-69.

About the authors Jane McCusker is Head, Department of Clinical Epidemiology and Community Studies and Quality Assessment Unit, St Mary’s Hospital, and Professor, Department of Epidemiology and Biostatistics, McGill University, Montreal, Canada. Nandini Dendukuri is Biostatistician, Department of Clinical Epidemiology and Community Studies, St Mary’s Hospital, and Assistant Professor, Department of Epidemiology and Biostatistics, McGill University, Montreal, Canada. Linda Cardinal is Quality Assessment Analyst, Quality Management, Johanne Laplante is Clinical Nurse Specialist, Mental Health Program and Linda Bambonye is Vice-President of Operations and Nursing, all at St Mary’s Hospital, Montreal, Canada.

322

A long journey always begins with a single step (Lao-tzu, 604 BC -531 BC ).

Quality management in health care: a 20-year journey

The issue of quality in health care

Ulises Ruiz

The author Ulises Ruiz is Head, TQM Division, Instituto Universitario de Evaluacio´n Sanitaria, Facultad de Medicina, Universidad Complutense de Madrid, Madrid, Spain.

Keywords Quality management, Patient care, Safety, Spain

Abstract In this article, the total quality programme in the Spanish healthcare system (1986-1992) and the subsequent quality improvement steps that have led to definition and implementation of such an integrated framework, seeking a quality management system and patient safety, are discussed.

Electronic access The Emerald Research Register for this journal is available at www.emeraldinsight.com/researchregister The current issue and full text archive of this journal is available at www.emeraldinsight.com/0952-6862.htm

International Journal of Health Care Quality Assurance Volume 17 · Number 6 · 2004 · pp. 323-333 q Emerald Group Publishing Limited · ISSN 0952-6862 DOI 10.1108/09526860410557570

Health care providers, that is the professionals of the health care system, and the health care financiers and the politicians (financiers and politicians may be consumers/patients, but they are not health care providers of professional services) are confronted with one of the most perplexing issues in today’s health care: the debate on how to improve the quality of care delivered to the community without losing historically acquired roles and responsibilities (BMA, 1988; Coile, 1990; Arndt and Bigelow, 1995; Bigelow and Arndt, 1995; Blumenthal, 1996). Although many approaches to improve quality in health care organisations have been implemented along the past century, little systematic research assessing their effectiveness has been reported (JCAHO, 1988; Harrigan, 1992; Øvretveit, 2003). In the past two decades the acknowledgement of either medical errors or system errors in health care organisations, leading to patient injury and even death, has generated an increasing body of knowledge found in the literature as well as potential health care legislation, media exposure and consumer awareness (Leape et al., 1991, Leape, 1994; Liang, 1999; IoM, 1999, 2001; Becher and Chassin, 2001). Patient safety, defined either as freedom from accidental injury due to medical care, or absence of medical errors or absence of misuse of services has become a major issue for health care quality. A number of questions remain unanswered worldwide: . Who is ultimately responsible for the care of people? . Which is the best health care system approach? . How to assure the quality and safety of care? . Which information is needed in health care systems for its improvement? . What are the expectations and demands of health care professionals? . What are the expectations and the demands of the community? Traditionally the technical knowledge of medical and nursing professionals has been considered sufficient for assuring quality and safety for the health care provided to the citizen. However, today’s health care centres are complex organisations where appropriate medical care requires administrative and managerial

323

Quality management in health care: a 20-year journey

International Journal of Health Care Quality Assurance

Ulises Ruiz

Volume 17 · Number 6 · 2004 · 323-333

support to get the patient safely discharged. Organisational change has to emphasise the links between quality systems and working practices, as well as appropriate and safe care. Developing and implementing such an approach in health care institutions and systems requires fundamental changes in the organisation’s culture, structure and working methods. Health system reforms in most industrialised countries are increasingly asking for active involvement of physicians and other health care professionals together with an increasing recognition that health care providers have to respond to the preferences and values of the patients as their customers (Thompson, 1999). Furthermore, interested parties such as service providers, service payers, customers and politicians are demanding reliable ways for assessing the services provided. Thus the attempts to increase the efficiency and effectiveness of health care systems have to deal with the equity issue (Relman, 1988) rather than that of competition (Thompson, 1998). The establishment of normalised criteria and standards for appropriate and safe health care provision as well as mechanisms for evaluating the systems and organisation’s performance, country and worldwide, is still a crucial issue in need of a valid and recognised answer (Scrivens, 1995; Shaw, 2000; Schyve, 2000a, b). Nevertheless, normalised criteria are being established through evidence-based medicine and practice guidelines – approaches which are beyond the scope of this article. A full review of the present state-of-the-art on these matters can be found in a recent supplement of the Medical Journal of Australia (2004). The specific state of these approaches in the USA is discussed in Preventive Medicine (Larson, 2003) and a comparison of European and American experiences is carried out in the Joint Commission Journal of Quality Improvement (Gross, 2000). Even though a common approach to accreditation is urgently needed there are some key differences which prevent the direct application of the traditional accreditation approaches in countries like Spain where health care has been, up to recently, mostly a public sector service (Simo´n and Ruiz, 1995a, b). A tradeoff of the classical technical quality assurance and professional power well known by health care-providers and the most recent approach of continuous improvement of organisations, as developed in the industrial and services sectors, can be achieved through an integrated approach. Such an approach should allow the measuring, assessing and comparing of organisations’ performance through a

combination of internationally accepted certification standards, accreditation schemes and excellence models criteria. The need for an organisational change in the health-care system Health care institutions are typical professional organisations where a specific profession establishes its rules for managing the whole of the organisation and does not accept willingly changes that might limit its power. To move from the old organisational structures of health care institutions to a new organisational and managerial design, deliberate transitional steps are necessary. It is important to emphasise that, at least in Spain, health care institutions do not have the required managerial maturity for seeking organisational excellence at once. Quality systems have to be established first as an initial step for building the process foundations needed in a modern organisation. Self-assessment and continuous quality improvement as a managing paradigm towards organisational excellence have to be pursued in a step-wise fashion in order to have reasonable probability for success. If carefully designed and well placed, these transitional measures will permit a policy to evolve along consensual lines and can signal the overall direction of change, reassuring stakeholders that change is taking place smoothly and that the most immediate problems are being dealt with. It also helps to ensure that longer-term systematic objectives are not compromised by short-term political imperatives. This is a long-term venture which, in the Spanish case, was started in 1986 with a total quality programme for the Spanish health care system (Ruiz et al., 1992) and is being pursued as an integrated approach for certification, accreditation and continuous improvement of the health care organisations focused on patient safety (Ruiz et al., 1999).

The Spanish health-care system Currently the health care system in Spain is a mixed public-private system where the public sector is responsible for about 80 per cent of the total cost which represents 6.5 per cent of the gross national product (GNP). The Spanish Constitution of 1979 recognises the right of all citizens to health care coverage and health protection, and places on the State the responsibility for transferring health care services management to the 17 autonomous communities and for coordinating the establishment of general

324

Quality management in health care: a 20-year journey

International Journal of Health Care Quality Assurance

Ulises Ruiz

Volume 17 · Number 6 · 2004 · 323-333

health care policies through a national health council. In 1986 the General Health Law was passed and a National Health System was created which was publicly funded, in large part from the state budget. This established a quality control and assessment framework for all health care organisations within the system. By the end of 1987 97.1 per cent of the Spanish population was covered for health care through the public health care system, which was financially managed and operated by the National Institute of Health, INSALUD, as the Government agency for the management of the Spanish health care system. This law provided the basic foundations for the subsequent development of all the elements needed to set up a system that would assure the quality of the health services. Therefore, according to the Constitution and the General Health Law the budget and managing responsibility for the regional health care services has been progressively transferred from the INSALUD to the existing 17 autonomous regions. The Ministry of Health coordinates all health care services through the National Health Council. In May 2003 a new bill on “Cohesion and Quality in the National Health Care System” was passed in Spain. In this bill a whole chapter deals with quality issues in health care, establishing that quality improvement should be the focus for any initiative in public and private health care organisations. Nevertheless the development of quality control and assessment mechanisms, as established by law, has faced major obstacles since the change to a new system has greatly suffered from existing shortcomings. First, overcoming the historical absence of adequate budgeting and accounting mechanisms is proving difficult. Second, the different political environment in the respective regions has made transferring managing responsibility a complex process with respect to the amount of expenditure to be transferred, the timing and procedures for transfer, and mechanisms for compensation. Third, accountability is still deficient due to chronic sliding of deficits; problems with personnel and rigid operational procedures still persist. Despite these difficulties significant achievements in the management of the regional and central health care services in Spain aimed at increasing consumer choice, purchasers of services information, expenditure control and creating incentives for performance, should be mentioned. Among those achievements the development of the Spanish health care system Total Quality Management Plan from 1986 through 1992 allowed the implementation of a normalised

patient data base system, a normalised costactivity/accountability information system and the pursuit of accepted self-assessment certification and accreditation schemes.

The first step: the Spanish Total Quality Management Plan In 1986, according to the General Health Law passed that year, a programme according to the Total Quality Management Plan, Phase One (Ruiz, 1988) was launched by the newly established Total Quality Unit under the General Directorate of the INSALUD, within the Spanish Ministry of Health and Consumer Affairs. The Plan was published as a benchmark by the regional office of WHO-Europe in 1990 (WHO-Europe, 1990). The aim and objectives of the Plan were established considering the deficiencies in the Spanish health care system, and resulted in the action lines and the operative framework of the Total Quality Management programme which are discussed in a previous publication (Ruiz et al., 1992). According to these action lines an operative framework of four cascading projects was designed and carried out step-wise between 1986 and 1992 as Phase One. Actions and landmark events were focused on defining a common database, an information system and a culture change for establishing quality management systems as well as certification/accreditation schemes in the health care system. Phase One encompassed four cascading projects that were managed as demonstration projects. The implementation was carried out in five to six hospitals at a time through pilot studies and working groups establishing a network of interrelated activities and cascading outputs. The hospital participation in the programme was established on voluntary basis, but requiring the formal commitment of the hospital managing body. Education and training were fulfilled as basic horizontal activity within all the projects: . NUBIS project. Definition and establishment of a minimum basic data set (demonstration project in five hospitals). . CODIGO project. Codification and implementation of the basic data set (demonstration project in six hospitals). . SICE project. Definition and establishment of a grouper system relating clinical data, resources used and cost. Definition and

325

.

Quality management in health care: a 20-year journey

International Journal of Health Care Quality Assurance

Ulises Ruiz

Volume 17 · Number 6 · 2004 · 323-333

implementation of an analytical accounting approach (demonstration project in five hospitals). GACSA project. Introduction of a standard framework for implementing quality improvement in 62 hospitals of the Spanish health care system applying quality management methodology and training as well as definition of process outcome indicators (Simo´n and Ruiz, 1995a, b).

.

Outcomes for the six-year program As established in the Plan, outcomes of Phase One from 1986 to 1992 focus on the establishment of an information system, both, clinically and result/ cost oriented which will allow the organisational assessment as well as improvement and benchmarking through quality improvement culture in the Spanish health care organisations (Ruiz and Simo´n, 1994). Outcomes are considered relevant if they have been instrumental for subsequent operations for improvement of the health care system or have been formally established as requirement in the Spanish health care system. We consider as most relevant outcomes the following: (1) Minimum basic data set (NUBIS and CODIGO projects): . A minimum basic data set (Conjunto Minimo Basico de Datos (CMBD) in Spanish) of 14 items was defined according to the one proposed by the European Council in 1982 and the one established in the USA as UHDDS. . The Clinical Modification (USA) of the 9th International Classification of Diseases (WHO) known as ICD-9-CM, was translated into Spanish and adapted to Spanish medical practice by the National Medical Specialties Board in order to codify diagnosis and procedures in the CMBD. . The National Health Council approved the two proposals as the potential Data Bank for the Spanish Health care Information System in December 1987. . In 1990 the Ministry of Health published the first Spanish version of the ICD-9-CM (ICD 9 CM, 1987) as well as a user’s manual (Unidad de Garantia de Calidad Total, 1990). . In February 1992 the Ministry of Health established the CMBD as a formal requirement for the Spanish health care system.

326

A formal working position, for ICD-9-CM codifiers in hospitals was established in all Health care System organisations.

(2) Patient grouping system (SICE project): . Evaluation of the two main groupers at that time (PMC and DRG) was carried out, and the results discussed at a national meeting in 1991. . The Ministry of Health establishes the Spanish adapted version of the diagnosis related groups (DRGs) as the case-mix measuring system relating clinical data and cost for the Spanish health care system (1992). . An analytical accounting system according to the DRG’s grouper, the SIGNO Program was devised. This approach and a companion manual were published by the Ministry of Health to be used for accounting in the health care organisations of the Spanish health care system (Ruiz, 1990, 1991a, b, c). . A central health care information system unit was established at the Ministry of Health and Consumer’s Affairs, where data of 98 per cent of all the admitted patients to Spanish hospitals are stored, processed, and published (1993). A central technical committee carries out the governance of the unit where all the Autonomous Health Care Services are represented. (3) Quality improvement methodology (GACSA project): . Partnership of 62 hospitals of the Spanish National Health Care System in a European Concerted Action research project (COMAC) on the advantages of quality assurance in health care (Klazinga, 1994; Simo´n and Ruiz, 1995a, b). . Training in quality improvement methodology and tools for the 62 hospitals participating in the COMAC project. . Quality in health care is established as a separate body of knowledge in the revised medical school curricula. (4) Quality assessment (GACSA project): . Initial standards and indicators were established in order to assess the quality in health care organisations as an input for Phase II of the Total Quality Management Plan. . The Ministry of Health publishes a Proposal of Guidelines for Accrediting

.

Quality management in health care: a 20-year journey

International Journal of Health Care Quality Assurance

Ulises Ruiz

Volume 17 · Number 6 · 2004 · 323-333

Hospitals in Spain (Ruiz, 1991b), and a software programme (EPIHOS) for recording and analysing hospital infection (Ruiz, 1991a). Three longitudinal surveys were carried out among the consumers of the Spanish health care system in 1985, 1988 and 1990 and the results analysed as input for indicators definition.

Lessons learned It is well accepted in the literature that barriers and obstacles for implementing quality systems in health care organisations reflect traditional existing culture as well as structural and functional deficiencies (Dale and Plunkett, 1990; Atkinson, 1990). The following factors seem to have been key to the success versus failure of specific actions carried out along the implementation of the Total Quality Plan in different hospitals of the Spanish health care system: . Professionals’ willingness for establishing a valid information system. Poor motivation is the result of an endemic absence of significant feedback information. The data generators, particularly physicians, should be convinced that the effort involved in collecting reliable data was worthwhile both for themselves and for the organisation as a whole. . Commitment of the top management and clinical staff. The prevailing climate of mistrust between managers and health care professionals, mainly physicians, made common objectives difficult to set and has been a primary obstacle to overcome. Formal commitment has been required from both of them as a trade off for satisfying existing expectations and demands. . Establishment of a quality improvement methodology based on uniform concepts and criteria. The so-called quality philosophies (i.e. control vs assurance, assurance vs improvement, reengineering vs continuous improvement, medical quality vs organisational quality, among others) have been used not infrequently for landmarking parochial interests. A normalised methodology should seek to identify the problems and propose the solutions; and the results obtained could be compared, both within and between different institutions and countries. The health professionals themselves should be actively involved in the development of process approach, quality system criteria and outcome indicators.

The health care system and health care organisations should establish a sound customer-provider relationship within it. The management and outcomes of the total quality program allowed the authors to draw unique lessons and definite operative conclusions that masterminded later developments and present initiatives such as the integrated approach towards excellence.

An integrated approach towards excellence Steps towards excellence In 1994 the University Institute for Health care Assessment (IUES) was founded in order to pursue the operative conclusions gathered by the authors from their direct involvement with the Spanish health care system Total Quality Plan previously described. Today there is a clear convergence of traditional external quality evaluation approaches like the Joint Commission on Accreditation of Health Care Organisations approach and managerial approaches for assessing performance like the Excellence Models and ISO 9000 (Schyve, 2000a, b). The normalisation of external evaluation in between countries and within a country is becoming a requirement for improving health care systems in developed countries. Applying recent theories on organisational change and learning organisation the IUES initiated the development of a stepwise methodology for self-assessment of health care organisations searching for quality improvement, organisational excellence and patient safety according with two basic considerations: (1) On the one hand, we should accept the fact that voluntary certification and accreditation represent, at present, a necessary recognition by service payers of the safety and suitability of health care organisations as providers. (2) On the other hand, self-assessment against an excellence model may be seen as a higher development stage, which is able to drive the organisation beyond the scheme of external evaluation and to establish a method for managing the collected knowledge gathered from the self-assessment exercise. Not to forget newer approaches like Balanced Scorecard and Six Sigma adding strategic and tactic dimensions to the excellence models. Consequently a two-level approach was developed. This approach was implemented as a three-year demonstration project in a 400-bed public sector hospital. The outcome of the first two

327

Quality management in health care: a 20-year journey

International Journal of Health Care Quality Assurance

Ulises Ruiz

Volume 17 · Number 6 · 2004 · 323-333

years’ experience has been published (Ruiz et al., 1999). This two-levels approach establishes criteria and standards to be used for self-assessment and improvement according to the degree of organisational maturity: (1) Level 1 for implementing quality management systems (ISO 9000:2000 (ISO, 2000a) and ISO 14000:1996) and specific quality assurance in health care organisations (JCAHO/CCHA). (2) Level 2 for progressing towards a model of excellence (European Foundation for Quality Management (EFQM)).

enhance interested party satisfaction by meeting interested party requirements (see ISO, 2000b).

Both levels apply the PDCA cycle as methodology for continuous improvement. Such a stepwise approach to certification/ accreditation and self-assessment facilitates initiation on the road of continuous learning and improvement and allows for a progressive implementation of knowledge management towards excellence in modern health care organisations and systems. Furthermore it allows health care organisations and systems to apply true managing approaches, tools and methodologies, which have been useful in the industry and services sectors, and have also proved their promise when applied to the health care sector. We consider mainly: . ISO International Standard for Quality Management (ISO 9000:2000). . ISO 14001:1996 Environmental Management Systems. Specification with Guidance for Use (ISO, 1996). . Baldrige National Quality Program (2001): Criteria for Performance Excellence. . Excellence Model, EFQM. . Balanced Scorecard. . Six Sigma. We will briefly comment on the application of these approaches to the health care sector.

ISO 9000:2000 standards The ISO 9000:2000 standard is being increasingly considered for implementation in the health care sector (Sweeney and Heaton, 2000; Staines, 2000). The standard specifies how to implement a quality management system that means a quality assurance system that prevents errors and therefore assures patient safety as a foundation for any other quality improvement approach in health care: This International Standard promotes the adoption of a process approach when developing, implementing and improving the effectiveness and efficiency of a quality management system to

The systematic identification and management of the processes employed within an organisation and particularly the interactions between such processes is referred to as the “process approach” (see ISO, 2000a).

At present two main initiatives for implementing ISO 9001:2000 (ISO, 2000b) and ISO 9004:2000 (ISO, 2001) in health care services are being developed. One by the International Standards Organization (ISO) itself and the other by the European Committee for Normalization (CEN). In September 2001, ISO published its first International Workshop Agreement (IWA 1) as an ISO 9004:2000, Guidelines for Process Improvement in Health Service Organisations (ISO, 2001) to be used for defining the fundamentals of the health care organisation’s quality management system and continuous improvement methodology, but not as substitute for traditional accreditation: The goal of this document is to aid in the development or improvement of a fundamental quality management system for health service organisations that provides for continuous improvement, emphasizing error prevention, the reduction of variation and organisationbal waste (see ISO, 2001).

In May 2002 CEN launched a Task Force, CEN/ BT TF 142, in order to develop a technical specification and a technical report for the implementation of ISO 9001:2000 and ISO 9004:2000, in the European health care sector. The technical specification will be a guide to the use of ISO 9001:2000 for health care services to improve quality management of clinical and nonclinical (support) services. The specification will be useful for health care organisations that are working with quality improvement, but it will be also possible to use the specification as a basis for certification. The technical report will be a guide to the use of 9004:2000. It is expected that the documents will be ready by 2005 and 2006. The secretariat of the CEN Project is at the Swedish Standards Institute (www.ssi.es).

ISO 14001:1996 This environmental standard is increasingly applied together with ISO 9000:2000 in order to establish an integrated management system. Environmental issues are very important in health care services and can be managed through this standard.

328

Quality management in health care: a 20-year journey

International Journal of Health Care Quality Assurance

Ulises Ruiz

Volume 17 · Number 6 · 2004 · 323-333

Baldrige National Quality Program: Criteria for Performance Excellence While health care organisations have traditionally integrated concepts of quality into their management, more and more they are being held accountable for demonstrating evidence to support this direction. Utilising the Baldrige Criteria for Performance Excellence can help health care organisations more fully integrate the concepts of quality throughout. The major practical benefit of using a common framework for all sectors of the economy is that it fosters cross-sector cooperation and sharing of best practices information. The framework of the Baldrige Business Criteria for Performance Excellence is adaptable to the requirements of all organisations including health care organisations. However, it is not assumed that these requirements are necessarily addressed in the same way. Therefore a specific way for addressing the Baldrige Criteria has been developed as Health Care Criteria for Performance Excellence. This adaptation to health care is largely a translation of the language and basic concepts of business excellence to similarly important concepts in health care excellence. The Baldrige Model in Health Care has been used in several health care organisations in the USA and its implications have been discussed along the past decade (Hertz et al., 1994; Gaucher and Kratochwill, 1995; Jensen, 1996; Weeks et al., 2000; Goldstein and Schweikhart, 2002). For the first time, in 2003, the Malcolm Baldrige National Quality Award was given to a health sector organisation, the SSMHC in St Louis, Missouri, USA (www.ssmhc.com).

customers, people and society are achieved through leadership driving policy and strategy, people, partnership, resources, and processes. The Model was initially used as a selfassessment tool. The organisation’s selfassessment allowed getting the picture of that “moment in time” where the organisation stood. The Excellence Model is being also used as a measurement tool. Organisations are realising that for the outcomes of the self-assessment process to have maximum value, it needed to be linked with other business planning process. The EFQM Excellence Model is a practical tool to help organisations establish an appropriate management system and measure where they are in their path towards excellence; helping them to understand the gaps; and then stimulating solutions (www.efqm.org). In 1996 EFQM developed a specific guide for using the European Excellence Model in health care but in 1999 the model was revised and a common framework is being applied to all sectors, industry as well as private and public service sectors, including health care. The European Excellence Model is being applied in health care organisations of several European countries (Arcelay et al., 1999; Breinlinger-O’Reilly, 2000; Nabitz et al., 2000; Jackson, 2001).

The EFQM Excellence Model The above considerations can be applied to the European Excellence Model. The EFQM Model is a non-prescriptive framework that recognises there are many approaches to achieving sustainable excellence. It has its roots in the philosophy of total quality management. This framework is structured on the following fundamental concepts: . results orientation; . customer focus; . leadership and constancy of purpose; . management by processes and facts; . people development and involvement; . continuous learning, innovation and improvement; . partnership development; and . public responsibility. The premise on which the Model is built is that excellent results with respect to performance,

Balanced Scorecard The Balanced Scorecard is a framework that helps organisations translate strategy into operational objectives that drive both behaviour and performance. It describes and helps implement and manage strategy at all levels of an organisation by linking objectives, initiatives, and measures to that organisation’s strategy. It is also a process that the organisation uses to foster consensus, alignment and commitment to the strategy by the management team and the people within the organisation at large. It provides an enterprise view of a health care organisation’s overall performance by integrating financial measures with other key performance indicators around customer (patient, physician, payer) preferences, internal clinical and business processes, and personal learning, development and growth (Pink et al., 2001; Radnor and Lovell, 2003; Zelman et al., 2003). The first Best Practices Conference held in Cambridge, Massachusetts, USA, in April 2002 focused on how health care organisations use the Balanced Scorecard to achieve results was organised by the Balanced Scorecard Collaborative (www.bscol.com). There, North American health care organisations like Duke Children’s Hospital, Blue

329

Quality management in health care: a 20-year journey

International Journal of Health Care Quality Assurance

Ulises Ruiz

Volume 17 · Number 6 · 2004 · 323-333

Cross/Blue Shield of Minesota, Montefiore Hospital Center, Alterra Health Care and St Mary’s/Duluth Clinic, offered their experience having used the Balanced Scorecard as a powerful performance framework.

The traditional approach to external evaluation of health care organisations was a breakthrough when the American College of Surgeons established the Hospital Standardisation Program in 1917. In 1950 the growing number and complexity of hospitals required revision of the standards and support of the entire medical and hospital field originating the Joint Commission on Accreditation of Hospitals (JCAH) in 1951 (Roberts et al., 1987). However, the performance of the health care organisations in the USA was challenged by the 1999 report of the Institute of Medicine of USA indicating that medical errors kill more than 44,000 people in the US hospitals each year, more than from motor vehicle accidents (43,458), breast cancer (42,297) or AIDS (16,516) and total national costs of preventable adverse events are estimated between $17 billion and $29 billion. There are no similar studies for any other country but the World Health Report 2000 states that “. . . many countries are falling short of their potential. There are serious shortcomings in the performance of one or more functions” (see World Health Organization, 2000). Therefore it seems that finding the appropriate management systems for health care organisations in technically developed countries, is becoming a high priority. It seems time to find an acceptable methodology for measuring, assessing and comparing organisational performance through valid standards as well as recognising self-assessment and accreditation results as valid measures of performance and efficiency. Integration of diverse approaches and methodologies by sequential steps appears to be the best way to achieve implementation of quality management systems and the appropriate culture to assure patient safety and organisational excellence.

Six Sigma Six Sigma is a quality initiative based on rigorous statistical process control. It augments traditional quality tools with exacting statistical analysis and a systematic problem-solving approach, targeting the root-cause of variations and redefining processes for long term improved results. It pursues to produce products and services with only 3.4 defects per million, meaning Six Sigma in statistical terms. Six Sigma can be, thus, considered a business quality improvement tool. It requires four steps, where data collection and analysis become the core of the Six Sigma Projects: (1) measure; (2) analyse; (3) improve; and (4) control. Six Sigma is also a business culture approach, since the goal is to establish a culture of quality improvement at all levels of the organisation by implementing improvement actions designed by specifically trained working teams. In the health care sector, Six Sigma, as a philosophy seeking near zero errors, and being, somehow, an extension of failure mode and effect analysis is ripe for its implementation by health care managers and practitioners concerned for patient safety and are already implementing a total quality management approach (Revere and Black, 2003; Ettinger and Van Kooy, 2003).

An integrated approach: ISO 9000:2000, traditional accreditation, Excellence Model, Balanced Scorecard and Six Sigma It seems therefore that health care organisations can benefit from learning from other sectors that have implemented quality assurance and quality control on a continuous improvement culture. Furthermore patient safety has become a priority issue after the IoM report and it appears that sectors like that of aviation can be a good benchmark for safety within the health care sector (Helmreich, 2000). A practical approach to improving patient safety would be the integration of approaches from other sectors that have proved to be useful in health care (Ruiz and Simo´n, 2004).

Initial step Early implementation of process thinking is a basic stepping-stone, better if it is the initial one, in the continuous quality improvement journey towards excellence. We understand “process thinking” as meaning that in order to function effectively and efficiently an organisation has to identify and manage all its linked activities, or processes, since the output of one is generally the input of another one. The system of processes within the organisation has to be defined and managed. The new ISO 9000:2000 promotes the adoption of a process approach and therefore, at present, ISO 9000:2000 is the best tool for implementing

330

Quality management in health care: a 20-year journey

International Journal of Health Care Quality Assurance

Ulises Ruiz

Volume 17 · Number 6 · 2004 · 323-333

“process thinking” and quality management systems in an organisation. Understanding, systematising and controlling processes is the best way available today for preventing processes and system failures as well as human errors as it is done in high-risk sector as that of aviation. It could be a reliable approach for increasing patient safety in the health care sector.

health providers, purchasers, payers, politicians and consumers. The patient management approach has to be revised and the health care management system should be overhauled in order to discover the system failures that allow for the findings that have been insistently denunciated in the medical literature for the past decade and finally reported by the Institute of Medicine. The author’s 20-year journey in total quality management has allowed him to draw some tentative hypothesis to work with in search of a safer and more efficient heath care system. Piece-meal application of quality control methods, quality assurance schemes, quality management philosophies and their assessment through diverse external evaluation approaches have proved their insufficiency for dealing successfully with quality management systems and with patient safety. Integration of the diverse methods, schemes, philosophies and approaches for establishing a culture of self-assessment and continuous quality improvement in a journey towards excellence, seems to be an alternative to be explored.

Second step Once the quality management system is in place, the health care specific standards like those offered in the traditional health care accreditation approaches, should be implemented.

Third step When the organisation is working effectively, efficiently and safely an excellence model can be established as reference for the organisation’s pursuit of excellence. Initially using the EFQM Model for selfassessment an organisation will have a good understanding of its own strengths and weaknesses at the process level. Scoring according to the Model will offer a reference point for internal comparisons along periodical self-assessments and improvement actions and also for external benchmarking. However, it may not have a strong sense of where to invest as a strategic priority, or where improvement will make the biggest impact in business performance and results. The Balanced Scorecard can be used at this point to provide the strategic focus needed to prioritise action and allocate resources. The Six Sigma approach prioritises resource allocation, facilitates internal comparisons and allows external comparisons for benchmarking. It can be easily integrated into existing quality management efforts through detailed data analysis, becoming part of the strategic plan. Benefits such as reduction in costs of poor quality and improved profitability will be obtained.

Conclusions Interest for quality in patient management has been formally present among the health care professionals since the American College of Surgeons established its standard in 1917. The importance of health care standardisation was recognised by the founding of the Joint Commission on Accreditation of Hospitals in 1951 in the USA. Today, after the Institute of Medicine report in 1999, patient safety is in the limelight of

References Arcelay, A., Sa´nchez, E., Herna´ndez, L., Incla´n, G. and Bacigalupe, M. (1999), “Self-assessment of all the health centres of a public health service through the European Model of Total Quality Management”, International Journal of Health Care Quality Assurance, Vol. 12 No. 2-3, pp. 54-8. Arndt, M. and Bigelow, B. (1995), “The implementation of total quality management in hospitals: how good is the fit?”, Health Care Manage. Rev., Vol. 20 No. 4, pp. 7-14. Atkinson, P.E. (1990), Creating Culture Change: The Key to Successful Total Quality Management, IFS, Kempston. Baldrige National Quality Program (2001), “Health-care criteria for performance excellence”, available at: www.quality.nist.gov Becher, E.C. and Chassin, M.R. (2001), “Improving quality, minimizing error: making it happen”, Health Affairs, Vol. 20 No. 3, pp. 68-81. Bigelow, B. and Arndt, M. (1995), “Total quality management: field of dreams?”, Health Care Manage. Rev., Vol. 20 No. 4, pp. 15-25. Blumenthal, D. (1996), “Quality of health care”, N. Engl. J. Med., No. 335, pp. 891-1331. Breinlinger-O’Reilly, J., Elser, J. and Mo¨ller, J. (2000), “Quality management in German health care – the EFQM Excellence Model”, International Journal of Health Care Quality Assurance, Vol. 13 No. 6, pp. 254-8. British Medical Association (BMA) (1988), Special Report on the Government White Paper: ”Working for Patients”, BMA, London. Coile, R.C. Jr (1990), New Medicine: Reshaping Medical Practice and Health-Care Management, Aspen Publishers, Rockville, MA.

331

Quality management in health care: a 20-year journey

International Journal of Health Care Quality Assurance

Ulises Ruiz

Volume 17 · Number 6 · 2004 · 323-333

Dale, B.G. and Plunkett, J.J. (Eds) (1990), Managing Quality, Philip Allan, London. Ettinger, W. and Van Kooy, M. (2003), “The art and science of winning physician support for Six Sigma change”, Physician Exec., Vol. 29 No. 5, pp. 34-8. Gaucher, E. and Kratochwill, E. (1995), “The Malcolm Baldrige National Quality Award: implications and uses for healthcare organisations”, Infection Control Hosp. Epidemiol., Vol. 16 No. 5, pp. 302-7. Goldstein, S.M. and Schweikhart, S.B. (2002), “Empirical support for the Baldrige Award framework in US hospitals”, Health Care Manag. Rev., Vol. 27 No. 1, pp. 62-75. Gross, P.A. (2000), “Implementing evidence-based recommendations for health care: a round table comparing European and American experiences”, Joint Commission Journal of Quality Improvement, Vol. 26 No. 11, November, p. 613. Harrigan, M.L. (1992), Quality of Care: Issues and Challenges in the 1990s: A Literature Review, Canadian Medical Association, Ottawa. Helmreich, R.L. (2000), “On error management: lessons from aviation”, BMJ, Vol. 320, pp. 781-5. Hertz, H.S., Reimann, C.W. and Bostwick, M.C. (1994), “The Malcolm Baldrige National Quality Award concept: could it help stimulate or accelerate health-care quality improvement?”, Quality Management in Health Care, Vol. 2 No. 4, pp. 63-72. ICD 9 CM (1987), Clasificacio´n Internacional de Enfermedades, 9ª¯ Revision, Modificacio´n Clı´nica, CIE 9 MC, Instituto Nacional de la Salud (INSALUD), Ministerio de Sanidad y Consumo, Madrid. Institute of Medicine (IoM) (1999), To Err Is Human: Building a Safer Health System, National Academy Press, Washington DC. Institute of Medicine (IoM) (2001), Crossing the Quality Chasm: A New Health System for the 21st Century, National Academy Press, Washington DC. ISO (1996), ISO ISO 14001:1996: Environmental Management Systems: Specification with Guidance for Use, ISO, Geneva. ISO (2000a), ISO 9000:2000: Quality Managements Systems: Fundamentals and Vocabulary, ISO, Geneva, December, available at: www.iso.ch ISO (2000b), ISO 9001:2000: Quality Managements Systems: Requirements, ISO, Geneva, December, available at: www.iso.ch ISO (2001), ISO 9004:2000: IWA 1, 2001, ISO, Geneva, available at: www.iso.ch Jackson, S. (2001), “Exploring the suitability of the European Foundation for Quality Management (EFQM) Excellence Model as a framework for delivering clinical governance in the UK National Health Service”, Quality Assurance Journal, Vol. 5, pp. 19-31. Jensen, L.A. (1996), “Improving health-care quality application of the Baldrige process”, J. Nurs. Adm., Vol. 26 No. 7-8, pp. 51-4. Joint Commission on Accreditation of Health-care Organizations (JCAHO) (1988), The Joint Commission Guide to Quality Assurance, JCAHO, Chicago, IL. Klazinga, N. (1994), “Concerted action programs on quality assurance in hospitals 1990-1993 (COMAC/HSR/QA), global results of the evaluation”, Int. J. Qual. Ass., Vol. 6, pp. 219-30. Larson, E. (2003), “Status of practice guidelines in the United States: CDC guidelines as an example”, Preventive Medicine, Vol. 36 No. 5, May, pp. 519-24.

Leape, L.L. (1994), “Error in medicine”, Journal of the American Medical Association, Vol. 272, pp. 1851-7. Leape, L.L., Brennan, T.A., Laird, N., Lawthers, A.G., Localio, A.R., Barnes, B.A., Hebert, L., Newhouse, J.P., Weiler, P.C. and Hiatt, H. (1991), “The nature of adverse events in hospitalised patients. Results of the Harvard medical practice study II”, The New England Journal of Medicine, Vol. 324 No. 6, pp. 377-84. Liang, B.A. (1999), “Error in medicine: legal impediments to US reform”, Journal of Health Politics, Policy and Law, Vol. 24 No. 1, pp. 27-58. Medical Journal of Australia (2004), Vol. 180 No. 6, Supplement, 15 March. Nabitz, U., Klazinga, N. and Walburg, J. (2000), “The EFQM Excellence Model: European and Dutch experiences with the EFQM approach in health care”, International Journal for Quality in Health Care, Vol. 12 No. 3, pp. 191-201. Øvretveit, J. (2003), What Are the Best Strategies for Ensuring Quality in Hospitals?, WHO Regional Office for Europe’s Health Evidence Network (HEN) report, WHO, Geneva, November. Pink, G.H., McKillop, I., Schraa, E.G., Preyra, C., Montgomery, C. and Baker, G.R. (2001), “Creating a balanced scorecard for a hospital system”, J. Health Care Finance, Vol. 27 No. 3, pp. 1-20. Radnor, Z. and Lovell, B. (2003), “Success factors for implementation of the balanced scorecard in an NHS multiagency setting”, International Journal of Health Care Quality Assurance incorporating Leadership in Health Services, Vol. 16 No. 2-3, pp. 99-108. Relman, A. (1988), “Assessment and accountability, the third revolution in medical care”, New Engl. J. Med., Vol. 319, pp. 1220-2. Revere, L. and Black, K. (2003), “Integrating Six Sigma with total quality management: a case example for medication errors”, Journal of Health-care Management, Discussion 392, Vol. 48 No. 6, pp. 377-91. Roberts, J.S., Coale, J.G. and Redman, R.R. (1987), “A history of the Joint Commission on Accreditation of Hospitals”, JAMA, Vol. 258 No. 7, pp. 936-40. Ruiz, U. (1988), Plan de Garantı´a de Calidad Total en Atencio´n Sanitaria. Programa Marco: 1a Fase, 1986-1990, Instituto Nacional de la Salud (INSALUD), Ministerio de Sanidad y Consumo, Madrid. Ruiz, U. (Ed.) (1990), Gestio´n Analı´tica. Hacia la Contabilidad Analı´tica en los Hospitales, Instituto Nacional de la Salud (INSALUD), Ministerio de Sanidad y Consumo, Madrid. Ruiz, U. (Ed.) (1991a), Manual del Programa Epihos, Instituto Nacional de la Salud (INSALUD), Ministerio de Sanidad y Consumo, Madrid. Ruiz, U. (Ed.) (1991b), Guı´a de Propuestas para la Acreditacio´n de Hospitales en Espan˜a, Ministerio de Sanidad y Consumo, Madrid. Ruiz, U. (Ed.) (1991c), Programa Signo. Manual del Usuario. Gestio´n Analı´tica. Hacia la Contabilidad Analı´tica en los Hospitales, Instituto Nacional de la Salud (INSALUD), Ministerio de Sanidad y Consumo, Madrid. Ruiz, U. and Simo´n, J. (1994), “La mejora Continua en el sistema sanitario: resultados de la 1a fase de implantacio´n del Plan de Calidad en Espan˜a (1986-1992)”, Rev. San. Hig. Pub., Vol. 68, pp. 535-48. Ruiz, U. and Simo´n, J. (2004), “Selecting successful health system managing approaches”, in Mayhall, C.G. (Ed.), Hospital Epidemiology and Infection Control, Ch. 13, Williams & Wilkins, New York, NY (forthcoming). Ruiz, U., Simo´n, J., Molina, P., Jimenez, J. and Grandal, J. (1999), “A two level integrated approach to self-

332

Quality management in health care: a 20-year journey

International Journal of Health Care Quality Assurance

Ulises Ruiz

Volume 17 · Number 6 · 2004 · 323-333

assessment in health-care organizations”, International Journal of Health Care Quality Assurance, Vol. 12 No. 4, pp. 135-42. Ruiz, U., Acedo, K., Buenaventura, R., Coll, J., Coronado, S., Rivero, A. and Rocillo, S. (1992), “Implementing total quality management in the Spanish health-care system”, Qual. Ass. Health Care, Vol. 4, pp. 43-59. Schyve, P.M. (2000a), “A trio for quality. Accreditation, Baldrige and ISO 9000 can play a role in reducing medical errors”, Quality Progress, June, pp. 53-5. Schyve, P.M. (2000b), “The evolution of external quality evaluation: observations from the Joint Commission of Accreditation of Health-care Organizations”, International Journal for Quality in Health Care, Vol. 12 No. 3, pp. 255-8. Scrivens, E. (1995), Accreditation – Protecting the Professional or the Consumer?, Centre for Health Planning and Management, Keele University, Keele, Scholarly Book Services and Garamond Press (Canada), Keele and Toronto. Shaw, C.D. (2000), “External quality mechanisms for health care: summary of the ExPeRT project on visitatie, accreditation, EFQM and ISO assessment in the European Union countries”, Int. J. Qual. Health Care, Vol. 12, pp. 169-75. Simo´n, J. and Ruiz, U. (1995a), “Accreditation trends in Spain”, International Journal of Health Planning and Management, Vol. 10, pp. 193-200. Simo´n, J. and Ruiz, U. (1995b), “Actividades de garantı´a de calidad en 62 hospitales del Sistema Nacional de Salud en Espan˜a”, Rev. Esp. Salud Pu´blica, Vol. 69, pp. 341-7.

Staines, A. (2000), “Benefits of an ISO 9001 certification – the case of a Swiss regional hospital”, International Journal of Health Care Quality Assurance, Vol. 13 No. 1, pp. 27-33. Sweeney, J. and Heaton, C. (2000), “Interpretations and variations of ISO 9000 in acute health care”, International Journal for Quality in Health Care, Vol. 12 No. 3, pp. 203-9. Thompson, A.G.H. (1998), “Competition and quality: looking for evidence for health-care reforms”, Int. J. Qual. Health Care, Vol. 10 No. 5, pp. 371-3. Thompson, A.G.H. (1999), “New millennium, new values: citizen participation as the democratic ideal in health care”, Int. J. Qual. Health Care, Vol. 11 No. 6, pp. 461-4. Unidad de Garantia de Calidad Total (1990), Clasificacio´n Internacional de Enfermedades, 9a Revisio´n, Modificacio´n Clinica, Manual de Uso, Instituto Nacional de la Salud, Ministerio de Sanidad y Consumo, Madrid. Weeks, W.B., Hamby, L. and Batalden, P.B. (2000), “Using the Baldrige management system framework in health care: the Veterans Health Administration experience”, Joint Commission Journal of Quality Improvement, Vol. 26 No. 7, pp. 379-87. WHO-Europe (1990), The Spanish Total Quality Assurance Programme, First Phase 1986-1990, EUR/ICP/HSR 031, 7977s, Regional Office for Europe of the World Health Organisation, Geneva. World Health Organization (2000), The World Health Report 2000, Health Systems: Improving Performance, Office of Publications, World Health Organization, Geneva. Zelman, W.N., Pink, G.H. and Matthias, C.B. (2003), “Use of the Balanced Scorecard in health care”, Journal of Health Care Finance, Vol. 29 No. 4, pp. 1-16.

333

Introduction

Quality improvement techniques to improve patient satisfaction E. Joseph Torres and Kristina L. Guo

The authors E. Joseph Torres is a Student and Kristina L. Guo is Assistant Professor, Health Services Administration, both at Florida International University, Miami, Florida, USA.

Health care is unique from any other type of industry in that health care professionals are highly dependent on each other to provide and coordinate services of high value for human beings. This is especially challenging for health care managers who are responsible for managing health care organizations (Shortell and Kaluzny, 2000). Since one of the main goals of any health care organization is not only to meet, but also to exceed the expectations of patients, improving levels of patient satisfaction is very critical to their longterm success. Quality improvement initiatives can be instrumental in attaining this goal. Indeed, one important focus of quality improvement in health care organizations involves patients. Specifically, using quality improvement techniques, managers strive to improve performance in key processes so that high levels of patient satisfaction are achieved. The intent of this paper is to discuss quality improvement techniques used by managers to enhance patient satisfaction.

Keywords Patients, Customer satisfaction, Quality improvement

Background

Abstract This paper describes several approaches for implementing quality improvement initiatives to improve patient satisfaction, which enables health-care organizations to position themselves for success in today’s global and increasingly competitive environment. Specifically, measuring the views of patients, improving patient satisfaction through a community-wide effort, and using a Six Sigma program are discussed. Each of these programs can be an effective mechanism for quality improvement. A key component to quality improvement techniques involves collaborative efforts by all health-care professionals and managers as they seek to increase patient satisfaction.

Electronic access The Emerald Research Register for this journal is available at www.emeraldinsight.com/researchregister The current issue and full text archive of this journal is available at www.emeraldinsight.com/0952-6862.htm

International Journal of Health Care Quality Assurance Volume 17 · Number 6 · 2004 · pp. 334-338 q Emerald Group Publishing Limited · ISSN 0952-6862 DOI 10.1108/09526860410557589

Quality improvement is now a perpetual objective of many organizations including those involved in the delivery of medical care. Quality improvement is essentially an organization’s attempts at improving its products and processes in terms of meeting the expectations of its customers. One facet of quality improvement initiatives involves customer satisfaction. In the case of health care organizations, customer satisfaction extends to patients as well as various medical staff such as physicians, nurses and medical technicians. The views and perceptions of these customers have an impact on the overall success of health care organizations, and have recently come more into prominence, since it is used as an indicator recognized by managers for making organizational changes and improvements in performance. Gathering the views of service users is a key feature of recent developments in society and the health care sector has identified methods for assessing the views of patients, especially in the last decade (Wensing and Elwyn, 2002). Milosevic and Bayyigit (1999) further stress the value of assessing patient satisfaction to health care organizations, where the organization must attempt to respond to reasonable expectations of patients. If health care organizations are in the business to provide service for their customers, then they must strongly consider the needs and expectations of their most important customers: patients. Furthermore, health care organizations

334

Quality improvement techniques to improve patient satisfaction

International Journal of Health Care Quality Assurance

E. Joseph Torres and Kristina L. Guo

Volume 17 · Number 6 · 2004 · 334-338

are in the business of caring for human beings. Patients entrust their lives and wellbeing to providers. Thus, monitoring patient satisfaction is a crucial element of an organization’s effectiveness and should be part of the quality improvement initiative. Efforts at quality improvement in health care organizations present their own unique set of challenges. As with all types of services, health care is an intangible product (Ford et al., 1997). The services that patients receive cannot physically be viewed or touched like manufactured products such as televisions and cars. It is easier for managers dealing with manufactured products to develop and implement quality improvement measures than it is for health care organizations (Ford et al., 1997). Another challenge concerns the manner in which health care organizations have traditionally been operated. For instance, many patients are not well informed about issues related to their health. They trust that physicians and other health care professionals are making decisions with their best interests at heart (Feldstein, 2003). However, physicians may not always act in the best interest of patients, as physicians are constrained by a lack of resources to do so (Feldstein, 2003). Nevertheless, many patients accept, without question, the decisions of their caregivers and accordingly receive medical services as specified by their providers. With this type of arrangement, there is little incentive for service providers to make efforts proactively to seek out dissatisfied patients. After all, patients will come regardless of whether or not such efforts are made. However, with the increasing popularity of the Internet and information from popular media, patients have access to more information and are now becoming more informed than they were before. They are learning that they have choices when it comes to their medical care, and they are demanding more choices as evidenced by the proliferation of preferred provider organizations and point of service organizations (Feldstein, 2003). With this shift in decision-making ability, health care organizations and management will find it necessary to place more emphasis on the views expressed by their patients, so that problem areas are addressed promptly and standardized procedures are put in place to prevent future conflicts.

about what should occur in the health care setting. There are both qualitative and quantitative methods for measuring preferences. Among the qualitative methods are individual interviews and focus groups. Quantitative measures include surveys, nominal group techniques, and consensus methods. Some examples of qualitative methods include evaluations and patient reports. Evaluation refers to the patient’s reaction to the service he/she received from a health care organization. Evaluations from the patients most often come in the form of questionnaires. Finally, patient reports are objective observations that patients make about an organization or processes. For example, a patient can usually indicate the number of times he/she was seen by a physician during a hospital stay regardless of whether or not he/she thinks it was a sufficient amount with an appropriate level of care. According to Wensing and Elwyn (2002), there are four approaches for utilizing the views of patients in the process of quality improvement. The following is a description of these components. First, health care organizations can provide data/information to seekers of health care. Better-informed patients can make decisions on whether or not to pursue care at a given organization. Thus, service providers who are considered better by patients receive a larger proportion of the patient population and competition ensues. As a result of competition, other health care organizations are forced to take actions to improve their programs in an effort to attract more patients. Second, Wensing and Elwyn (2002) emphasize patient involvement. For example, shared decision-making strategies or patient-held records identify patient preferences. They ensure care providers give adequate information on relevant options, assess patient preferences regarding these options, and make decisions with or seek approval from the patient. Thus, by respecting a patient’s desire and right for self-determination, health care organizations contribute to the flow of information that patients gain and actively involve them in the communication chain so that they are better informed to make decisions. A third approach involves using devices such as surveys, written complaints and questionnaires to obtain patient feedback. This way, the views of the patients can be analyzed and appropriate actions can be taken to improve on weak areas. Here, health care organizations can implement processes to meet their patients’ needs more reliably and efficiently. Furthermore, this approach can provide indication of the direction and intensity of staff training and education to improve specific areas. The final approach concerns eliciting the help of

Measuring the patient’s perspective Wensing and Elwyn (2002) emphasize three components of patients’ views on health care: preferences, evaluations and reports. Preferences are essentially a patient’s desires and expectations

335

Quality improvement techniques to improve patient satisfaction

International Journal of Health Care Quality Assurance

E. Joseph Torres and Kristina L. Guo

Volume 17 · Number 6 · 2004 · 334-338

customers/patients in planning and design of systems of health care. Involving customers and patients in this process would allow them to incorporate processes and designs according to their own expectations and result in preferred patient features that gain higher levels of satisfaction.

Improving patient satisfaction To improve patient satisfaction, health care providers must focus on quality improvement strategies. That is, health care professionals must demonstrate attributes consistent with organizational culture. Since patients entrust their lives to the competence of hospitals and its medical staff, then patients expect that everything that can be done to maximize their chances of successful treatment and survival (patient outcomes) are indeed done. With such a perspective, a quality improvement initiative was conducted in Dayton, Ohio to illustrate how patient outcomes can be improved using a community-based approach (Snow et al., 2003). Five competing hospitals along with local businesses and hospital associations in the area collaborated through the Greater Dayton Area Hospital Association (GDAH) to participate in the Hospital Performance Reports Project (HPRP), an effort at quality and performance improvement. HPRP identified that the county hospital along with other individual hospitals have significantly higher that predicted mortality rates associated with acute myocardial infarction (AMI). Furthermore, it was determined that there was a direct link between whether or not patients, with an ST-segment elevated myocardial infarction (STEMI) received reperfusion and hospital specific mortality rates. In response, a group of medical directors along with a steering committee then developed several expanded process of care measures appropriate for AMI (Snow et al., 2003). Reperfusion is the process of restoring blood supply to an organ, in this case: the heart, or tissue that has lost blood flow (The American Heritage Dictionary, 2000). Also, hospital-specific performance of patients having STEMI selected for reperfusion was reviewed by a cardiology subcommittee. The results of the effects of the project and mortality associated with AMI showed a mortality decline from an average of 9.68 percent in 1999 to 6.3 percent for the first three quarters of 2002. This represents a 36 percent relative reduction in AMI mortality over this time frame (Snow et al., 2003). HPRP was successful for several reasons. First of all, the program focused on those who provide health care and not the customers (Snow et al., 2003).

Many prior community-based programs have assumed that customers have adequate information to make informed decisions about the quality of care that they received. However, this notion, in general does not appear to be accurate. Thus, by taking an approach in which the physicians and other health care professionals monitor and take actions to improve the quality of care that they are giving, patient outcomes can be improved. Moreover, this project enlisted the help of the entire community in a collaborative effort versus a competitive approach in identifying the gaps in improvement of the health care. This approach has the obvious advantage of having many different people involved with different perspectives and levels of experience. Thus, in this way, many more ideas about causes and potential solutions can be evaluated. Better patient care and healthy outcomes are the preferred results. This collaborative approach also encouraged and enabled clinicians to understand the need to monitor quality in improving patient outcomes. Thus, by forming committees for peer reviews to discuss the problems and develop a model to improve the quality of patient care, physicians and other health care professionals learn from the past, and this leads to a better standard of patient care.

Improving patient satisfaction with Six Sigma Six Sigma began in the 1980s as a quality improvement plan for Motorola. The approach has since grown into efforts adopted by companies. As a methodology and measurement, Six Sigma evaluates the capability of a process to perform defect free, where a defect is defined as anything that results in customer dissatisfaction. The innovativeness of Six Sigma is that it combines improved methods with a new management philosophy to significantly reduce defects, thereby strengthening a company’s market position and improving the profit line (Harry and Schroeder, 2000). Six Sigma is a process that can be used in meeting the needs and expectations of the customers in health care organizations along with improving profitability and cash flow (Samuels and Adomitis, 2003). Specifically, it seeks to identify, quantify and eliminate errors in business processes (Gale, 2003). Six Sigma is also identified as critical to quality (CTQ) defects using measures that indicate the effectiveness of a given process (Samuels and Adomitis, 2003). The first of these measures is defects per million opportunities (DPMOs): the quantity of defective CTQ factors per 1 million opportunities for a defect to occur. Second, error-free yield (EFY) is the percent of a

336

Quality improvement techniques to improve patient satisfaction

International Journal of Health Care Quality Assurance

E. Joseph Torres and Kristina L. Guo

Volume 17 · Number 6 · 2004 · 334-338

process without defects. Finally, the sigma level is similar to the concept of standard deviation: as the sigma level increases, the number of defects decreases. For a point of reference, the average industry runs at a sigma level of 4.0; whereas a perfect, defect-free process would have a sigma level of 6.0. According to Samuels and Adomitis (2003), in implementing Six Sigma, managers should execute the following process: (1) define the purpose and scope of the project; (2) create a performance baseline to compare data evidencing errors; (3) analyze root causes quantified by actual data; (4) implement procedures to abolish root causes of errors and improve performance; and (5) evaluate the performance of the process before and after to make attempts at improvements.

focus on an health care organizational philosophy that promotes quality improvement.

Utilizing this process can be effective in providing better customer satisfaction, as well as reducing costs and improving profitability (Samuels and Adomitis, 2003). For example, if a hospital’s goal was to improve patient satisfaction, management would want use a quantifiable indicator of patient satisfaction against which performance can be measured. Data would then be gathered to analyze the root causes of patient dissatisfaction. Then, procedures would be implemented to eliminate the identified causes of patient dissatisfaction. Finally, the impact of the procedure can be obtained evaluating the levels of patient satisfaction before and after implementation of Six Sigma. In 2000, Mount Carmel Health System in Columbus, Ohio demonstrated that a Six Sigma program can be effective for a health care organization (Gale, 2003). Specifically, by implementing the program, Mount Carmel managed to cut costs and save several million dollars in operating expenses every year since the program began (Gale, 2003). If other health care organizations can achieve results such as these, then they can avoid negative actions such as employee layoffs and spending cuts in the customer/patient service arena. As a result, the focus of the organization can stay on where it should be: meeting and exceeding the needs and expectations of the patients. Revere and Black (2003) examined the use of Six Sigma for the purpose of reducing medical errors and increased profitability. They suggest that Six Sigma is more effective than the traditional total quality management methods, since it offers more precise quantifiable measurements. Furthermore, they recommend that Six Sigma can be successful when used as a managerial tool for reducing medication errors because of its focus on identifying, analyzing, and monitoring errors. However, the key lies in extensive training and a

Discussion In implementing a quality improvement initiative to meet and exceed the expectations of the patients, there are several approaches that can be used be health care organizations. The approach and success of any quality improvement program is determined by corporate philosophy, and initiation, involvement and support of senior level management. For any quality improvement effort to be effective, it is imperative to have support from all of an organization’s top management personnel. With such support, it will be easier to manage employee resistance to change when efforts at quality improvement do indeed require change. There were three approaches to using quality improvement to increase levels of patient satisfaction discussed in this paper: measuring the patient’s perspective, improving patient outcomes, and using a Six Sigma program. When using the views of the patients in quality improvement efforts, there are both advantages and disadvantages. Specifically, the direct feedback provided by patients is perhaps the biggest advantage of this approach. Patients are able to inform providers of areas of satisfaction and dissatisfaction. Qualitative approaches to getting the patient’s feedback such as evaluations and questionnaires have the advantage of allowing more open-ended questioning allowing managers to get a better feel for why a patient may feel a certain way. However, qualitative approaches are not as easy to apply statistical evaluations to as quantitative approaches. Thus, when managers wish to establish a standard against which future efforts can be evaluated on a more objective basis, quantitative methods, such as surveys, nominal group techniques, and consensus methods, may be better. When coupled with methods of obtaining patient feedback, the act of involving patients in decisions related to their treatment and the overall design of health care systems can serve to improve levels of patient satisfaction. Improving patient outcomes also serves to increase levels of patient satisfaction. HPRP demonstrated that collaboration by many different organizations could be effective in improving the standard of patient care. Better patient outcomes are the main advantage of this approach. However, a community-wide effort can also result in more effective solutions being identified because more people with different backgrounds and experiences are involved. On the other hand, competition between different organizations could be a

337

Quality improvement techniques to improve patient satisfaction

International Journal of Health Care Quality Assurance

E. Joseph Torres and Kristina L. Guo

Volume 17 · Number 6 · 2004 · 334-338

drawback to this approach. Tension between personnel from competing organizations might inhibit all aspects of the effort from generation of ideas to implementation of a final plan. Also, this approach could be very costly and timeconsuming. For example, it takes a great deal more effort to schedule meetings and form committees with personnel from different organizations. If some of these drawbacks can be overcome, then a community-wide initiative can be instrumental in improving the standard of care for patients and thus increasing levels of patient satisfaction. Finally, use of a Six Sigma program may help an organization to achieve higher levels of patient satisfaction. Six Sigma provides a formal procedure to follow in quality improvement eliminating the need for health care organizations to expend the effort in developing a program from scratch. Six Sigma has also proven to be effective for many types of industries outside of health care (Samuels and Adomitis, 2003). However, it has not yet been heavily utilized in the health care field. Nevertheless, current studies have shown that Six Sigma has been more effective than previous quality improvement techniques, and without requiring additional resources or producing more stress to the organization. In fact, it can be incorporated into existing quality management efforts (Revere and Black, 2003).

professionals and managers as they seek to increase patient satisfaction. Implementing quality improvement initiatives to improve patient satisfaction can enable health care organizations to position themselves for success in today’s global and increasingly competitive environment.

Conclusion Achieving high levels of patient satisfaction through quality improvement should be one of the top priorities of any health care organization. After all, without patients, these organizations would cease to exist. There are several approaches that can be utilized to meet and exceed the expectations of the patients. Of these, measuring the views of the patients, improving patient outcomes with a community-wide effort, and using a Six Sigma program were discussed. Each of these programs can be an effective mechanism for quality improvement if used properly. One of the key components of quality improvement techniques involves collaborative efforts by all health care

References (The) American Heritage Dictionary (2000), available at: http:// education.yahoo.com/reference/dictionary/entries/91/ r0159150.html (accessed March 3, 2004). Feldstein, P.J. (2003), Health Policy Issues: An Economic Perspective, 3rd ed., Health Administration Press, Chicago, IL, pp. 16-40. Ford, R.C., Bach, S.A. and Fottler, M.D. (1997), “Methods of measuring patient satisfaction in health-care organizations”, Health Care Management Review, Vol. 22 No. 2, Spring, p. 74. Gale, S.F. (2003), “Building frameworks for Six Sigma success”, Workforce, Vol. 82 No. 5, May, pp. 64-8. Harry, M. and Schroeder, R. (2000), Six Sigma, Doubleday Publishers, New York, NY, pp. 108-15. Milosevic, D. and Bayyigit, M. (1999), “Quality improvement: what is in it for the patient?”, IEEE Transactions on Engineering Management, Vol. 46 No. 3, pp. 346-7. Revere, L. and Black, K. (2003), “Integrating Six Sigma with total quality management: a case example for measuring medication errors”, Journal of Health-care Management, Vol. 48 No. 6, November-December, pp. 377-92. Samuels, D.I. and Adomitis, F.L. (2003), “Six Sigma can meet your revenue-cycle needs: Six Sigma is far from being the latest quality-improvement fad; it is a proven technique grounded in principles that will endure as long as there are processes that require improvement”, Health-Care Financial Management, Vol. 57 No. 11, November, pp. 70-5. Shortell, S.M. and Kaluzny, A.D. (2000), “Organization theory and health services management”, in Shortell, S.M. and Kaluzny, A.D. (Eds), Essentials of Health Care Management, Delmar Publishers, New York, NY, pp. 4-33. Snow, R.J., Engler, D. and Krella, J.M. (2003), “The GDAHA hospital performance reports project: a successful community-based quality improvement initiative”, Quality Management in Health Care, Vol. 12 No. 3, July-September, pp. 151-8. Wensing, M. and Elwyn, G. (2002), “Research on patients’ views in the evaluation and improvement of quality of care”, Quality and Safety in Health Care, Vol. 11 No. 2, June, pp. 153-7.

338

Identification of seniors at risk: process evaluation of a screening and referral program for patients aged $75 in a community hospital emergency department Rebecca N. Warburton, Belinda Parke, Wynona Church and Jane McCusker The authors Information about the authors can be found at the end of the article.

Keywords Elderly people, Emergency services, Hospitals, Patient care, Quality improvement, Canada

Abstract Reports on the authors’ experience with a patient safety quality improvement program, intended to reduce the incidence and severity of adverse outcomes for emergency department (ED) patients aged $ 75. The Identification of Seniors at Risk scale was used for screening, and those at high risk were referred for appropriate intervention. The plan-do-study-act improvement cycle was followed, conducting process evaluation to diagnose and correct implementation difficulties. Reports that: implementing an ED screening and referral program is deceptively difficult; process evaluation multidisciplinary working group meetings are an essential improvement tool; screening inclusion criteria had to be adapted to the subject population in order to make efficient use of staff time; the screening questions and process required ongoing assessment, revision, and local adaptation in order to be useful; and high-risk screening in the ED is critical to a hospital system’s ability to anticipate clinical problems; the plan-do-study-act improvement cycle is a practical and useful tool for improving quality and systems in a real care setting.

Electronic access The Emerald Research Register for this journal is available at www.emeraldinsight.com/researchregister The current issue and full text archive of this journal is available at www.emeraldinsight.com/0952-6862.htm International Journal of Health Care Quality Assurance Volume 17 · Number 6 · 2004 · pp. 339-348 q Emerald Group Publishing Limited · ISSN 0952-6862 DOI 10.1108/09526860410557598

Introduction In 1998, Vancouver Island Health Authority (VIHA) inaugurated the Elder-Friendly Hospital Initiative, a wide-ranging evidence-based quality improvement project aimed at enhancing the acute care experience and improving the health outcomes of elderly patients (Parke and Stevenson, 1999). An “Elder Alert” is one of the customized strategies created in an elder-friendly hospital to address the needs of a vulnerable older population (Parke and Brand, 2004). We report here process evaluation results for one pilot patient safety project, the “Elder Alert” screening program at Saanich Peninsula Hospital (SPH) Emergency Department (ED). Starting in May 2003, this program applied a high-risk screening, referral, and follow-up process to all patients aged $75 attending at Emergency. The results from nine months’ experience with this program are being The authors gratefully acknowledge the assistance of: Dawn Nedzelski, Patient Care Manager, Acute Care, Saanich Peninsula Hospital; Margaret Tennant, Clinical Nurse Educator, Saanich Peninsula Hospital; Dr Marilyn Bater, Geriatrician, Program Director (Seniors Health Program) Vancouver Island Health Authority; Dr Ambrose Marsh, Chief of Medicine, Saanich Peninsula Hospital; Renate Nahser-Ringer, Registered Nurse, Vancouver Island Health Authority; Research Assistant, University of Victoria; and· Dr Jose´e Verdon, Internist and Geriatrician, Department of Geriatrics, McGill University; Royal Victoria Hospital, Montreal. This paper reflects the views of the authors, who are solely responsible for any errors or omissions. This is not an official document of Saanich Peninsula Hospital, the Vancouver Island Health Authority, the Fraser Health Authority, the University of Victoria, McGill University, or any other organization. The Elder-Friendly Hospital Initiative and the Elder Alert Screening Program have been funded by Vancouver Island Health Authority as part of continuing efforts to improve quality of care. Dr Warburton holds salary and operating funds for research into the costs and effects of patient safety improvements from the Michael Smith Foundation for Health Research of Vancouver, British Columbia. The evaluation of the Elder Alert Screening Program has been funded as part of Dr Warburton’s patient safety research. BP initiated and designed the clinical program in consultation with Saanich Peninsula Hospital staff. RNW and BP initiated and designed the evaluation of the clinical program. WC conducted the process evaluation with guidance from RNW. RNW wrote and edited the final paper in consultation with other authors. All authors have read and approved the final paper, and are responsible for its conclusions.

339

Identification of seniors at risk

International Journal of Health Care Quality Assurance

Rebecca N. Warburton et al.

Volume 17 · Number 6 · 2004 · 339-348

used to guide expansion of the screening program to other British Columbia hospitals.

an independent second cohort of older patients was assembled. Functional decline occurred in 34 of 142 (24 percent) of these patients, with low-risk, intermediate-risk, and high-risk rates of 6 percent, 29 percent and 83 percent respectively (p , 0:001). The study concluded that multiple impairments have cumulative or synergistic effects. In another study, Inouye and Charpentier (1996) found that a predictive model based on five precipitating factors (use of physical restraints, malnutrition, more than three medications, use of bladder catheter, and any iatrogenic event) indicated vulnerability and could assist in the identification of medical patients at risk for delirium during hospitalization. Baseline vulnerability and precipitating factors were documented to be interdependent and statistically significant (p , 0:001). The authors concluded that risk of delirium depended on pre-hospital patient-specific risk factors (vulnerability) interacting with additional factors occurring during hospitalization (precipitating factors). Similarly, Fisher and Flowerdew (1995) found patients 60 years and older seen in a preadmission clinic prior to elective orthopedic surgery were successfully identified for risk of delirium pre-operatively using similar risk factors. McCusker et al. (1999) developed and validated the six-question Identification of Seniors at Risk (ISAR) tool to screen elderly patients in the ED for risk of adverse outcomes during the six months after the ED visit, such as lengthened acute stay, functional decline, admission to long term care, or death,. The ISAR tool has also been shown to predict high rates of hospitalization, repeat ED visits, and use of community services (McCusker et al., 2000a, b; Dendukuri et al., 2004). Improved patient safety and better outcomes occur only when screening and effective follow-up are combined; the effectiveness of improved individual care plans depends on successful implementation. A multi-center trial at four large, urban, teaching hospitals (McCusker et al., 2001) showed that a two-step intervention – screening using the ISAR tool followed by a short, standardized nursing assessment and referral intervention significantly reduced (adjusted OR 0.53, 95 percent CI ¼ 0:31
0:91) rates of subsequent functional decline. A systematic review (Aminzadeh and Dalziel, 2002) identified the ISAR as a valid screening tool and noted that risk factors for adverse health outcomes in older adults visiting the ED are well established. However, the review called for more

Background Rates of hospitalization are high among older adults. In 1997/1998, patients aged $ 65 accounted for 35 percent of the 3 million discharges reported by Canada’s in-patient hospitals, 52 percent of the 21 million acute patient days, and nearly one-third of all primary diagnostic and surgical procedures performed in hospitals during this time. Older adults (. 65) are three times as likely to be hospitalized as adults aged 45-64 (CIHI, 2000). Not only are older adults high users of hospital services, but also the experience of hospitalization increases certain risks for this population, including functional decline - loss of mental and physical ability (Creditor, 1993; Dudek, 2000; Palmer, 1995; McCusker et al., 2002). Risk of an adverse outcome is increased by a combination of factors including advanced age, residence in a long-term care facility, cognitive and nutritional deficits, vision or functional impairment, alcoholism, and polypharmacy – taking multiple medications (McCusker et al., 2000a, b; 2001, 2002, 2003; Clark, 2001; Thomas and Richie, 1995; Karp and Koval, 1998; Boult et al., 1993; Heruti et al., 1999; Inouye, 2000; Inouye et al., 1993; Inouye and Charpentier, 1996; Liebergall et al., 1999; Winograd et al., 1991). Some have advocated improving outcomes for hospital-attending older adults by developing screening tools to predict risk, and targeting appropriate interventions based on level of risk. The goal of screening and intervention is to prevent the preventable, reverse the reversible, and support and palliate the patient during their hospital stay (Parke and Brand, 2004). Inouye et al. (1993) conducted two prospective cohort studies to develop and validate a predictive risk index for functional decline based on the personal characteristics of patients. Using four independent baseline risk factors for functional decline (decubitus ulcer, cognitive impairment, functional impairment, and low social activity level) a risk-stratification system was developed by adding the total numbers of existing risk factors identified on admission to hospital. Functional decline occurred among 51 of 188 (27 percent) of patients in the first cohort. Observed rates of functional decline for low-risk (zero risk factors), intermediate-risk (one to two risk factors) and high-risk (three to four risk factors) patients were 8 percent, 28 percent and 63 percent respectively (p , 0.001). To validate the predictive model,

340

Identification of seniors at risk

International Journal of Health Care Quality Assurance

Rebecca N. Warburton et al.

Volume 17 · Number 6 · 2004 · 339-348

research on the characteristics of successful interventions, since some studies report beneficial effects and others do not. This paper reports process evaluation results for such a program, a quality improvement initiative to develop a screening and referral intervention for elderly ED patients at a small community hospital. This evaluation complements earlier studies that applied the ISAR screening tool in a large urban teaching hospital.

(plan). as the improvement is implemented (do), it is monitored (study) to determine effectiveness. Finally, it may be modified (Act) as necessary based on experience. The Elder Alert program was designed (plan an improvement) by a clinical nurse specialist with expertise in gerontological nursing (author BP), leading a multidisciplinary Elder Alert Working Group at SPH that included the clinical resource nurse educator for acute care and the ED; ED, ward, and preadmission clinic nurses; as well as physicians, social workers, physiotherapists, pharmacists, and nutritionists. Local university researchers (RNW, WC, and RNR) attended as many Working Group meetings as possible. The Elder Alert program was seen as an essential patient safety quality improvement project by Saanich Peninsula Hospital. Administrative data indicated that older adults where high users of hospital services, entered hospital most often through the ED, and remained in hospital longer than younger patients. Local data also indicated that some critical incidents may have been the result of preventable problems such as delirium. Design and implementation of the Elder Alert Program was multidisciplinary and inclusive, jointly supported by professional practice, medicine, and administration. Individuals selfselected to participate in a working group, and were encouraged to become involved by the ED patient care manager. Internal systems of communication were established to ensure information was shared with staff not actively involved in the project but who held important information to inform the process. The specific goals of Elder Alert are: . to detect patients at elevated risk for adverse outcomes of hospitalization; . to design a care plan for appropriate intervention; and . to provide targeted, coordinated, preventive services.

Clinical setting Like many health authorities, VIHA has been through a prolonged period of organizational change and acute care downsizing, beginning with amalgamation of community hospitals in the 1980s, extending through creation of the Capital Health Region in the 1990s, and culminating in the amalgamation of several smaller regions into VIHA as one of six large health authorities in December 2001. VIHA now delivers services to over 700,000 residents, and is geographically organized into south, central and north island areas. Three acute care hospitals (total 734 beds) serve the VIHA-South area, population 300,000. The smallest of these, Saanich Peninsula Hospital, has 38 acute beds, ten palliative beds, and serves the semi-rural peninsula north of the capital city. Adults over age 65 are the primary users of SPH. The new SPH ED opened in November 2003, and (like the previous department) has ten beds and two chairs, one physician, three nurses working days and evenings, and two nurses at night. Referral services include Pharmacy, Physiotherapy, Social Work, Liaison to community services, and Nutrition. Referral services are available only on weekdays, except for Liaison (available evenings, not nights, with limited capacity on weekends).

Screening program design Development of the Elder Alert program followed the plan-do-study-act model for improvement (Langley et al., 1996). The model begins with three questions: (1) What are we trying to accomplish? (2) How will we know that a change is an improvement? (3) What changes can we make that will result in an improvement? Quality improvement begins with discussion of these questions and design of an improvement

The program is intended to prevent or ameliorate adverse outcomes such as delirium, which can result in functional decline and subsequent long term care admission or death. The intention was to devise a screening and referral program for ED patients aged $ 75, who without screening might not be recognized as high risk. It was expected that 30 percent to 50 percent of screened elderly ED patients would be identified as high risk through the screening program. Patients coming to the ED for scheduled blood transfusions, and also those coming from and returning to extended care or palliative care units, were believed not to require

341

Identification of seniors at risk

International Journal of Health Care Quality Assurance

Rebecca N. Warburton et al.

Volume 17 · Number 6 · 2004 · 339-348

high-risk screening, and were excluded from the screening program. Specific interventions for patients designated Elder Alert (high risk) included “Delirium Watch” and a referral “fan-out”. Delirium Watch is an interdisciplinary surveillance approach used to quickly identify and respond to delirium in older patients; components include assessment checks three times daily, and protocols related to pain management, constipation, and disruptive behaviour (Parke and Brand, 2004). At-risk inpatients also received a referral fan-out notifying the patient’s physician, pharmacy, physiotherapy, and other hospital departments or community services as necessary. High-risk patients not admitted received a “community fanout”. A community fan-out is a hospital communication process specifically for nonadmitted patients that facilitates the communication of their needs to their doctor, their family, and, if necessary, to community services (i.e. social work). Based on clinical judgment, admitted elderly patients not identified as high risk through Elder Alert screening could also be put on Delirium Watch. The intention was to facilitate gerontological care for all elderly patients. Initially, the Working Group considered available screening tools, including the ISAR, but decided instead to develop a new tool based on evidence from the literature. After several weeks of tool development, however, a consensus emerged among Working Group members that using an existing, validated tool would be simpler and more efficacious than designing a new tool. According, the ISAR was adopted as the screening tool. Prior to implementing the program, the clinical nurse specialist and the clinical resource nurse educator provided staff with basic Elder Alert training. Training mechanisms consisted of memorandums, videos, journal articles, in-service training, and a large poster board. The goal of the training was to engage staff in understanding the physical changes that occur in the older adult when hospitalized, to enable them to utilize a gerontological approach to their practice. Although the Elder Alert Program implemented at SPH used the same screening tool as that used in the first step of the two-step intervention assessed in a reported multi-centre trial (McCusker et al., 2001), the SPH intervention was designed based on sound gerontological practice and local conditions, and was not patterned on any specific program elsewhere. The SPH program provides a useful test of the practicality of an ED screening program in a small community hospital with limited resources. Based on SPH results as well as literature, Royal Jubilee Hospital (VIHA-south),

Peace Arch Hospital (Fraser Health Authority) and Chilliwack General Hospital (Fraser Health Authority) will soon be developing and implementing Elder Alert programs in their Emergency Departments. The SPH Elder Alert program was implemented on May 24, 2003.The initial process had four major steps: (1) Triage nurse administered a two-question pre-screen (asking whether the patient was taking more than three medications a day; or had a previous Elder Alert) to all ED patients age $ 75, except those coming in for a scheduled blood transfusion; noted responses; and put the ISAR screening form on the chart of all patients who scored positive for either pre-screen risk factor. (The pre-screen was intended to minimize the time required for ISAR screening and to assist in integrating the new high-risk screening program into existing ED procedures.) (2) ED nurse administered ISAR to all patients who pre-screened positive, and stamped “Elder Alert” on charts (and added care plan forms) for all patients with ISAR score $2. (3) ED clerk created and filed an Elder Alert card for all patients with ISAR score $ 2 (to facilitate tracking should these patients return to the ED at a later date). (4) Once the ED physician decided whether or not to admit the patient, the nurse responsible for the patient (either in the ED or on the ward) initiated referrals, and referral disciplines carried out referrals and followed up as necessary. These steps are shown in Figure 1, used in the extensive staff orientation and in-service training at SPH. Note that in this initial process, the ED in-patient fan-out required referrals to Pharmacy, Physiotherapy, Social Work, and Nutrition for all Elder Alert patients. As the program proceeded (do the improvement), regular Working Group meetings were held. These meetings provided a means to inform staff of the progress of the Elder Alert program and any process changes, and to receive and discuss suggestions for improving the program (study performance). Based on the experience gained, many changes were made to the program process and to the audit process (act on study results). The original ISAR questions were used during the evaluation period reported here. They are shown in Table I. A “yes” answer is scored as 1, while a “no” answer is scored as zero, except for Question 4 (vision), where the scaling is reversed because of the wording of the question.

342

Identification of seniors at risk

International Journal of Health Care Quality Assurance

Rebecca N. Warburton et al.

Volume 17 · Number 6 · 2004 · 339-348

Figure 1 Elder Alert Program – initial process diagram

Table I ISAR screening questions (original) 1.

4.

Before the illness or injury that brought you to the Emergency, did you need someone to help you on a regular basis? Since the illness or injury that brought you to the Emergency, have you needed more help than usual to take care of yourself? Have you been hospitalized for one or more nights during the past six months (excluding a stay in the Emergency Department)? In general, do you see well?

5.

In general, do you have serious problems with your memory?

6.

Do you take more than three different medications every day?

2. 3.

Evaluation design (study the change) The evaluation of Elder Alert has six proposed phases: (1) process evaluation; (2) simple before-after comparison of length of stay for high-risk versus other patients; (3) more careful before-after outcome comparison using more sophisticated outcome indicators;

Yes No Yes No Yes No Yes No Yes No Yes No

01 00 01 00 01 00 00 01 01 00 01 00

(4) assessment of effects on staff and staff opinions about the Elder Alert program; (5) assessment of patient experience of care that results from the Elder Alert program; and (6) cost utility analysis. Process evaluation Process evaluation (to assess the completeness of implementation of the intended program) was

343

Identification of seniors at risk

International Journal of Health Care Quality Assurance

Rebecca N. Warburton et al.

Volume 17 · Number 6 · 2004 · 339-348

seen as an essential first step; here it has been used as a formative evaluation tool to improve the screening and referral processes. Once the program design has been finalized, and it is found to be operating as intended, then assessment of impacts on patient outcomes, costs, and staff and patient experience will proceed. Process evaluation to date has consisted of a series of audits, examining how closely actual procedures at SPH followed the intended Elder Alert process: (1) Audit 1: May 24-28, 2003. (2) Audit 2: July 3-8, 2003. (3) Audit 3: July 14-17, 2003. (4) Audit 4: July 23-31, 2003. (5) Audit 5: September 30-October 6, 2003. (6) Audit 6: October 28-November 3, 2003. (7) Audit 7: November 25-December 1, 2003.

scored $2 and were admitted, hence who should have had Elder Alert implemented while in hospital; and the number who received key Elder Alert elements (file card, care plan, referrals sent). Particularly in early audits, many patients had the ISAR partially completed or completed with an error; and many had Elder Alert procedures partially implemented. Later audits examined whether care plans were fully completed and whether referrals were completed, though referral completion is not shown in the table. By the latest audit reported here all admitted patients had some elements of Elder Alert, although few had all, and care plans often remained incomplete. There were numerous program process changes after each audit. The most significant were: (1) Audit 1: May 24-28, 2003 – stopped doing pre-screen (found not to be clinically useful as virtually all patients screened positive). (2) Audit 2: July 3-8, 2003 – ED clerk (not triage nurse) placing ISAR form on chart. (3) Audit 3: July 14-17, 2003 – some relaxation from required referrals in ED, instead the registered nurse caring for the patient to use clinical judgement; admitted patients still required all referrals. (4) Audit 4: July 23-31, 2003 – most pre-booked patients excluded; audits began more carefully checking referrals, admission status, care plan completeness. (5) Audit 5: September 30-October 6, 2003 – no ED referrals after hours except to Liaison; weekday ED referrals to be based on the clinical judgement of the nurse caring for the patient; admitted patients still required all referrals. (6) Audit 6: October 28-November 3, 2003 – triage nurse to put the EA screening form on the patient chart; ED or ward nurse to complete ISAR questions, stamp chart, and create file card if needed. (7) Audit 7: November 25-December 1, 2003 – decided to re-word screening questions to reduce screening and scoring errors; new procedure for patients with chest pain.

Each process audit led to modifications to both Elder Alert program processes and to the audit forms and processes used for subsequent audits.

Study limitations Our process evaluation was designed to capture essential, objective, information on the application of the screening program. This first set of plan-dostudy-act improvement cycles has not monitored or assessed clinical leadership, nor has it collected systematic information on the views of affected staff, or on impacts on patient outcomes. Process audits alone could not collect anecdotal evidence or “rich” but subjective observations (“telling stories”) about the progress of the Elder Alert program. Therefore we cannot yet assess the clinical utility of the screening program, nor assess its costeffectiveness. Later phases of the evaluation will address these important questions through future plan-do-study-act improvement cycles.

Results The results reported here are for only the first phase, process evaluation. Table II summarizes results of the audits for admitted patients. Much more detail was collected (including information on non-admitted patients) and is available upon request. The table shows the number of eligible patients for the audit period (age $ 75, not extended care, palliative care, or scheduled blood transfusion); the number of days in the audit period; the number of patients who had the ISAR form on their chart and (for later audits) the number where the ISAR was completed in full and correctly; the number of those ISAR-screened who

Interim findings Our experience revealed a number of interim findings: . While the Working Group had realized during the PLAN stage that patients coming in for a scheduled blood transfusion, extended care patients, and palliative care patients did not require Elder Alert screening and should be excluded, it became clear as the program was

344

Identification of seniors at risk

International Journal of Health Care Quality Assurance

Rebecca N. Warburton et al.

Volume 17 · Number 6 · 2004 · 339-348

Table II Process evaluation summary

Audit 1 2 3 4 5 6 7c

Days in period

Eligible patients

5 6 4 9 7 7 7

46 50 42 73 51 55 62

Patients receiving ISAR screening n Correct score 16 17 7 16 17 25 30

Admitted, need Elder Alert

Admitted, Elder Alert implementeda

Referrals initiatedb

5 2 0 0 8 10 10

1 2 0 0 3 2 4

6 6 5

21 24

Notes: a File card in box, care plan forms on chart and at least partly completed; b To Physiotherapy, Social work, Liaison, and Nutrition; referral completion not verified; c First audit since opening of new ED in November 2003

.

.

.

implemented that exception procedures were needed for most pre-booked patients as well. Workable procedures for patients with chest pain are still being developed; these patients currently are not being screened on arrival at the ED because of the urgency of their clinical situation, but may require screening later. Despite extensive staff training, nurses frequently misunderstood Question 2 (Since the illness or injury that brought you to the Emergency, have you needed more help than usual to take care of yourself?), hence left the question unanswered. This question was intended to ask about the patient’s state, after developing the problem that brought them to the ED, but before actually coming to the ED. The audits discovered missing answers, and discussion at the Working Group revealed several different interpretations of the question, including a belief by some staff that it referred to the period while actually in the ED. Nurses and patients were confused by the reference to the ED in Question 3 (Have you been hospitalized for one or more nights during the past six months (excluding a stay in the Emergency Department)?), with some believing that a previous overnight visit to the ED should be included and others believing it should be excluded. The reverse scaling of Question 4 (In general, do you see well?) was confusing to nurses and patients, and the initial forms developed at SPH actually scored “yes” as 1, and “no” as 0, which is correct for all other questions but is backwards for Question 4. This mistake was at first hand-corrected on the screening forms, then corrected on the originals; but with both the hand-corrected and the properly amended forms, many ISAR scores were added up (by nurses) simply by counting the “yes” answers, leading to a number of scoring errors related to Question 4.

.

.

Nurses and patients were confused by the wording of Question 6 (Do you take more than three different medications every day?), answering “yes” for patients taking three or more medications instead of (as intended) answering “yes” only for those taking four or more different medications daily. In addition, this question was not clinically useful as a screening tool, since virtually all patients aged $ 75 answered “yes”. This meant that one other positive ISAR answer resulted in a positive screen for Elder Alert, and in practice meant that almost all patients aged $ 75 were being designated Elder Alert, overwhelming available referral resources. Limited referral resources proved problematic. Pharmacy staff found referrals to be very time-consuming, and were not able to complete them in a timely fashion; by later audits the pharmacy did not attempt to complete any referrals. With few referral services available on nights and weekends, and some available only part-time on weekdays, it proved to be impractical for the ED to make referrals except at times when referral services were actually available.

Recent changes The Working Group reviewed the experience to date in January 2004, and decided that revision of the ISAR screening tool was essential to reduce screening and scoring errors. According, RNW contacted JM and asked for her assistance in refining the questions while retaining their validity. Based on consultation between RNW, JM and colleagues, and Working Group members, four revisions were made to the original questions, as shown in Table III. Questions 2, 3, and 4 were revised to avoid the misunderstandings that had been revealed in the process evaluation. Question 6 was revised to reflect

345

Identification of seniors at risk

International Journal of Health Care Quality Assurance

Rebecca N. Warburton et al.

Volume 17 · Number 6 · 2004 · 339-348

Table III Original question

Revised question

2.

2.

In the last 24 hours, have you needed more help than usual?

3.

Have you been hospitalized for one or more nights during the past six months?

4.

In general, do you have serious problems with your vision, that cannot be corrected by glasses? Do you take six or more different medications every day?

3.

4. 5.

Since the illness or injury that brought you to the Emergency Department, have you needed more help than usual to take care of yourself? Have you been hospitalized for one or more nights during the past six months (excluding a stay in the Emergency Department)? In general, do you see well? Do you take more than three different medications every day?

6.

the greater prevalence of polypharmacy, with the intention of screening in (as high-risk) only 30 percent to 50 percent of patients aged $75. The revised questions (shown in Table III) were implemented 2 February 2004. The success of these revisions will be tested in future process audits. Our results (including the revised wording for ISAR screening questions) are also being used to guide implementation at three more British Columbia hospitals this spring. The question revisions are intended to increase the clinical utility of the ISAR screening tool. Making the questions easier to understand should reduce screening errors and improve the targeting of preventive services to patients at highest risk. These changes have good face validity because they were based on both user feedback and consultation with the researchers who designed the original ISAR screening tool. Longer-term, it would be valuable to validate the revised questions as was done for the original questions, by statistical testing on the correlation between question responses and adverse outcomes in elderly patients; this work cannot proceed until program procedures have stabilized, however.

Conclusions Six main conclusions emerge from our work to date: (1) Despite putting together an excellent multidisciplinary Working Group and providing extensive staff training, implementing the Elder Alert program (the do stage) was much more complex than had been realized in the plan stage. The time required for nurses and other staff to process referrals and keep records had been greatly underestimated, as was the difficulty of integrating gerontological screening with usual ED triage, diagnosis, and treatment activities. It turned out to be essential to limit referrals in order to avoid overwhelming

346

available referral resources. Because every procedure change affected many staff, managing change was more challenging than had been anticipated. (2) The process evaluation audits (study the change) emerged as an essential improvement tool. Without the audits, the program almost certainly would have failed, and managers would not have understood why; the audits made it possible for the Working Group to identify problems staff were encountering with the planned program, and adjust procedures (act) to make them practical. Unrecognized implementation difficulties, in programs without process evaluation, may explain why some ED screening programs for seniors are effective at reducing adverse outcomes, and others are not. (3) Regular meetings of the multidisciplinary Elder Alert Working Group, including academic collaborators, were an equally essential improvement tool. Most problems revealed in the audits could not be resolved by any single individual or department, and Working Group discussions were essential for deep understanding of the reasons for the problems, and to design appropriate and workable solutions. Working Group meetings also allowed staff to share their success by “telling stories”, confirming the value of the high-risk screening approach, and its usefulness as a way of anticipating and preventing clinical problems before they occur, rather than simply reacting to them after the fact. (4) Screening criteria had to be adapted to the specific patient population in order to have clinical utility, and inclusion criteria at Saanich Peninsula may require further adjustment to ensure that patients who do not require high-risk screening are appropriately excluded from the screening program. The goal is efficient use of ED and ward nursing time; screening should not be applied to patients who (because of their

Identification of seniors at risk

International Journal of Health Care Quality Assurance

Rebecca N. Warburton et al.

Volume 17 · Number 6 · 2004 · 339-348

clinical situation) are clearly high-risk or clearly low-risk, but instead should be used as a tool to detect high-risk patients who would otherwise be missed. Once patient eligibility criteria are correctly specified, 100 percent of eligible patients should be screened and approximately 30 percent to 50 percent should screen positive for high risk; these objective goals will be assessed in subsequent plan-do-study-act improvement cycles. (5) Our experience underlines the importance of ongoing monitoring, assessment, and local adaptation in order for screening tools to be practical and useful. Although the ISAR questions had been extensively pre-tested in the original population of elderly ED patients at four Montreal hospitals (McCusker et al., 1999), some wording changes were required for their use in a different setting. Furthermore, one question had to be changed because of changes in medical practice. In the original study, conducted in 1996, less than half of the study population reported taking more than three different medications every day. In the current study, virtually all patients took more than three medications, requiring an upward revision of the number of medications to allow this question to identify higher risk patients. We will continue to assess and revise the screening program as needed, combining knowledge from the literature with insights gained from local practice. The revised screening questions will need to be validated through statistical analysis, as was done for the original questions. (6) High-risk screening in the ED is critical to a hospital system’s ability to anticipate clinical problems. Further, the plan-do-study-act improvement cycle is a practical and useful tool for improving quality and systems in a community hospital ED. Lessons learned from the SPH Elder Alert program can serve to reduce barriers and resistance to change in the future.

hospital admission”, Journal of American Geriatrics Society, Vol. 41, pp. 811-17. Canadian Institute of Health Information (CIHI) (2000), Canada’s Elderly Primary Users of Hospitals, CIHI, Ottawa, pp. 1-4. Clark, E. (2001), “Preoperative assessment: primary care workup to identify surgical risks”, Geriatrics, Vol. 56 No. 7, pp. 36-40. Creditor, M.C. (1993), “Hazards of hospitalization of the elderly”, Annals of Internal Medicine, Vol. 118, pp. 219-33. Dendukuri, N., McCusker, J. and Belzile, E. (2004), “The identification of seniors at risk screening tool: further evidence of concurrent and predictive validity”, J. Am. Geriatr. Soc., Vol. 52 No. 2, pp. 290-6. Dudek, S.G. (2000), “Malnutrition in hospitals: who’s assessing what patients eat?”, American Journal of Nursing, Vol. 100 No. 4, pp. 36-43. Fisher, B.W. and Flowerdew, G. (1995), “A simple model for predicting postoperative delirium in older patients undergoing elective orthopedic surgery”, American Geriatric Society, Vol. 43, pp. 175-8. Heruti, R.J., Lusky, A., Barell, V., Ohry, A. and Adunsky, A. (1999), “Cognitive status at admission: does it affect the rehabilitation outcome of elderly patients with hip fractures?”, Arch. Physical Medical Rehabilitation, Vol. 80 No. 4, pp. 432-6. Inouye, S.K. (2000), “Prevention of delirium in hospitalized older patients: risk factors and targeted intervention strategies”, Annals of Medicine, Vol. 32 No. 4, pp. 257-63. Inouye, S. and Charpentier, P.A. (1996), “Precipitating factors for delirium in hospitalized elderly persons”, Journal American Medical Association, Vol. 275 No. 11, pp. 852-7. Inouye, S.K., Wagner, D.R., Acampora, D., Horwitz, R.I., Cooney, M.L., Hurst, L.D. and Tinetti, M.E. (1993), “A predictive index for functional decline in hospitalized elderly medical patients”, Journal of General Internal Medicine, Vol. 8, pp. 645-52. Karp, A.H. and Koval, K.J. (1998), “Preoperative medical evaluation of the elderly patient”, Archives of the American Academy of Orthopaedic Surgeons, Vol. 2 No. 1, pp. 81-7. Langley, G., Nolan, K., Nolan, T., Norman, C. and Provost, L. (1996), The Improvement Guide: A Practical Approach to Enhancing Organizational Performance, Jossey-Bass, San Francisco, CA. Liebergall, M., Soskolne, V., Mattan, Y., Feder, N., Segal, D., Schneidman, G., Stern, Z. and Israeli, A. (1999), “Preadmission screening of patients scheduled for hip and knee replacement: impact on length of stay”, Clinical Performance Quality Health Care, Vol. 7 No. 1, pp. 17-22. McCusker, J., Kakuma, R. and Abrahamowicz, M. (2002), “Predictors of functional decline in the hospitalized elderly: a systematic review”, J. Gerontol. A. Biol. Sci. Med. Sci., Vol. 257A No. 9, pp. M569-77. McCusker, J., Cardin, S., Bellavance, F. and Belzile, E´. (2000a), “Return to the emergency department among elders: patterns and predictors”, Acad. Emerg. Med., Vol. 7 No. 3, pp. 249-59. McCusker, J., Cole, M.G., Dendukuri, N. and Belzile, E. (2003), “Does delirium increase hospital stay?”, J. Am. Geriatr. Soc., Vol. 51 No. 11, pp. 1539-46. McCusker, J., Bellavance, F., Cardin, S., Belzile, E. and Verdon, J. (2000b), “Prediction of hospital utilization among elderly patients during the 6 months after an emergency

References Aminzadeh, F. and Dalziel, W. (2002), “Older adults in the emergency department: a systematic review of patterns of use, adverse outcomes, and effectiveness of interventions”, Annals of Emergency Medicine, Vol. 39 No. 3, pp. 238-47. Boult, C., Dowd, B., McCaffrey, D., Boult, L., Hernandez, R. and Krulewitch, H. (1993), “Screening elders for risk of

347

Identification of seniors at risk

International Journal of Health Care Quality Assurance

Rebecca N. Warburton et al.

Volume 17 · Number 6 · 2004 · 339-348

department visit”, Ann. Emerg. Med., Vol. 36 No. 5, pp. 438-45. McCusker, J., Bellavance, F., Cardin, S., Trepanier, S., Verdon, J. and Ardman, O. (1999), “Detection of older people at increased risk of adverse health outcomes after an emergency visit: the ISAR screening tool”, J. Am. Geriatr. Soc., Vol. 47 No. 10, pp. 1229-37. McCusker, J., Verdon, J., Tousignant, P., de Courval, L.P., Dendukuri, N. and Belzile, E. (2001), “Rapid emergency department intervention for older people reduces risk of functional decline: results of a multicenter randomized trial”, J. Am. Geriatr. Soc., Vol. 49 No. 10, pp. 1272-81. Palmer, R. (1995), “Acute hospital care of the elderly: minimizing the risk of functional decline”, Cleveland Clinical Journal of Medicine, Vol. 62 No. 2, pp. 117-28.

Parke, B. and Brand, P. (2004), “An elder friendly hospital: translating a dream into reality”, Canadian Journal of Nursing Leadership, Vol. 17 No. 1, pp. 62-76. Parke, B. and Stevenson, L. (1999), “Creating an elder friendly hospital: one organization’s experience”, Healthcare Management Forum, Vol. 12 No. 3, pp. 45-8. Thomas, D.R. and Richie, C.S. (1995), “Preoperative assessment of older adults”, Journal of American Geriatrics Society, Vol. 43, pp. 811-21. Winograd, C.H., Gerety, M.B., Chung, M., Goldstein, M.K., Dominguez, F. and Vallone, R. (1991), “Screening for frailty: criteria and predictors of outcomes”, Journal of American Geriatrics Society, Vol. 39, pp. 778-84.

About the authors Rebecca N. Warburton is Assistant Professor, School of Public Administration, University of Victoria, Victoria, Canada. Belinda Parke is Clinical Nurse Specialist: Older Adult Health, Research and Clinical Development, Fraser Health Authority, Chilliwack, Canada. Wynona Church is a Master of Public Administration Graduate, University of Victoria, Victoria, Canada. Jane McCusker is Professor, Department of Epidemiology and Biostatistics, Faculty of Medicine, McGill University, Montreal, Canada.

348

Introduction

Completing the circle: from PD to PDSA Paul Walley and Ben Gowland

The authors Paul Walley is Lecturer in Operations Management, Warwick Business School, University of Warwick, Coventry, UK. Ben Gowland is Programme Director, NHS Modernisation Agency, London, UK.

Keywords Continuous improvement, Team working, Employees, Health services, United Kingdom

Abstract Problem-solving teams, involving front-line staff, are widely used to achieve continuous process improvement. Approaches such as “plan-do-study-act” (PDSA) cycles, are now a core element of many health-care improvement initiatives. This paper evaluates the use of PDSA improvement cycles within the UK National Health Service, using emergency care improvement activity as a source of research evidence. It was found that, despite an abundance of information on how to implement this type of change, many senior professionals still misinterpret how this should work. This has implications for how such methodologies are implemented. There is a long way to go in allowing greater employee involvement, moving much further away from the “management committee” style of change. Care has to be taken to ensure that empowered employees are working to consistent and appropriate objectives. It is important that senior personnel develop process understanding alongside the workforce, rather than simply providing distant support.

Electronic access The Emerald Research Register for this journal is available at www.emeraldinsight.com/researchregister The current issue and full text archive of this journal is available at www.emeraldinsight.com/0952-6862.htm

A recent Modernisation Agency (2003) directory lists over 80 change programmes that are being implemented within the UK National Health Service (NHS). A considerable proportion of these apply process redesign tools that have been adopted widely in manufacturing and service organisations for many years. This approach is heavily influenced by the US-based organisation, the Institute for Healthcare Improvement (IHI). They have taken the work of the quality gurus, such as Deming (1986) and Juran (1989), and refocused their methodologies so that the tools and techniques can be adopted by healthcare organisations. In particular, they have taken quality improvement methods and adopted these as their improvement implementation strategy. Most organisations have experienced some difficulty in the adoption of quality improvement methods, with high failure rates often quoted (Beer et al., 1990). Healthcare organisations are often slow to adopt external management practices and they also seem to struggle to sustain good practice. It is therefore important to assess how well quality improvement methodologies are being implemented. This paper investigates the use of the plan-do-study-act (PDSA) approach to continuous quality improvement. The approach is an important element of most quality improvement initiatives because it is used to focus and structure all improvement activity within each programme. The experiences of the ten healthcare regions involved in the “Ideal Design of Emergency Access” (IDEA) project were used as case study sites for research evidence. The IDEA project was started in the spring of 2001 as a means of developing the UK’s emergency care system, taking a whole systems approach to improvement. The project finished in April 2003. The tools and techniques used on the project included process mapping, capacity and demand theory and “lean thinking” (Womack and Jones, 1996). Like other Modernisation Agency initiatives, the improvement work was based around PDSA improvement cycles, using local teams. The three objectives of the IDEA project were: (1) reducing delays and waits for patients requiring emergency care; (2) reducing inequity and improving the quality of local services; and (3) improving the experience for patients and carers. The research reported here was part of a larger study to evaluate the effectiveness of the

International Journal of Health Care Quality Assurance Volume 17 · Number 6 · 2004 · pp. 349-358 q Emerald Group Publishing Limited · ISSN 0952-6862 DOI 10.1108/09526860410557606

The authors would like to acknowledge the help of Wendy Joberns and Mandy Sankey at Birmingham Children’s Hospital NHS Trust.

349

Completing the circle: from PD to PDSA

International Journal of Health Care Quality Assurance

Paul Walley and Ben Gowland

Volume 17 · Number 6 · 2004 · 349-358

Emergency Services improvement initiatives. The broader findings are reported elsewhere (Walley, 2003). In this paper, we investigate the: . improvement roles played by senior managers and clinicians; . response of senior managers and clinicians to incremental improvement activities; and . relationship between roles and improvement outcomes.

Professional

Empowerment

Individual responsibility Professional leadership Autonomy Administrative authority Goal expectations Rigid planning Responses to complaints Retrospective performance appraisal Quality assurance

Collective responsibilities Managerial leadership Accountability Participation Performance and process expectations Flexible planning Benchmarking Concurrent performance appraisal Continuous improvement

Source: Zbabada et al. (1998) after Short and Rahim (1995)

Making quality improvement work There is a wealth of literature that defines total quality management (TQM) approaches in healthcare and the general factors that make TQM successful. This literature will not be fully reviewed here. Instead, we will focus on the issues relating to management and clinican involvement and behaviour during the implementation of improvement programmes. The majority of the literature considers management and clinician commitment and involvement as essential for successful implementation (see Joss, 1994a; Nwabueze and Kanji, 1997; Jackson, 2001). Zbabada et al. (1998) highlight key issues in the translation of improvement programmes from commercial organisations to healthcare. TQM assumes there is hierarchical control of management over the technical core of the organisation and there is dominance of rational decision-making processes (after Arndt and Bigelow, 1995). The powerful clinician subculture makes quality definition difficult to define with a consensus agreement. They see clear obstacles unique to healthcare: . clinicians’ perception of lack of applicability; . the under-involvement of clinicians; . the lack of attention to the presence of the patient within processes; . the opposition to consumer involvement in the system; . rigid, hierarchical structures; and . the lack of consumer power. They contrast traditional hospital professional and empowerment models of management, shown in Table I. The contrast in approaches is further discussed by Rago (1996) in relation to the transformation of a US mental health organisation. He identifies a number of organisational challenges that have immediate personal implications for senior personnel, summarised in Table II. Rago (1996, p. 233) emphasises the need for management to have faith in what they are doing: The manager’s lack of faith leads to his or her wanting to add or subtract something from the

Table I Hospital professionals and empowerment models of management

decisions of others . . . The senior manager almost reflexively alters this decision in some way or other.

The contrasts we have established above highlight the change in culture and management style that is required when implementing participative styles of working. Imai (1986) demonstrated the contrasting approaches of traditional Western and Japanese approaches to improvement. Imai proposed that, in most traditional organisations change tends to be large step, “breakthrough” change, usually involving new technology and capital investment. This change is very much management-lead. By contrast, the Japanese approach to continuous improvement, kaizen, emphasises small-scale changes. Table III summarises the characteristics of each approach. In summary, it is a key question whether senior managers and clinicians can adapt to the changes in managerial style. The research conducted here established the extent to which this change is style was achieved at the study sites.

Structured quality improvement Most authors (e.g. Roberts, 1993; Joss, 1994b; Jackson, 2001; Nwabueze, 2001) see that structures for both implementation and the change process are necessary if an empowerment is to be successfully implemented. A key element of this is the array of problem-solving methodologies used to steer improvement teams. Many programmes have used a methodology now referred to as the “Deming Wheel” or the PDCA cycle. The framework has been marginally adapted for use in healthcare and is called the PDSA cycle. Figure 1 shows the framework. It is widely recognised that a structured problemsolving approach is necessary when attempting to tackle improvement in a participative style (Pescod, 1994). Problem-solving systems involve: . create logical steps to solve a problem, ensuring the root cause is found;

350

Completing the circle: from PD to PDSA

International Journal of Health Care Quality Assurance

Paul Walley and Ben Gowland

Volume 17 · Number 6 · 2004 · 349-358

Table II The challenges of empowerment as a leadership activity Organizational barrier

Personal struggle

Inability to communicate the [organisation’s] purpose and to communicate this and coordinate activity Lack of preparedness to assume responsibility associated with empowerment Lack of understanding of how empowerment works Difficulty in providing information to decision-makers

Lack of faith that others will make the right decisions Difficulty in transcending personal point of view Difficulty in relinquishing decision-making authority

Source: Rago (1996, p. 233)

Table III A comparison of breakthrough and continuous improvement Effect Pace Time-frame Change Approach Stimulus Requirements Orientation

Breakthrough improvement

Continuous improvement

Short-term, dramatic Big steps Intermittent Abrupt and volatile Individualism Technological change Investment Technology

Long-term and undramatic Small steps Continuous, incremental Gradual and constant Collectivism Conventional know-how Effort to maintain People

Source: Adapted from Imai (1986)

Figure 1 The PDSA cycle

of early improvement suggestions can cause a considerable loss of employee motivation. Hence, the planning phase of an improvement cycle must be used to establish the appropriateness of the intended improvement activity. In complex environments, it can be difficult to establish whether or not any single change has caused an improvement to a process. This is even more relevant where the intended scale of changes is relatively small. The study phase of the improvement forces measurement to take place. This ensures that measurable objectives are set and also encourages experimentation of alternative solutions. The final phase of the cycle is to either withdraw or embed the proposed change depending upon the results of the study. Withdrawal of a change is not seen as a team failure if results do not support the proposal.

Research method

. . . . .

prevent solutions tackling symptoms; prevent recurrence of problems; ensure solutions are based on hard data; establish a common language; and create the environment that facilitates empowerment.

If teams do not follow a structure, a number of problems may appear. For instance, teams needs to be given direction, so that the problems they tackle are consistent with the overall objectives for the organisation. It is known that many teams can focus attention on issues that do not make an appropriate impact on the system or the ideas are inconsistent with strategy. Although management control systems can keep teams in check, this is usually very de-motivating for the team personnel. The rejection

The principal author was involved in action research at the sites over the duration of the project. Progress data were continually assessed from selfreporting systems. These reports highlight progress against objectives and provide some formal measurement of outcomes, such as waiting time. The authors visited each site to validate the selfassessment report and to investigate the other aspects of the study. The following participants were interviewed, where possible: . the project manager; . the lead clinician; . at least one project assistant (where relevant); . members of staff in the emergency care system not working full-time directly with the project, but who had been affected by any changes; and . other representatives of the emergency care system, where this was deemed relevant (e.g. Primary Care physicians, ambulance representatives etc.). Table IV shows the sites involved in the study. In the table, the regions’ names have been simplified for clarity. Each region is usually represented by one or occasionally two hospital sites covering a

351

352

15,188

25

4

No. emergency admissions

% emergency admissions

No. of FTE A&E

N/A

N/A

4

99,300

Inner city

245,000

South London

3

49.6

27,430

55,300

Affluent town

250,000

North West England (Hospital 2)

4

22.4

16,220

72,550

City

500,000

East Midlands

7

27.7

38,385

138,700

City with affluent suburbs

500,000

Central England

3

22.0

11,986

54,500

Large town + rural

350,000

East England

3

45.2

25,000

55,200

City within a tourist area

350,000

South West England

2

28.8

13,641

47,400

Town with tourists and retired

170,000

South West England (North Coast)

2

21.9

12,219

55,700

Affluent town

600,000

Central East England

2

17.4

13,052

74,900

Large, affluent town

600,000

North of London

6

11.4

9,332

81,800

Town + rural. Isolated

500,000

North East England

2

16.8

11,901

70,800

City suburb

500,000

North London

Note: These figures are for illustrative purposes only. There may be differences in how the measures are collected at site level. For example, some sites may include all acute admissions in their figures, whereas others may only count those that have been through an A&E process

60,500

Town + rural

Area type

A&E attendances/ year

309,000

Area population

Site location

North West England (Hospital 1)

Table IV Site information for all regions involved in the IDEA project

Completing the circle: from PD to PDSA International Journal of Health Care Quality Assurance

Paul Walley and Ben Gowland Volume 17 · Number 6 · 2004 · 349-358

Completing the circle: from PD to PDSA

International Journal of Health Care Quality Assurance

Paul Walley and Ben Gowland

Volume 17 · Number 6 · 2004 · 349-358

relatively small geographic area of England. Where regions include a city, there may be several other hospitals not included in the study that cover separate parts of the same city. There are approximately 200 such regions within England. Wales, Scotland and Northern Ireland are not represented in the study.

admitted to hospital. There is a risk, especially for chronically ill patients, that they become institutionalised if they are admitted to hospital. These patients potentially have a better quality of life if they can be assessed and returned to their existing homes with appropriate community support. Regrettably, the role of the MAU has been misunderstood by many people and can be misused because patients admitted to an MAU via Accident and Emergency (A&E) are considered to have been “treated” by A&E, helping the hospital to meet its four-hour A&E throughput time target, even though they may not have received any treatment within A&E. In one hospital studied, managers perceived the “trolley waits” in A&E as evidence of the need for an MAU without performing any process mapping or analysis of the situation. The solution was created by a “gut feel” that A&E lacked space and the hospital needed more bed space for emergency patients. An MAU was created, at the cost of £2 million per annum, closing a treatment ward to fund the running costs. As soon as the MAU opened, A&E patients seemed to move out more quickly in the short term and the decision was perceived to be a success. Our analysis of the situation reveals this to have been a bad decision, where key personnel are unaware of the true impact of their actions. The lack of flow from A&E was not due to a lack of space, nor a genuine shortage of beds. Two factors contributed to the delays: (1) One junior doctor was given sole responsibility for all medical admissions (typically 50 per day). Without the time to complete proper assessment, the doctor would routinely admit anyone where the diagnosis was either uncertain or contained clinical risks. (2) Most consultants (and their teams) only conducted two discharge decision rounds each week. Not only did this unnecessarily add up to four days to each patient’s length of stay, but also there were some days when few discharges occurred. Weekends would see between zero and five patients leaving the hospital, and on Wednesdays only one out of nine medical consultants conducted a discharge round. This partially explained why it was nearly impossible to admit all emergency medical patients on that day of the week.

Results Table V shows the levels of activity that were reported and the number of changes that were embedded during the project’s duration. Two locations were used as pilot sites and had more staff and additional support than the rest. Their projects covered a wider span of the emergency care system. The remaining sites focused on particular process streams of activity, such as “minor injury” patient categories. The results showed that two clusters of sites readily appeared. One small cluster viewed the PDSA approach as a high-level improvement process that generally mainly involved senior staff. Junior staff were used as suppliers of information (e.g. for process mapping exercises) but kept out of most of the team deliberations. The improvement teams took on large, single projects where the source of problems was usually assumed without prior analysis. The solutions were usually resource intensive (i.e. requiring structural change, the purchase of new equipment or the hiring of new staff). This cluster had a “P-D” mentality. In other words, a solution to a “known” problem was implemented without extensive research and the performance of the implemented solution was not measured rigorously. Every solution was politically and organisationally benign. A characteristic of these changes was that front-line staff were usually not aware of the reason for the change and did not associate the changes with any improvement programme. Although most of the changes had some degree of commitment amongst front-line staff, they did not own the solution. There was a high degree of ownership, occasionally bordering on the possessive, amongst the senior managers. An example of a “P-D” approach – the under-achieving medical assessment unit One of the most common developments within the emergency care system in the UK over the period of study has been the introduction of medical assessment units (MAUs). The intention of these units is to take selected patients out of the A&E departments and provide an intensive period of assessment. The purpose is accurately to establish which patients need to be admitted to an acute medical bed, so that patients are not unnecessarily

Consequently, the MAU acted only as a “buffer” between the A&E and hospital. It took a month for the MAU to fill up with patients whose journey had been blocked by lack of discharges. Soon A&E performance was back to pre-MAU levels of delay. Patients’ expected journey time from A&E to hospital ward has increased by 72 hours, potentially harming the quality of their treatment. (One other MAU created in similar circumstances

353

No. of PDSA cycles No. of patient/carer experience-related changes No. of implemented changes

Site location 227

3 13

6

14

South London

99

North West England (Hospital 1)

354 8

10

61

North West England (Hospital 2)

Table V Reported site activity for all regions involved in the IDEA project

3

15

14

East Midlands

4

15

12

Central England

18

9

25

East England

5

12

71

South West England

2

0

46

South West England (North Coast)

7

12

39

Central East England

8

8

65

North of London

11

4

3

North East England

8

8

11

North London

Completing the circle: from PD to PDSA International Journal of Health Care Quality Assurance

Paul Walley and Ben Gowland Volume 17 · Number 6 · 2004 · 349-358

Completing the circle: from PD to PDSA

International Journal of Health Care Quality Assurance

Paul Walley and Ben Gowland

Volume 17 · Number 6 · 2004 · 349-358

has doubled the death rate for coronary care patients, compared to pre-existing systems). In this case, the PD has created the wrong solution to the problem because of the lack of analysis. We should emphasise that not all MAUs operate in the way described above and some approaches are more successful. Furthermore, many PDs are successful where they happen to adopt an appropriate response to an improvement opportunity. The larger cluster of sites cascaded PDSA activity more extensively. High-level process mapping activity identified problem areas, but these problems were broken down into lower-level PDSA mini-projects over an extended period of time. At these sites, the managerial and clinical roles were very different. Managers were often used as a source of resource (e.g. to find a spare computer to work on data gathering) and they were also used as a conduit for multi-agency working. Similarly, clinicians were used to screen solutions in terms of clinical risk and other aspects of feasibility. Although both management and clinicians did contribute suggestions, they did not impose these solutions. In a number of cases, improvement teams actually disproved the effectiveness of such proposals. The majority of the work was conducted by front-line staff. In the most successful example, over 50 people routinely participated in small team PDSA activity. The level of involvement became so great a “kaizen coordinator” was appointed to control levels of activity and manage potential conflicts of priority across improvement teams. These sites managed to maintain a PDSA cycle, where problems were diagnosed effectively and solutions were carefully evaluated. The top-performing teams used the PDSA cycle as a true experimental methodology. In one case, not only did teams use PDSA cycles to redesign processes, they fine-tuned their designs using statistical process control (SPC) measurement. For example, one team experimented with alternative team structures for rapid assessment of major patients and also for “see-and-treat” minor injury clinics. Useful data were gathered about the behaviour of the processes when teams were lead by senior clinicians or senior nursing staff. This team could identify the effects of single changes of personnel in teams from the process behaviour on their SPC control charts.

transport or take-home pharmacy supplies, even through they may have been medically discharged early in the morning. The lack of synchronisation between arrivals and discharges over each 24-hour period creates very significant problems where both emergency and elective patients are waiting for a well patient to leave a bed. One solution to the problem described has been to create discharge lounges, which should be safe, comfortable and convenient spaces for patients to use while they wait for transport etc. This allows the hospital to create the bed-space up to eight hours sooner than it has traditionally achieved. Birmingham Children’s Hospital NHS Trust (not one of the regions in the main study) successfully implemented a PDSA to investigate the potential value of a discharge lounge. The PDSA was originally identified from analysis of patient throughput time data. This revealed the following causes of delay for medical patients: . waiting for bed (49 per cent of primary causes); . waiting for diagnosis; . waiting for a specialist; and . waiting for treatment.

An example of a PDSA approach – assessing the value of a discharge lounge Emergency admissions occur with moderate demand seasonality, with a relatively predictable pattern of arrivals over a 24-hour period. Discharges from wards have very different 24-hour patterns, with a high proportion of patients staying until late in the afternoon/early evening waiting for

Similarly, for surgical admissions, 42 per cent of delays were caused by lack of availability of a bed. The PDSA was structured in the following manner. Plan A written proposal was submitted detailing an operational policy that would establish a temporary discharge lounge, complete with a protocol for its operation. This proposal detailed what facilities were needed and the staff that would be required for its temporary operation. The protocol deliberately built in a “demand pull” flow, so that arrivals to the discharge lounge would trigger a signal to send a replacement patient to the appropriate ward. Do The facilities were provided in the school building (within the hospital grounds) and a test of the idea took place over the half-term holiday (three days), when the facilities were not needed for other purposes. In practice, this stage meant briefing bed managers, liaising with wards where discharge rounds took place, establishing protocols in pharmacy for prompt supply of prescriptions, a welcome desk for transport coordination and refreshments and entertainment for patients and carers within the discharge lounge itself. Study All incoming patient waiting times were accurately measured over the three-day period and waiting times ranged from 20 to 235 minutes, preventing any patient breaching the four-hour waiting target. The study revealed the following advantages:

355

. . . .

.

.

Completing the circle: from PD to PDSA

International Journal of Health Care Quality Assurance

Paul Walley and Ben Gowland

Volume 17 · Number 6 · 2004 · 349-358

improved patient/family carer satisfaction; improved communication with pharmacy; lighter nursing load; an opportunity for reverse usage as a waiting area for elective patients coming in; children were less upset about being in hospital; and beds were freed up earlier.

The study did identify further opportunities for improvement. For example, ward staff changeovers caused problems with the consistency of operation. The design of the discharge lounge needed some careful attention, as the test space did not have seating for special needs patients. Act The test suggested that the idea had potential and so a longer test was scheduled for the Easter holidays. This later test revealed similar results in terms of patient waiting times and really highlighted the potential of the approach. Implementation issues concerning the consistency of use of the protocol were raised during the second study. The combined PDSAs found that the lounge was mainly used by patients waiting for their take-home drugs. This identified another improvement opportunity. The PDSA revealed that it was possible to improve patient quality, improve flow and reduce A&E congestion and workload through the use of a discharge lounge. It also gave the staff a clear set of options about how to proceed. The significance of the delays in pharmacy may not have been identified by A&E staff if they had not conducted the experiment. The example is typical in its smallscale, low-risk, incremental approach that uses a team to conduct the experiment.

Implications The differences between these two approaches had implications for the change process, the effectiveness of decisions and the sustainability of improvements. It was clear that senior management and clinicians in the P-D group had not especially contemplated passing ownership of improvement activity to others. Most sites had ill-defined strategy, especially where this related to emergency care work and this did contribute to a reluctance to pass responsibility to others. There was also a feeling amongst some senior staff that large changes tied to resource increases were the only solution to their problems. These people did not have the time or opportunity to develop a detailed process view of the system and therefore could not consistently make appropriate decisions. For example, the project coincided with

widespread adoption of medical and surgical assessment units, as a solution to “trolley waits” in A&E. In some cases, this did not help to create better flow of patients through the system. Instead, patient journey times lengthened due to the extra “buffering” between A&E and wards. Analysis of the performance of these units was sometimes very limited, leading to a false impression that flow had improved. The risk of this approach is that changes are sometimes adopted as an act of faith. Where PDSA activity remained structured, analysis of problems was more thorough, but this did lead to implementation issues. Most changes tended to be incremental adaptations of existing systems that did not require significant extra resource. Where new resources were requested, this was often due to the changes requiring a different skill mix and hence either a change of personnel, job design or further training. PDSAs sometimes disproved the need for larger scale changes – the solutions preferred by senior managers and clinicians. For example, one team established that a “rapid assessment unit”, comprising highly-skilled (new) staff was not required if existing processes and procedures were improved. This was not a popular finding among medical consultants. The risk that the organisation takes is that the changes are inconsistent with strategy or that the pattern of PDSA decisions moves away from the senior managers’ assumed model of how the system should work. For example, one wellintentioned PDSA change involved putting a nurse practitioner in an ambulance to visit minor injury patients in their home. This prevented those patients from having to be taken to A&E and back home in an ambulance. Although this did reduce the numbers of patients waiting in A&E for ambulances, it ring-fences resources and substantially changes the service delivery concept. The PDSA cluster of sites developed extremely good process knowledge amongst front-line staff. As time progressed, teams accelerated their own rate of change and started to become a resource for the health care community as a whole. Staff working on other change initiatives started to use the emergency care flow analysis, PDSA data and SPC charts for their own work. At one site, the level of knowledge became so great that it became threatening to senior staff. One senior manager, with responsibility for process improvement, started to deliberately undermine the PDSA teams because of the perceived threat from more knowledgeable staff. During the review, this manager reported that the PDSA activity had not resulted in any performance improvement and that the 90 per cent reduction in throughput time for some patients would have happened anyway during the manager’s tenure. At another site, a hospital director similarly dismissed the PDSA achievement, claiming that he had introduced the

356

Completing the circle: from PD to PDSA

International Journal of Health Care Quality Assurance

Paul Walley and Ben Gowland

Volume 17 · Number 6 · 2004 · 349-358

ideas ten years previously at another hospital and merely transferred to his new site. There was no evidence to support this claim at either site.

emergency admissions with the elective booking system, ensuring that elective admissions did not crowd out all emergency demand. The PDSA approach to policy deployment, known as hoshin kanri, would clearly be a good option to consider when investigating policy deployment methodologies (see Witcher, 2002). Our research showed that resistance to this style of working does not consistently come from one stakeholder group across all sites. Sometimes senior managers were the main source of resistance where they felt threatened by the loss of their control. Senior managers should actively appraise their own style of working, establishing whether or not they have relinquished sufficient control of PDSA activity. A common mistake is to prevent junior colleagues from receiving credit for the success of improvement activity. Where there is argument over who thought of an idea or who contributed to a solution, one of the main points of teamwork has been missed. Equally, there should not be a highlevel inquest if a PDSA fails to work in the manner intended. In this programme, the most successful PDSAs happened where managers and clinicians created an environment that supported occasional experimentation with new process designs without the fear of failure. This creates the need for an understanding of the relationship between risk and potential benefit for a PDSA. In general, small PDSAs usually carry lower risks associated with failure and therefore need fewer controls. Where resistance from senior clinicians was seen, this could be caused by pre-conceptions of what patients’ needs are. In these cases, work to research patients’ needs and patient experience is recommended as a means of establishing desired outcomes. Many clinicians admitted that PDSA work in this area was valuable to establish desired outcomes. Clinicians frequently under-estimated the extent to which patients see errors, or are simply not advised properly about what is happening. It can also come as a surprise that patients often do not want to remain in hospital under the care of the consultant! Comments that “we already understand patients’ needs” were usually found to be wrong where PDSAs were used to research the topic further. Finally, we would recommend that senior managers and clinicians participate in the development of process knowledge. All too often, senior staff make assumptions about where process bottlenecks are. This does result in poor decisions being taken and gives others very little chance of achieving successful change. We would encourage the use of “process rooms”, where high and low level process maps are visible, allowing quick and accurate access to knowledge of how the system should work. This provides a central point where changes can be logged and activity coordinated.

Recommendations These findings help to define further the appropriate behaviour and roles of senior managers and clinicians. Existing management literature on the role of managers in quality improvement tends to emphasise the less tangible aspects of leadership behaviour. It is always quoted that senior management and clinician commitment is required. Their roles are far more substantial both in terms of the implementation process and the content. We would offer the following recommendations concerning the roles of senior managers and clinicians. Senior managers must ensure that PDSA activity is consistent with the overall objectives of the organisation and the wider healthcare system. This means that all employees have a clear view of what shape and structure the improved organisation will look like. This requires a delicate balance between the freedom to work on low-level PDSAs as the teams see fit and an understanding of the types of changes that would be deemed inappropriate for the overall strategy. Managers must not allow PDSAs to proceed and then withdraw support at a late stage because of unspecified objections. Similarly, objectives and targets need to be managed carefully. For example, the “four-hour” target throughput time for A&E departments can be a double-edged sword. While it does provide a focus for improvement, it can also encourage change that is directed towards cosmetic improvement, i.e. the achievement of the target can be used to hide underlying chaos. At this level, it is probably better to emphasise the achievement of process capability, i.e. the mean and standard deviation of process throughput times is such that the process is control and is capable of meeting set targets. The range of initiatives within the UK NHS also means that one improvement project may inadvertently conflict with other activity. Methods of policy deployment can be used to cut across improvement initiatives, to highlight where changes may trade one initiative for another. In these circumstances, the conflict has to be recognised and procedures put in place for reconciling these. For example, one site used a single project coordinator to work on both the A&E project and another for “booked admissions”. At first, this may seem to be a strange decision, but it proved to be very useful in resolving emergency vs elective issues. The project manager was able to link forecast demand for

357

Completing the circle: from PD to PDSA

International Journal of Health Care Quality Assurance

Paul Walley and Ben Gowland

Volume 17 · Number 6 · 2004 · 349-358

The results of the study have shown that the lessons from about 20 years of TQM research still have to be learned at a hospital site level. Like managers in other sectors of the economy, senior personnel in healthcare have traditionally worked with a management model that emphasises infrequent, step-based change as an improvement process. When continuous improvement methods are introduced, care has to be taken that the techniques are not simply superimposed on the existing systems and style of management. If this happens, the outcome can be to create improvement committees that do not involve junior staff. There is an additional risk that the PDSA methodology is used in name only, with little proper assessment or understanding of the process that is being improved. The organisation has to move from the P-D mentality of step-based change towards completing the PDSA circle, using front-line teams to assess the issues properly and monitor the performance of changes. It is too easy to assume that the role of senior personnel is purely to act as figureheads for the activity. Bottom-up improvement activity needs to be complemented by top-down policies that are consistent with the improvement objectives. This requires clear, well-communicated strategy, good policy deployment and a senior team that truly understands the process issues their organisation is facing. As individuals, possibly the biggest challenge for some senior managers is to relinquish a degree of control of the change process and to allow others to share in the credit for performance improvement. Although the need to achieve the commitment of senior clinical staff has been widely acknowledged for many years, the nature of their involvement has been less well articulated before. Consultants can be seen as powerful and influential stakeholders who need to be managed during the implementation process. However, they are much more valuable as advocates of PDSA improvement cycles, once they have seen their potential benefit. Much of the initial reluctance to become involved is because PDSAs challenge their inherent knowledge of the causes of problems and solutions. They frequently accept the PDSA style of working more readily than managers.

Deming, W.E. (1986), Out of the Crisis, Cambridge University Press, Cambridge. Imai, M. (1986), Kaizen – The Key to Japan’s Competitive Success, McGraw-Hill, London. Jackson, S. (2001), “Successfully implementing total quality management tools within healthcare: what are the key actions?”, International Journal of Health Care Quality Assurance, Vol. 14 No. 4, pp. 157-63. Joss, R. (1994a), “What makes for successful TQM in the NHS?”, International Journal of Health Care Quality Assurance, Vol. 7 No. 7, pp. 4-9. Joss, R. (1994b), “Converging implementation strategies in commercial TQM initiatives: implications for the NHS”, International Journal of Health Care Quality Assurance, Vol. 7 No. 2, pp. 4-9. Juran, J.M. (1989), Juran on Leadership for Quality, Free Press, New York, NY. Langley, G.J., Nolan, K.M., Nolan, T.M., Norman, C.L. and Provost, L.P. (1996), The Improvement Guide. A Practical Approach to Enhancing Organizational Performance, Jossey-Bass, San Francisco, CA. Modernisation Agency (2003), The Modernisation Agency Map, Modernisation Agency, London. Nwabueze, U. (2001), “Chief executives – hear thyselves: leadership requirements for 5-S/TQM implementation in healthcare”, Managerial Auditing Journal, Vol. 16 No. 7, pp. 406-10. Nwabueze, U. and Kanji, G.K. (1997), “The implementation of total quality management in the NHS: how to avoid failure”, Total Quality Management, Vol. 8 No. 5, pp. 265-80. Pescod, W.D.T. (1994), “Effective use of a common problemsolving process as an integral part of TQM”, International Journal of Health Care Quality Assurance, Vol. 7 No. 7, pp. 10-13. Rago, W.V. (1996), “Struggles in transformation: a study in TQM, leadership and organizational culture in a government agency”, Public Administration Review, Vol. 56 No. 3, May-June. Roberts, I.L. (1993), “Quality management in health care environments”, International Journal of Health Care Quality Assurance, Vol. 6 No. 2, pp. 25-35. Short, P.J. and Rahim, M.A. (1995), “Total quality management in hospitals”, Total Quality Management, Vol. 6 No. 3, pp. 255-63. Walley, P. (2003), A Report into the Implementation of the Modernisation Agency’s “Ideal Design of Emergency Access” (IDEA) Project, Warwick University, Coventry. Witcher, B. (2002), “Hoshin kanri: a study of practice in the UK”, Managerial Auditing Journal, Vol. 17 No. 7, pp. 390-6. Womack, J. and Jones, D. (1996), Lean Thinking: Banish Waste and Create Wealth in Your Organisation, Simon & Schuster, New York, NY. Zbabada, C., Rivers, P.A. and Munchus, G. (1998), “Obstacles to the application of total quality management in health-care organizations”, Total Quality Management, Vol. 9 No. 1, pp. 57-66.

Summary

References Arndt, M. and Bigelow, B. (1995), “The implementation of total quality management in hospitals: how good is the fit?”, Health Care Management Review, Vol. 20 No. 4, pp. 7-14. Beer, M., Russell, E. and Spector, B. (1990), “Why change programs don’t produce change”, Harvard Business Review, November-December, pp. 158-66.

Further reading Koch, H. (1992), “Sustaining commitment”, Managing Service Quality, March, pp. 157-61.

358

Call for papers International Journal of Health Care Quality Assurance For 16 years the International Journal of Health Care Quality Assurance has covered all issues related to quality in health care. It provides a forum for the international exchange of theoretical and practical aspects of quality assurance, innovation, management, continuous improvement and performance management in health care. The journal will be of interest to all professionals in health care aiming to develop knowledge about quality assurance and process innovation and their implementation in health-care systems. IJHCQA is uniquely placed to cover all recent developments concerning health-care quality and clinical governance. The topics addressed are both clinical and non-clinical, encouraging the translation of theory and tried and tested approaches into practice. As part of its unique placement, the journal aspires to provide a network for practitioners to improve quality within their own organisation. In particular, the journal encourages new writers to publish their work.

The journal explores topics surrounding: Successful quality/continuous improvement projects. The use of quality tools and models in leadership management development such as the EFQM Excellence Model, Balanced Scorecard, Quality Standards and issues relating to Process Control, Leadership, Managing Change, Pareto Analysis, Process Mapping, Theory of Constraints. A key principle is the support of management in leadership in bringing about the changes in quality cultures. . Improving patient care through quality-related programmes and/or research. . .

Articles submitted may be of a theoretical nature, be based on practical experience, provide stimulus for debate, report a case study situation, or report on experimental results. As a guide, articles should be between 3,000 and 6,000 words in length. Please send papers to the Editors: Robin Gourlay: [email protected] or Keith Hurst: [email protected]