Using SERVQUAL Model to Assess Hospital Information System Service Quality

AUTHORS

Mohamad Jebraeily 1 , Bahlol Rahimi 1 , Zahra Zare Fazlollahi 1 , Hadi Lotfnezhad Afshar 1 , *

1 Department of Health Information Technology, School of Allied Medical Sciences, Urmia University of Medical Sciences, Urmia, Iran

How to Cite: Jebraeily M, Rahimi B, Zare Fazlollahi Z, Lotfnezhad Afshar H. Using SERVQUAL Model to Assess Hospital Information System Service Quality, Hormozgan Med J. 2019 ; 23(1):e86977. doi: 10.5812/hmj.86977.

ARTICLE INFORMATION

Hormozgan Medical Journal: 23 (1); e86977
Published Online: February 13, 2019
Article Type: Research Article
Received: December 1, 2018
Revised: January 8, 2019
Accepted: February 4, 2019
Crossmark

Crossmark

CHEKING

READ FULL TEXT
Abstract

Background: Assessment of hospital information system (HIS) service quality helps to meet the needs of users and a strategy to expand the interaction between HIS developers and the users. SERVQUAL is an extensively used technique to measure the service quality of information systems.

Objectives: The purpose of this study was to assess HIS service quality by the SERVQUAL model in the teaching hospitals affiliated to Urmia University of Medical Sciences (UMSU).

Methods: This study is a descriptive analysis carried out in UMSU teaching hospitals in 2017. The sample was comprised of 270 users selected randomly via multi-stage cluster sampling. The modified SERVQUAL questionnaire, which included five dimensions, was used to collect data. The gap between the perceptions and the expectations of the users was calculated and the significance of scores was tested.

Results: The highest quality gaps in the five dimensions were related to responsiveness (-1.52) and reliability (-1.34) and also the lowest quality gap was related to tangibles (-0.95). There were significant differences between perceptions and expectations of the users in all SERVQUAL dimensions (P < 0.001). This implied that the quality of the delivered services was lower than what the users expected.

Conclusions: Given the greatest gap in responsiveness and reliability, it is essential that HIS vendors focus on providing HIS support and updating services and allocate IT staff with the right knowledge and skills to provide the trust needed to use HIS services in the users.

Keywords

Hospital Information Systems SERVQUAL Model Service Quality Information Technology Information Systems Assessment

Copyright © 2019, Hormozgan Medical Journal. This is an open-access article distributed under the terms of the Creative Commons Attribution-NonCommercial 4.0 International License (http://creativecommons.org/licenses/by-nc/4.0/) which permits copy and redistribute the material just in noncommercial usages, provided the original work is properly cited.

1. Background

Hospitals are one of the most important facilities providing healthcare services all over the world (1). The quality and cost-effectiveness of healthcare delivered are major issues to be continuously improved in order to have higher patient satisfaction (2). This requires up-to-date, accurate, and timely information supplied by information systems implemented in hospitals for clinical and administrative decision making (3, 4).

The hospital information system (HIS) is a computerized system that has been designed to manage information and provide supportive services for performing hospital activities (5). The various researches indicate that the implementation of HIS has a wide range of benefits for hospitals. HIS can be useful and effective in improving the quality of patient care services and increasing hospital staff efficiency. Also, it can reduce treatment costs and medical errors (6-8).

There are three main aspects of the success of information systems: Information quality, system quality, and service quality (9). The earlier studies conducted on the evaluation of HIS have focused on information and system qualities (10-12). Given the impact of HIS services quality on its success, it is essential that further research should be performed on HIS service quality assessment (9, 13).

The findings of previous studies have shown that HIS users are the customers of system, service, and information. Hence, the quality of HIS services should satisfy users’ expectations and guarantee usage continuity (13-15). The vendors and developers of ISs should provide appropriate support services to ensure that users effectively and satisfactorily use their products (16-18).

Assessment of HIS services quality can be a requirement to meet the needs of the users and a strategy to expand the interaction between HIS developers and the users (14, 19, 20). It is imperative that HIS is continuously evaluated to reach such high-caliber services (15). Accordingly, a validated instrument is required to measure HIS service quality (6, 7, 17, 18).

A widely applied method of assessing service quality is the SERVQUAL or gap analysis model. This model measures the service quality as the gap between a customer’s expectations of service and the customer’s perceptions of the service delivered (21, 22).

The SERVQUAL tool was used for the first time through modification of the items in the specific context of information system service quality measurement (20). Over the past few years, SERVQUAL has been widely accepted and used as a valid instrument for measuring IS service quality such as the quality of website, software, and e-banking services (22-24). Studies have demonstrated that SERVQUAL can be an effective and appropriate tool for measuring the quality of information system services (20, 22-25).

2. Objectives

The use of HIS is spreading more and more in Iranian hospitals. Given the great cost invested in designing and implementation of HIS, it is necessary to evaluate the quality of the services provided by HIS. The purpose of this study was to assess HIS service quality by the SERVQUAL model in the university-related hospitals of Urmia, Iran.

3. Methods

The current study is a descriptive analysis conducted in 2017. Using a multi-stage sampling method, the study participants were selected randomly. They consisted of 270 users who used HIS for more than one year. Four university-related hospitals were clusters. The selection was done according to the number of the users from the different types of occupations that have been directly associated with HIS in each cluster. Those occupations were: Physicians, nurses, paramedical staffs, health information technology staffs, and administrative staffs.

SERVQUAL questionnaire was used to collect data. This model tries to measure the quality of service by comparing customers’ expectations and their perceptions of actual performance from five dimensions (tangibles, reliability, responsiveness, assurance, and empathy) and 22 items (22). The first part of the questionnaire is dedicated to the respondents’ demographic information such as gender, age, occupation, work experience, computer skills, and use of HIS services. The second part measures the customer expectations concerning a service and the third part is the perception of the actual service delivered. The participants’ responses were rated by a seven-point Likert scale ranging from strongly disagree (one) to strongly agree (seven).

The validity of the modified instrument was verified via valid scientific texts (26, 27) and comments of the experts in the health information management and the medical informatics fields. Internal consistency was used to test the reliability and the values of Cronbach’s alpha were estimated to be 0.83.

The gap between the scores of perceptions and the expectations was simply calculated by subtracting them.

Data analyses were performed using descriptive statistics and analytical statistics using SPSS software.

4. Results

Of 270 surveys distributed, 165 were returned (overall response, 61.1%). The frequency of the female respondents were (59.4%) and their mean age and work experience were 37.3 and 11.4 years, respectively.

Most of the respondents were nurses (35.2%). The majority of HIS users (34.6%) had work experience between five and ten years. Most of them affirmed that their computer skills were intermediate (53.4%) and have used moderately (55.7%) HIS services (Table 1).

Table 1. Demographic Characteristics of HIS Users (N = 165)
CharacteristicsNo. (%)
Gender
Male67 (40.6)
Female98 (59.4)
Age, y
< 3031 (18.8)
30 - 4063 (38.2)
40 - 5048 (29.1)
≥ 5023 (13. 9)
Occupation
Physician35 (21.2)
Nurse58 (35.2)
Health information management25 (15.1)
Paramedical staff20 (12.1)
Administrative staff27 (16.4)
Work experience
< 523 (13.9)
5 - 1057 (34.6)
10 - 1549 (29.7)
≥ 1536 (21.8)
Computer skills
Fundamental41 (24.8)
Intermediate88 (53.4)
Advanced36 (21.8)
Use of HIS services
Low28 (17)
Medium92 (55.7)
High45 (27.3)

The mean scores of expectations in all of the items were higher than perceptions. The highest gap (-1.84) between users’ perception and expectation was an item that belonged to the reliability dimension referred to provide accurate and error-free services from HIS vendors. The lowest gap (-0.69) was modern IT equipment and up-to-date software from the tangibles dimension (Table 2).

Table 2. Mean Scores of Expectation, Perception, and Quality Gap of Services Provided by HISa
Items of Quality DimensionsExpectationPerceptionGap
Tangibles
Modern IT equipment and up-to-date software5.84 ± 1.355.15 ± 1.42-0.69 ± 1.38
User-friendly and visually attractive HIS6.17 ± 1.125.05 ± 1.45-1.12 ± 1.29
Understandable HIS reports6.26 ± 1.055.18 ± 1.40-1.08 ± 1.22
Easy-to-use HIS6.32 ± 0.775.44 ± 1.23-0.88 ± 1.32
Reliability
Providing services according to HIS vendor commitments6.29 ± 0.835.15 ± 1.46-1.14 ± 1.22
Real interest of HIS vendor to solve users’ problems6.37 ± 0.755.05 ± 1.55-1.32 ± 1.15
Providing accurate and error-free services6.75 ± 0.534.91 ± 1.74-1.84 ± 1.13
Providing HIS services at the promised time6.09 ± 0.865.02 ± 1.52-1.07 ± 1.34
Responsiveness
Maintain and update HIS services6.77 ± 0.615.20 ± 1.33-1.57 ± 1.09
Determining a schedule to do HIS services6.23 ± 0.724.91 ± 1.86-1.32 ± 1.35
HIS vendor priority for responding to users’ requests6.58 ± 0.695.13 ± 1.46-1.35 ± 1.33
Providing HIS services quickly and accurately6.86 ± 0.535.25 ± 1.42-1.61 ± 1.05
Availability of IT staff when users required6.69 ± 0.865.05 ± 1.51-1.64 ± 0.97
Assurance
Create confidence of users by IT staff to use HIS services6.37 ± 0.755.05 ± 1.55-1.32 ± 1.14
Comfortable feeling in communication with IT staff6.09 ± 0.865.02 ± 1.52-1.07 ± 1.36
Courteous interaction with HIS users6.29 ± 0.835.15 ± 1.46-1.14 ± 1.25
Suitable knowledge and skill of IT staff6.75 ± 0.534.93 ± 1.84-1.82 ± 0.79
Empathy
Individual attention to users for customizing HIS services6.11 ± 1.085.05 ± 1.55-1.06 ± 1.31
Understand the specific needs of users6.06 ± 1.105.05 ± 1.51-1.11 ± 1.18
The tendency of IT staff for helping willingly to users5.90 ± 1.254.78 ± 1.67-1.12 ± 1.15
The proper attention to resolve users’ problems6.25 ± 1.025.02 ± 1.56-1.23 ± 1.07
Respect for cultural values and user desires5.88 ± 1.234.85 ± 1.62-1.03 ± 1.29

a Values are expressed as mean ± SD.

The highest and lowest quality gaps in all the items of five dimensions were related to responsiveness (-1.52) and tangibles (-0.95), respectively. There were significant differences between the perceptions and expectations of the users in all SERVQUAL dimensions (P < 0.001) (Table 3). These differences are shown by error bars in Figure 1.

Table 3. Paired-Sample t-Test to Compare Users’ Expectations and Perceptions of HIS Service Quality
Quality DimensionsExpectationPerceptionGapTP Value
Tangibles6.15 ± 1.155.20 ± 0.85-0.95 ± 1.30-11.090.001
Reliability6.37 ± 0.835.03 ± 0.96-1.34 ± 1.21-12.960.000
Responsiveness6.62 ± 0.755.10 ± 0.89-1.52 ± 1.16-15.220.000
Assurance6.26 ± 1.045.09 ± 0.93-1.17 ± 1.15-14.130.009
Empathy6.04 ± 1.134.95 ± 1.04-1.09 ± 1.20-11.870.003
The general differences between users’ perception and expectation in all items of five dimensions
Figure 1. The general differences between users’ perception and expectation in all items of five dimensions

5. Discussion

The results of this research showed that there are significant gaps between users’ perception and expectation in all five SERVQUAL dimensions. These gaps are increased from tangibles (-0.95) to responsiveness (-1.52) dimensions in ascending order. The items with maximum gaps in all dimensions were: user-friendly and visually attractive HIS (tangibles), providing accurate and error-free services (reliability), availability of IT staff when they were required by the users (responsiveness), suitable knowledge and skill of IT staff (assurance), and the proper attention to resolve users’ problems (empathy). These items generally reveal that users of HIS are not satisfied with HIS services quality and system characteristics.

Chang et al. (13) found that HIS offers services that are intangible and negatively affect the work performance of the users. This is consistent with our results. Although the gap score of tangibility is less than other dimensions in the present study, the users significantly are dissatisfied with tangibility of HIS. Wibawa et al. (28) applied the SERVQUAL to evaluate a HIS in Indonesia. The system reliability and response time in their study had the highest gap score. These dimensions were the most unsatisfactory among the users of HIS in the study of Chang et al. (13) and the highest gap in a research conducted by Amelia et al. (29) that assessed the quality of IS/IT services provided in higher education.

Among five dimensions of SERVQUAL, responsiveness and reliability, which are chiefly about the human aspects of service delivery (30) and also they are more objective (25), have the highest gap in our study and this confirms previous findings in the literature. The users can evaluate these aspects of HIS better than other dimensions (25, 30). Hence, the vendors should consider more the feedback of HIS users about reliability and responsiveness dimensions.

In the present study, the gap between perception and expectation of “modern IT equipment and up-to-date software” is the least among other items of tangibles. This shows that the infrastructure of the studied HIS is approximately acceptable but are not user-friendly and visually attractive nonetheless.

HIS that is user-friendly and has modern technologies can be used effectively by the users (31). Involving users during the design and implementation of HIS and receiving feedback from them are the key solutions to increase user-friendliness and attractiveness of HIS (32).

The findings of Khalifa and Alswailem (33) revealed that the pharmacists and physicians were the least satisfied with the user-friendliness of HIS. They believed that attractiveness and user-friendliness are the influential factors of technology acceptance.

The users of HIS in our study feel a less gap between their perception and expectation in “providing HIS services at the promised time” than other items in the reliability domain. Providing incorrect services at the promised time is definitely useless. In the first contact, HIS problems in our study are resolved by the hospitals’ IT staff. They might easily be able to solve the problems at the promised time, but most of the time, the problems are not solved and are referred to the vendor that is located in the capital (Tehran).

The results of the present study show that among all items of the responsiveness domain, the users are the least satisfied with the availability of IT staff when they are faced with problems of the system. The staff of hospital’s IT department is responsible for many other tasks in the hospital and many problems are referred to the staff of HIS vendor. These factors have led to the unavailability of IT staff when required by the users.

Assurance domain is generally about knowledge of IT employees and their ability to create trust in HIS users (14). Empathy measures the personal attention and caring received by HIS users from IT staff (31). Both domains are more subjective than other domains (25). In our study, there is a less gap between expectation and perception of users about easy communication with IT staff in the domain of assurance. Nevertheless, the perception of HIS users has the highest gap with their expectation about creating confidence in the users by IT staff to use HIS services.

The hospital employees do not have good familiarity with information systems (13). They should be motivated by IT staff and their confidence kept at a good level for using the benefits of HIS. The substantial difference is observed between expectation and perception of HIS users about the proper attention of IT staff to resolve their problems. The examination of all items of empathy domain shows that the gap scores between expectation and perception are close to each other. This reveals that HIS users state a low deviation that IT staffs do not have enough empathy.

To the best of our knowledge, this study is the first about the assessment of HIS services of Iranian hospitals. However, the modification of SERVQUAL model based on cultures of our country could lead to better results.

5.1. Conclusions

This study showed that in the five domains of quality of HIS services, there was a negative gap between the users’ expectations and perceptions. This implies that the quality of the delivered services was lower than what the users expected. Therefore, HIS vendors should seek to eliminate the gap between the users’ expectations and perceptions.

Given that there were high gaps in responsiveness and reliability dimensions, it is recommended to consider the provision of accurate services, maintenance, and updating of HIS. Obviously, it is critical that IT staff should be available when is required by the users and they should create trust in users for the use of HIS services. Moreover, HIS vendors should allocate IT staffs that have suitable knowledge and skill to improve the responsiveness and assurance.

It is thus recommended that the health information management department of hospitals should have continuous communication with HIS vendors, in order to improve benefits and gain more achievements generated by hospital information systems.

Acknowledgements

Footnotes

References

  • 1.

    Carraher SM, Carraher SC, Enterprises C. Customer service, entrepreneurial orientation, and performance: A study in health care organizations in Hong Kong, Italy, New Zealand, the United Kingdom, and the USA. J Appl Manag Entrep. 2006;11(4):33-48.

  • 2.

    Rahimi B, Safdari R, Jebraeily M. Development of hospital information systems: User participation and factors affecting it. Acta Inform Med. 2014;22(6):398-401. doi: 10.5455/aim.2014.22.398-401. [PubMed: 25684849]. [PubMed Central: PMC4315630].

  • 3.

    Menachemi N, Chukmaitov A, Saunders C, Brooks RG. Hospital quality of care: Does information technology matter? The relationship between information technology adoption and quality of care. Health Care Manage Rev. 2008;33(1):51-9. doi: 10.1097/01.HMR.0000304497.89684.36. [PubMed: 18091444].

  • 4.

    Song M, Spallek H, Polk D, Schleyer T, Wali T. How information systems should support the information needs of general dentists in clinical settings: Suggestions from a qualitative study. BMC Med Inform Decis Mak. 2010;10:7. doi: 10.1186/1472-6947-10-7. [PubMed: 20122272]. [PubMed Central: PMC2843644].

  • 5.

    Kuperman GJ, Gardner RM, Pryor TA. HELP: A dynamic hospital information system. 1st ed. Springer Publishing Company; 1991.

  • 6.

    Bardhan IR, Thouin MF. Health information technology and its impact on the quality and cost of healthcare delivery. Decis Support Syst. 2013;55(2):438-49. doi: 10.1016/j.dss.2012.10.003.

  • 7.

    Goldschmidt PG. HIT and MIS: Implications of health information technology and medical information systems. Commun ACM. 2005;48(10):68-74. doi: 10.1145/1089107.1089141.

  • 8.

    Ovretveit J, Scott T, Rundall TG, Shortell SM, Brommels M. Improving quality through effective implementation of information technology in healthcare. Int J Qual Health Care. 2007;19(5):259-66. doi: 10.1093/intqhc/mzm031. [PubMed: 17717038].

  • 9.

    Safdari R, Ghazisaeidi M, Jebraeily M, Masarat E, Shikhtayefeh M, Farajolahi S. Hospital information systems success: A study based on the model adjusted DeLone and McLean in UMSU hospitals. Eur J Exp Biol. 2014;4(5):37-41.

  • 10.

    Kimiafar K, Moradi GR, Sadoughi F, Sarbaz M. [Views of users towards the quality of hospital information system in training hospitals affiliated to Mashhad University of Medical Sciences]. Health Inf Manage J. 2007;4(1):43-50. Persian.

  • 11.

    Petter S, DeLone W, McLean E. Measuring information systems success: Models, dimensions, measures, and interrelationships. Eur J Inf Syst. 2017;17(3):236-63. doi: 10.1057/ejis.2008.15.

  • 12.

    Yusof MM, Kuljis J, Papazafeiropoulou A, Stergioulas LK. An evaluation framework for Health Information Systems: Human, organization and technology-fit factors (HOT-fit). Int J Med Inform. 2008;77(6):386-98. doi: 10.1016/j.ijmedinf.2007.08.011. [PubMed: 17964851].

  • 13.

    Chang CS, Chen SY, Lan YT. Motivating medical information system performance by system quality, service quality, and job satisfaction for evidence-based practice. BMC Med Inform Decis Mak. 2012;12:135. doi: 10.1186/1472-6947-12-135. [PubMed: 23171394]. [PubMed Central: PMC3538068].

  • 14.

    Gorla N. An assessment of information systems service quality using SERVQUAL+. ACM SIGMIS Database. 2011;42(3):46-70. doi: 10.1145/2038056.2038060.

  • 15.

    Zheng Y, Zhao K, Stylianou A. The impacts of information quality and system quality on users' continuance intention in information-exchange virtual communities: An empirical investigation. Decision Support Systems. 2013;56:513-24. doi: 10.1016/j.dss.2012.11.008.

  • 16.

    Aggelidis VP, Chatzoglou PD. Hospital information systems: Measuring end user computing satisfaction (EUCS). J Biomed Inform. 2012;45(3):566-79. doi: 10.1016/j.jbi.2012.02.009. [PubMed: 22426283].

  • 17.

    Kettinger WJ, Park SHS, Smith J. Understanding the consequences of information systems service quality on IS service reuse. Inf Manage. 2009;46(6):335-41. doi: 10.1016/j.im.2009.03.004.

  • 18.

    Kim DR, Chen MJ, Aiken M. Towards an understanding of the relationship between IS outsourcing vendors' service quality and outsourcing effects. Int J Inf Tech Manage. 2005;4(1):12. doi: 10.1504/ijitm.2005.006402.

  • 19.

    Jiang JJ, Klein G, Parolia N, Li Y. An analysis of three SERVQUAL variations in measuring information system service quality. Electron J Inf Syst Evaluat Volume. 2012;15(2):149-62.

  • 20.

    Jiang JJ, Klein G, Carr CL. Measuring information system service quality: SERVQUAL from the other side. MIS Q. 2002;26(2):145-66. doi: 10.2307/4132324.

  • 21.

    Zeithaml VA, Parasuraman A, Malhotra A. Service quality delivery through web sites: A critical review of extant knowledge. J Acad Mark Sci. 2002;30(4):362-75. doi: 10.1177/009207002236911.

  • 22.

    Han SL, Baek S. Antecedents and consequences of service quality in online banking: An application of the SERVQUAL instrument. Adv Consum Res. 2004;31:208-14.

  • 23.

    Lee GG, Lin HF. Customer perceptions of e‐service quality in online shopping. Int J Retail Distrib Manage. 2005;33(2):161-76. doi: 10.1108/09590550510581485.

  • 24.

    Kuo T, Lu IY, Huang CH, Wu GC. Measuring users' perceived portal service quality: An empirical study. Total Qual Manag Bus. 2006;16(3):309-20. doi: 10.1080/14783360500053824.

  • 25.

    Landrum HT, Prybutok VR, Zhang X, Peak DA. Measuring IS system service quality with SERVQUAL: Users' perceptions of relative importance of the five SERVPERF dimensions. Int J Emerg Transdiscipl. 2009;12:17-35. doi: 10.28945/426.

  • 26.

    Parasuraman A, Zeithaml VA, Berry LL. Servqual: A multiple-item scale for measuring consumer perceptions of service quality. J Retailing. 2005;7(3):12-40.

  • 27.

    Parasuraman A, Zeithaml VA, Berry LL. A conceptual model of service quality and its implications for future research. J Marketing. 1985;49(4):41. doi: 10.2307/1251430.

  • 28.

    Wibawa J, Widjaja HAE, Hidayanto AN. Integrating IS success model, SERVQUAL and Kano model into QFD to improve hospital information system quality. International Conference on Information Management and Technology (ICIMTech). 16-18 Nov. 2016; Bandung, West Java, Indonesia. IEEE; 2016. p. 29-34.

  • 29.

    Amelia L, Hidayanto AN, Hapsari IC. Analysis of IS/IT service quality in the higher education with SERVQUAL: A case study of STMIK MDP Palembang. The 2nd International Research Symposium in Service Management. Jul 26-Jul 30; Yogyakarta, Yogyakarta Special Region (DIY), Indonesia. 2011.

  • 30.

    Saraei S, Amini AM. A study of service quality in rural ICT renters of Iran by SERVQUAL. Telecommun Policy. 2012;36(7):571-8. doi: 10.1016/j.telpol.2012.03.002.

  • 31.

    Gorla N, Somers TM, Wong B. Organizational impact of system quality, information quality, and service quality. J Strategic Inf Syst. 2010;19(3):207-28. doi: 10.1016/j.jsis.2010.05.001.

  • 32.

    Handayani PW, Hidayanto AN, Pinem AA, Hapsari IC, Sandhyaduhita PI, Budi I. Acceptance model of a hospital information system. Int J Med Inform. 2017;99:11-28. doi: 10.1016/j.ijmedinf.2016.12.004. [PubMed: 28118918].

  • 33.

    Khalifa M, Alswailem O. Hospital information systems (HIS) acceptance and satisfaction: A case study of a tertiary care hospital. Procedia Comput Sci. 2015;63:198-204. doi: 10.1016/j.procs.2015.08.334.

  • COMMENTS

    LEAVE A COMMENT HERE: