Patient satisfaction with anaesthesia care - Development, pilot testing and validation of a survey questionnaire

Introduction

For several decades anaesthesia practice has been refined with focus on improving patient safety. Discussions on morbidity and mortality have aided this by fostering positive attitudes and ongoing physician education. The practice of medicine has changed over past 20 years, and healthcare providers now face a variety of new challenges posed by the emergence of medical insurance firms, rising patient awareness, easy access to information online, raised patient expectations, and, ultimately, legal action for unsatisfactory outcomes.

Correspondingly, our approach also needs to expand beyond the analysis of adverse outcomes to an analysis of the “Quality of care” provided, as it is the single most important metric of patient satisfaction.^[1] In developed countries, information from surveys is used to benchmark hospitals, inform customers, accredit health plans, and influence new payment techniques, in addition to encouraging quality improvement among healthcare personnel and ensuring accountability.^[1] A similar strategy will also be deployed in emerging nations.

There are a few validated questionnaires addressing the quality of peri-operative anaesthesia care. However, they either focus on a certain patient subgroup (obstetrics, ICU care, pre-operative clinic, ambulatory anaesthesia) or are less appropriate for Indian subcontinent.^[2], ^[3], ^[4], ^[5] Moreover, the entire gamut of anaesthesia care is not addressed. In the present study, our aim was to develop and validate a questionnaire for the assessment of patient satisfaction with anaesthesia care.

Materials and Methods

Approval from Institutional Human Ethics Committee (MGMCRI/Res/01/2020/08/IHEC/265) was obtained for development and validation of the questionnaire; and its utilisation for conducting the survey. The study was registered with CTRI (CTRI/2022/01/039359) and conducted as per the principles laid down in Declaration of Helsinki, 2013. The model for questionnaire development described by Tsang et al. and Alsaif et al. was used as a guideline in the development process.^[6], ^[7]

We describe the stepwise methodology followed in the development of the questionnaire. To generate items and create questions, we looked at the current literature, gathered ideas from expert anaesthesiologists, and interviewed patients. This helped us to establish the factors that influence patient satisfaction with anaesthesia care. A thorough review of the literature was done and questionnaires like LPPSq, Iowa scale, etc were referred.^[2], ^[4] A brainstorming session was conducted with twenty anaesthesiologists of all cadre from senior residents to professors at our institute and their input was gathered. Input from patients was obtained through Focussed Group Discussion (FGD) with post-operative patients before their discharge. Each group had five patients, and the discussion was conducted by one of the authors in a pre-determined role as facilitator. Over the course of two months, twenty such FGDs (13 groups for male patients and 7 groups for female patients) were conducted. Additionally, two subjects who underwent gender reassignment surgery were individually interviewed.

This led to the formation of 36 preliminary questions which could be grouped under six categories (dimensions): Anaesthetist-patient communication, adequacy of information provided, involvement of the patient in decision making, compassionate care, continuity of care through the peri-operative period and addressal of peri-operative discomforts. A few items pertaining to participant demographics, background, and overall satisfaction were added to the questionnaire that was prepared. It was then enlisted in the order that involved routine perioperative anaesthesia care for elective surgery. During this process, questions that addressed first three categories (anaesthetist-patient communication, adequacy of information provided, and involvement of patient in decision-making) were clubbed.

Each item in questionnaire was generated to record the participant’s response to only one issue. All patient satisfaction-related responses were graded using a 5-point Likert scale. We assigned each question the same weight since we believed that all the dimensions were equally significant, and a higher score meant that the answer was positive. At the end of survey, three additional questions were added; one asking respondents to rate their overall anaesthesia experience on a scale of 0 to 10, one asking them whether they would recommend this anaesthesia service to family and friends, and one asking them to specify which aspect would increase their satisfaction.

The questionnaire was reviewed by six experts, of which, three were senior anaesthesiologists, two were community medicine experts and a statistician. Based on their suggestions, questions were changed to be brief and straightforward, redundant questions were eliminated, and reverse item scoring was fixed. We ended up with 24 questions ([Table 1], [Table 2]).

Questionnaire translation was done by two separate bilingual translators who carried out forward translation from source language (English) to target language (Tamil) and the reverse translation. The lack of discrepancies between translated questions with original questions was established and the readability level of questions was lowered so that local population could easily understand them.

Pre-pilot testing to determine feasibility, practicality of questions and presence of ‘floor or ceiling response’, was performed on twenty patients. In the opinion of participants, majority of questions were clear, and time taken for completing it was reasonable. During pre-pilot testing, we identified that participants could not interpret question number 9 and 10 and missing values were noted. Hence, explanations for these questions were added in parenthesis. Responses to open-ended question at the end of survey were analysed. A few patients brought out “thirst” as a reason for peri-operative discomfort, thus it was added to question number 18 ([Table 1], [Table 2]).

A pilot study (Preliminary Questionnaire testing) was carried out over a course of one month. It was designed as a self-administered questionnaire. Both English and Tamil hard copies were made available. Patients who were unable to express their opinions owing to mental retardation, psychiatric illness, inability to read and write in Tamil or English, or those who were shifted to ICU or HDU on ventilator support after surgery were excluded. All patients who were willing to participate were asked for their written informed consent after being explained the purpose of the study and assuring them that their responses would be kept anonymous. The forms were given to patients on postoperative day (POD) 1 and they were asked to fill them out either on their own or with assistance from their family and friends and hand it over to the post-operative ward nurse. If not filled by POD 1, they got one more reminder the following day. Response obtained from preliminary pilot study were entered into an excel spreadsheet. Statistical validation for internal consistency (reliability) was done by estimating Cronbach’s alpha using SPSS 16.0 statistical software.

Results

36 preliminaries questions were reduced to 24 after expert evaluation to increase response and completion rates. Eighty-five patients were included in the pilot study. We received 60 responses, which amounted to a response rate of 70%. The demographic data of the pilot study population is shown in ([Table 3]).

The Tamil version of questionnaire was used by 93% of patients. We received 50% of responses on POD 1 and remaining 50% on POD 2. Sufficient variance in participant’s response was confirmed.

Twenty questions related to patient satisfaction which were graded using a 5-point Likert scale were subjected to internal consistency testing (Cronbach’s alpha) using SPSS 16.0. The Cronbach’s alpha was estimated as 0.697. The test also ruled out any negative correlation between items in the questionnaire. The mean score (scored from 0 to 100) was 87.29 ± 4.65, indicating that our questionnaire reliably measured patient satisfaction.

Discussion

Most of us as working doctors are unaware of or unable to commit to the hard and time-consuming process of developing questionnaires. We developed a questionnaire for assessing patient satisfaction with anaesthesia care for Indian subcontinent in English and Tamil languages.

The domain of interest for construction of questionnaires can be determined by direct observations, expert judgement, content analysis, review of research or critical incidents.^[2], ^[6] The degree to which patient expectations and achievements match determines how satisfied they are, and knowing this enhances content validity.^[8] Patient’s expectations were acquired through focused group discussions and one-on-one interviews using open-ended questions.^[9] No queries related to participant’s privacy were raised.

We did not use the questionnaire developed by Ambulkar and colleagues as majority of their questions were bipolar and were not statistically validated.^[10] Corollaries based on surrogate endpoints like nausea, vomiting and postoperative pain were avoided.^[11], ^[12] An overall satisfaction summary score for patient satisfaction assessment was included in addition to the questions and not as a substitute as it can result in a misleading representation if used as a sole question.^[13] Multivariate statistical analysis and identification of individual dimensions with highest weightage (beta weight) can determine what contributes to satisfaction.^[14] Hence, in our questionnaire, all patient satisfaction-related responses were graded using a 5-point Likert scale so that statistical validation could be done. To confirm consistency in patients' opinions and determine the aspects that contributed to satisfaction, questions pertaining to overall satisfaction and one question rating the anaesthesia care on a 0–10 scale were added. However, in our pilot study, we did not receive any feedback for the open-ended question (question number 24).

Leiden (LPPSq) questionnaire was developed in Dutch, the Heidelberg in German language, and EVAN-G in French.^[2], ^[3], ^[5] LPPSq questionnaire was developed to measure patient experience with entire perioperative care rather than their satisfaction; of which, anaesthesia care was only an element. Ours is the first questionnaire suitable for the Indian subcontinent.

To reduce selection bias, interviewer bias, social desirability bias, and to ensure acceptable divergent validity, a self-administered questionnaire was used for pilot testing. However, this carries the disadvantage of low response rates. Participant’s response rate in our pilot study was comparable to that reported in other studies where self-administration format was used.^[2], ^[6] For an upcoming study on patient satisfaction with anesthesia care utilising this questionnaire, we will employ Google Forms for data collection, as we aim to gather responses from a broad demographic.

Barnett and colleagues in their systematic review found that most anesthesia-related studies did not employ validated techniques resulting in possibility of bias thereby yielding unreliable and meaningless results.^[14], ^[15] In the present study, we estimated the Cronbach’s alpha; where a value of 0.61 to 0.80 represents substantial correlation and 0.81 to 1.00, a good correlation. Internal consistency testing using Cronbach’s alpha showed that the score could be increased to ≥ 0.7 from the mean value of 0.697 if question number 9, 11 and 16 were deleted ([Table 4]). We choose to retain the three questions as they were important and the improvement in Cronbach’s alpha was only minimal even if deleted. No ceiling or floor responses were noted in the pilot study.^[8]

Inter-rater testing for reliability must be used when questionnaires are filled by multiple observers. The calculation of inter-rater reliability or Kappa coefficient testing was not necessary in our study because our questionnaire was self-administered. Several other tests have also been used to validate questionnaires. Item discriminant validity and inter-item correlation were used by Caljouw and colleagues to measure reliability.^[2] Tsang et al have described conducting test-retest reliability where participant’s consistency in response across repeated questionnaire administration was evaluated.^[6] Dongare et al have calculated item-wise content validity index, and scale-wise content validity index.^[16] Mui and colleagues have determined content validity coefficient and homogeneity reliability coefficient by conducting Bardett’s test of sphericity and chi-square test for Exploratory Factor analysis on pilot questionnaire and Confirmatory factor analysis applied to final version of questionnaire respectively.^[17] Regular audits using validated questionnaires are an easy tool that can help us identify the areas of care amenable to improvisation and thereby enhance the quality of anesthesia services provided.

The limitation of our study was that the response rate in our pilot study was only 70%. Although this falls within the range described in various previous satisfaction surveys, a response rate >80% is desirable to reflect the exact facts.^[18]

Conclusion

Ours is the first validated questionnaire for assessment of patient satisfaction with anaesthesia care suitable for the Indian population. The questionnaire can further be translated into appropriate regional languages and utilized.

Sources of Funding

None.

Conflict of Interest

The authors declare no conflicts of interest.