Original Articles

Assessment of learning environment in anesthesia by using ATEEM tool

Huma Zeb1, Abdul Rehman2, Abid Ullah Khan Niazi3

1Medical Officer,2 Consultant Anesthesiologist

Quaid-e-Azam International Hospital, Islamabad (Pakistan)

Correspondence: Dr. Abdul Rehman, Consultant Anesthesiologist, Quaid-e-Azam International Hospital, Islamabad (Pakistan); E-mail: rehman222@hotmail.com; Phone: +92 51 8449100

ABSTRACT

Objectives: To assess the learning environment for anesthesia trainees and compare the results between two leading public and private hospitals of Islamabad region
Study Design: A prospective descriptive study involving postgraduate anesthesia trainees.
Duration: January 2016 to February 2016
Setting: Anesthesia Department of a leading public and private hospital, Islamabad (Pakistan)
Methodology: Anesthesia Theater Education Environment Measure (ATEEM) questionnaire was administered to trainees of public and private teaching hospital after taking informed consent. ATEEM total score was calculated using descriptive statistics while comparisons were made using Student’s t-test.
Results: Thirty trainees responded each from one public and one private hospital. The mean total ATEEM score was 117.16/160. Mean score of trainees from private hospital was higher in all the domains of ATEEM. A total of 18 variables out of total 40 resulted in statistically significant results (p value < 0.05).
Conclusion: The learning environment of anesthesia trainees is positively perceived by all the participants. However, the ATEEM scores in four of the five domains showed that the trainees’ perception of their educational environment varies profoundly between a public and the private hospital under study. There were no differences on the basis of gender and the scores are similar to international studies.
Key words: Anesthesia Theater Education Environment Measure; ATEEM; Learning Environment; Anesthesiology residents; Operating room
Citation: Zeb H, Rehman A, Niazi AUK. Assessment of learning environment in anesthesia by using ATEEM tool. Anaesth Pain & Intensive Care 2017;21(3):354-359
Received – 23 Dec 2016; Reviewed – 30 Mar, 29 Jul 2017; Corrected – 27 Dec 2016, 4 Aug 2017; Accepted – 20 Aug 2017

INTRODUCTION

Educational system in medical institutes requires monitoring of its quality, outcome and determination of how the students value and perceive the educational environment. Evaluation of an educational environment can be done by the accomplishment or failure of a program. There are certain characteristics of any training environment that support learning whiles other features may put a stop to learning. Such factors must be acknowledged and assessed to promote learning and to achieve the required goals and objectives.1,2
The learning environment of a teaching hospital consists of three parts: the physical/substantial (mainly safety, food and comfort); affective (personal support, the prevention of bullying and harassment); and the intellectual (learning with patients, motivation and structured education). A positive environment motivates students to achieve their goals and objectives. On the other hand, a negative environment leads to a hindrance in the accomplishment of those goals.3,4,5,6

The ATEEM (Anesthesia Theater Education Environment Measure) [Appendix I]

Appendix I: Anesthetic Theatre Education Environment Measure (ATEEM) Questionnaire

Gender            Male:  Female:

What grade are you?   PG 1   PG 2  PG 3 PG 4

Please indicate whether you:

Strongly Agree (SA), Agree (A), Unsure (U), Disagree (D), Strongly Disagree (SD) with the following statements as it applies to your own feelings about theatre teaching in this hospital. Circle the appropriate response.

  1. There are opportunities for learning all desired clinical skills
  2. The teaching helps to develop my confidence
  3. I receive effective supervision from the clinical teachers
  4. Surgeons do not like the noise of theatre teaching*
  5. Teaching is done at appropriate time not affecting vigilance
  6. I receive theatre teaching in anesthetic specialty areas targeted at my learning needs
  7. The teacher helps to develop my competence
  8. My clinical teachers are accessible for advice
  9. I experience friendly relations with my teachers in theatre
  10. I am aware of my anesthetic role in theatre
  11. I have opportunities to learn and practice a variety of clinical procedures
  12. The clinical teachers in this hospital interact well with trainees
  13. There is an informative anesthetic trainee handbook
  14. The people I work with are friendly
  15. I feel responsible and accountable for the care given to my patients
  16. I am able to acquire adequate technical skills in this post
  17. My clinical teachers are fair in their evaluations
  18. At this hospital I have access to help from more experienced colleagues
  19. My clinical teachers promote an atmosphere of mutual respect
  20. I have an appropriate level of clinical responsibility
  21. There are good opportunities for trainees who fail to complete their training satisfactorily
  22. My clinical teachers are clear in their teaching
  23. Whenever I should participate in formal educational programs I get relief from theatre duties
  24. There is sex discrimination in this post*
  25. I am clear about the learning objectives of the theatre teaching session
  26. There is a clinical training program here that allows me to get first-hand experience in a range of procedures
  27. I receive the necessary clinical supervision
  28. I feel part of a team working here
  29. I discuss the anesthetic plan of cases with the theatre teacher
  30. I have the opportunity to acquire the appropriate practical procedures for my level of training (e.g. fibreoptic intubation/subtenons
    nerve block)
  31. My workload in this job is fine
  32. I have good collaboration with theatre staff
  33. I am encouraged to visit patients pre-operatively
  34. I have the opportunity for on the job learning
  35. My clinical teachers have established good rapport with me
  36. I am encouraged to participate in the theatre setting
  37. There is a systematic clinical training program
  38. I feel able to ask the questions I want
  39. Much of what I learn seems relevant to my career
  40. I feel comfortable in theatre sociallyNote: * Reversed score items.

is a specific instrument developed to objectively measure the educational environment for anesthetic theatre teaching.7 The questionnaire consists of 40 items that cover five domains including autonomy, perception of atmosphere, workload / supervision / support, perception of teachers and teaching, learning opportunities and orientation to learning. It can be used as a simple, prompt and a cost effective way of assessing and contrasting learning environments. ATEEM can serve as a valuable instrument in the judgment of teaching and learning. It can encourage early identification of the problems and ensure sufficient preparation and training of teachers to assist the learners to pursue their objectives.7

There are some other similar instruments like Dundee Ready Educational Environment Measure (DREEM),8 Surgical Theater Educational Environment Measure (STEEM)9 and Postgraduate Hospital Educational Environment Measure (PHEEM)10,11,12 that particularly assess the quality of educational atmosphere in hospital settings other than anesthesia.

All evaluations of learning environment of anesthesia training are being done subjectively and so far no objective assessment has been done. Consequently, the rationale of this study is to objectively compare the postgraduate learning environment of one leading private and one public teaching hospitals of Islamabad.

 METHODOLOGY

This study was conducted at anesthesia departments of a public and a private hospital in Islamabad after getting approval from hospital ethics committees and informed consent of the postgraduate trainees. Shifa International Hospital, Islamabad is a leading private hospital of Islamabad and was selected to represent private hospital setting. Pakistan Institute of Medical Sciences (PIMS) Islamabad was selected as public hospital for this study since it is a leading public hospital in this region. Convenience sampling was used. Postgraduate trainees of fellowship program (FCPS) in anesthesiology were part of the study. To ensure the confidentiality of responses the names of these institutions have been removed from the study and are identified as institution A (Public) and institution B (Private). Those trainees not willing to participate were excluded from the study. All trainees, with a training period of less than 6 months in that particular hospital, were also excluded from the study.

Printed copies of the ATEEM questionnaire with consent forms were sent to two major institutions of Islamabad in January 2016 and returned during the same month. These were distributed to the trainees during a face to face session and ATEEM questionnaire was explained. All anesthesia training residents of both genders were included in this study. The questionnaire consisted of a variety of matters directly related to the learning environment. The participants were asked to show their agreement by means of a 5 point scale. Completed questionnaire was collected later. Incomplete questionnaires were excluded from the study.

Statistical analysis: All the data were assessed in Statistical Package for Social Science (SPSS version 19). Descriptive statistics for the total score of the instrument as well as for each of the five domains were calculated. Differences among domains on the basis of institution (public and private) and gender were calculated using Student’s t test. Where differences were observed which was statistically significant, effect size was calculated to see the magnitude of the difference. Effect size (Cohen’s d) is the magnitude of the effect. It can be interpreted as small (0.2), moderate (0.5) and large (0.8).  Effect size is the difference between two means (e.g., treatment minus control) divided by the standard deviation of the two groups.

RESULTS

There were sixty postgraduate trainees who participated in the study and returned the questionnaire. Of these, thirty were each from the public and private institutions. Among the participants, 58% of the respondents were male. Figure 1 shows distribution of participants in each of the four years of postgraduate training.

Figure 1: Distribution of participants in four years of postgraduate training
Data

3Assessment of learning environment - Fig-1
The ATEEM questionnaire consisted of 40 items with a maximum score of 160. There are five domains in the questionnaire with an internal consistency of 0.81, which was measured using Cronbach’s alpha. Nunally considered Cronbach’s alpha of 0.7 or more as a reliable estimate of the internal consistency of an instrument.14 The mean total ATEEM score was 117.16 out of a possible maximum score of 160.

Table 2 presents the total mean scores that show a significant difference between the total scores of private and public institutions and a p value of 0.000. Cohen’s Effect size suggests a moderate practical difference.

Table 1: Mean differences of each domain on the basis of Institution

 

Domain Public Sector

Mean Score

 Private  Sector

Mean Score

 p value
Autonomy 24.5 26.3 0.20
Perceptions of atmosphere 26.0 30.0 0.00
Workload/Supervision/Support 17.7 23.0 0.00
Perception of Teachers and Teaching 13.4 17.7 0.11
Learning opportunities and orientation to learning 27.7 32.1 0.00
Total 109 (17.6) 129 (16.8) 0.000

Table 1 shows the scores of each domain and a comparison is made between both institutions. ATEEM Score of private score was much better than public hospital in nearly all variables. It can be seen that three domains have a significant p value (< 0.05).

Table 2 shows 18 items out of 40 that have a statistically significant p value.

Table 2: Mean differences for each item on the basis of institute

S. No. Variable Mean Score

(Private)

 Mean Score (Public) p value
1 There are opportunities for learning all desired clinical skills 3.5 2.7 0.000
2 The teaching helps to develop my confidence 3.4 2.7 0.002
3 I receive effective supervision from the clinical teachers 3.3 2.4 0.002
4 I am clear about the learning objectives of the theatre teaching session.  

3.2

  

2.5

 0.001
6 I receive theater teaching in anesthetic specialty areas targeted at my learning needs  

3.2

  

2.6

 0.004
12 Clinical teachers in this hospital interact well with the trainees 3.2 2.7 0.04
17 My clinical teachers are fair in their evaluations 4.5 2.7 0.03
18 At this hospital I have access to help from more experienced colleagues  

3.3

  

2.7

 0.04
19 My workload in this job is fine 3.0 2.1 0.01
22 My clinical teachers are clear in their teaching 3.2 2.4 0.01
25 I am clear about the learning objectives of the theater teaching session  

3.2

  

2.5

 0.001
26 There is a clinical training program here that allows me to get first-hand experience in a range of procedures 3.2 2.6 0.02
27 I receive the necessary clinical supervision 3.3 2.5 0.003
32 I have good collaboration with theater staff 3.3 2.8 0.02
35 My clinical teachers have established good rapport with me 3.3 3.0 0.01
37 There is a systematic clinical training program 3.2 2.6 0.003
38 I feel able to ask the questions I want 3.2 2.6 0.02
39  Much of what I learn seems relevant to my career 3.3 2.7 0.006

None of the domains showed any difference on the basis of gender that was statistically significant. When study was compared to pilot study done previously; scores were found comparable7 (Table 3).

Table 3: Comparison with International study

Subscales Current Study

Mean Scores

 Pilot Study in UK

Mean Scores
Autonomy 25.1 24
Perceptions of atmosphere 27.5 31
Workload/Supervision/Support 20.1 20
Perception of Teachers and Teaching 15.3 15
Learning opportunities and orientation to learning 29.1 27


DISCUSSION

The American Medical Association (AMA) defines the learning environment as “A social system that includes the learner (including the external relationships and other factors affecting the learner), the individuals with whom the learner interacts, the setting(s) and purpose(s) of the interaction, and the formal and informal rules/policies/norms governing the interaction”.2

The educational environment; variously referred to as climate, atmosphere, ambiance or tone; is a set of factors that gives each situation a personality, a spirit, a culture.7 The learning environment (LE) includes the instructive, substantial, societal, and emotional context in which trainees are involved, and plays a very important role in their progress and improvement.

Most of the anesthesia syllabus is taught in the theater and its atmosphere has not been investigated earlier in Pakistan objectively. Problems can be highlighted, thus, paving the path for improvement. A number of questions emerge regarding the educational climate whether the teaching is very strict and dictatorial. Is the atmosphere relaxed or is it stressful and frightening for the trainees to learn? These are all key questions in determining the nature of the learning experience.14 It is imperative to know that there are certain factors that make the atmosphere of a learning place comfortable while certain other features would make it unpleasant.15

This study shows that learning environment of private hospital was more positively perceived by the trainees as compared to the public hospital. Trainees in private hospital perceive themselves as more confident than those being trained in a public hospital. These differences can be attributed to a number of reasons. A big private hospital in Islamabad region is expected to be well equipped with all basic and advanced facilities and the environment to be more supportive of teaching and learning. Supervisors provide more time and are concerned with the learning of their students in private hospital. Similarly, clarity of teaching and perception of clear learning objectives regarding each teaching session is more pronounced among private hospital trainees. There is good collaboration with theater staff and the training program, which allows the trainees of private hospital to learn skills in a variety of procedures. Since private hospital is quite expensive and out of reach of the common population, the flow of patients is much less. Therefore, reduced workload is perceived by the trainees. It may be the main factor of a high confidence level of trainees due to effective supervision and better interaction of supervisors with the trainees.

On a different note, it is quite possible that the government hospital lacks those facilities and one to one interaction of teachers and students is not in practice. The inflow of patients in public hospital is more, thereby more workload is perceived by the trainees. Learning objectives of each session are not clearly defined and teaching is not directed towards their learning needs. Lack of confidence can be due to decreased collaboration and interaction of trainees with their supervisors. This study showed results that are almost identical when compared with an International study of ATEEM.7

There was no local or regional study done in similar manner to assess anesthesia training so no comparisons could be done. While study was compared to pilot study done previously; scores were found comparable7 (Table 4).

LIMITATIONS

This was a limited study with a small sample size and only two hospitals were evaluated and compared. A larger study with a number of public and private hospitals would show true picture of our anesthesia training.

CONCLUSION

To conclude, postgraduate trainees at a public and private hospital perceive the learning environment positively. However, the ATEEM scores in four of the five domains showed that the trainees’ perception of their educational environment varies profoundly between a public and the private hospital under study. There were no differences on the basis of gender and the scores are similar to international studies.

Conflict of interest: None declared by the authors
Author contribution:

HZ – Concept, conducted the study, literature search

AR – compiled the results, manuscript editing

AUKN – Statistical analysis, manuscript review

REFERENCES

  1. Hoff TJ, Pohl, Bartfield J. Creating a learning environment to produce competent residents: the roles of culture and context. Acad Med. 2004;79:532-539. [PubMed]
  2. Educational Environment. www.gmc-uk.org (accessed 22 June 2016).
  3. Colbert-Getz JM, Kim Sooyoun, Goode VH, Shochet RB, Wright SM. Assessing medical students’ and residents’ perceptions of the learning environment: exploring validity evidence for the interpretation of scores from existing tools. Acad Med. 2014. [PubMed] doi: 10.1097/ACM.0000000000000433.
  4. Genn J.M. AMEE Medical Education Guide No. 23 (Part 2): Curriculum, environment, climate, quality and change in medical education—a unifying perspective. Med Teach. 2001;23(5):445–54. [PubMed] DOI: 10.1080/01421590120075661
  5. Roff S, Mcaleer S. What is educational climate? Med Teach. 2001;23(4):333–334. [PubMed] DOI: 1080/01421590120063312
  6. Spencer N. The clinical teaching context: a cause for concern. 2003;37:182-183. [PubMed] DOI: 10.1046/j.1365-2923.2003.01460.x
  7. Holt M and Roff S. Development and validation of the anesthetic theater educational measure (ATEEM). Med Teach. 2004;26:553-558. [PubMed] DOI: 10.1080/01421590410001711599
  8. Miles S, Swift L, Leinster SJ. The Dundee Ready Education Environment Measure (DREEM): A review of its adoption and use. Med Teach. 2012;34(9):e620-34. [PubMed] doi: 10.3109/0142159X.2012.668625.
  9. Nagraj S, Wall D, Jones E. The development and validation of the Mini-surgical theatre educational environment measure. Med Teach. 2007;29:192-197. [PubMed] DOI: 10.1080/01421590701299249
  10. Roff S, Mcaleer RS, Skinner A. Development and validation of an instrument to measure the postgraduate clinical learning and teaching educational environment for hospital-based junior doctors in the UK. Med teach. 2005;27(4):326-31. [PubMed] DOI: 10.1080/01421590500150874
  11. Clapham M, Wall D, Bachelor A. Educationalenvironment in intensive care medicine—use of postgraduate hospital educational environment measure (PHEEM). Med Teach. 2007;29(6):184-191. [PubMed] DOI: 10.1080/01421590701288580
  12. Aspegren K, Bastholt L, Bested K, Bonnesen T, Ejlersen E, Fog I, et al. Validation of the PHEEM instrument in a Danish hospital setting. Med Teach. 2007;29(5):498-500. [PubMed] DOI: 10.1080/01421590701477357
  13. Nunnally, J. C. (1978). Psychometric theory (2nd Ed.). New York: McGraw-Hill.
  14. Jawaid M, Raheel S, Aijaz H. Students’ perception of educational environment at public sector medical university of Pakistan. J Res Med Sci. 2013;18(5):417–421. [PubMed] [Free full text]
  15. Linda Hutchinson. Educational environment. BMJ. 2003;326(7393):810-812 [PubMed] [Free full text] DOI: 10.1136/bmj.326.7393.810