Introduction

A generational cohort consists of people of approximately the same age who share specific sociohistorical experiences and who differ from those lived by groups that preceded or followed¹. The “Generation Y” or “Millennials,” those born between 1979 and 1994², have been stereotyped as self-centered, demotivated, disrespectful, and disloyal. On the other hand, this generational cohort also possesses a unique set of well-identified qualities³. Among the positive attributes of this generation are the tendency to work as a team, optimism, greater civil awareness, a desire for immediate access to technology, and efficiency⁴.

According to the population pyramid of the National Institute of Statistics and Geography (INEGI), the “Millennials” represent around 35% of the Mexican population and many of them are already part of the country’s workforce.

In the current context of postgraduate medical education, there is a coexistence between at least three generations, “Baby Boomers” (1946-1964), “Generation X” (1965-1978), and “Millennials.” The next generation, the “Centennials” or “Generation Z” will be incorporated in the coming years.

Medical specializations are governed by a curricular plan called the Unique Medical Specialization Plan (Plan Único de Especializaciones Médicas [PUEM]), which defines the purposes to which academic action should be directed and the strategies to achieve its objectives. The PUEM contemplates that the student must dedicate 40 h/week for the realization of the academic activities (subjects) that comprise it and a minimum of 15 h/week of individual study. The specialty of neurology is considered to be usually 5 years long (2 years of internal medicine and 3 years of neurology) and the academic load is distributed as follows: health-care work 7280 h (980 credits), medical care seminar 690 h (85 credits), research seminar 460 h (60 credits), and education seminar 230 h (30 credits). That is, 92.5% of the hours are dedicated to medical care, 5% to research, and 2.5% to education. The PUEM proposes an active teaching methodology focused on problem-solving.

The objective of this review is to reinforce some basic concepts of the teaching-learning process and competency-based learning and to suggest some specific teaching and evaluation models and techniques applicable to the field of neurology with particular emphasis on the generation of Millennials.

Bloom’s taxonomy

The development of a curricular program has as its cornerstone the definition of the objectives. Since 1956, Benjamin Bloom and collaborators established a classification for the different objectives and skills that the teacher seeks to convey to his students. This classification of learning domains, better known as Bloom’s taxonomy, is made up of three dimensions: affective, psychomotor, and cognitive. In turn, each of these dimensions or domains has different levels, which are hierarchical in nature, thus defining lower-order and higher-order levels⁵.

The cognitive domain starts from the fact that it is first necessary to learn and remember the concept and then understands it. Once this is done, it is possible to apply this concept in an analysis, to finally evaluate an impact and generate a new product or result. A contribution of this taxonomy is the differentiation in the use of verbs as indicators of the objective.

The original classification has undergone several revisions as a reflection of the evolution and changes in educational theories. The 2001 revision stands out for the change from noun to verb to name the different categories and the inversion of the sequence of the last two higher-order levels⁶.

Figure 1 shows Bloom’s taxonomy and some examples of the verbs suitable for each level assessment. More recently, Bloom’s taxonomy has been integrated with the Substitution, Augmentation, Modification, and Redefinition (SAMR) system; the latter is a model whose purpose is to facilitate the integration of Information and Communication Technologies (ICTs) in education and will be described in greater detail in the corresponding section.

Figure 1 Bloom’s taxonomy and examples of the verbs suitable for each level assessment. 

Miller’s pyramid

The PUEM refers to the Miller pyramid as an integral part. This model is used for the evaluation of clinical competencies. It consists of four levels; two lower levels “Know” (knowledge) and “Know How” (describe) and two higher levels “Show how” (simulation) and “Do.” Based on this categorization, a student can be classified as a novice or as an expert. Miller’s pyramid, although useful, only includes knowledge and operational aspects, leaving aside other characteristics of competencies such as attitude and values. Recently, adjustments have been proposed to cover domains not evaluated⁷. At present, the evidence regarding the usefulness of this model in medical education is limited.

Dreyfus model of skill acquisition

A third approach uses the Dreyfus model of skill acquisition. The Dreyfus model consists of five levels: novice, advanced beginner, competent, proficient, and expert. However, when applying this model to medicine, the first two levels correspond to an undergraduate student, the third level to the resident, the fourth to the newly graduated specialist, and the last to the professional in the middle of his career. Perhaps, one of the most relevant limitations of this model is that the expert level works primarily on intuition and not on reasoning⁸.

Kirkpatrick Model

The Kirkpatrick Model is used specifically for analyzing and evaluating the results of training and educational programs, thus making it suitable for neurology training assessment. It is based on four levels: Level 1 – reaction which measures how the student reacts to the training serving as a surrogate of satisfaction; Level 2 – learning analyses not only knowledge but also skills; Level 3 – behavior assesses if the training reflects in changes in how the student behaves; and Level 4 – results determine if the impact on the overall learning program and even on the organizational outcomes, the Kirkpatrick Model has been used extensively in medical education⁹.

Competency-based education model

Competency-based education is a model that prioritizes knowledge learning and skills development without established time goals. That is, this model emphasizes how much students learn (learning-based system) instead of how long it takes to learn such knowledge and skills (time-based system)¹⁰. The foregoing still contrasts with the European Directive 2013/55/EU of Recognition of Professional Qualifications that still consider that the basic training of the doctor must consist of at least 5 years and a minimum of 5500 h of theoretical and practical teaching; parameters have not changed in the past 15 years.

In the field of medical education, there is no clear definition; however, there is consensus that the competence of the doctor must be multidimensional, dynamic, and contextual. The initial change in this education paradigm is to replace the question “What should the resident know?” To “What skills does the resident require?” Once these skills have been identified, the competencies (and their components) must be explicitly defined, set goals throughout the process as a reference, select the appropriate educational activities and methods, and design the instruments to assess the outcome. The Accreditation Council for Graduate Medical Education (ACGME) of the United States requires six basic competences: (1) medical knowledge, (2) patient care, (3) interpersonal communication skills, (4) professionalism, (5) practice-based learning and improvement, and (6) system-based practice¹¹. These skills focused on neurosciences are in the curriculum of the American Academy of Neurology¹².

On the other hand, this same Accreditation Council has tools to evaluate the quality of the programs in different specialties through the fulfillment of milestones. The system consists of five levels; the teacher selects whether the resident already has the competencies of each level or is in the process of acquiring them¹³. Figure 2 shows the general goals for movement disorders as an example.

Figure 2 Example of competency-based education assessment through the use of milestones. 

Active education and passive education

Broadly speaking, learning methods can be divided into passive and active. Passive learning is one that occurs when listening to the expert gives a lecture or conference. In this scenario, the resident does nothing but listen (convergent thinking). In active education, the teacher facilitates interaction and discussion (divergent thinking); the above empowers and motivates students resulting in a better learning process.

Within the active teaching strategies, the method called problem-based learning (PBL) has been the one that has traditionally been used in medicine. The problem-based method promotes critical thinking and communication skills and is useful when you have a small number of students¹⁴. At present, this is the method described in the PUEM. An alternative is the method called team-based learning (TBL); this method is more structured and is appropriate when you have a significant number of students as it works through creating small work teams. The TBL consists of three phases or modules: preparation before class (reading of pre-assigned material), assurance of learning (multiple-choice examination), and application of the concepts analyzed (discussion and consensus). This method has also demonstrated better academic performance and greater learning compared to the passive method¹⁵. Recently, it has been suggested to use a combination of both methods¹⁶.

It is well recognized that Millennials are characterized by changing the focus of their attention if they are not interested in the subject. Therefore, active learning methods are the best choice.

The “flipped classroom”

Millennials are characterized by collaborative work, familiar with technology and driven by feedback. This is why the flipped classroom has been suggested as the ideal model to teach residents of this specific generation¹⁷.

The educational model known as “flipped classroom” is to reverse the traditional teaching of sessions or classes. The time devoted to this activity should not focus on teaching a class based on textbooks or review articles, but rather focus on clarifying points of confusion, explaining the main concepts, and even encouraging the application of these concepts in hypothetical clinical settings. For this, it is required that residents complete basic learning at home beforehand. This model has been shown to improve levels of satisfaction while maintaining at least the same levels of knowledge acquisition as traditional methods¹⁸.

More specifically, the flipped classroom educational model focuses on the development of higher-order thinking skills. Based on the Bloom’s taxonomy described previously, this model privileges the levels of analyzing, synthesizing, and evaluating. In comparison, in the traditional model, the teacher transmits the knowledge to the residents during the class (lower-order levels knowledge and understanding), and the learning is passive. The resident is then responsible for developing skills individually and without home orientation that corresponds to the higher levels of applying, analyzing, evaluating, and creating. In contrast, in the flipped classroom model, the resident acquires knowledge at home asynchronously and at his own pace, while in class, jointly develops higher-order skills with the teacher.

From the point of view of the operatic program of a residency course, its content must be identified asynchronously. Conventionally, it is recommended that this knowledge is acquired through the reading of textbooks in a programmed way; however, it is currently necessary to include digital media such as blogs and podcasts previously identified by the tutor. Once this content is established, class sessions may follow a model of PBL, TBL, or audience response system.

On the other hand, it is recommended that the lectures or presentations given by the teacher are not to be completely eliminated; however, the objective is not to present the existing information in textbooks or review articles but to cement the knowledge that residents obtained asynchronously. It should be considered that only 5% of the material received at a lecture is properly retained and that the attention span is only about 15-20 min¹⁹. Therefore, it is suggested to divide the presentations into 15 min of class and 30 min of active learning²⁰. For example, the Anesthesiology Program at Stanford University replaced the traditional 45 min classes at seven in the morning with 15 min classes supported with slides 3 times a week; the result was an increase in the quality of education²¹.

At the undergraduate level of medicine, this model has been successful²², although there are still some controversies as to whether the entire curriculum should use this model. A recent study showed that approximately 20% of students believe that 50% traditional model and 50% flipped classroom would be ideal; however, almost 70% of them felt that attending classes to review knowledge acquired at home should not be mandatory²³.

The PUEM contemplates the seminar modality focused on individual study and inquiry and analytical discussion in peer groups, for subsequent reflection. As previously mentioned, traditional classes are no longer considered very useful; however, a viable option is the update in the form of imparting them. Since Millennials prefer a less formal learning environment and appreciate interaction, it is advisable to use tools to include activities that required the participation of residents during presentations. There are free tools such as Google Forms or Pear Deck that can be integrated into PowerPoint presentations, Keynote, or Google Slides and provide results in real time.

Resident as teacher

It should not be forgotten that the word doctor comes from the Latin docere, to teach. That is, why another of the requirements of the ACGME is that the development of residents as educators is encouraged. The “resident as teacher” programs have been identified as such since the 1970s. However, the parameters to define its effectiveness have not been studied specifically until recently. In general, the benefits are both for students, usually medical students or residents of lower academic hierarchy, and for the resident who serves as a teacher²⁴. In fact, a study conducted between 2007 and 2010 showed that the evaluation of medical students on a clinical rotation depends mainly on academic interaction with residents²⁵.

Feedback to residents about their ability to transmit knowledge to their peers is also relevant. One study showed that 94% of residents rated the feedback received immediately as very useful, and 91% of them mentioned that such feedback would impact the way they gave their presentations. The recommendations focused on the amount of content, focus on clinical reasoning, the use of visual support, and encouraging interaction²⁶.

The PUEM establishes time requirements that are equivalent to only 2.5% of the training; however, it is suggested to favor this role given the evidence presented. In the case of the specialty in neurology, there are usually 1^st year and 3^rd year residents, to whom are added rotating other specialties, interns, and even students resulting in large groups. However, if the number of students is very small, the tutelary model can be favored.

Resident as researcher

As previously mentioned, medical specialty courses are also intended to involve the specialist in research. A fundamental part of this is the publication of scientific articles in peer-reviewed journals. It has been shown that 75% of residents who published an article during their training will continue publishing later. In contrast, only 55% of residents who did not publish during their residency published after the end of their residency²⁷. Moreover, 71% of residents of curricular programs that actively contemplated research published compared to only 41% of programs focused solely on traditional education of knowledge and skills. In general, residents who published one or two articles during their training increased their number of publications to seven after the end of their residency. Another study conducted in urology residents of Canada reported that 94% had at least one publication during their residency, the average being four publications. However, 44% of them reported that publishing did not improve their competence in this area mainly due to issues such as lack of time and attention by a mentor and poor active participation in the investigation²⁸. In the United States, a study of surgical residents graduated between 1993 and 2013 reported that 54% of residents had published at least one article as the first author. In addition, there was an increase of twice as many publications per resident as of 2003²⁹. Finally, a study conducted at the Mayo Clinic from 2006 to 2012 found that the number of original articles published and the total number of publications during residency predicts better clinical performance not only in skills assessments such as the MiniCEX but also in the examination of knowledge of the corresponding specialty board³⁰.

Based on the above, specific programs have been developed. For example, the Penn State Hershey Medical Center Academic Resident Project contemplates the following actions: during the 1^st year of residence, the resident is assigned a mentor (associate or principal professor) and a counselor (another faculty member) with whom a research project is chosen and the proposal is written. During the 2^nd year, the project is completed and in the 3^rd year, its results must be presented in the form of a manuscript that can be published along with a 10 min oral presentation³¹.

A study in otorhinolaryngology residents showed that the implementation of a schedule restriction at a maximum of 80 h/week along with two rotations of 20-24 weeks in research resulted in the increase of one to five publications per resident³². The PUEM requires the preparation of a thesis as a termination requirement; it should be noted that in general, it is expected that around 30-40% of the specialty theses will be subsequently published as an original article³³.

On the other hand, it has been reported that < 30% of the papers presented at congresses by residents end up with the publication of an article in extense³⁴. Finally, the previously cited studies agree that publishing during residency doubles the likelihood that the resident will be integrated into academic and research activities in the future. It is worth mentioning another of the Millennials stereotypes, not staying in the same job. In this regard, it has been shown that they are effectively better prepared for changes in their occupational area; however, this does not happen for those with high levels of education such as postgraduate studies³⁵. Therefore, encouraging the continuation of studies with masters or doctorates in research can increase job stability.

A final aspect to consider in relation to publications is ethics. Millennials show a high concern for ethics and a search for role models. It has been reported that up to 57.3% of publications made by residents included an honorary authorship, euphemism for gift authorship. That is, more than half of the publications include coauthors who do not meet the international criteria to be recognized as such³⁶. The above can subtract the interest of residents in the investigation and dissemination of it, so the teacher must always show an impeccable ethic. The PUEM only requires 5% of the total program time allocated to this activity; however, the benefits described justify a greater dedication to this area.

Assessment and evaluation of skills

Conventionally, the evaluation of skills is carried out through different methods, including the direct observation of procedural skills (DOPS) and clinical evaluation exercises (MiniCEX). The DOPS evaluates the resident while performing diagnostic and treatment procedures in clinical practice. The evaluator uses a structured checklist. In the MiniCEX, the resident’s skills, attitudes, and behavior are evaluated before a routine clinical encounter with an approximate duration of 15 min. It should be noted that the MiniCEX has shown a very good correlation with the Objective Structured Clinical Examination (OSCE), where knowledge, skills, behaviors, and attitudes are evaluated throughout a series of stations with different clinical scenarios³⁷. Since the OSCE requires more time and resources, the MiniCEX is an acceptable option. In addition to the relative brevity of the assessment, the MiniCEX as well as the DOPS require that immediate feedback be offered to the resident. In the specific case of neurology, both instruments have proven useful³⁸.

Other methods include multi-source feedback (MSF or 360° evaluation) in which professionals working with the resident answer a structured form. In the case of Millennials, it is important to remember that it is a generation with a history of success, achievements, and self-confidence; as a result, they are accustomed to positive feedback. On the other extreme, the generations that precede them grew with the maxim “if there is no news, it is good news.”

The methods mentioned above differ from formal written examinations where it is only possible to evaluate knowledge. In addition, formal oral examinations are characterized by poor content validity, high inter-rater variability, and grade inconsistencies³⁹.

Another recommended method for assessing skills is called Reporter, Interpreter, Manager, and Educator. At the reporter level, the resident is expected to be able to interrogate and explore, as well as present the clinical findings. At the interpreter level, it is expected to be able to use clinical reasoning, analyze, and prioritize differential diagnoses. At the manager level, the resident must generate reasonable patient-centered treatment options; finally, at the educator level, he is able to contribute to the education of patients, family members, and other members of the medical team⁴⁰.

Designing questions for evaluation

When generating the questions for evaluations, the six levels of Bloom’s taxonomy should be considered: knowledge, understanding, application, analysis, synthesis, and evaluation. It should be considered that this taxonomy is hierarchical and that the terminology used corresponds to a level of complexity of the educational objectives. There are studies that compare performance among those who use the flipped classroom model or the traditional lecture model; some of them show superiority of the first⁴¹, while others have shown no difference, including a study in a neuroanatomy course⁴². A recent study suggests that the difference or absence of this is related to the development of the questions since if there is no concordance between the question and the Bloom’s level that is expected to evaluate. Thus, the results will be inadequate⁴³. In summary, if the objective is to evaluate the flipped classroom model, questions aimed at higher-order levels should be used. On the other hand, if the model is traditional, a better evaluation will be obtained using questions directed at lower-order levels.

ICT

To properly understand the role of ICT, the concepts of synchronous and asynchronous formats must be defined. These terms are related to the online learning method used for educational materials and virtual training, included in courses, diplomas, masters, and doctorates. The above are encompassed under the term of e-learning. Broadly speaking, synchronous learning occurs when students and teachers are online while respecting pre-established schedules and duration times. The asynchronous one allows the student to access the didactic material when it suits him best, without restrictions of schedule or duration (although there are deadlines). Residents have been shown to prefer the asynchronous format over the synchronous in relation to e-learning, even placed alongside the traditional face-to-face class format⁴⁴. The ACGME recommends that the program includes a combination of synchronous and asynchronous learning. The asynchronous component should cover at least 20% of the program and ideally should be monitored through a learning management system as described below⁴⁵.

The second aspect to consider is that with the involvement of ICTs in education, it has been necessary to establish new frameworks to guide the educator. The most recent model is the SAMR. It is a hierarchical model; the two lower levels are called potentiation and include substitution and augmentation. The upper two levels correspond to transformation and include modification and redefinition⁴⁶. For example, at the substitution level, it is only necessary to change paper examination to one in digital format. On the augmentation level, electronic devices are used to improve the educational experience. At the modification level, a redesign of the way in which the material is presented (interactive simulations) is required, and in the redefinition level, new tasks are created that would result impossible to perform without the use of ICT.

Millennials are the first generation considered as digital natives, that is, they were born and grew up in a digital environment. At the same time, the Millennials seek flexibility in their schedules to reach the balance between life and work. The above facilitate a paradigm shift of reserving learning moments only for scheduled sessions or during clinical activities as a visit step or outpatient consultation. The use of technology in smartphones or tablets such as messaging systems and social networks allows the delivery of brief and timely information at any time of the day. This can be done spontaneously or in a pre-established way. This information can be images or videos of quick visualization. A recent study evaluated the effect of using Twitter as part of a medical residency program in internal medicine. The authors created a specific account which was managed by two of the professors, as well as by the chief resident. The content of the publications was established in thirds: education of the residents, promotion of academic activities of the program, and promotion of other events. At the end of the academic year, 69% of residents indicated that this tool generally improved their medical education⁴⁷.

Another option is text messages or short message service (SMS). A study in urology residents compared learning through traditional classes compared to questions sent through SMS. Final grades and the level of satisfaction with learning were higher with the SMS method⁴⁸. Recently, studies on the usefulness of other messaging systems such as WhatsApp have been published. The Duke Cardiology Program implemented a “WhatsApp group,” in which clinical cases of interest and images were published under teacher supervision. About 66.7% of the students in the program participated actively and 85.7% said that the platform enriched the teaching process⁴⁹. It is worth mentioning that the use of this platform was based on four rules: “what happens in WhatsApp, stays in WhatsApp,” there is no consequence for wrong answers, it is only used during working days and hours, and there is a maximum of five cases per day to avoid fatigue in interactions.

Finally, the use of ICT with all its potential necessarily requires a platform for learning management system, generally known as LMS. The LMS allows to administer, distribute, and evaluate training activities. LMS costs for a residency or fellowship program can vary from free (Schoology) to hundreds of dollars per user (Blackboard, Canvas). However, on many occasions, academic or university institutions grant access to these systems (Moodle, Google Classroom). The National Institute of Neurology and Neurosurgery has its own LMN named as Neurocampus (www.neurocampus.cuaed.unam.mx), which through the LMS Moodle offers continuous medical education for its residents of specialty, subspecialty, and high specialty in virtual formats or mixed (blended education).

It is important to understand that it is currently considered that the responsibilities of the program to the resident and vice versa extend beyond the date of completion of the course. ICTs favor a learning ecosystem that allows these interactions to continue.

Conclusions

It should not be forgotten that generations are the product of the social environment, economic aspects, world and regional events, and technological changes, in which they grow. “Technological” generations such as Millennials and Generation Z require teaching-learning environments congruent with their needs. Diversity in educational models must be the norm, integrating multicultural talent. The models and techniques presented here are not exhaustive; the National University Autonomous University mandatory pedagogical training course for professors of medical specialties recommends additional tools such as assessment rubric, appreciation checklist, and portfolios. Recommendations on education for the high specialty courses have been also published elsewhere⁵⁰.

Mexico is an academic destination for the training of specialists, particularly from Central America and, more recently, from South America so that academic expectations force us to adopt modern teaching, learning, and evaluation models. Generation Y or Millennials are considered the generation with the highest level of education in history, and unlike Generation X, they tend to be more optimistic and literally believe that they can change the world. It is the responsibility of teachers to keep up to date in pedagogy and adopt the strategies with the highest probability of success to promote the development of skills and professionalization of their residents.