The use of the tracer methodology to assess the quality of care for patients enrolled in Medical Insurance for a New Generation

El uso de la metodología de trazadores para la evaluación de la calidad de la atención en pacientes afiliados al Seguro Médico para una Nueva Generación

Luis Durán-Arenas, MD, PhD^I; Onofre Muñoz-Hernández, MD, MSc^II; Luz del Carmen Hernández-Ramírez, MD^II; Luis Jasso-Gutiérrez, MD, MSc^II; Samuel Flores, MD^II; Vanessa Aldaz-Rodríguez, MSc^II; Juan Manuel Zurita, BSc.^II

^IDivisión de Estudios de Posgrado, Facultad de Medicina, Universidad Nacional Autónoma de México. México
^IIGrupo de trabajo del proyecto de Evaluación del Seguro Médico para una Nueva Generación, Hospital Infantil de México Federico Gómez. México

ABSTRACT

OBJECTIVE: To assess the quality of care provided at medical units that provide services to Medical Insurance for a New Generation (SMNG) enrollees.
MATERIALS AND METHODS: The tracer methodology was used in a sample of 82 medical units selected in fifteen states of Mexico and data collected in November 2009.
RESULTS: Problems were found to locate the minimal number of the 18 medical charts requested in three of the tracers. The first level of care on the average reports that the quality of the process of care is 6, in a 10 point scale. In the second level improves and the third level of care is better qualified.
CONCLUSIONS: The tracer methodology has enabled us to assess the quality of care. There is room for improvement in the medical units of the state health services, to that end should be directed the efforts in the health system in Mexico.

Keywords: Tracers; evaluation of quality of medical care; Mexico

RESUMEN

Palabras clave: trazadores; evaluación de la calidad de la atención; México

Almost 40 years ago Kessner published his seminal work on tracer methodology to evaluate quality of care. His original work established that the evaluators of programs could use a set of specific health problems, he labeled tracers, to assess the quality of care of medical services. These tracers were selected based on a set of criteria of pertinence and relevance for the health care provided at the medical unit or system. Through medical chart reviews, the researchers evaluated the diagnostic, therapeutic, and follow-up processes of the set of tracers and the treatment outcome. The main premise was that the results on these tracers would reflect the quality of the care provided in a health care unit or even a health care system.¹

The work of Kessner is in fact a health services research classic. Quoting Neuhauser: "this article is a member of the hall of the fame of health services research…the pertinence of the tracer methodology for the evaluation of quality of health care in the USA is widely recognized".²

Proof of this recognition is the work of Osterweis et al, who developed and tested methods for assessing the quality of health care that would be broadly applicable to diverse ambulatory care organizations for periodic comparative reviews.³ In their work they used Kessner's tracer methodology (including not only acute and chronic diseases but also screening and immunization rates as indicators), and a fixed tracer matrix at all test sites. They found that the actual process of care at all three sites for the three acute conditions (streptococcal pharyngitis, urinary tract infection, and iron deficiency anemia) often differed from the expected process in terms of both diagnostic procedures and treatment outcomes.

In Mexico, the use of this methodology either in research projects or in health program evaluations has been limited. In a review of the last five years of the health systems literature there were no references about the use of this methodology. However, based on the reports of the use of this methodology in the international literature we decided to use in order to assess the quality of health care of services delivered to the enrollees of the Medical Insurance for a New Generation (Seguro Médico para una Nueva Generación SMNG), a new component of Seguro Popular, given that the quality of care of the newborns enrolled in the insurance program is a high priority. In fact, a great effort has been dedicated to the development of clinical guidelines and other tools to evaluate the health results in the services provided by Seguro Popular.^5-8 The tracer methodology is an appropriate tool to use criteria developed in this guidelines to evaluate the quality of health care.^9,10

The purpose of this article is to report the results of quality assessment of care provided at medical units that deliver services to SMNG enrollees, and to identify improvement opportunities in the health care processes, through the use of five tracers among the services paid for by the SMNG.

Materials and methods

The methods used in this study are based in the application of the tracer methodology in a sample of medical units selected in fifteen Mexican states in November and December 2009. The research team developed the medical audit instruments and they were validated in a pilot study in medical units outside of the sample in Mexico City.

Tracers were selected based on the relevance to the priority programs for children care in the Federal Ministry of Health. In general it was made sure that all the tracers were considered either as part of the catalogue of diseases of popular insurance (CAUSES) or in the SMNG catalog of catastrophic diseases.

Tracer Description:

Five tracers were selected, two to be used in first level of care units, two in second level of care units, and one in third level of care units. They are described next.

]]> 1. Medical chart audits in first level of care medical units.

This was done with the following tracers:

Iron supplementation for anemia prevention: sufficiency of health care structure; qualification and training of health care personnel; service quantity and quality.

Child Growth surveillance in children less than 2 years old: health care structure sufficiency; qualification and training of health care personnel; service quantity and quality.

2. Medical chart audits in second level of care medical units.

This was done with the following tracers:

Low birth weight: health care structure sufficiency; qualification and training of health care personnel; service quantity and quality.

Congenital hypothyroidism screening: health care structure sufficiency; qualification and training of health care personnel; service quantity and quality.

]]> 3. Medical chart audits in third level of care medical units. In children with Respiratory Distress Syndrome in Neonatal Intensive Care Units: sufficiency of the health care structure; qualifications and training of the health care personnel; service quantity and quality.

Sample design

The study population consisted of all the medical units delivering health services to enrollees of the Seguro Popular in Mexico. A multiple stage sampling process was done for definition and selection of the medical units to be sampled. First, the medical units were grouped according to the regions of Mexico (using geographic, social development and economic criteria); in a second stage, within the regions, a random sampling of hospitals and ambulatory health care units in the states belonging to the region was performed (units that according to legislation are obliged to have birth delivery services). Finally, within the medical units a census of medical providers was carried out in the morning shift, and medical charts were randomly selected in order to retrieve the five tracers.

For the medical chart audits, sample size was calculated according to the health care occupancy rate per person and year taken from the 2006 National Health and Nutrition Survey (ENSANUT 2006), in order to select enough medical charts to have enough statistical power to spot differences among units regarding health care quality. The former was done for the five tracers, assuming that this rate is a good approximation of SMNG enrollees use rate.

Table I shows the medical units selected by level of care and state for this study. A total of 82 medical units were selected, 29 at the first level of care, 30 at the second level and 23 at the third level of care that were located within 15 states in the country. Those 15 states were the result of the random selection of medical units within the regions.

Table II shows the final number of charts expected to be reviewed by tracer and level of care.

Instruments

Three sets of instruments were used in this evaluation:

1. Structure of the medical units at three levels of care including: capacity of the physical infrastructure and its quality.

]]> 2. Characteristics of health care providers at the three levels of care including: availability of health care resources, qualifications and training.

3. Medical Chart review, at the three levels of care in five tracer diseases. It was designed that the charts were randomly selected with a quota of 18 charts per tracer, per medical unit. However, in most cases all the charts available in the unit were used, given that the activity on the tracers was low (this was the case with congenital hypothyroidism screening). This is an interesting finding, that is presented in the results section.

Data analysis

Tracers were evaluated for this article using process indicators, and in one case (third level of care) an outcome indicator. This decision was taken since there was not sufficiently good information on the medical charts in regard to the intermediate results that were selected as outcomes in the first and second level of care tracers.

All the medical units within each level of care were ranked according to the quality of the process of medical care in each one of the tracers. To define the scores in the tracer for each unit a set of algorithms for score calculation were developed. These algorithms contrasted the values in each of the medical files reviewed for process variables of each of the tracers, against standards defined by experts at Hospital Infantil de México Federico Gómez. The algorithms are presented next:

Low birth weight: a linear combination of the variables "antenatal care" and "number of antenatal consultations", assuming an equal importance of the variables. The scores of each file were added and averaged for the medical unit in a scale metric that varies between 0 and 10.

Congenital hypothyroidism screening: a linear combination of the variables "date of screening", "date of tentative diagnosis" and "date of diagnostic confirmation", assuming an equal importance of the variables. The scores of each file were added and averaged for the medical unit in a scale metric that varies between 0 and 10.

Iron supplementation for the prevention of anemia: a linear combination of the variables "age of start of supplementation" and "duration in months of the supplementation" assuming an equal importance of the variables. The scores of each file were added and averaged for the medical unit in a scale metric that varies between 0 and 10.

Surveillance of Child normal growth in children less than 2 years old: a linear combination of the variables "registration of birth date", "date of the last consultation", "number of times with registry of weight until the last consultation", "number of times with registry of height until the last consultation", "registry of a note of the growth" and "development status of the child in the last consultation", assuming an equal importance of the variables. The scores of each file were added and averaged for the medical unit in a scale metric that varies between 0 and 10.

Children with Respiratory Distress Syndrome in Neonatal Intensive Care Units: In this case three indicators were used: two regarding process ("appropriateness of the reference"; "hospital care quality") and one an outcome measure, "survival rate". The scores of each file were added and averaged for the medical unit indicators and standardized to create a scale metric between 0 and 10.

Results

Even though in general the results suggest problems regarding health care quality at all the levels of care, it is important to point out a persisting gradient of quality, being the lowest quality at the first level of care, improving at the second level of care units, and reaching the best overall scores in the tertiary health care units. We now present the results within each level of care.

Only the child growth surveillance tracer seems to be a working program in the first level of care medical units, and in general is being carried out with a good quality level according to the medical files reviewed (mean scores between 7.3 and 9.6).

In the case of the iron supplementation tracer, in 19 of the medical units reported activity (They have medical charts for the tracer). However in only six units were the 18 files requested found, the other 13 provided between 1 and 16 medical files. In this case the tracer quality was less homogeneous, with mean scores between 2.3 and 8.8 with only one unit reaching a score of 10.

The expected results in primary and secondary care are different, in the case of the second level of care, the low birth weight tracer shows a higher level of activity, only two medical units did not have files of this condition. However, only five medical units out of the 30 provided the 18 medical files requested for low birth weight. Mean quality scores ranged between 2.2 y 10. In only one unit, a mean score higher than 9 was reported.

It was found that hipothyroidism is a program with a very low activity. In this case only 13 units reported to have medical files for this condition and none of them reported the 18 files requested (therefore, in this case it was conducted a census of the medical charts for Hypothyroidism). The scores tend to be low with 5 of the 13 units reporting mean scores under 6; only two units have scores higher than 9.

Table III shows the analysis of hipothyroidism cases, assessing in particular the opportunity in the critical activities of the different processes of the screening. The results suggest that screening was not optimal. Initial screening was carried out timely during the first week of life only in 53.1% of the cases. In only 21.9% a tentative diagnosis was done in the first two weeks of life. Finally, only 21.9% of the children had a confirmed diagnosis before 4 weeks elapsed. Without doubt, there is room for improvement in the performance of this preventive program.

Finally, the tertiary health care units showed in general evidence that they carry out activities on the tracer selected for this level of care, and they have better quality scores in contrast to other levels of care. It is important to notice that in all the units the 18 files requested were found and reviewed.

Nevertheless, we still found an important variation between units in the three indicators that were used to evaluate the quality of care using the medical files of children with respiratory distress syndrome. Regarding the outcome indicator (survival rate) the units tend to have high scores, 15 of the 23 units at this level of care reported scores of more than 8 out of 10, which indicates a good quality of care in this tracer.

On the other hand, the two process indicators had considerably lower scores. In the case of appropriateness of the reference to this level of care, only four units reported scores over 8, and 11 reported scores below 6. A worst situation is observed in the case of the process of hospital care, where only one unit reported scores over 6, and 11 reported scores below 4.

Discussion

It is important to remember that all the tracers were selected because they are part of the priority programs for children care in the Ministry of Health. In the case of respiratory distress syndrome, it is part of the list of catastrophic expenses of the Seguro Popular. Therefore all the tracer programs should be active in the levels of care studied.

In the first place the results suggest that there is a low quality of care when analyzing process measures on the tracers selected at all the different levels of care. This finding demands attention. The Seguro Popular should not only be concerned about affiliation but also about the quality services that are provided to its enrollees and the quality of the care delivered to them.

Secondly, there is evidence that the first level of care in general is not carrying out the health programs on the tracers and when there is evidence of activity, the quality of the process of care is bad. It is important to discuss further this finding, the simple fact that there was no evidence of medical charts for this priority programs is an indication of the lack of clinical governance in these medical units. All of them were supposed to have active programs regarding these diseases, and the lack of medical charts is a strong indication of the low performance of activities in these programs. Therefore we can use it as an indirect measure of the process quality. This is congruent with reports in Mexico about the quality of the programs aimed at Hipothyroidism screening.¹¹

Even though the activity on the tracers improved in second and third level of care, the quality in the process indicators still tended to be low. These results suggest that even at these levels there are important limitations to assure access to, and quality of care. Once again this should be of great concern for Seguro Popular management.

Only in the case of the outcome indicator of the third level of care tracer there is evidence of better quality of care. However, as it is reported in other article of this special number, there is evidence of a faulty structure and processes of care even at this level.

There are strengths and weaknesses of the tracer methodology used in this article. On the first hand, it enabled us to assess the quality of care in 82 medical units of the three levels of care in Mexico. The results presented here show that there are opportunities for improvement. In particular training and monitoring of the programs need reinforcement. The joint commission of health care organizations of the United States of America has recognized this, and the tracer methodology is a component of the certification process to improve quality in that country.

One of the more important weaknesses of this methodology is that it depends of the quality of the medical charts. It is well recognized in Mexico the need for the development of electronic medical charts that could be used for gathering better information as well as for monitoring and evaluation of the priority health care programs.

Finally, we would like to make a reflection on the achievements of the Seguro Popular and the SMNG. Even though a great success has been documented regarding affiliation, affiliation is not a synonymous of coverage, and the results of this study support the idea that even coverage is not enough without quality of health care. There is room for improvement in the medical units of the state health services, to that end should be directed the efforts of the second wave of reform of the health care system in Mexico.

Declaration of conflicts of interest: The authors declare that they have no conflict of interests.

References

1. Kessner DM, Kalk CE, Singer J. Assessing health quality. The case for tracers. N Eng J Medicine 1973; 288:189-194.         [ Links ]

2. Neuhauser D. Assessing health quality: the case for tracers. J Health Serv Res Policy 2004; 9: 246-247.         [ Links ]

3. Osterweis M, Bryant E. Assessing technical performance at diverse ambulatory care sites. J Community Health 1978;4(2):104-119.         [ Links ]

4. American Health Consultants. Tracer methodology: how it can help you improve quality: quality pros see opportunity to identify system, process problems. 2004. American Health Consultants, Inc. [consulted 2010 August 4]. Available at: http://findarticles.com/p/articles/mi_m0NUZ/is_6_11/ai_n6183790/pg_2/?tag=content;col1        [ Links ]

5. OPS. Intervenciones basadas en evidencia en el contexto del continuo materno-recién nacido-niño menor de dos meses. Intervenciones Basadas en Evidencia". Washington, DC: OPS, 2009.         [ Links ]

6. Marquis S, Long S. The role of public insurance and the public delivery system in improving birth outcomes for Low-income pregnant women. Medical Care 2002; 40 Issue (11):1048-1059.         [ Links ]

7. SSA. Norma Oficial Mexicana NOM-007-SSA2-1993, Atención de la mujer durante el embarazo, parto y puerperio y del recién nacido. Criterios y procedimientos para la prestación del servicio. Secretaría de Salud, México; 1995. (Consultado 2009 mayo 29). Disponible en: http://www.salud.gob.mx/unidades/cdi/nomssa.html.         [ Links ]

8. SSA. Lineamientos para la suplementación de hierro y vitamina A en recién nacidos, niños y adolescentes. Centro Nacional para la Salud de la Infancia y la Adolescencia, México: Secretaría de Salud, 2009.         [ Links ]

9. Norma Oficial Mexicana NOM-031-SSA2-1999, Para la atención a la salud del niño. Secretaría de Salud, México; 2001. (Consultado 2009 mayo 29). Disponible en: http://www.salud.gob.mx/unidades/cdi/nomssa.html.         [ Links ]

10. Estrategia y plan de acción regionales sobre la salud del recién nacido en el contexto del proceso continuo de la atención de la madre del recién nacido y del niño. Washington, DC: OPS, 2008.         [ Links ]

11. Aguilera N, Marrufo GM. Can better infrastructure and quality reduce hospital infant mortality rates in Mexico? Health Policy 2007;80(2):239-252.         [ Links ]

12. Rendón-Macías ME, Morales-García I, Huerta-Hernández E, Silva-Batalla A, Villasís-Keever MA. Birth prevalence of congenital hypothyroidism in Mexico. Paediatr Perinat Epidemiol 2008;22(5):478-485.         [ Links ]

Corresponding author:
Dr. Luis Durán Arenas
División de Estudios de Posgrado, Facultad de Medicina, Universidad Nacional Autónoma de México
Dr. Márquez 162, col. Doctores
06720 México DF, México
E-mail: lduran19@liceaga.facmed.unam.mx