Validity of underlying cause of death statistics in Hermosillo, Mexico

Validez de las estadísticas de causa básica de muerte en Hermosillo, México

Mary H Freire de Carvalho, MPH, PhD,^I; Gerardo Álvarez-Hernández, MD, MPH, PhD,^II; Catalina Denman, PhD,^III; Siobán D Harlow, PhD.^I

^ISchool of Public Health, University of Michigan.Ann Arbor, Michigan, USA
^IIDepartamento de Medicina y Ciencias de la Salud, Universidad de Sonora. Hermosillo, Sonora, México ]]> ^IIICentro de Estudios en Salud y Sociedad, El Colegio de Sonora. Hermosillo, Sonora, México

Correspondence author

ABSTRACT

OBJECTIVE: This paper assesses the quality of the underlying cause of death (COD) statistics in Hermosillo, Mexico in a random sample of 300 in-hospital adult deaths.
MATERIAL AND METHODS: A "gold standard" COD, determined by a systematic review of hospital medical charts, was compared to the COD reported by the vital registry system.
RESULTS: Overall agreement between the reviewer and original COD at the ICD-10 chapter block was 69.2%, with a weighted kappa of 0.62. Agreement varied greatly by ICD-10 chapter. Mutual misclassification among common co-morbidities,such as diabetes mellitus and circulatory disease, minimized the net change in the mortality fraction assigned to each ICD-10 chapter after physician review.
CONCLUSIONS: The ICD-10 chapter level underlying COD codes can be used to estimate disease burden in the population. Caution is recommended for use of vital registry statistics in Hermosillo for individual level or disease-specific analyses.

Key words: Cause of death;data analysis;mortality registries; Mexico

OBJETIVO: Examinar la validez de la causa básica de muerte (CBM) de certificados de defunción de un sistema digital de vigilancia epidemiológica en Hermosillo, México.
MATERIAL Y MÉTODOS: Se comparó la CBM originalmente codificada en una muestra aleatoria de 300 certificados de defunción de muertes adultas ocurridas en al año 2005, con la CBM determinada por una revisión de expedientes médicos.
RESULTADOS: Se identificó una concordancia global de 69.2% entre la CBM original y la determinada por la revisión, con un coeficiente de kappa de 0.62. La concordancia varió,con errores de codificación en padecimientos como diabetes mellitus, enfermedades circulatorias y respiratorias y neoplasias. El efecto de esos errores fue compensado por una mutua asignación equivocada en el proceso de codificación de la CBM empleado por el sistema.
CONCLUSIONES: Las estadísticas generadas por el sistema examinado pueden usarse con cautela para estimar la carga poblacional de mortalidad en Hermosillo, México.

Palabras clave: causa básica de muerte; análisis de datos; registros de mortalidad; México

Birth and death certificates provide essential data for health research and policy, as well as for program implementation and evaluation.^1-3 In developing and transition economy countries, vital registry data is often the only data available for tracking population health over time and for making evidence-based policy decisions to effectively address community health needs.

Mexico has a well-established vital registry system dating back to 1950, with underlying cause of death available since 1955. When compared to the 192 countries with registries globally, Mexico's national vital registry system is of high quality, with 96% coverage of deaths and just 5% of deaths coded as ill-defined causes.¹ In 2000, the Secretary of Health of the state of Sonora began to digitalize its death certificate information, creating a valuable source of epidemiologic data for research and policy.

This paper assesses the quality of underlying cause of death statistics in Hermosillo, Mexico in a sample of adult deaths. This analysis directly addresses goals established for the betterment of the Mexican Health System's National Health Information System (Sistema Nacional de Información en Salud; SINAIS), including: 1) improving the quality of health information and 2) strengthening the Mexican Center for the Classification of Disease.²¹

Material and Methods

This cross-sectional study assesses the reliability of in-hospital adult (age >15) mortality statistics from the Epidemiological and Death Statistics System (Sistema Epidemiológico y Estadístico de Defunciones; SEED) for 2005 in Hermosillo, the capital of Sonora (population 662 924). The SEED uses the international system for the classification of diseases and related health problems (ICD-10) as recommended by the World Health Organization. At the time this study was completed, the cause of death code was determined manually. The study was approved by the University of Michigan Institutional Review Board, the Ethics Committee of El Colegio de Sonora, the hospitals where medical record abstraction took place and the Secretary of Health of the state of Sonora.

In 2005, a total of 2 586 deaths occurred in adults 15 years of age and older residing in Hermosillo. Death certificate information was provided in a de-identified database from the SEED system, which included information on the underlying cause of death (ICD-10 code), age at death, sex (male/female), neighborhood of residence, certifying personnel (attending physician/other physician), medical insurance enrollment (yes/no), type of hospital facility (for insured/for uninsured) and time of death based on usual hospital employment shifts (07:00-14:30 hrs./14:31-21:30 hrs./21:31-06:59 hrs.).

A sample of 300 in-hospital, non-accidental deaths in 2005 was selected at random from the SEED database. The study was limited to the two largest hospitals in Hermosillo, which in 2005 accounted for approximately 90% of total deaths in Hermosillo; one provides care for uninsured patients and one provides care for those with social security.²² Of the 300 deaths sampled, 111 (37%) occurred in the hospital for the uninsured and 189 (63%) in the social security hospital.

Two physicians from the Secretary of Health, blindedto original underlying cause of death (COD) coding, used a standard form to review and abstract information on cause of death from the medical records. They underwent training to identify causes of death and to complete death certificates using the Center for Disease Control trainingmanual.²³ A standard chart abstraction form included: 1) a section for systematically reviewing test results and physician notes in medical charts and 2) a section identical to the cause of death section of the WHO-approveddeath certificate used by the hospitals in Mexico.

To reach a final underlying cause of death, subsequently referred to as the "reviewer underlying COD,"the physicians compared their independent certifications and filled out a consensus death certificate to be used for nosologist coding. For 29 charts in which their independent death certificates did not agree, the physicians reviewed their medical chart abstractions, or the medical chartitself,andcametoconsensus. Athirdphysicianwas consulted once. A nosologist, trained in ICD-10 coding and blinded to the original underlying COD, assigned an ICD-10 code to the new consensus death certificate. The original and the new reviewer ICD-10 codes were then compared. Of the 300 deceased sampled, a medical chart was available for 258, 250 (83%) of which had sufficient data to determine an underlying cause of death.

This study was designed to mimic the process by which the vital registry underlying cause of death is assigned in Sonora, using fully trained and experienced physicians and a single nosologist trained in ICD-10 coding rules. The outcome of this process is the most accurate underlying COD that can be abstracted from hospital data, thus we use the reviewer underlying COD as the gold standard.

The original underlying COD was compared to the reviewer underlying COD at the ICD-10 chapter level by calculating sensitivity, specificity, positive predictive values and negative predictive values. Cohen's kappa was calculated to determine the level of agreement between the reviewer and the original COD, beyond that expected by chance. Agreement was analyzed at the level of ICD-10 chapters and of 2-digit ICD-10 codes. AKappa value greater than 0.75 was considered excellent agreement beyond chance; values from 0.61 to 0.74 were considered to reflect substantial agreement; from 0.41 to 0.60 moderate, and; values below 0.40 poor agreement.False-positive and false-negative percentages higher than 10% and 40%, respectively, were considered unacceptably high. To evaluate the effect of errors in the coding of underlying CODs for mortality statistics, changes in cause-specific mortality fractions were estimated and a misclassification matrix was created.

Logistic regression was used to assess the association between the accuracy of the underlying COD and hypothesized predictor variables: time of death, whether the death certificate was filled out by the treating physician, the hospital where death occurred, and age and sex of the deceased.²⁴ Underlying CODs were categorized into four broad categories: neoplasms, disease of the circulatory system, communicable diseases and other. All analyses were done in SAS 9.1.

Results

Table I shows the demographic characteristics of the full study population (n=300), the analysis population (n=250) and the missing data population (n=50). The mean age of the analytical sample was 66.2 years, 26.8% were from the hospital for the uninsured and 88.8% were certified by the treating physician. The three leading causes of death, as originally coded, were diseases of the circulatory system (27.6%), neoplasms (20.4%) and diabetes mellitus (10.8%). Deaths with incomplete or missing charts were more likely to have occurred at the hospital for the uninsured, to have been under the age of 65 and certified by the treating physician (p <0.05).

The false-positive rate was highest for diseases of the circulatory system (9.9%) and the false-negative rate was highest for hypertensive diseases (62.50%). The positive predictive value was highest for neoplasms (94.23%) and lowest for hypertensive disease (30.00%). Sensitivity was the highest for neoplasms (85.1%), diseases of the nervous system (87.5%) and cerebrovascular diseases (85.2%) and lowest for diabetes mellitus (44.0%), hypertensive diseases (42.9%) and acute respiratory disease (25.0%). The lowest specificity observed was for diabetes mellitus (92.4%) and diseases of the circulatory system (89.6%).

Underlying COD category was the only statistically significant predictor of agreement between the original and reviewer classification (Table III). Compared to all other disease categories, neoplasms were over 12 times more likely (OR 12.5, 95% CI 3.7, 42.6) and circulatory disease were more than twice as likely (OR 2.2, 95% CI 1.2, 4.2) to be in agreement.

Table IV provides a cross-tabulation showing the misclassification of selected diseases. Of the diseases with low sensitivity, underlying COD discrepancies between the original and reviewer coding tended to involve common co-morbidities. Diabetes mellitus was commonly misclassified on the death certificate as diseases of the circulatory system, infectious diseases or diseases of the genitourinary system. Hypertensive disease was most commonly misclassified as diabetes mellitus or respiratory diseases. Respiratory diseases were misclassified as diabetes mellitus, neoplasms and disease of the circulatory system.

Change in the cause-specific mortality fraction for a given disease category in the study population was minimal (Table II). While agreement was poor for a number of disease categories, the mutual misclassification of disease among these categories minimized the effect of poor agreement on the estimated distribution of disease at the population level. The largest changes occurred in diseases of the respiratory system (- 2.4%), neoplasms (+2%) and ischemic heart disease (-1.6%).

Vital registry systems are crucial sources of data on population disease burden.^3;25,26 This study is the only identified evaluation of the quality of cause of death coding of adult mortality in the Sonoran vital registration system. The study used a systematic review of hospital medical charts to determine a "gold standard" for underlying COD and compared this COD to the data reported by the vital registry system.²⁷ The electronic vital registry database, released annually, is the primary data source for assessing mortality in the northern Mexican border region.

Overall agreement between the reviewer and original COD at the ICD-10 chapter block level was 69.2%, with a weighted kappa of 0.62. This study focused on chapter level agreement because the majority of Mexico's national statistics are published and health policy priorities are set at this level. Agreement at the 2-digit classification level dropped to 50%, a proportion similar to that reported for infant deaths in Hermosillo.²⁰ In contrast, a study of adults in Brazil found a higher rate of overall agreement (83%; kappa=0.76) at the 3-digit ICD-10 level. However, 30% of the original statements of death were based on autopsy results, which likely increased the accuracy of the original COD.¹⁷ Although autopsy isthe gold standard for determining the underlying cause of death,^28,29 it is not a common practice in Mexico.²⁴ Although previous studies report higher percentage agreements than those found in the Sonora data by the present study, trends by cause of death are similar, with high rates of agreement for neoplasms and lower levels of agreement for hypertension related deaths.³⁰

Currently, the medical record is the best source of data for cause of death studies in Hermosillo. In the present study, 83% of hospital medical charts had sufficient data to identify a confirmed or probable underlying cause of death. Agreement between the underlying COD reported by the vital registry system and the underlying COD determined by trained physician reviewers varied widely across ICD-10 chapters. Excellent agreement and sensitivity was found for neoplasms, diseases of the nervous system, cerebrovasculardisease and ischemic heart disease. Poor agreement was found for diabetes mellitus, hypertensive disease and acute upper and lower respiratory disease. A recent study in China also reported high sensitivities for cancers and low sensitivity for diabetes mellitus and hypertensive diseases.⁷ In contrast to our findings, two large community cohort studies found overestimation of coronary heart disease,^3,4 while a study in Taiwan and another in the Mexico/US border region of the United States observed a low rate of agreement for cerebral infarction.^30,31

Nonetheless, the net change in mortality fraction after physician review was minimal across all chapters, as mutual misclassification occurred among common co-morbidities. This "compensatory effect of errors" has been documented in some studies,^30,32 but contrasts with earlier studies that consistently reported over-estimation for specific categories of disease, such as coronary heart disease.^5,10 Misclassification between common co-morbidities, such as diabetes mellitus and circulatory disease was common.

COD analyses can have profound effects on policy makers' understanding of the cause-specific mortality burden in a population, and may result in misguiding public health interventions. Nonetheless, identifying the principal cause of death among one or more competing causes or co-morbidities has been identified as a barrier to accurate death certification.^33,34 Mackenbach et al. report that competing causes were most often present for respiratory disease and cardiovascular disease, two categories with low agreement in our study.³⁵ Much of this misclassification, with particular reference to diabetes mellitus, has been attributed to coder preference for particular ICD rules.^30,36 Investigations of specific categories of disease that have common co-morbidities or competing causes of death should consider a multiple cause of death analysis, which uses both the underlying and contributing causes of death as recorded on the death certificate.³⁷ Using multiple causes of death to identify "stroke-related" deaths increased population stroke mortality estimates by 100%.³⁸

A common fallacy in cause of death studies is the lack of reproducibility of the research methods, including the failure to use a standard form for medical chart reviews.²⁷ In order to increase potential reproducibility, this study mimicked the process by which the official COD is assigned by the Sonoran vital registry. By utilizing a systematic approach to review medical charts, including a standardized abstraction form, our study demonstrated good overall agreement for underlying COD (88%) between independent physician reviewers, higher than that reported in previous studies.^38,39

A total of 17% of medical charts could not be located. Missing medical records may limit the generalizability of the study, as the younger and uninsured patients were more likely to have missing charts and be excluded from the study. Specialization of the hospital has previously been shown to be a determinant of death certification quality.³⁵ Uninsured and younger patients may be less likely to have sought care previously, are more likely to present with acute, late-phase illness and may have minimal information in their medical chart.

This study did not consider out-of-hospitaldeaths. Studies of out-of-hospital deaths due to coronary heart disease have shown that the accuracy of death certification for said deaths is lower than that of in-hospital deaths.⁴⁰ Future research should include assessment of out-of-hospital deaths and considermethodologies such as physician or family interviewsto improve the quality of the cause of death coding forthese deaths.

While this study assessed cause of death coding at the local level in the Sonoran registry, mortality statistics are subject to final coding before publication at the national level. Since national statistics are most widely used, future studies might incorporate the final cause of death as determined at the national level in order to increase external validity of comparisons.

While the vital records system in Hermosillo provides complete adult mortality data, some inaccuracies in the coding of the underlying cause of death statements exist that may negatively affect reporting of mortality statistics. Suggested improvements to the system include increasing physician training (in 2001, 25.5% of Sonoran physicians reported formal training in death certification),⁴¹ querying of ill-defined causes of death, installing an automated classification program and strengthening the research capacity in the region to effectively use, analyze and distribute data from the vital registry system.

Regular analysis and dissemination of data from vital registries, on both quality and prevalence, is crucial for increasing physician and policy makers' awareness of the importance of the vital registry system and its role in improving health systems.² Mortality statistics are one of the few digitalized, systematically-collected health databases available to researchers tracking the health of populations in the northern Mexico border region. Quality vital registry systems are an essential component for evidence-based health policy decisions that foster sustainable development. This research shows that, while caution is recommended for use of vital registry statistics in Sonora for analysis by specific diseases (ICD-10 2- and 3-digit codes), the ICD-10 chapter level underlying cause of death codes can be used to estimate the disease burden in the population.

Acknowledgements

The authors would like to acknowledge the cooperation of the Secretary of Public Health of the state of Sonora in the completion of this study.

Declaration of conflict of interests: The authors declare that they have no conflict of interests.

References

1. Mathers CD, Ma Fat D, Inoue M, Chalapati R, Lopez AD. Counting the dead and what they died from: an assessment of the global status of cause of death data. Bull World Health Organ 2005;83:171-7.         [ Links ]

2. Lozano R. Burden of disease assessment and health system reform: Results of a study in Mexico. J Int Dev 1995;7:555-63.         [ Links ]

3. Coady SA, Sorlie PD, Cooper LS, Folsom AR, Rosamond WD, Conwill DE. Validation of death certificate diagnosis for coronary heart disease: the Atherosclerosis Risk in Communities (ARIC) Study. J Clin Epidemiol 2001;54:40-50.         [ Links ]

4. Lloyd-Jones DM, Martin DO, Larson MG, Levy D. Accuracy of Death Certificates for Coding Coronary Heart Disease as the Cause of Death. Annals of Int Med 1998;129:1020-6.         [ Links ]

5. Thomason MJ, Biddulph JP, Cull CA, Holman RR. Reporting of diabetes on death certificates using data from the UK Prospective Diabetes Study. Diabetes Med 2005;22:1031-6.         [ Links ]

6. Stalioraityte E, Pangonyte D, Kazlauskaite D. Reliability of cause-specific mortality rate statistics: case of Lithuania. Public Health 2005;119:799-807.         [ Links ]

7. Rao C,Yang G, Hu J, Ma J, Xia W, Lopez AD.Validation of cause-of-death statistics in urban China. Int J Epidemiol 2007;36:642-51.         [ Links ]

8. Morton L, Omar R, Carroll S, Beirne M, Halliday D,Taylor KM. Incomplete and inaccurate death certification--the impact on research. J Public Health Med 2000;22:133-7.         [ Links ]

9. D'Amico M,Agozzino E, Biagino A, Simonetti A, Marinelli P. Ill-Defined and multiple causes on death certificates - A study of misclassification in mortality statistics. Eur J Epidemiol 1999;15:141-8.         [ Links ]

10. Mackenbach JP, Kunst AE, Lautenbach H, Oei YB, Bijlsma F. Competing causes of death:A death certificate study. J Clin Epidemiol 1997;50:1069.         [ Links ]

11. Lahti RA, Penttila A. Cause-of-death query in validation of death certification by expert panel; effects on mortality statistics in Finland, 1995. Forensic Sci Int 2003;131:113-24.         [ Links ]

12. Franca E, de Abreu DX, Rao C, Lopez AD. Evaluation of cause-of-death statistics for Brazil, 2002-2004. Intl J Epidemiol 2008;37:891-901.         [ Links ]

13. Department of Health and Human Services: Centers for Disease Control and Prevention's National Center for Health Statistics. Physicians' handbook on the medical certification of death. Report No. 2003-1108. Hyattsville, Maryland: DHHS, CDCP, 2003.         [ Links ]

14. Landis JR, Koch GG.The measurement of observer agreement for categorical data. Biometrics 1977;33.         [ Links ]

15. Johansson LA,Westerling R, Rosenberg HM. Methodology of studies evaluating death certificate accuracy were flawed. J Clin Epidemiol 2006;59:125.         [ Links ]

16. Schnitman A.Análise da fidedignidade da declaração da causa básica de morte por câncer em Salvador, Brazil. Rev Saude Publica 1990;24:490-6.         [ Links ]

17. Fonseca LAM, Laurenti R. Qualidade da certificação médica da causa de morte em São Paulo, Brasil. Rev Saude Publica 1974;8:21-9.         [ Links ]

18. Laurenti R, Jorge MH, Gotlieb SLD. Mortalidade segundo causas: considerações sobre a fidedignidade dos dados. Pan Am J Public Health 2008;23:349-56.         [ Links ]

19. Dirección General de Información y Evaluación del Desempeño. Secretaría de Salud.México. Estadísticas de mortalidad en México: muertes registradas en el año 2003. Salud Publica Mex 2005;47:171-87.         [ Links ]

20. Alvarez G, Harlow SD, Denman C, Hofmeister MJ. Quality of causeof-death statements and its impact on infant mortality statistics in Hermosillo, Mexico. Rev Panam Salud Publica 2009;25:120-7.         [ Links ]

21. Secretaria de Salud. Programa de Acción: Sistema Nacional de Información en Salud 2001-2006. Mexico DF: Coordinación General de Planeación Estratégica, Dirección General de Información y Evaluación del Desempeño, 2002.         [ Links ]

22. Secretaria de Salud. Proyecciones de Población sin Seguridad Social, Según Municipio CML, Jurisdicción y Edad Quinquenal, Sonora, 2006. Mexico: Dirección General de Información en Salud, 2006.         [ Links ]

23. Lu TH, Lee MC, Chou MC.Accuracy of cause-of-death coding in Taiwan: types of miscoding and effects on mortality statistics. Int J Epidemiol 2000;29:336-43.         [ Links ]

24. Instituto Nacional de Estadística y Geografía. Principles resultados del xii censo general de poblacion y vivienda.Aguascalientes, Mexico: INEGI, 2000.         [ Links ]

25. Mathers CD, Ma Fat D, Inoue M, Chalapati R, Lopez AD. Counting the dead and what they died from: an assessment of the global status of cause of death data. Bull World Health Organ 2005;83:171-7.         [ Links ]

26. Lozano R. Burden of disease assessment and health system reform: Results of a study in Mexico. J Int Dev 1995;7:555-63.         [ Links ]

27. James G, Patton RE, Heslin AS.Accuracy of cause-of-death statements on death certificates. Public Health Report 1955;70:39-51.         [ Links ]

28. Myers KA, Farquhar DRE. Improving the accuracy of death certification. Can Med Assoc J 1998;158:1317-23.         [ Links ]

29. Cirera Suárez L, Martínez López C, Contreras Gil J, Navarro Sánchez C.Aprendizaje y satisfacción de los talleres de pre y postgrado de medicina para la mejora en la certificación de las causas de defunción, 1992-1996. Rev Esp Salud Publica 1998;72:185-95.         [ Links ]

30. Lu TH, Shih TP, Lee MC, Chou MC, Lin CK. Diversity in death certification:A case vignette approach. J Clin Epidemiol 2001;24:1086-93.         [ Links ]

31. Brown DL,Al-Senani F, Lisabeth LD, Farnie MA, Colletti LA, Langa KM, et al. Defining Cause of Death in Stroke Patients:The Brain Attack Surveillance in Corpus Christi Project. Am J Epidemiol 2006;165(5):591-6.         [ Links ]

32. Franco LJ, Mameri C, Pagliaro H, Iochida LC, Goldenberg P. Diabetes como causa básica ou associada de morte no Estado de São Paulo, Brazil, 1992. Rev Saude Publica 1998;35:237-45.         [ Links ]

33. Mant J,Wilson S, Parry J, Bridge P,Wilson R, Murdoch W, et al. Clinicians didn't reliably distinguish between different causes of cardiac death using case histories. J Clin Epidemiol 2006;59:862-7.         [ Links ]

34. Smith Sehdev A, Hutchins G. Problems with Proper Completion and Accuracy of the Cause-of-Death Statement.Arch Intern Med 2001;131:277-84.         [ Links ]

35. Cordeiro R, Olivencia Peñaloza ER, Cardoso CF, Cortez DB, Kakinami E, Souza JJGd, et al.Validade das informações ocupação e causa básica em declaraçoes de õbito de Botucatu, São Paulo. Cad Saude Publica 1999;15:719-28.         [ Links ]

36. Johansson LA,Westerling R. Comparing Swedish hospital discharge records with death certificates: implications for mortality statistics. Int J Epidemiol 2000;29:495.         [ Links ]

37. Nashelsky MB, Lawrence CH.Accuracy of cause of death determination without forensice autopsy examination.Am J Forensic Med Pathol 2003;24:313-9.         [ Links ]

38. Jougla E, Papoz L, Balkau B, Maguin P, Hatton F.The Eurodiab Subarea CSG. Death Certificate Coding Practices Related to Diabetes in European Countries--The 'EURODIAB Subarea C' Study. Int J Epidemiol 1992;21:343-51.         [ Links ]

39. Carr JS, Scott SM,Wagner BM.The Necessary Role of the Autopsy in Cardiovascular Epidemiology. Human Pathol 1998;29:1469-79.         [ Links ]

40. Fox CS, Evans JC, Larson MG, Lloyd-Jones DM, O'Donnell CJ, Sorlie PD, et al.A comparison of death certificate out-of-hospital coronary heart disease death with physician-adjudicated sudden cardiac death.Am J Cardiol 2005 Apr 1;95(7):856-9.         [ Links ]

41. Dirección General de Información y Evaluación del Desempeño. Secretaría de Salud.México. Estadísticas de mortalidad en México: muertes registradas en el año 2001. Salud Publica Mex 2002;(6)44.         [ Links ]

Correspondence author:
Mary H. Freire de Carvalho, MPH, PhD.
4900 Floyd #4 Houston TX, 77007 USA
E-mail: maryjohofmeister@gmail.com

Received on: August 2010
Accepted on: May 2011