Artículo original

 

Screening mexicans for psychosocial and behavioral problems during pediatric consultation

 

Detección de problemas psicosociales y de comportamiento en población pediátrica originaria de México

 

Marie A. Leiner,^{*, ****} Hector Balcazar,^** David C. Straus,^*** Pratibha Shirsat,* Gilbert Handal*

 

* Department of Pediatrics. Texas Tech University Health Sciences Center- El Paso, Texas,

** El Paso Regional Campus. UT School of Public Health at Houston,

*** Department of Microbiology and Immunology, Texas Tech University Health Sciences Center, Lubbock, Texas,

**** El Colegio de Chihuahua. Ciudad Juarez, Chih.

 

Correspondence and reprint request:
Marie A. Leiner Ph.D.
Department of Pediatrics.
Texas Tech Univeristy Health Sciences Center
El Paso, Texas
E-mail: marie.leiner@ttuhsc.edu

Recibido el 7 de diciembre de 2006.
Aceptado el 15 de febrero de 2007.

 

ABSTRACT

Background. Psychosocial and behavioral problems should be detected early during the pediatric consultation to prevent more severe psychosocial impairment as children move into adolescence. The pediatric visit offers an excellent opportunity to assess possible problems using screening tools. The aim of this study is to assess whether an adaptation of a screening tool (Pediatric Symptom Checklist) using visual aids is valid and suitable for the early detection of psychosocial problems among a sample of Mexican children and adolescents.

Methods. We included 411 of 468 (87.8%) Mexican mothers or female caretakers of children and adolescents aged 4-16 undergoing routine health assessments in two Preventive Child Health Clinical Services in El Paso, Texas, USA. Mothers or female caretakers completed the Pictorial Pediatric Symptom Checklist and a psychological comprehensive test known as the Child Behavior Checklist (CBCL). Sensitivity and specificity was established using the CBCL as criteria for the validity of the PPSC.

Results. The PPSC screening tool performed well as demonstrated by its detection rates of psychosocial and behavioral problems among the sample of Mexican children. Sensitivity and specificity at the standard cut-off scores was 69.6% and 95.2% for children 4-5 years old and 61.8% and 91.8% for children 6-16 years old respectively. The results were slightly lower than those found using the PSC among other ethnic groups in the United States, with a sensitivity and specificity of 71.7% and 93.0%, respectively.

Conclusions. The PPSC is a simple, effective tool that can detect on average more than 65% of the children and adolescents with possible psychosocial problems during pediatric consultations. This free screening tool detected psychosocial problems in pediatric populations of Mexican origin living on the border and may be also effective for Mexican children living in Mexico.

Key words. Mental Health. Pediatrics. Screening. Pictorials.

 

RESUMEN

Antecedentes. Los problemas psicosociales y de comportamiento deben ser detectados durante la consulta pediátrica para prevenir problemas irreparables y severos cuando el niño pasa a la adolescencia. La visita pediátrica ofrece una excelente oportunidad para detectar posibles problemas utilizando "pruebas rápidas" de diagnóstico. El objetivo de este estudio es determinar la validez diagnóstica de una adaptación de una prueba rápida conocida como "Pediatric Symptom Checklist" utilizando apoyos visuales para la detección de problemas psicosociales en una muestra de niños y adolescentes mexicanos.

Métodos. Se trabajó con una muestra de 411 (87.8%) madres mexicanas de niños y adolescentes de 4 a 16 años, asistiendo a servicios preventivos en dos clínicas en El Paso, Texas, Estados Unidos. Las madres completaron la adaptación de la prueba rápida "PPSC" y una prueba psicológica comprensiva conocida como el "Child Behavior Checklist". Se utilizó el método de sensibilidad y especificidad para establecer la validez diagnóstica del PPSC.

Resultados. La prueba rápida del PPSC se comportó favorablemente como se muestra en los resultados de detección de problemas psicosociales y de comportamiento en la muestra de niños y adolescentes. Los puntos de corte convencionales para el PPSC dieron una sensibilidad de 69.6% y especificidad de 95.2% para los niños en edades 4 a 5 años y de 61.8% y 91.8% para las edades de 6 a 16 años, respectivamente. Los resultados cuando se comparan con los reportados en otros grupos étnicos en los Estados Unidos (sensibilidad 71.7% y especificidad 93.0%) son ligeramente menores.

Conclusiones. La prueba rápida del PPSC es una herramienta sencilla y efectiva que detectó en promedio más de 65% de los niños y adolescentes con posibles problemas psicosociales durante la consulta pediátrica. Esta "prueba rápida" que es gratuita detectó problemas psicosociales en poblaciones pediátricas de origen mexicano viviendo en la frontera y puede ser efectiva para niños y adolescentes que viven en México.

Palabras clave. Detección. Problemas psicosociales. Pictoriales. Pruebas psicológicas rápidas.

 

INTRODUCTION

Early detection and treatment of psychosocial problems may prevent more severe psychosocial impairment as children move into adolescence. Most available literature states that systematic behavioral screening should be implemented for routine use in pediatric settings. However, some debate exists on how and when it should be done, and what instruments are optimal.¹

The use of systematic screening for behavioral concerns during routine child health supervision faces several challenges and barriers embedded in a multidimensional context. On one hand, the pediatrician or primary care physician may lack adequate training² and time during the office visit.³ On the other hand, the parent may be hesitant to discuss psychosocial or mental problems due to cultural, educational, and/or literacy issues.⁴ Finally, evidence suggests that only 50% of children referred to a mental health professional actually attend the first appointment, despite their socioeconomic status, which may be an additional limitation for further referral.⁵

The need for early detection and prevention of psychosocial and behavioral problems has been proposed for more than three decades by primary care physicians and professional organizations.^6-8 Traditionally, the systematic use of behavioral screening measures completed by parents has been used successfully by child mental health professionals (psychologists, psychiatrists, counselors, etc). Although these mental health professionals have used behavior rating scales derived from questionnaires, or screening instruments completed by a parent about the child's problematic behaviors, this practice remains uncommon in pediatric settings. Evidence suggests that pediatricians can correctly identify more problems when provided with a classification system that is more appropriate for primary care.² Thus, screening tools that can be easily scored and give a positive/negative result are much easier to interpret than those that give different syndromes or categorized problems that require further interpretation.

There are several available screening tests to identify behavioral problems that are simple to use, free-of-charge, and require no professional expertise to score.⁹ Among these, the Pediatric Symptom Checklist (PSC) is recommended to facilitate the recognition of cognitive, emotional, and behavioral problems during the pediatric consultation. The sensitivity and specificity of the PSC as reported by the authors indicate that the PSC has a sensitivity of 0.68 and a specificity of 0.95.¹⁰

The PSC has demonstrated consistent results to detect psychosocial and behavioral problems in different pediatric populations in the United States^11-20 and other countries.¹⁰ However, in the United States, among Hispanic populations, including Mexican-Americans, this screening tool has typically yielded unexpected substantially lower rates of detection.^14,21 Even though this ethnic group is considered at a higher risk for psychosocial and behavioral problems prevalence studies indicate rates that vary from 1 to 7%.^14,22 Mexican/Mexican American pediatric patients living on the border between Mexico and the United States reported a prevalence of 5 to 7% using the PSC in a previous study.²³

We hypothesized that the lower detection of problems among Hispanics using the original PSC could be attributed to the format of the instrument itself. The format might lack elements that capture the attention of the parent/caretaker which are required to facilitate accurate and precise responses.

The PSC is composed of one page containing 35 questions arranged in columns.

Questions are related to children's emotional and behavioral problems that reflect parents' impressions of their children's psychosocial functioning. Responses use a scale: 0 = not true; 1 = somewhat or sometimes true; 2 = very true or often true. We converted this written questionnaire into a "user-friendly" format by adding visual descriptions (pictorials) to each one of the questions (see sample of PSC and adapted PSC Figure 1). The original content of the PSC (questions) was kept intact. We added pictorials to make the questionnaire visually attractive, and keep the attention of the parents. This was accomplished by mapping words to pictures in a coordinated way, which has been demonstrated to be more effective in promoting creative problem solving than separate verbal and visual explanations.

The purpose of this study was to evaluate the use of the Spanish version of the pictorial PSC in identifying behavioral problems of children in a population of Mexican children attending well-child and non-emergency visits in university-based clinical settings.

 

METHODS

Setting

This study was conducted at two university-based clinics in El Paso, Texas, USA border city with Ciudad Juarez, Chihuahua, Mexico from December 2004 to December 2005.

 

Sample

The population consisted of Mexican mothers or female caretakers of Mexican pediatric patients attending the clinics for non-emergency, well-child visits.

A total of 468 Mexican mothers or female caretakers (ages 17 years and older) of eligible children and adolescents (ages 4-16 years) were invited to participate in this study. During registration of the pediatric patient, a bilingual researcher invited mothers or female caretakers that indicated to be born in Mexico to participate in this study using the following inclusion criteria: a) the mother or female caretaker had children that were attending clinic for a non-emergency or well-child consultation, b) the age of children was between 4 to 16 years old, c) the children were NOT previously diagnosed with either a chronic, mental, neurological, life threatening disease or disability d) that wanted to participate in the study.

The population attending these two clinics included pediatric patients living in the vicinities of El Paso, Texas, USA, of lower socioeconomic status. The majority of the parents of patients attending the clinic are of Hispanic origin (76%) and are composed of Mexican and Central or South American immigrants. For this study, only mothers or female caretakers that were born in Mexico were included in the study to avoid bias due to the respondent's gender, because fathers attend clinic services less frequently. The study was conducted over a 13-month period and mothers/female caretakers participated only once during this period.

 

Measurements

Mothers or female caretakers completed two questionnaires; the Pictorial Pediatric Symptom Checklist (PPSC) and the reference standard or "gold standard" the Child Behavior Checklist (CBCL).

 

Pictorial Pediatric Symptom Checklist

The PPSC is an adaptation using pictorials of the Pediatric Symptom Checklist (PSC) developed by Dr. Michael Jellinek and Dr. Michael Murphy.²⁴ The PSC is a standardized, brief screening questionnaire that is used by pediatricians and other healthcare professionals to improve the detection and treatment of psychosocial problems in children ages 4-16 years. This free, brief questionnaire requires only less than ten minutes for the parent/caretaker to complete and evaluates the functioning of the child in several psychosocial areas: feelings, behavior, family, school, and friends. The screening tool is not designed to make a particular diagnosis, but instead, serves as an indicator of possible problems that require further evaluation. The PSC has been referenced in a large number of studies, including those using the same minority group that was used in the present study.^{14,15,18,25-28} The PSC contains 35 short statements of problem behaviors, including externalizing (conduct, attention, etc.) and internalizing (depression, anxiety, etc.) behaviors. The parent/caretaker rates each item for how truly it describes the child (as far as the parent/caretaker knows), using the following scale: 0 = not true; 1 = somewhat or sometimes true; 2 = very true or often true. A total score is calculated by adding the individual scores for each item, yielding a range of possible scores from 0 to 70 points. For school-aged children 6-16 years, a total score of 28 or higher indicates significant psychosocial impairment. For children ages 4-5 years, the scores on items 6, 7, 14, and 15 are ignored and a total score of 24 points or higher (based on the 31 remaining items) indicates a psychosocial impairment. Spanish and English language versions of this screening tool are available.

 

Child Behavior Checklist

This standardized screening tool has been used as the reference standard or "gold standard" in numerous studies in different populations, including Hispanics²⁹ and other low-income minority populations.^{12,14,15,30-32} The CBCL was chosen because this in-depth screening instrument has shown good internal consistency to measure the PSC and has been used with Hispanics populations.²⁹

The Child Behavior Checklist CBCL³³ is a self-response parent/caretaker questionnaire that is applicable to children ages 18 months to 18 years and is widely used in the clinical assessment of children's behavior problems. The CBCL can be completed in 15 to 20 minutes and is available for use in children ages 18 months to five years (110 questions) or children ages 6-18 years (120 questions).

Behaviors are scored on a 3-category Likert scale. Both forms of the CBCL consist of two broadband factors of behavioral problems: internalizing and externalizing. The CBCL for ages 6-16 years old gives information about eight syndrome scales grouped into externalizing behaviors (aggressive and delinquent) and internalizing behaviors (anxious/depressed, somatic complaints, and withdrawn). The other three syndromes are social, thought, and attention problems. The CBCL for children ages 18 months to 5 years gives information about 7 syndrome scales grouped into externalizing behaviors (attention and aggression) and internalizing behaviors (emotionally reactive, anxious depressed, somatic complaints and withdrawn) and sleep problems. A total problems score is computed by summing all problem items. Higher scores are associated with more problem behaviors. Total scores of 60 are considered borderline and clinical cutoff points for the syndrome/total including externalizing, internalizing and total problems, are >63.

 

Data analysis

Demographic characteristics of the study population were described using standard indicators for the parent/caretaker and child. The sensitivity and specificity of the PPSC were calculated using the CBCL as the criterion standard. We assessed the validity of the PPSC using dichotomized values for the PPSC and CBCL based on recommended scores. PPSC was considered positive (presence of psychosocial and behavioral problems) with a score of 28 for 6-16 years, and 24 for 4-5 years respectively. CBCL was considered positive (presence of psychosocial and behavioral problems) if total problem score, internalizing score, or externalizing score were equal to or greater than 60.

A receiver operator characteristics (ROC) curve was performed, plotting all possible pairs of true-positive rates (sensitivity) against false-positives (1-specificity) to determine the best cut-off for the PPSC. The likelihood ratio was used as another indicator of screening performance. It incorporates both the sensitivity and specificity of the test and provides a direct estimate of how much a test result will change the odds of having a disease.

All analyses were done with SPSS 14.0 for Windows. The study was approved by the University Institutional Review Board.

 

RESULTS

Demographics

During the study period, 468 eligible mothers of children ages 4-16 years were invited to participate in the study. Only questionnaires responded to by a Mexican mother or female caretaker, and completed in Spanish were included in this study, leaving a total of 411 (87.8%) participants.

The average age of the children was 8.3 years (SD, 3.6 years). There were more girls, 226(55.0%) than boys 185(45.0%). The distribution of medical coverage for the participants was Medicaid (75.5%), State health Insurance Program (9.6%), private insurance (5.4%) participants was 9.4 years. There were no significant differences in participant's age, education, and children's insurance coverage.

 

Scores on PPSC and CBCL

Mean scores on the PPSC were similar for boys and for girls for both age groups (Table 1). The same was true for the CBCL scores. There were no significant differences in any of the total scores or scales internalizing, externalizing or total scores.

 

Prevalence of psychosocial and behavioral problems

According to the PPSC, the prevalence of psychosocial and behavioral problems for this sample was 16.5% (21 of 127, 95% CI: 10.0 to 23.1) for 4-5 year olds, and 16.2% (46 of 284, 95% CI: 11.9 to 20.5) for 6-16 year olds.

The CBCL detected 18.1% (23 of 127, 95% CI: 11.3 to 24.9) for 4-5 year olds, and 23.9% (68 of 284, 95% CI: 19.0 to 29.0) for 6-16 year olds.

 

Validity

The results of the sensitivity and specificity are described in figure 2. Using the PPSC as a screening device and the CBCL for the standard assessment, the sensitivity for detecting psychosocial and behavioral problems in 4-5 year olds was 69.6% (95% CI 47.1-86.7), with specificity of 95.2% (95% CI 89.1-98.4) for 6-18 year olds, the sensitivity was 61.8% (95% CI 49.2-73.3) with a specificity of 98.1% (95% CI 95.3-99.5).

Using the "best cut-off points" as suggested by the ROC the best score to determine positive cases using the PPSC was for ages 4-5 years old score >17 yielding a sensitivity of 82.6% (CI = 61.2-94.9), and a specificity of 82.7% (CI = 74.0-89.4) with a LR+ of 4.77 and LR- of 0.21. For the 6-16 year old children, the optimal PPSC cut-off point was > 20, yielding a sensitivity of 83.8% (CI = 72.9-91.6) and specificity of 85.2% (CI = 79.7-89.6), and LR+ of 5.66 and LR- of 0.19 (Table 2).

As a rough guide, the accuracy of a diagnostic test can be considered excellent when the ROC values are between 0.90 and 1.00. The ROC using the CBCL as a criterion was 0.91(95% confidence interval 0.85 to 0.95) for age 4-5 year olds and 0.93(95% confidence interval 0.90 to 0.96) for ages 6-16 year olds. ROC curves for the PPSC were close to the upper-left corner, indicating a high validity.

 

DISCUSSION

While some mental health problems originate from exposure to behavioral risks later in life,^34,35 many common chronic and mental health conditions arise during childhood.^36,39 All mental, emotional, and behavioral disorders that may occur during childhood and adolescence can have a serious impact on the quality of life of children, their families, and society in general.^40-42

For pediatricians, the ability to accurately identify children with possible psychosocial problems using simple and valid screening tools is essential. Using screening tools that parents can easily understand is an important step when evaluating psychosocial and behavioral concerns in children. Adding pictorials to a screening tool to capture the attention of the parent and perhaps improving the understanding of the task (in this case to improve health literacy) is a promising area of clinical practice. The success of these pictorial descriptions is attributed to their flexibility to connect actions and events in order to describe or explain procedures.⁴³ We believe our PPSC is a promising screening tool to be used with -Mexican families.

The results on the PPSC showed good internal consistency and validity using the CBCL as a reference standard. Based on our results, we found that by using slightly lower cut-off scores (when compared to the cut-off scores suggested by the PSC authors), it was possible to achieve similar results previously attained among other ethnic groups. In fact, a study using the PSC with a high percentage of Mexican American participants suggested the best cut-off score to be at 12 for 6-16 year olds (sensitivity 0.74; specificity 0.94) [14]. Our adaptation (PPSC) using pictorials indicates that the a cut-off point of =>23 for ages 4-5 and =>26 for ages 6-16 can detect problems in 69.6% of the 4-5 year old children and 61.6% of the 6-16 year old children. This finding indicates that the use of our PPSC with a more moderate reduction of the suggested cut-off scores gives similar detection results. We feel that using the original cut-off scores suggested by the authors of 24 for 4-5 years old and 26 for 6-16 may be a good start for future studies with Mexican children living in Mexico.

The PPSC is a simple, effective tool that can detect on average more than 65% of the children and adolescents with possible psychosocial problems during pediatric consultations. Even though among Hispanics this tool has reported an unexpected lower detection, in our study, the adaptation using pictorials PPSC seemed to be effective in a Mexican sample and detected children who would otherwise be missed. The PPSC is available at www.dbpeds.org and the original PSC is available at psc.partners.org.

It is important to consider some limitations when applying the screening tool. This tool was tested with Mexican mothers or female caretakers and their children living in the United States and results may be different with Mexican mothers and children living in Mexico. Wide confidence intervals around the estimates indicate that the sample was small. In addition, the PPSC does not cover the same array of behavioral scales (e.g. internalizing, externalizing, etc.) that can be elicited using the CBCL instrument. In addition, only the CBCL was used as a measure for psychosocial problems instead of clinical interviews because of complexity and high cost.

Like the PSC, the PPSC is not a diagnostic tool. It is a screening tool and should be used with the knowledge that although more children will be detected according to the cut-off points (sensitivity), some children will not be detected (specificity), or will be misclassified. The PPSC should be used as a screening tool to help pediatricians detect possible problems during the pediatric visit. The application of the instrument does not require an excessive amount of time and can help pediatricians to provide further referral as needed.

 

REFERENCES

1. Perrin E, Stancin T. A continuing dilemma: whether and how to screen for concerns about children's behavior. Pediatr Rev 2002; 23(8): 264-76.        [ Links ]

2. Horwitz SM, et al. Identification and management of psychosocial and developmental problems in community-based, primary care pediatric practices. Pediatrics 1992; 89(3): 480-5.        [ Links ]

3. Halfon N, et al. Assessing development in the pediatric office. Pediatrics 2004; (6 Suppl): 1926-33.        [ Links ]

4. Sen B. Adolescent propensity for depressed mood and help seeking: race and gender differences. J Ment Health Policy Econ 2004; 7(3): 133-45.        [ Links ]

5. Stein MA, et al. Sleep and behavior problems in school-aged children. Pediatrics 2001; 107(4): E60.        [ Links ]

6. Lewis M. Child psychiatric consultation in pediatrics. Pediatrics 1978; 62(3): 359-64.        [ Links ]

7. Dodge WF, et al. Pediatric practice: patterns of change. Tex Med 1978; 74(7): 49-54.        [ Links ]

8. Arredondo A, Ramos R, Zuniga A. Economic evaluation of the demand of medical care for mental health in Mexico: schizophrenia and depression, 1996-2000. Rev Invest Clin 2003; 55(1): 43-50.        [ Links ]

9. Glascoe FP. Early detection of developmental and behavioral problems. Pediatr Rev 2000; 21(8): 272-9; quiz 280.        [ Links ]

10. Reijneveld SA, et al. Use of the Pediatric Symptom Checklist for the detection of psychosocial problems in preventive child healthcare. BMC Public Health 2006; 6: 197.        [ Links ]

11. Gardner W, Kelleher KJ, Pajer KA. Multidimensional adaptive testing for mental health problems in primary care. Med Care 2002; 40(9): 812-23.        [ Links ]

12. Borowsky IW, Mozayeny S, Ireland M. Brief psychosocial screening at health supervision and acute care visits. Pediatrics 2003; 112(1 Pt 1): 129-33.        [ Links ]

13. Wren FJ, Bridge JA, Birmaher B. Screening for childhood anxiety symptoms in primary care: integrating child and parent reports. J Am Acad Child Adolesc Psychiatry 2004; 43(11): 1364-71.        [ Links ]

14. Jutte DP, et al. Use of the Pediatric Symptom Checklist in a low-income, Mexican American population. Arch Pediatr Adolesc Med 2003; 157(12): 1169-76.        [ Links ]

15. Navon M, et al. Use of the pediatric symptom checklist in strategies to improve preventive behavioral health care. Psychiatr Serv 2001; 52(6): 800-4.        [ Links ]

16. Gall G, et al. Utility of psychosocial screening at a school-based health center. J Sch Health 2000; 70(7): 292-98.        [ Links ]

17. Anderson DL, et al. Use of the pediatric symptom checklist in the pediatric neurology population. Pediatr Neurol 1999; 20(2): 116-20.        [ Links ]

18. Jellinek MS, et al. Use of the Pediatric Symptom Checklist to screen for psychosocial problems in pediatric primary care: a national feasibility study. Arch Pediatr Adolesc Med 1999; 153(3): 254-60.        [ Links ]

19. Murphy JM, et al. The family APGAR and psychosocial problems in children: a report from ASPN and PROS. J Fam Pract 1998; 46(1): 54-64.        [ Links ]

20. Murphy JM, et al. Utility of the Pediatric Symptom Checklist as a psychosocial screen to meet the federal Early and Periodic Screening, Diagnosis, and Treatment (EPSDT) standards: a pilot study. J Pediatr 1996; 129(6): 864-69.        [ Links ]

21. Pasick RJ, et al. Quality of data in multiethnic health surveys. Public Health Rep 2001; 116(Suppl 1): 223-43.        [ Links ]

22. Pagano M, et al. Screening for psychosocial problems in 4-5-year-olds during routine EPSDT examinations: validity and reliability in a Mexican-American sample. Clin Pediatr (Phila) 1996; 35(3): 139-46.        [ Links ]

23. Leiner MA, Shirsat P. Improving Detection of Psychological Problems in a Hispanic Population Using a Pictorial version of the PSC. Developmental and Behavioral News 2005; 14(2) Autumn 2005 (cited). Available from: http://wwww.dbpeds.org/ articles/detail.cfm?TextID=612.        [ Links ]

24. Jellinek M, Murphy M, Pediatric Symptom Checklist 2006.        [ Links ]

25. Murphy JM, et al. Screening for psychosocial dysfunction in pediatric practice. A naturalistic study of the Pediatric Symptom Checklist. Clin Pediatr (Phila) 1992; 31(11): 660-7.        [ Links ]

26. Murphy JM, et al. Screening for psychosocial dysfunction in innercity children: further validation of the Pediatric Symptom checklist. J Am Acad Child Adolesc Psychiatry 1992; 31(6): 1105-11.        [ Links ]

27. Murphy JM, Jellinek M, Milinsky S. The Pediatric Symptom Checklist: validation in the real world of middle school. J Pediatr Psychol 1989; 14(4): 629-39.        [ Links ]

28. Jellinek M, et al. The Pediatric Symptom Checklist. Support for a role in a managed care environment. Arch Pediatr Adolesc Med 1995; 149(7): 740-6.        [ Links ]

29. Rubio-Stipec M, et al. The internal consistency and concurrent validity of a Spanish translation of the Child Behavior Checklist. J Abnorm Child Psychol 1990; 18(4): 393-406.        [ Links ]

30. Duke N, Ireland M, Borowsky IW. Identifying psychosocial problems among youth: factors associated with youth agreement on a positive parent-completed PSC-17. Child Care Health Dev 2005; 31(5): 563-73.        [ Links ]

31. Backman ML, et al. Psychiatric symptoms of children and adolescents with juvenile neuronal ceroid lipofuscinosis. J Intellect Disabil Res 2005; 49(Pt 1): 25-32.        [ Links ]

32. Simonian SJ, Tarnowski KJ, Utility of the Pediatric Symptom Checklist for behavioral screening of disadvantaged children. Child Psychiatry Hum Dev 2001; 31(4): 269-78.        [ Links ]

33. Achenbach TM, RLA. Manual for the ASEBA Preschool and School-Age forms and Profiles. Burlington, VT: University of Vermont, Department of Psychiatry; 2001.        [ Links ]

34. Vineis P, et al. Environmental tobacco smoke and risk of respiratory cancer and chronic obstructive pulmonary disease in former smokers and never smokers in the EPIC prospective study. BMJ 2005; 330(7486): 277-89.        [ Links ]

35. Simonen RL, et al. Factors associated with exercise lifestyle—a study of monozygotic twins. Int J Sports Med 2003; 24(7): 499-505.        [ Links ]

36. Forrest CB, AW Riley. Childhood origins of adult health: a basis for life-course health policy. Health Aff (Millwood) 2004; 23(5): 155-64.        [ Links ]

37. Artiola IFL, et al. Research with spanish-speaking populations in the United States: lost in the translation. A commentary and a plea. J Clin Exp Neuropsychol 2005; 27(5): 555-64.        [ Links ]

38. Goodwin RD, Stein MB. Association between childhood trauma and physical disorders among adults in the United States. Psychol Med 2004; 34(3): 509-20.        [ Links ]

39. Blackwell DL, Hayward MD, Crimmins EM. Does childhood health affect chronic morbidity in later life? Soc Sci Med 2001; 52(4): 1269-84.        [ Links ]

40. Castañeda X. Promising outreach practices: enrolling low-income children in health insurance programs in California. Health Promot Pract 2003; 4(4): 430-8.        [ Links ]

41. Sawaya AL, et al. The link between childhood undernutrition and risk of chronic diseases in adulthood: a case study of Brazil. Nutr Rev 2003; 61(5 Pt 1): 168-75.        [ Links ]

42. Tsang HW, et al. Sources of burdens on families of individuals with mental illness. Int J Rehabil Res 2003; 26(2): 123-30.        [ Links ]

43. Fuller R, Dudley N, Blacktop J. How informed is consent? Understanding of pictorial and verbal probability information by medical inpatients. Postgrad Med J 2002; 78(923): 543-4.        [ Links ]