SciELO - Scientific Electronic Library Online

vol.58 issue6Satisfaction and perceived quality of people insured by the Social Health Protection in Mexico. Methodological foundationsHealth workers cohort study: methods and study design author indexsubject indexsearch form
Home Pagealphabetic serial listing  

Services on Demand




Related links

  • Have no similar articlesSimilars in SciELO


Salud Pública de México

Print version ISSN 0036-3634

Salud pública Méx vol.58 n.6 Cuernavaca Nov./Dec. 2016 

Artículos originales

The intellectual developmental disorders Mexico study: situational diagnosis, burden, genomics and intervention proposal

Estudio sobre trastornos del desarrollo intelectual en México: diagnóstico situacional, carga, genómica y propuesta de intervenciones

Eduardo Lazcano-Ponce, D en C1 

Gregorio Katz, MC1  2 

Rocío Rodríguez-Valentín, PhD1 

Filipa de Castro, PhD1 

Betania Allen-Leigh, PhD1 

María Elena Márquez-Caraveo, PhD3 

Miguel Ángel Ramírez-García, MC3 

Eduardo Arroyo-García, MC3 

María Elena Medina-Mora, PhD4 

Gustavo Ángeles, PhD5 

José Edmundo Urquieta-Salomón, PhD6 

Luis Salvador-Carulla, MC, PhD7 

1 Centro de Investigación en Salud Poblacional, Instituto Nacional de Salud Pública. Cuernavaca, México.

2 Departamento de Salud Mental, Facultad de Medicina, Universidad Nacional Autónoma de México. Ciudad de México, México.

3 Hospital Psiquiátrico Infantil Dr. Juan N. Navarro. Ciudad de México, México.

4 Instituto Nacional de Psiquiatría Ramón de la Fuente Muñiz. Ciudad de México, México.

5 Department of Maternal and Child Health, Gillings School of Global Public Health, University of North Carolina at Chapel Hill. North Carolina, USA.

6 Dirección de Indicadores y Análisis de Información Gubernamental, Instituto Nacional de Estadística y Geografía (Inegi). Ciudad de México, México.

7 Brain and Mind Centre, The University of Sydney. Sydney, Australia.



This study aims to generate evidence on intellectual development disorders (IDD) in Mexico.

Materials and methods:

IDD disease burden will be estimated with a probabilistic model, using population-based surveys. Direct and indirect costs of catastrophic expenses of families with a member with an IDD will be evaluated. Genomic characterization of IDD will include: sequencing participant exomes and performing bioinformatics analyses to identify de novo or inherited variants through trio analysis; identifying genetic variants associated with IDD, and validating randomly selected variants by polymerase chain reaction (PCR) and sequencing or real-time quantitative PCR (qPCR). Delphi surveys will be done on best practices for IDD diagnosis and management. An external evaluation will employ qualitative case studies of two social and labor inclusion programs for people with IDD.


The results will constitute scientific evidence for the design, promotion and evaluation of public policies, which are currently absent on IDD.

Keywords: intellectual developmental disorders; ADHD; autistic disorder; burden; Mexico



Esta investigación busca generar evidencia sobre trastornos del desarrollo intelectual (TDI) en México.

Material y métodos:

La carga de la enfermedad por TDI se estimará con un modelo probabilístico usando encuestas poblacionales. Se estimarán costos directos e indirectos de gastos catastróficos de familias con un integrante conTDI. La caracterización genómica deTDI incluirá secuenciar exomas, realizar análisis bioinformático para identificar variantes de novo o heredadas a través de análisis de tríos, identificar variantes genéticas asociadas con TDI, y validar variantes aleatoriamente seleccionadas con reacción en cadena de polimerasa y secuenciación o qPCR. Se harán encuestas Delphi sobre mejores prácticas de diagnóstico y manejo de TDI. Una evaluación externa empleará estudios cualitativos de caso de dos programas de inclusión social y laboral para personas con TDI.


Los resultados serán evidencia científica que podrá ser la base para el diseño, promoción y evaluación de políticas públicas, actualmente ausentes para TDI.

Palabras clave: trastornos del desarrollo intelectual; TDAH; trastorno autista; gasto; México

The Intellectual development disorders (IDD) Mexico study will combine epidemiological, clinical, economic and social science methods; the objective is to determine the true dimension of IDD as a public health problem in Mexico. The results of this study will constitute scientific evidence that can be the basis in order to design, promote and evaluate public policies, which are currently absent from public policy on IDD.

Intellectual developmental disorders in Mexico: a public health perspective

According to the 2014 National Survey of Demographic Dynamics in Mexico1 there are approximately 2 million people under 18 years old with serious IDD. Moreover, according to the first disability screening performed in Ensanut 2012, there is a 15% increased risk of IDD in children under 9,2 in whom development should be evaluated through a comprehensive approach regarding their cognitive, emotional and motor abilities and functioning in terms of social integration. IDD are a group of developmental conditions characterized by significant limitations in cognitive functions associated with learning abilities or disorders and adaptive behaviors. IDD, previously grouped under the term mental retardation, constitute a poorly studied public health problem in Mexico and Latin America, and their magnitude has not been quantified.3 Although the prevalence of IDD in children is not extremely high, its consequences can be devastating given the lifetime impairment of fundamental aspects of cognitive, motor and social functioning, as well as language impairments. Interventions focused on prevention, rehabilitation, community integration and labor inclusion are essential. IDD are a forgotten public health problem that are absent from legislative, health, education and legal field policies, and they do not benefit from governmental strategies of social development and poverty reduction. The diagnostic criteria for IDD have been the subject of ongoing international discussion, and significant discrepancies persist regarding issues such as scales and weights. A relevant clinical criterion for the previous characterization of IDD worldwide was their inclusion in the 10th version of the International Classification of Diseases.4 In this document, IDD are predominantly related to mental retardation of unknown cause without association to dysmorphism or physical anomalies. Currently, several experts believe that a definition of IDD including not only intelligence quotient but also the adaptive abilities of people with this disability is more useful.5 In addition to categorization by severity in terms of the intelligence quotient level, or development of adaptive abilities, IDD can also be grouped into syndromic IDD and non-syndromic IDD, although this distinction can be difficult and non-conclusive.6

Quantification of the population impact and determinants of IDD

Over the last decade, the rights and needs of persons with disabilities have received renewed consideration under the global agenda on population health and development, with the 2006 UN Convention on the Rights of Persons with Disabilities (CRPD),7 and a new bio-psycho-social approach to understand disability, functioning and participation.8

Within this paradigm shift, disability is understood as the result from the interaction between a person's impairment and obstacles and barriers in the environment. Contextual obstacles include legal, structural and physical barriers as well as prevailing attitudes that prevent participation in society. The more obstacles there are the more disabled a person becomes.

The availability of appropriate information, including statistical and research data that enable policy formulation and implementation related to disability, has also been recognized as an obligation within the 2006 convention. On the other hand, the agenda for the new Sustainable Development Goals requires the identification of the population experimenting functional difficulties and disability through censuses and surveys along data disaggregation according to key variables to address the barriers faced by persons with disabilities in exercising their rights.

Within the spectrum of disabling conditions, persons with IDD face the most severe restrictions to participation. Adequate epidemiological approaches are essential to identify the population at risk of not participating, estimate the magnitude and factors associated with these restrictions and generate recommendations of policies oriented at removing contextual barriers of all sorts.

The economic burden of IDD

Disability, whether physical or intellectual, is a condition that limits the interaction of a person with their environment and generates a series of economic and social costs that are assumed by the person that suffers from it, by their family and by society in general. These costs can be classified as direct and indirect. Some of these costs, as the World Report on Disability9 recently noted, could be reduced if barriers that prevent the adequate development of the person with the disability are removed.

Despite its importance, few studies have sought to quantify the economic burden associated with disability, even in developed countries. This lack of evidence is primarily a result of the limited scope of information available on the various components of the cost of disability, which adds to the limitations present in surveys and administrative records to properly operationalize a definition of disability.

There are several components of cost related to disability. There are direct and indirect costs, private costs and public expenditure on action programs targeted to the population that suffers from disabilities. According to the classification of the World Health Organization,9 direct costs can be grouped into two categories: the additional private cost that people with disabilities and their families must cover to ensure an acceptable standard of living that is comparable to people without this condition; and the cost that is covered by the government, through public programs, in providing benefits to people with disabilities.

Private costs are related to expenses associated with the demand for health services, purchase of medications and the use of other goods and services related to the care of the person with the disability; however, there are also indirect costs, both tangible and intangible, related to this health condition. One of the main indirect costs is related to the loss of productivity and work time, both from the people with the disability and from family members who are involved in their care, whose employment status results are often affected. To a large extent, this loss of well-being, in the medium and long term, results from the lack of accumulation of human capital, which in the life course of the affected people translates into a lower education, a lower likelihood of access to the job market and, in the absence of social protection mechanisms, a lack of access to health insurance, whether public or private. In the case of family members, costs are incurred by the reduction of working time or choice of occupations that allow flexibility to be able to care for the person with IDD. In some cases, the indirect costs also affect the education level of the people responsible for caring for the person with IDD. The few studies on the subject estimate that the aggregate economic impact due to disability may represent up to 6.7% of the GDP.10 In the case of IDD, a recent study documented that families absorb most of the economic burden, representing an opportunity cost of approximately 85%, even after discounting benefits received through social programs.11

In Mexico, data from the 2010 Population Census show that people with at least some mental limitations showed significant lags compared to the rest of the population in aspects such as coverage of medical insurance, employment status and educational attainment. For literacy alone, it was found that approximately 50% of people 15 years and older with some mental limitations were illiterate.12 An analysis in Mexico's poor population found that an inability to perform daily activities was associated with greater out-of-pocket expenses.13

Because of the economic and social involvement of IDD in the population's wellbeing, it is important to estimate the costs of IDD in Mexico. Understanding how the cost varies in different population groups, its composition and trends in recent years constitute essential inputs to improve public policy instruments that address the problem such that they become genuine social protection mechanisms that maximize the population's general wellbeing. In sum, there is a great need to estimate the magnitude, composition, distribution and change over time of the economic burden that IDD represent in families in Mexico.

Genetic determinants of IDD

IDD, also known as intellectual disability (ID), can be caused by genetic or non-genetic factors; however, is estimated that genetic causes are present in more than 50% of individuals with intellectual disability.14 Genetic causes of IDD can be chromosomic (e.g., Down Syndrome), recognizable genetic syndromes such as Fragile X, or innate errors in metabolism and current techniques allow genomic characterization. IDD represent a huge challenge for clinical geneticists in the search for molecular diagnosis because they are heterogeneous conditions; this can mean families are left without accurate genetic or reproductive counseling.

In a classical view, considering syndromic ID, individuals present with one or multiple dysmorphisms, severe anomalies or co-morbidities in addition to ID. While non-syndromic ID and ID of unknown cause have intellectual disability as the sole clinical feature. However, ruling out the presence of more subtle physical dysmorphisms, neurological anomalies and psychiatric disorders can be challenging.15

For example, it has been observed that IDD, coexist with autism spectrum disorders (ASD) in most patients.16 Around 25 and 70% of children with ASD have some level of ID,17 while at least 10% of individuals with ID have ASD. Additionally, it has been reported that both the ID and autism can coexist with attention deficit disorder (ADD, also known as attention deficit/ hyperactivity disorder or ADHD)16 or hyperkinetic disorders.4 Some studies have consistently shown high rates of ADHD in children with ID and vice versa.18,19 Besides, it has been estimated that around 14 and 78% of children with ASD also meet criteria for ADHD.20,21 Coexistence of these disorders results in further deterioration of cognitive, motor and social skills, further reducing quality of life of patients.

In addition to the comorbidities described above, it has been reported that children and adolescents with ASD and ADHD show an increased frequency of overweight and obesity.22,23 There is also evidence that the prevalence of obesity and overweight is greater in adolescents with ID than those without.23 Recently, a chromosomal deletion associated with autism, ID and overweight/obesity was identified,24 suggesting that genetic factors have a role in disorders such as obesity in individuals with IDD.

The genetic determinants of IDD are well recognized in specific syndromic forms such as chromosomal anomalies and single gene disruptions that contribute significantly to these disorders. Trisomy 21 (Down syndrome), for example, is considered the most important chromosomal cause of syndromic ID while FMR1 gene mutations (Fragile X syndrome) are the most common cause of inherited syndromic ID.25,26

On the other hand, progress in genetic research has allowed the identification of changes in genome at the level of a single nucleotide (SNV, single nucleotide variation) and copy number (CNV copy number variation), either deletions or duplications of DNA, associated with ID of unknown cause.27-29 It has been observed that single nucleotide mutations or CNVs implicated in ID are also associated with ASD, suggesting that genetic causes for IDD and ASD are similar.30 CNVs for ADD also have been identified, and overlap with the variants identified in ID and autism has been reported.31-33 Together, these observations support the participation of similar molecular processes and point out the contribution of genetic component in the development of these disorders.

Many groups have focused on identifying the specific genetic causes of ID and ASD. As a result, between 400 and 700 genes have been associated with these conditions, many of which converge on pathways involved in regulation of neuronal function.29,30,34-36 Particularly, it has been shown that most of the genes associated to ID and autism have a role in the control of formation and function of synapses.37,38 Interestingly, a biological systems study indicates that there are up to 4 000 genes that may contribute to neurodevelopmental and neuropsychiatric disorders39 suggesting that a large number of genes associated with disorders as ID and autism have not been identified yet.

New genomic technologies such as chromosomal microarray (e.g. microarray-based comparative genomic hybridization (array CGH)) and massive sequencing have improved the discovery of genetic variants with an etiologic role in IDD and other neurodevelopmental disorders.40-43 Chromosomal microarray analysis has been routinely utilized as a first-tier genetic test in patients with non-syndromic ID (providing a molecular diagnosis in 10 to 20% of cases);28,44 however, in the last years, the usage of exome sequencing has raised because it has proved to have the potential to increase molecular diagnosis yield27,43,45,46

The use of these genomic tools for a better understanding of the etiology of complex disorders such as IDD, including identification of genes, molecular mechanisms and pathways involved, is extremely important to provide a better diagnosis and treatment option, particularly in countries where these conditions are poorly studied, such as Mexico.

Materials and methods

This project is organized into two stages of work that include eight activities and their corresponding academic outputs (Figure 1). The proposed development of a 24-month project is described below.

Figure 1 Overview of study scheme. The phases, activities and respective products are described 

Systematic review of the literature

A protocol will be developed for the systematic extraction and classification of the available scientific literature in national and international databases on the subject of IDD in children under 10 years old. The identified literature will be classified from a quantitative and qualitative classification matrix in a previously designed Excel sheet, from which it will be possible to disaggregate information identified in the obtained literature in terms of theoretical, clinical, methodological and epidemiological aspects of IDD reported in the extracted articles. A descriptive analysis of the data obtained will be performed in Stata version 13.

Review and harmonization of databases

Potential data sources will be identified and reviewed to estimate the disease burden and quantify the population impact and determinants of IDD. Data sources will be explored, and study periods, strata, sub-groups and weightings included in the designs and population representativeness will be determined. The following data sources are proposed for review:

  1. National Survey on Perception of Disability 2010

  2. National Survey of Health and Nutrition 2012

  3. National Survey on Living Standards of Households 2002/2005/2012

  4. Survey for the Study of Global Aging and Adult Health 2009/2013

  5. Census of Schools, Teachers and Alumni of Basic and Special Education 2013

  6. National Survey of Household Income and Expenditures 2014

  7. National Survey of Boys, Girls and Women in Mexico 2015

The methodology used for the identification and measurement of IDD and measurement instruments will be reviewed, and the data sources will be harmonized with respect to proxy variables of IDD, considering possible normalization, transformation or standardization of variables.

Estimation of disease burden and quantification of population impact and IDD determinants

To estimate the disease burden, quantify the population impact and identify the determinants of IDD, exploratory and descriptive analyses will be performed as follows: nationally, by entity and other relevant stratified analyses, by socio-demographic variables, such as age, sex, indigenous condition by self-ascription, size of locality (rural, urban, metropolitan), household characteristics, socio-demographic level by income deciles, etc. The prevalence of IDD will be analyzed by birth year cohorts in population databases to evaluate the proportion of IDD cases throughout life to approximate a prevalence of cases not related to congenital cases and to estimate the number of years that an individual lives with an IDD. Finally, the disease burden of IDD in Mexico will be estimated from the prevalence of IDDs by the number of expected years that an individual will live with an IDD by the weight associated with disability47 to calculate the DALYs for IDD. All of the analyses will be performed in Stata version 13. The results will provide fundamental input to guide decision-making in related public policies.48

Estimation of economic burden

Given the economic and social involvement of mental disability on the wellbeing of the population, it is very important to estimate the costs of intellectual disability in Mexico. Knowing how the cost varies in different population groups, as well as composition and trend in recent years, constitutes a fundamental input to improve public policy instruments49 oriented towards addressing this problem, in such a way that they comprise authentic social protection mechanisms that maximize the general wellbeing of the population. Therefore, the objective of this component is to estimate the magnitude, composition, distribution and change over time of the economic burden that IDD represent in families in Mexico.

A conceptual framework will be developed which takes into account the areas of family wellbeing affected by the presence of a member with IDD. This implies taking into account the activities that are carried out at the domestic and professional level in order to promote the family, social (or community) and work-related integration of the family member with IDD. This framework will structure the identification of mechanisms that cause additional economic burden related to having a family member with IDD.

The principal data source will be the National Survey of Household Income and Expenses (ENIGH, per the initials in Spanish) for 2010-2016. The ENIGH is the standard survey for determining the distribution, amount and structure of household income and expenses. It also gathers socio-demographic and workrelated data on household members, including those with an intellectual disability, and the time household members use in caregiving. This survey is bi-annual and the data collected is representative at the national level, for urban and rural areas, and for some years is representative at the state level. A strength of this survey is that, through the Socio-economic Conditions Module, it is the official source for the multidimensional measurement of poverty, which covers not just poverty in terms of income but also access to basic needs.

The analysis will be done in the following stages:

  1. We will describe the expense and income profiles of households with a member with an IDD in comparison to homes without, and change over time. Analysis will focus especially on the existence of differences in: the level of certain expenses, income level and work-related characteristics of household members.

  2. Based on information on time use by household members taken from the National Survey on Time Use 2014 and data from the National Survey on Perception of Disability 2010, we will estimate the time spent by household members in the care of other members with an intellectual disability. Using the satellite counting of unremunerated work method, we will estimate the economic burden in homes related to this type of activities.49

  3. Using the methodology for the multi-dimensional measurement of poverty in Mexico we will estimate poverty in households that have a member with IDD.50 We will identify, through a matching technique, households with and without a member with IDD that share socio-demographic characteristics. The goal is to estimate what would be the level of poverty that homes would have if they did not have a member with IDD. Information for 2010, 2012, 2014 and 2016 from the Socio-economic Conditions Module will allow us to identify whether the poverty level of these households has changed in relative terms.

  4. Once the income level has been calculated for households, we propose to estimate the level of economic equality associated with those that include a member with IDD. To this end we will stratify households by income deciles and use the Gini coefficient and the concentration coefficient to explore if the presence of a household member with IDD is distributed differently by household income level and if it is, estimate the inequality gap.

Genomic characterization of IDD in México

The identification of genetic variants associated with IDD of unknown etiology in children, adolescents and young adults stratifying by ADHD and ASD (in children and adolescents between 6 and 15 years old) and overweight/obesity (in those over 18 years old), will be carried out as described below.

Study population

Participating subjects will be recruited from the Dr. Juan N. Navarro Children's Psychiatric Hospital (HPIDJNN) of Mexico City and the Integral Training and Development Center (CADI A. C.) of Mexico City. In case of healthy infants (controls) recruitment will be carry out in schools of Mexico State. Written informed consent will be obtained from the parents of children, adolescents and young adults. Assent will also be obtained from all of the minors and young adults of appropriate development and with capacity to granting it. All procedures using in this protocol will be approve by the IRB (Institutional Review Board) and the ethic committee of participating institutions.

Triads of healthy parents and offspring (children and adolescents between 6-15 years old and young adults over 18 years old) with and without IDD of unknown etiology will be recruited to form 5 groups and 10 strata, as described below:

Stratification by ADHD and ASD

Group I-10 triads of parents and a child under 15 years old without an IDD and without ASD

  1. 5 triads with a child without an IDD, without ASD and without ADHD

  2. 5 triads with a child without an IDD, without ASD and with ADHD

Group II-10 triads of parents and a child under 15 years old with an IDD of unknown etiology without ASD

  1. 5 triads with a child with an IDD, without ASD and without ADHD

  2. 5 triads with a child with an IDD, without ASD and with ADHD

Group III-10 triads of parents and a child under 15 years old without an IDD and with ASD

  1. 5 triads with a child without an IDD, with ASD and without ADHD

  2. 5 triads with a child without an IDD, with ASD and with ADHD

Group IV-10 triads of parents and a child under 15 years old with an IDD of unknown etiology and with ASD

  1. 5 triads with a child with an IDD, with ASD and without ADHD

  2. 5 triads with a child with an IDD, with ASD and with ADHD

Stratification by overweight/obesity

Group V-10 triads of parents and child older than 18 with an IDD of unknown etiology and the absence of ASD and ADHD

  1. 5 triads with a child with an IDD and without overweight/obesity

  2. 5 triads with a child with an IDD and with overweight/obesity

The diagnostic algorithm for the evaluation of subjects with IDD of unknown etiology and comorbidities is shown in Figure 2.

* Recruiting and evaluation of triads with children and adolescents 6-15 years of age will be carried out at the Children's Psychiatric Hospital Dr. Juan N. Navarro (HPIDJNN) in Mexico City and recruitment and evaluation of triads of people over 18 years old will be done at the Integral training and development center (CADI) in Mexico State

Formal diagnostic evaluation will be done through application of tools described in the section on clinical diagnosis

§ Metabolic screening will be done in children and adolescents with IDD or autism spectrum disorders, to exclude cases with inborn metabolism errors

IDD: Intellectual Developmental Disorders

ADHD: Attention Deficit Hyperactivity Disorder

ASD: Autism Spectrum Disorders

Figure 2 Diagnostic algorithm for the evaluation of subjects with IDD of unknown etiology 

The basic inclusion criteria are as follows: 1) Triad consisting of a father, mother and a child between 6 and 15 years (from HPIDJNN) or a child over 18 years old (from CADI), 2) Presumptive clinical diagnosis of the disorder suspected by a health or infant care professional, 3) Formal diagnostic assessment realized by specialized physicians using the international criteria DSM-516 and CIE-10,4 4) Neurological evaluation, 5) Child or adolescent with an Intelligence Quotient (IQ) under 70 for the triads with IDD alone or with a comorbidity.

The exclusion criteria are as follows: 1) Cases without a confirmed clinical diagnosis, which are not classified in the triad groups described previously, with a neurological evaluation which verifies a diagnosis of a defined etiology, 2) Cases with suspicion of syndromic IDD or autism spectrum disorder and cases with a familial aggregation of IDD, ADHD and ASD, 3) Cases with suspected inborn metabolism error, and 4) Cases with a positive diagnosis of fragile X syndrome.

Clinical diagnosis

A clinical history will be done for patients requesting care for the first time. The mental health status of each person (child, adolescent or young adult) will be determined with a psychiatric evaluation. A formal diagnostic evaluation will be done using appropriate tools. To confirm or refute ADHD the Mini International Neuropsychiatric Interview for Children and Adolescents (Minikid) will be used.51,52 To confirm or refute an IDD, IQ will be measured using the WISC-IV53 and for children not enrolled in school the Stanford-Binet.54 The adaptive level will be evaluated with the Vineland II adaptive behavior scal55 and functioning with the WHO-DAS 2.0 questionnaire for disability evaluation.56 The Childhood Autism Rating Scale, Second Edition (CARS-2)57,58 will be used as well as the Autism Diagnostic Interview, Revised (ADI-R)59 to confirm or refute a disorder on the autism spectrum. In addition, a neurological evaluation will be done through evaluation of eye movements, pupil reflexes, facial mimicking, hearing, ability to manipulate the soft palate, swallowing, phonation and tongue movements. Trophism, tone and strength in the neck and the four limbs will be determined as well as tendon reflexes. Walking will be evaluated and abnormal movements described. Mental health specialists will carry out all evaluations. Results will be included in the patient's clinical file and reported to the family.

Cases without a confirmed clinical diagnosis as well as cases that are not classified in the previously described triads, and cases with a neurological evaluation that verifies a defined etiology will be excluded. Participants who leave the study at this point will continue with their habitual clinical treatment regime. In the case of young adults with IDD, overweight and obesity will be defined according to standard cutoff points. NormaI weight as BMI= 18.5-24.9 kg/m2, overweight as BMI= 25.0-29.9 kg/m2, and obesity as BMI ≥ 30 kg/m2.60 For the study, subjects with a BMI ≤ 20 (normal weight control group) and subjects with a BMI ≥ 25 (overweight/obese group) will be selected.

Clinical genetic diagnosis

For analysis of family background, a history of IDD, ADHD and autism spectrum disorder will be explored. For patients with IDD or an autism spectrum disorder a dismorphological clinical evaluation will be done. Weight and height will be measured and cranial shape, eyebrows, eyeballs, nose, oral cavity, ears, neck, thorax, abdomen, genitals (if necessary) and upper and lower limbs will be examined. In cases with family aggregation or with multiple dismorphisms suggestive of a known chromosomic or monogenic syndrome will be excluded from the study. Participants excluded from the study at this time will receive genetic counseling, specific therapeutic recommendations and the corresponding referrals.

Metabolic screening

Participants diagnosed with an autism spectrum disorder or IDD who continue in the study will have a 2 ml peripheral blood sample taking to perform metabolic screening. Cases with suspected inborn metabolism errors will be excluded, receive genetic counseling and therapeutic recommendations and care and the corresponding referrals.

Fragile X screening

A saliva sample will be obtained from each member of the father-mother-child triad with Oragene DNA Kit from DNA Genotek (Cat. OG-500). An aliquot of the saliva sample from each child will be sent to a specialized laboratory, which will test fragile X, by the method of quantifying the expansion of CGG repeat in the FMR1 gene by PCR. The samples will be classified as: normal (<55 repeats), premutation (55-200) and full mutation (> 200). Cases that do not have normal results (positive for fragile X) will be excluded, receive genetic counseling and therapeutic recommendations and care and the corresponding referrals.

In case of normal result (negative for fragile X chromosome) genomic DNA of the triad will be isolated using the prepIT L2P kit from DNA Genotek (Cat. PTL2P5). Samples will be stored at -70 °C until exome sequencing is performed. A part of the genomic DNA from each participant will be preserved and if necessary used for experiments related to the research proposed.

Exome sequencing and detection of variants

For genomic characterization, the coding regions (exome) of the genome of each integrant of de 50 triads previously described will be analyzed, using 1-5 μg de DNA and the Nextera Rapid Capture Exome kits of the company Illumina (Cat. FC-140-1001, Cat. FC-140-1003). Each sample will be sequenced in the Illumina HiSeq 2000 platform with a configuration that allows us to obtain paired readings of 100 bases and a yield of up to 300 million readings per sequencing line. For this study, at least 1.5 G bases of information per individual will be generated to obtain a coverage of ~50X (considering that the size of the exome is 30 million bases). With this depth, it is possible to search for punctual variants (SNV) with high reliability in addition to searching for discrepancies in the number of copies that correspond to structural variants (CNV). After passing a quality control, alignment will be performed using the reference human genome and punctual variants will be searched for with the FreeBayes program.61 CNV will be searched for using the CNV-tools of Bioconductor62 and CoNIFER program.63 A comparison will be made between individuals of each family (trio analysis) to identify de novo or inherited variants in offspring (Figure 3). Subsequently, a comparison will be performed from the variables identified in unrelated individuals with the same condition (10 previously described strata). With the intersection of these variants and by comparisons between groups with and without disorders, it will be possible to select those variants potentially associated with each disorder. A search in public databases such as dbsnp (, OMIM ( and other genomic projects will be performed, to identify which of these variants already has an assigned phenotype and the type of inheritance followed. Finally, those variants that do not have a characterization or associated phenotype in the databases, will be new variants for which a functional prediction (pathogenic impact) using PolyPhen264 and SIFT65 will be realized. The identified variants will be validated by PCR and Sanger sequencing or by qPCR. New variants will be candidates for future screenings in larger populations of people diagnosed with IDD to statistically validate the association.

The circles in the upper part of the figure represent the different clinical conditions for the 10 strata described in the text. The individual genomic characterization steps are shown in white letters. Steps in the typification include comparisons between individuals and between conditions and are shown in black letters.

ADHD: Attention Deficit Hyperactivity Disorder

IDD: Intellectual Developmental Disorders

ASD: Autism Spectrum Disorders

* Subjects with normaI weight (BMI= 18.5-20.0 kg/m2) or subjects with overweight/obesity (BMI ≥ 25.0 kg/m2)

Figure 3 Working scheme for the detection of variants associated with IDD of unknown etiology 

Summary of the evidence base and generation of consensus on screening and diagnostic tools for IDD and on guidelines for social and labor inclusion programs for people with IDD

Through a realist synthesis of the evidence base and a two-round Delphi study with clinical experts, an updated consensus will be reached on which tools to use for psychiatric diagnosis of IDD and screening for IDD for referral to services. The same process will be used with a wider group of stakeholders to reach a consensus on guidelines for services and supports aimed at social and labor inclusion of people with IDD. We will do a systematic search of: 1) tools for screening and diagnosis of IDD and 2) guidelines and literature about services and supports aimed at social and labor inclusion of people with IDD. Then, we will do a realist synthesis of each body of documents, develop questionnaires and use them in Delphi surveys of the opinions of expert stakeholders for each topic.

A realist approach to a literature search and synthesis takes into account that no one tool or program works the same way in every context,66 thus providing information about how the context influences the success, quality and general functioning of tools, programs or interventions in general.67 A realist synthesis of the literature often seeks to identify an intervention's context, mechanism and outcome.68 Therefore, we will do a realist literature search and synthesis which seeks to identify effective patterns in terms of the context, the mechanism used and the outcome achieved through a specific tool, program or intervention.67

We will search the diagnostic and screening tools on the one hand, and guidelines and literature about social and labor inclusion for people with IDD on the other. Both types of documents will be searched for systematically and we will use an additional snowballing approach, as recommended by the Cochrane Collaboration Guidelines for systematic reviews.67,69 Snowballing implies that in addition to a systematic search, individuals identified as experts in the Mexican, Latin American and to some degree the global context in the field of diagnostic and screening tools, guidelines and literature about social and labor inclusion for people with IDD will be contacted by email to request recommendations for documents relevant to the literature search.67,68

Systematic extraction of data will include assessment of methodological quality.70,71 Technical aspects of tools for screening and diagnosis of IDD will be extracted into a database. Data from guidelines and literature about services and supports for social and labor inclusion of people with IDD will be extracted and classified into three categories: context-mechanismoutcome (using a data analysis matrix based on these three categories, with coding to achieve content and thematic analysis).67 This data will be grouped under themes or domains in order to develop statements for the Delphi survey. After grouping into domains, we will operationalize the synthesis of the findings into specific statements, to potentially be included in the survey. Before formulating the final version of this survey, the statements will be reviewed (using qualitative methods)72 by three types of stakeholders: service providers (professionals who provide support for social and work inclusion to people with IDD); family members of people with IDD and services users (people with IDD themselves).71 Individual interviews or focus groups will be used with service providers and family members to review statements, while the nominal group technique will be used with people with IDD.

The nominal technique is a method for achieving consensus which is highly structured, uses very small groups, does not require literacy (is applied verbally) and focuses on a single, unambiguous question.73 This technique has been used with people with ID and with dementia.72,74 A nominal group discussion goes through four steps: 1.) generation of ideas by each individual; 2.) one-by-one statement (taking turns) of ideas which are noted down immediately; 3.) structured and timelimited discussion of the ideas; 4.) voting on consensus or agreement about ideas expressed.72

The two different panels for the Delphi surveys on screening and diagnostic tools for IDD and on guidelines for services and supports aimed at social and labor inclusion of people with IDD will be recruited through purposeful sampling, including snowball sampling. The research team will generate the initial contact list; each panel may have overlap (participants in both panels). All potential participants on the list will be asked about other possible participants they would suggest (with a maximum of 2-3 suggestions each).72,75 Participants will be emailed a link to the survey for each round, where they will be able to register their level of agreement with names of diagnostic and screening tools or with statements about components of services and supports aimed at social and labor inclusion of people with IDD. They will also be able to make comments about each element. There will be 2-3 iterations (subsequent surveys) that will report to participants the level of consensus in the previous stage of the survey.72,76,77

External evaluation of programs for social and labor inclusion for people with IDD in Mexico

The objective of this component of the study is to generate scientific evidence needed to develop evidencebased policy aimed at promotion of independent living and social and labor inclusion programs for people with IDD, through the external evaluation of the actual implementation in Mexico of such guidelines.

In order to evaluate the functioning of programs for social and labor inclusion for people with IDD in Mexico, we will do two qualitative case studies78,79 on actual implementation of such programs, one a program assumed to be a "best practice" and one a program assumed to be "lower functioning" or with more problems and barriers impeding good practice. The two programs will be selected purposefully and each case study80-84 will include the following:

  • Review of program documents;

  • Individual interviews with program personnel;

  • Use of data from the nominal groups with service users (people with IDD), generated during the previous activity;

  • Observation of program functioning, which will be registered on checklists.

Program documents will be analyzed through content analysis and traditional qualitative thematic analysis.85-87 Individual interviews with program personnel and data from the nominal groups with service users (people with IDD), and checklists of observation will be analyzed through qualitative thematic analysis.88,89 First, an intra-case analysis will be done of the data for each case study. Second, a comparative, inter-case analysis will be done comparing the two cases.90 The analysis will focus on generating scientific evidence for developing evidence-based policy aimed at promotion of independent living and social and labor inclusion programs for people with IDD.91-93


IDD and ID in general have not been included in the public health agenda in Mexico, in terms of research, evidence-based public policy or the evaluation of public, charitable or private services offered to people with IDD and their families. This research project will generate scientific evidence around IDD in a variety of areas. Although there is survey data on disability in Mexico, recent surveys have yet to be analyzed with a specific focus on ID. Therefore, statistical data on the burden of IDD at the population level will be processed and presented in order to provide this type of epidemiological information. Little data exist about the catastrophic costs IDD may have for families and so the direct and indirect costs families must cover when one of their members has an IDD will be estimated. Genomic analysis is not generally carried out to achieve diagnosis of IDD in Mexico. This project will contribute with the genomic characterization of IDD, including the identification of de novo or inherited variants as well as validation of selected variants. Consensus is lacking in Mexico on best practices in terms of which tools should be used for diagnosing IDD and what type of services can help people with IDD achieve social and labor inclusion. Through a realist systematic review and Delphi surveys, this project will contribute in this area as well. Another gap in the data is how such services for social and work-related inclusion are provided in Mexico, and this project will seek to generate evidence on this topic through qualitative case studies. There is much to be done in the field of IDD in Mexico and this research project proposes important first steps in a series of fields, using appropriate approaches for each.


The authors gratefully acknowledge to Dr. Angélica Ángeles Llerenas, Psychologist Verónica Pérez Barrón, Dr. Fidel Alejandro Sánchez Flores and Dr. Katy Sánchez Pozos for critical review of this manuscript, and to Conacyt-FOSISS 2016 (Project No. 272137) for providing funding for this work.


1. Instituto Nacional de Estadística y Geografía (INEGI). Encuesta Nacional de la Dinámica Demográfica 2014 [internet document]. México: INEGI, 2014 [accessed: October 30, 2016]. Available in: ]

2. De Castro F, Allen-Leigh B, Katz G, Salvador-Carulla L, Lazcano-Ponce E. Indicadores de bienestar y desarrollo infantil en México. Salud Publica Mex 2013;55(supl 2):S267-S275. [ Links ]

3. Lazcano-Ponce E, Katz G, Allen-Leigh B, Magaña Valladares L, Rangel-Eudave G, Minoletti A, et al. Trastornos del desarrollo intelectual en América Latina: un marco para establecer prioridades políticas de investigación y atención. Rev Panam Salud Publica 2013;34(3):204-209. [ Links ]

4. Organización Mundial de la Salud. CIE 10: Trastornos mentales y del comportamiento. Descripciones clínicas y pautas para el diagnóstico. Clasificación Internacional de las Enfermedades Mentales. 10a Revisión. Madrid: Meditor, 1992. [ Links ]

5. Salvador-Carulla L, Reed GM, Vaez-Azizi LM, Cooper SA, Martinez-Leal R, Bertelli M, et al. Intellectual developmental disorders: towards a new name, definition and framework for "mental retardation/intellectual disability" in ICD-11.World Psychiatry 2011;10(3):175-180. ]

6. Kaufman L, Ayub M, Vincent JB. The genetic basis of non-syndromic intellectual disability: a review. J Neurodev Disord 2010;2(4):182-209. ]

7. United Nations. Convention on the rights of persons with disabilities [internet document]. UN Web Services Section, 2006 [accessed: September 30, 2016].Available in: convention.shtmlLinks ]

8. Madans JH, Loeb ME, Altman BM. Measuring disability and monitoring the UN Convention on the Rights of Persons with Disabilities: the work of the Washington Group on Disability Statistics. BMC Public Health 2011;11(Suppl 4):S4. [ Links ]

9. World Health Organization. World report on disability. Malta: WHO, 2011. [ Links ]

10. Stephens T, Joubert N. The economic burden of mental health problems in Canada. Chronic Dis Can 2001;22(1):18-23. [ Links ]

11. Doran CM, Einfeld SL, Madden RH, Otim M, Horstead SK, Ellis LA, Emerson E. How much does intellectual disability really cost? First estimates for Australia. J Intellect Dev Disabil 2012;37(1):42-49. ]

12. Instituto Nacional de Estadística y Geografía (INEGI). Las personas con discapacidad en México: una visión al 2010. México: INEGI, 2010. [ Links ]

13. Urquieta-Salomón JE, Figueroa JL, Hernández-Prado B. El gasto en salud relacionado con la condición de discapacidad: un análisis en población pobre de México. Salud Publica Mex 2008;50(2):136-146. ]

14. Curry CJ, Stevenson RE, Aughton D, Byrne J, Carey JC, Cassidy S, et al. Evaluation of mental retardation: recommendations of a Consensus Conference: American College of Medical Genetics. Am J Med Genet 1997;72(4):468-477. ]

15. Tzschach A, Ropers H. Genetics of Mental Retardation. Dtsch Arztebl 2007;104(20):A1400-A1405. [ Links ]

16. American Psychiatric Association (APA). Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition (DSM-5). Arlington, VA (USA): American Psychiatric Publishing, 2013. [ Links ]

17. Fombonne E. Epidemiology of pervasive developmental disorders. Pediatr Res 2009;65(6):591-598. ]

18. Kuntsi J, Eley TC, Taylor A, Hughes C, Asherson P, Caspi A, et al. Cooccurrence of ADHD and low IQ has genetic origins. Am J Med Genet B Neuropsychiatric Genet 2004;124B(1):41-47. ]

19. Simonoff E, Pickles A, Wood N, Gringras P, Chadwick O. ADHD symptoms in children with mild intellectual disability. J Am Acad Child Adolesc Psychiatry 2007;46(5):591-600. ]

20. Gadow KD, Devincent CJ, Pomeroy J, Azizian A. Comparison of DSM-IV symptoms in elementary school-age children with PDD versus clinic and community samples. Autism 2005;9(4):392-415. ]

21. Gargaro BA, Rinehart NJ, Bradshaw JL, Tonge BJ, Sheppard DM. Autism and ADHD: how far have we come in the comorbidity debate? Neurosci Biobehav Rev 2011;35(5):1081-1088. ]

22. Kummer A, Barbosa IG, Rodrigues DH, Rocha NP, Rafael MD, Pfeilsticker L, et al. Frequency of overweight and obesity in children and adolescents with autism andattention deficit/hyperactivity disorder. Rev Paul Pediatr 2016;34(1):71-77. ]

23. Krause S, Ware R, McPherson L, Lennox N, O'Callaghan M. Obesity in adolescents with intellectual disability: Prevalence and associated characteristics. Obes Res Clin Pract 2016;10(5):520-530. ]

24. Biamino E, Di Gregorio E, Belligni EF, Keller R, Riberi E, Gandione M, et al. A novel 3q29 deletion associated with autism, intellectual disability, psychiatric disorders, and obesity. Am J Med Genet B Neuropsychiatr Genet 2016;171(2):290-299. ]

25. Mundhofir FE, Winarni TI, van Bon BW, Aminah S, Nillesen WM, Merkx G, et al. A cytogenetic study in a large population of intellectually disabled Indonesians. Genet Test Mol Biomarkers 2012;16(5):412-417. ]

26. Reiss AL, Hall SS. Fragile X syndrome: assessment and treatment implications. Child Adolesc Psychiatr Clin N Am 2007;16(3):663-675. ]

27. Rauch A, Wieczorek D, Graf E, Wieland T, Endele S, Schwarzmayr T, et al. Range of genetic mutations associated with severe non-syndromic sporadic intellectual disability: an exome sequencing study. Lancet 2012;380(9854):1674-1682. ]

28. Sagoo GS, Butterworth AS, Sanderson S, Shaw-Smith C, Higgins JP, Burton H. Array CGH in patients with learning disability (mental retardation) and congenital anomalies: updated systematic review and meta-analysis of 19 studies and 13,926 subjects. Genet Med 2009;11(3):139-146. ]

29. Vissers LE, Gilissen C, Veltman JA. Genetic studies in intellectual disability and related disorders. Nat Rev Genet 2016;17(1):9-18. ]

30. Srivastava AK, Schwartz CE. Intellectual disability and autism spectrum disorders: causal genes and molecular mechanisms. Neurosci Biobehav Rev 2014;46(Pt 2):161-174. ]

31. Lionel AC, Crosbie J, Barbosa N, Goodale T,Thiruvahindrapuram B, Rickaby J, et al. Rare copy number variation discovery and crossdisorder comparisons identify risk genes for ADHD. Sci Transl Med 2011;3(95):95ra75. ]

32. Hiroi N, Takahashi T, Hishimoto A, Izumi T, Boku S, Hiramoto T. Copy number variation at 22q11.2: from rare variants to common mechanisms of developmental neuropsychiatric disorders. Mol Psychiatry 2013;18(11):1153-1165. ]

33. Ramalingam A, Zhou XG, Fiedler SD, Brawner SJ, Joyce JM, Liu HY, et al. 16p13.11 Duplication is a risk factor for a wide spectrum of neuropsychiatric disorders. J Hum Genet 2011;56(7):541-544. ]

34. van Bokhoven H. Genetic and epigenetic networks in intellectual disabilities. Annu Rev Genet 2011;45:81-104. ]

35. Xu LM, Li JR, Huang Y, Zhao M, Tang X, Wei L. AutismKB: an evidence-based knowledgebase of autism genetics. Nucleic Acids Res 2012;40:D1016-D1022. ]

36. Delorme R, Ey E, Toro R, Leboyer M, Gillberg C, Bourgeron T. Progress toward treatments for synaptic defects in autism. Nat Med 2013;19(6):685-694. ]

37. Gilman SR, Iossifov I, Levy D, Ronemus M, Wigler M, Vitkup D. Rare de novo variants associated with autism implicate a large functional network of genes involved in formation and function of synapses. Neuron 2011;70(5):898-907. ]

38. Humeau Y, Gambino F, Chelly J, Vitale N. X-linked mental retardation: focus on synaptic function and plasticity. J Neurochem 2009;109(1):1-14. ]

39. Cristino AS, Williams SM, Hawi Z, An JY, Bellgrove MA, Schwartz CE, et al. Neurodevelopmental and neuropsychiatric disorders represent an interconnected molecular system. Mol Psychiatry 2014;19(3):294-301. ]

40. Jiang YH, Wang Y, Xiu X, Choy KW, Pursley AN, Cheung SW. Genetic diagnosis of autism spectrum disorders: the opportunity and challenge in the genomics era. Crit Rev Clin Lab Sci 2014;51(5):249-262. ]

41. Li Z, Chang SH, Zhang LY, Gao L, Wang J. Molecular genetic studies of ADHD and its candidate genes: a review. Psychiatry Res 2014;219(1):1024. ]

42. Mefford HC, Batshaw ML, Hoffman EP. Genomics, intellectual disability, and autism. N Engl J Med 2012;366(8):733-743. ]

43. Willemsen MH, Kleefstra T. Making headway with genetic diagnostics of intellectual disabilities. Clin Genet 2014;85(2):101-110. ]

44. Miller DT, Adam MP, Aradhya S, Biesecker LG, Brothman AR, Carter NP, et al. Consensus statement: chromosomal microarray is a first tier clinical diagnostic test for individuals with developmental disabilities or congenital anomalies. Am J Hum Genet 2010;86(5):749-764. ]

45. de Ligt J, Willemsen MH, van Bon BW, Kleefstra T, Yntema HG, Kroes T, et al. Diagnostic exome sequencing in persons with sever intellectual disability. N Engl J Med 2012;367(20):1921-1929. ]

46. Gilissen C, Hehir-Kwa JY, Thung DT, van de Vorst M, van Bon BW,Willemsen MH, et al. Genome sequencing identifies major causes of severe intellectual disability. Nature 2014;511(7509):344-347. ]

47. World Health Organization. WHO methods and data sources for global burden of disease estimates 2000-2011. Geneve: WHO, 2013. [ Links ]

48. Gregori D. Evidences to validate public policies: a review with an international research perspective. Salud Publica Mex 2014;56(Suppl 2):s157-s161. [ Links ]

49. Instituto Nacional de Estadística y Geografía (INEGI). Sistema de Cuentas Nacionales. Cuenta Satélite del Trabajo no Remunerado de los Hogares. Fuentes y Metodología [internet document]. México: INEGI, 2013 [accessed: September 15, 2016]. Available in: ]

50. Consejo Nacional de Evaluación de la Política de Desarrollo Social. Metodología para la medición multidimensional de la pobreza en México (2a ed.). México, DF: Coneval, 2014. [ Links ]

51. Sheehan DV, Lecrubier Y, Shytle D, Milo K, Hergueta T, Colón SM. Mini International Neuropsychiatric Interview for children and adolescents [M.I.N.I. KID].Version 1.1 Medical Outcome Systems, Inc. 2000. [ Links ]

52. De la Peña OF, Esquivel AG, Pérez GAJ, Palacios CL. Validación Concurrente para Trastornos Externalizados del MINI-Kid y la Entrevista Semiestructurada para Adolescentes. Rev Chil Psiquiatr Neurol Infanc Adolesc 2009;20(1):8-12. [ Links ]

53. Weschler D. Escala de inteligencia para escolares [Intelligence scale for schoolchildren].WISC-IV. México: El Manual Moderno, 2005. [ Links ]

54. Thorndike RL, Hagen EP, Sattler JM. Stanford-Binet intelligence scale. Chicago: Riverside Publishing Company. 1986. [ Links ]

55. Sparrow SS, Balla DA, Cicchetti DV. Vineland-II adaptive behavior scales. Circle Pines, MN: American Guidance Service, 2005. [ Links ]

56. Ustun TB, Kostanjsek N, Chatterji S, Rehm J. Measuring health and disability. Manual for WHO Disability Assessment Schedule, WHODAS 2.0. Geneva: WHO, 2010. [ Links ]

57. Schopler E, Reichler RJ, Renner BR. The childhood autism rating scale (CARS). Los Angeles: Western Psychological Services, 1988. [ Links ]

58. Flores RY, Albores GL. Validity of Mexican cars version. J Am Acad Child Adolesc Psychiatry 2016;55(10): S110. [ Links ]

59. Rutter M, Le Couteur A, Lord C. Autism diagnostic interview-revised. Los Angeles: Western Psychological Services, 2003. [ Links ]

60. Kumar A, Karmarkar AM,Tan A, Graham JE,Arcari CM, Ottenbacher KJ, et al. The effect of obesity on incidence of disability and mortality in Mexicans aged 50 years and older. Salud Publica Mex 2015;57(Suppl 1):S31-S38. [ Links ]

61. Garrison E, Marth G. Haplotype-based variant detection from shortread sequencing. arXiv: 1207.3907. Cornell University Library, 2012 [accessed October 30, 2016]. Available at: http// ]

62. Barnes C, Plagnol V. CNVtools: A package to test genetic association with CNV data. R package version 1.64.0. 2009. [ Links ]

63. Krumm N, Sudmant PH, Ko A, O'Roak BJ, Malig M, Coe BP, et al. Copy number variation detection and genotyping from exome sequence data. Genome Res 2012;22(8):1525-1532. ]

64. Adzhubei IA, Schmidt S, Peshkin L, Ramensky VE,Gerasimova A, Bork P, et al. A method and server for predicting damaging missense mutations. Nat Methods 2010;7(4):248-249. ]

65. Kumar P, Henikoff S, Ng PC. Predicting the effects of coding non-synonymous variants on protein function using the SIFT algorithm. Nat Protoc 2009;4(7):1073-1081. ]

66. Pawson R, Greenhalgh T, Harvey G, Walshe K. Realist review - a new method of systematic review designed for complex policy interventions. J Health Serv Res Policy 2005;10(1):21-34. ]

67. Rycroft-Malone J, McCormack B, Hutchinson AM, DeCorby K, Bucknall TK, Kent B, et al. Realist synthesis: illustrating the method for implementation research. Implement Sci 2012;7(33):1-10. ]

68. McVeigh J, MacLachlan M, Gilmore B, McClean C, Eide AH, Mannan H, et al. Promoting good policy for leadership and governance of health related rehabilitation: a realist synthesis. Global Health 2016;12(1):49. ]

69. The Cochrane Collaboration. Cochrane Handbook for Systematic Reviews of Interventions [internet document]. London: The Cochrane Collaboration, 2015 [accessed November 8, 2016]. Available in: ]

70. Kuipers P, Wirz S, Hartley S. Systematic synthesis of community-based rehabilitation (CBR) project evaluation reports for evidence-based policy: a proof-of-concept study. BMC Int Health Hum Rights 2008;8:3. ]

71. Kuipers P, Hartley S. A process for the systematic review of community-based rehabilitation evaluation reports: formulating evidence for policy and practice. Int J Rehabil Res 2006;29(1):27-30. ]

72. Petry K, Maes B, Vlaskamp C. Operationalizing quality of life for people with profound multiple disabilities: a Delphi study. J Intellect Disabil Res 2007;51(Pt 5):334-349. ]

73. Tuffrey-Wijne I, Bernal J, Butler G, Hollins S, Curfs L. Using Nominal Group Technique to investigate the views of people with intellectual disabilities on end-of-life care provision. J Adv Nurs 2007;58(1):80-89. ]

74. Dening KH, Jones L, Sampson EL. Preferences for end-of-life care: a nominal group study of people with dementia and their family carers. Palliat Med 2013;27(5):409-417. ]

75. Geist MR. Using the Delphi method to engage stakeholders: a comparison of two studies. Eval Program Plann 2010;33(2):147-154. ]

76. Linstone HA, Turoff M, eds. The Delphi Method. Techniques and Applications. Portland, New Jersey: Portland State University, New Jersey Institute of Technology, 2002. [ Links ]

77. Okoli C, Pawlowski SD. The Delphi method as a research tool: an example, design considerations and applications. Information & Management 2004;42:15-29. ]

78. Yin RK. The Case Study Method as a Tool for Doing Evaluation. Current Sociology 1992;40:121-137. ]

79. Egan MY, Kubina LA, Lidstone RI, Macdougall GH, Raudoy AE. Acritical reflection on occupational therapy within one Assertive Community Treatment team. Can J Occup Ther 2010;77(2):70-79. ]

80. Cornielje H, Nicholls PG, Velema J. Making sense of rehabilitation projects: classification by objectives. Lepr Rev 2000;71(4):472-485. ]

81. Cornielje H, Velema JP, Finkenflügel H. Community based rehabilitation programmes: monitoring and evaluation in order to measure results. Lepr Rev 2008;79(1):36-49. [ Links ]

82. Velema JP, Finkenflügel HJ, Cornielje H. Gains and losses of structured information collection in the evaluation of 'rehabilitation in the community' programmes: ten lessons learnt during actual evaluations. Disabil Rehabil 2008;30(5):396-404. ]

83. Velema JP, Cornielje H. Reflect before you act: providing structure to the evaluation of rehabilitation programmes. Disabil Rehabil 2003;25(22):1252-1264. ]

84. Cornielje H, Nicholls PG, Velema JP. Avoiding misperceptions: classifying rehabilitation projects using letters rather than numbers. Lepr Rev 2002;73(1):47-51. [ Links ]

85. Kuipers P, Foster M, Carlson G, Moy J. Classifying client goals in community-based ABI rehabilitation: a taxonomy for profiling service delivery and conceptualizing outcomes. Disabil Rehabil 2003;25(3):154-162. ]

86. Doig E, Fleming J, Kuipers P, Cornwell PL. Comparison of rehabilitation outcomes in day hospital and home settings for people with acquired brain injury-a systematic review. Disabil Rehabil 2010;32(25):2061-2077. ]

87. Hsieh HF, Shannon SE. Three Approaches to Qualitative Content Analysis. Qual Health Res 2005;15(9):1277-1288. ]

88. Ryan GW, Bernard HR. Techniques to Identify Themes. Field Methods 2003;15:85-109. ]

89. Charmaz K. Constructing grounded theory: A practical guide through qualitative analysis. London: Sage, 2006. [ Links ]

90. Ayres L, Kavanaugh K, Knafl KA. Within-case and across-case approaches to qualitative data analysis. Qual Health Res 2003;13(6):871-883. ]

91. Doig E, Fleming J, Cornwell PL, Kuipers P. Qualitative exploration of a client-centered, goal-directed approach to community-based occupational therapy for adults with traumatic brain injury. Am J Occup Ther 2009;63(5):559-568. ]

92. Kuipers P, Allen O. Preliminary guidelines for the implementation of Community Based Rehabilitation (CBR) approaches in rural, remote and Indigenous communities in Australia. Rural Remote Health 2004;4(3):291. [ Links ]

93. Kuipers P, Kendall E, Hancock T. Developing a rural community-based disability service: (I) service framework and implementation strategy. Aust J Rural Health 2001;9(1):22-28. ]

Received: October 10, 2016; Accepted: November 18, 2016

Corresponding author: Dra. Rocío Rodríguez-Valentín. Centro de Investigación en Salud Poblacional, Instituto Nacional de Salud Pública. Av. Universidad 655, col. Santa María Ahuacatitlán. 62100 Cuernavaca, Morelos, México. E-mail:

Declaration of conflict of interests. The authors declare that they have no conflict of interests.

Creative Commons License This is an open-access article distributed under the terms of the Creative Commons Attribution License