Internal consistency, test-retest reliability and construct validity of the Frost Multidimensional Perfectionism Scale

Consistencia interna, confiabilidad test-retest y validez de constructo de la Escala Multidimensional de Perfeccionismo de Frost

Karina Franco^{1, 3}, Felipe Díaz^{1, 3}, Patricia Torres¹, Yolanda Telléz¹ y Carlos Hidalgo-Rasmussen^{1, 2 3}

¹ Centro Universitario del Sur, Universidad de Guadalajara, México.

³ Miembro del Grupo de Investigación Comportamiento, Salud y Calidad de Vida.

Correspondencia:
Karina Franco Paredes,
Centro Universitario del Sur de la Universidad de Guadalajara.
Av. Enrique Arreola Silva 883, Colonia Centro, C.P. 49000,
Ciudad Guzmán, Jalisco, México.
E-mail: karina.franco@cusur.udg.mx

Abstract

The purpose of this research was to evaluate the internal consistency, the test-retest reliability and the construct validity of the Frost Multidimensional Perfectionism Scale (FMPS) in women. The total sample was made up of 325 university women, with an average age of 20.75 years (SD = 2.81). The scale was administered twice (n = 189) with an interval of one or two months between the first and second time of administration. Results showed that Cronbach's Alpha for the total score of instrument was .87 and for the factors ranged from .66 to .80. The one-month test-retest reliability was .80 and for the two-months was .67. People with symptomatology of eating disorders showed significantly higher scores than the control group in the total score of the FMPS and three of its factors, Concerns about Mistakes, Doubts about Actions and Parental Expectations. These findings provide favorable evidence for internal consistency, test-retest reliability and construct validity of the FMPS.

Key words: Reliability, validity, FMPS, women, eating disorders symptoms.

Resumen

El propósito de esta investigación fue evaluar la consistencia interna, la confiabilidad test-retest y la validez de constructo de la Frost Multidimensional Perfectionism Scale (FMPS) en mujeres. Se trabajó con una muestra de 325 mujeres universitarias, con una edad promedio de 20.75 años (DE = 2.81). Una muestra (n = 189) de participantes contestó la FMPS en dos ocasiones para el test-retest, con una diferencia de un mes o dos meses entre la primera y la segunda aplicación. Los resultados mostraron que el Alpha de Cronbach para el total del instrumento fue de .87 y para los factores el rango fue de .66 a .80. La confiabilidad test-retest a un mes fue de .80 y a dos meses fue de .67. Las personas con sintomatología de trastornos del comportamiento alimentario presentaron puntuaciones significativamente mayores que el grupo control en el total de la escala y tres de sus factores, Preocupación por los Errores, Indecisión de Acción y Expectativas Paternas. Se concluye que existe evidencia favorable sobre la consistencia interna, confiabilidad test retest y validez de constructo de la FMPS.

Introducción

Research interest focused on perfectionism has grown over the last two decades in the clinical and personality areas, recognizing its potential negative effects, such as sense of failure, indecision, procrastination and shame (González, Ibañez, Rovella, López, & Padilla, 2013). Taking into account the effects of perfectionism on health, a dual conceptualization has been proposed throughout a number of early writings in the clinical literature. For example, Shafran, Cooper, and Fairburn (2002) classified perfectionism as functional vs dysfunctional.

Functional perfectionism is associated with several features, such as the high personal expectation, self-esteem, order, organization, good performance, planning and facing tasks with a high intellectual level (Parker, 2000; Sastre-Riba, 2012), positive attachments (Bieling, Israeli, & Anthony, 2004), and life satisfaction (Bergman, Nyland, & Burns, 2007). Whereas, dysfunctional perfectionism is associated with negative affects like anxiety, depression, social anxiety (Stoeber & Otto, 2006), as well as, eating disorders (Franco-Paredes, Mancilla-Díaz, Vázquez-Arévalo, López-Aguilar, & Alvarez-Rayón, 2005; González et al., 2013).

Scappatura, Arana, Elizathe, and Rutsztein (2011) stated that perfectionism has been associated with unrealistic expectations regarding physical appearance oriented toward extreme thinness, therefore, this construct is considered as an important risk factor and maintenance factor of eating disorders (Fairburn, Cooper, & Shafran, 2003; Franco-Paredes et al., 2005; Lilenfed, Wonderlich, Riso, Crosby, & Mitchell, 2006; Rutsztein, Scappatura, & Murawski, 2014). In addition, a high level of perfectionism after recovery may be considered as an important element for relapse in eating disorders (Bardone-Cone, 2007). In this sense, perfectionism supposes a transdiagnostic process, as a common risk factor in the etiology and in the maintaining of eating disorders.

Given the importance of perfectionism within the psychology field, different instruments have been developed and used in several researches carried out in the last two decades, including the Frost Multidimensional Perfectionism Scale (FMPS; Frost, Marten, Lahart, & Rosenblate, 1990). The FMPS includes 35 items that evaluate six dimensions of perfectionism: Personal Standards, Concerns over Mistakes, Organization, Doubts about Actions, Parental Expectations and Parental Criticism. The internal consistency of the FMPS have been assessed in adult population with Cronbach's Alpha coefficients ranging from .85 to .93 (Cheng, Chong, & Wong, 1999; Franco, Mancilla-Díaz, Vázquez, Álvarez, & López, 2010; Frost et al., 1990; Gelabert et al., 2011; Parker & Adkins, 1995; Stöber, 1998). Temporal one month stability was also evaluated, in two studies test-retest correlation ranged from .76 (Monteiro et al., 2013) to .82 (Gelabert et al., 2011).

Empirical evaluation of the factorial structure of the FMPS has confirmed the original six factor solution (Gelabert et al. 2011; Parker & Adkins 1995; Purdon, Antony, & Swinson, 1999; Rhéaume, Freeston, Dugas, Letarte, & Ladouceur, 1995; Stöber, 1998). However, three studies reported a five factor solution (Cox, Enns, & Clara, 2002; Cheng et al. 1999; Franco et al., 2010); and finally, four studies identified a four factor solution (Franco, Santoyo, Díaz, & Mancilla-Díaz, in press; Harvey, Pallant & Harvey 2004; Hawkins, Watt, & Sinclair, 2006; Khawaja, & Armstrong, 2005; Stumpf, & Parker, 2000), which can be considered the most parsimonious structure taking into account the fit indexes.

The development and constant updating of questionnaires is an important task in any area of study and it is desirable to have the greater amount of evidence regarding the psychometric properties of this tools. The assessment of the psychometric properties of the questionnaires is a continuous process in which the evidences that confirm or complement existing findings are generated. The questionnaires have allowed the advance of knowledge about the effects of perfectionism on health and behavior. The majority of the studies have examined the psychometric properties of the FMPS in school-aged people, and only two studies have used clinical samples (Cox et al., 2002; Purdon et al., 1999). Also, the reliability has been assessed with Cronbach's Alpha method, and there is a few evidence about temporal stability of the FMPS (Gelabert et al., 2011; Monteiro et al., 2013). On the other hand, the two studies that have been carried out in Mexico to validate the FMPS revealing that internal consistency for the total score of the scale was adequate (Franco et al., 2010) and the four factors structure was the most parsimonious (Franco et al., in press). Because of this, the purpose of this research was to generate more evidence about psychometric properties of the FMPS, specifically regarding the test-retest reliability, the internal consistency and construct validity in women.

Sample

The total sample consisted of 325 undergraduate women with an average age of 20.75 years (SD = 2.81), range 18-25 years old. The majority of the students were in their second year of undergraduate studies (61.3%), and 38.7% were third-year undergraduate students. The great majority of the students were single (96.62%). To analyze the temporal stability of the FMPS, 189 women answered the questionnaire at two different moments, separated by one (n = 115) and two months (n = 74).

Instruments

The Frost Multidimensional Perfectionism Scale (FMPS) includes 35 items, with a five-point Likert type scale (from strongly disagreement = 1 to strongly agree = 5). The first validation in Mexico (Franco et al., 2010) revealed an adequate internal consistency (Alpha = .86). A four-factor structure was recently confirmed in a Mexican sample: Concern over Mistakes, Organization, Doubts about Action and Parental Expectations (Franco et al., in press).

Eating Attitudes Test (EAT-40) was designed to assess the presence of symptoms and features of the eating disorders (Garner & Garfinkel, 1979), it includes 40 items with six options for answer in a Likert type scale. The first validation in Mexico revealed a good internal consistency (Alpha = .90) and a cut-off point of 28 was established (Alvarez, Vázquez, Mancilla, & Gómez-Peresmitre, 2002).

Procedure

Students answered the questionnaires in the classroom. One of the researchers explained to the participants the purpose of the research and those who voluntarily decided to participate answered the questionnaires in approximately ten minutes. To analyze the temporal stability of the FMPS, 115 women answered the scale a month after the first application and 74 did it two months later.

Data were analyzed using SPSS for Windows (version 15). Internal consistency of the FMPS for total sample (n = 325) was measured using Cronbach's Alpha coefficient. Pearson correlation was used to assess test-retest reliability and the homogeneity of the test. The discrimination between eating disorders symptomatology sample and control sample was analyzed with Student t test for unrelated samples.

Results

Internal consistency

The Chronbach's Alpha coefficients were calculated for the total score of the FMPS and the four factors derived in Mexican samples (see table 1) for the first time (n = 325). The Cronbach Alpha for the total of the FMPS was .87. The coefficients for the individual factors fluctuated among .66 for Parental Expectations and .80 for Concern over Mistakes.

Homogeneity of the test

Test-retest reliability

The one-month test retest reliability was r = .80 (p < .001) and the two-month test-retest reliability was r = .67 (p < .001) for the total of the FMPS. Additionally, a comparison between the mean score between test and retest was conducted. There were no significant differences between means for one-month test-retest (t = 1.67, p > .05) and for two-month test retest (t = 1.34, p > .05).

Group differences

Considering the total sample (n = 325) and the cutoff point of EAT-40, a group with symptomatology of eating disorders was conformed (n = 33) and a control group was randomly selected from the rest of the sample (n = 50). A comparison was conducted between both groups and it was found that eating disorders symptomatology sample scored significantly higher than control sample in total score of FMPS and three of its factors, Concern over Mistakes, Doubts about Action and Paternal Expectations (see table 2).

The purpose of this study was to assess the internal consistency, the test-retest reliability, and construct validity of FMPS in women. Concerning the reliability of the FMPS it was found that, in general, the internal consistency was adequate, with Cronbach's Alpha values above .70 for the total score and for three of the four factors (Concern over Mistakes, Organization and Doubts about Actions), indicating that the questions of the scale converge to the same construct. This finding replicates the results obtained in previous researches in which internal consistency has been evaluated with the Cronbach's Alpha coefficient (Cheng et al., 1999; Franco et al., 2010; Frost et al., 1990; Gelabert et al., 2011; Parker & Adkins, 1995; Stöber, 1998). The internal consistency of the Parental Expectations factor was low; the internal consistency of this factor was also low in a previous validation conducted in Mexico, therefore, it is necessary that future research analyze this factor in order to determine its reliability. In general, the internal consistency was adequate regarding the criteria published in the literature of Cronbach's Alpha (Cortina, 1993; Cronbach, 1951; Cronbach, & Shavelson, 2004; Oviedo, & Campo-Arias, 2005). It is necessary to remark that even though the concept of internal consistency, and mathematical methods to obtain it have been criticized, in this research the standard procedure was employed.

The repeated application of the test in two different moments is the most rigid method for analyzing the reliability because it assesses the temporal stability of the measurement (Gregory, 2001). The test-retest method is more adequate to assess the reliability of an instrument; however, there are only two researches in which this method was employed to assess the reliability of the FMPS. In this study, when the FMPS was applied a month after the first administration, it was found a similar results that obtained in two previous studies (Gelabert et al., 2011; Monteiro et al., 2013). This finding indicates that the instrument scores are stable after a month; nevertheless, the value of the coefficient decreased within two months and it was situated below the recommended value (.70) for the reliability of the instrument (Gregory, 2001; Nunally, & Bernstein, 1995), which indicates that the stability of the measurement decreased lightly with the passing of time. Perfectionism appears to be a negative predictor of outcome for anorexia nervosa (Bizuel, Sadowsky, & Rigaud, 2001), however, the use of the FMPS to evaluate the effects of treatment perhaps is not recommended; although future research is required in order to contribute to generate evidences about temporal stability of the scale using longitudinal designs.

The homogeneity analyses showed that 27 of the 35 items of the FMPS reached an acceptable correlation with total score, therefore, it contributes to the homogeneity of the questionnaire. The items 2, 6, 7, 8, 16, 27, 29, and 31 have a weak correlation with the total score. This finding is consistent with other research in some of these items (2, 6 and 31) were inadequate (Franco et al., 2010; Hawkins et al., 2006; Stober, 1998). Taking into account this finding, it is necessary to evaluate short versions of the instrument in future studies.

Perfectionism is a characteristic that is usually present in people with eating disorder or its symptoms. Initial studies have related perfectionism mainly with anorexia nervosa (Bruch, 1973; Castro-Fornieles et al., 2007; Shafran et al., 2002; Sutandar-Pinnock, Woodside, Carter, Olmsted, & Kaplan, 2003) and recently, with bulimia nervosa (Bardone-Cone, Abramson, Vohs, Heatherton, & Joiner, 2006; Bardone-Cone et al., 2007; Franco, Mancilla-Díaz, Vázquez, Alvarez, & López, 2011). Therefore, it is expected that people with eating disorders symptoms obtain high scores on perfectionism measures. In the present study, the total score of the FMPS, in addition of three of its four factors, differentiated in the expected direction between eating disorders symptomatology sample and control sample. That is, women with symptomatology of eating disorders had greater scores in perfectionism than women without symptomatology.

This study revealed that FMPS is a good and useful questionnaire for research purposes. One of the limitations of this study was the small sample for two-month test-retest. Lack of data on male participants is the second shortcoming of the present study.

This study contributes to the body of knowledge of psychometric properties of FMPS. The findings highlight that FMPS has an adequate internal consistency, good short term (one month) test-retest reliability, and good discriminant capacity between eating disorders symptomatology sample and control sample.

In the future research, it would be important to assess the temporal stability of the scale in order to analyze its utility as predictor of treatment outcomes, as well as, to analyze if a reduced version of the scale should be used. Also, it is necessary to apply the FMPS in several clinical samples in order to contribute to assess its clinical relevance and generalizability.

References

Alvarez, G., Vázquez, R., Mancilla, J. M., & Gómez-Peresmitre, G. (2002). Evaluación de las propiedades psicométricas del Test de Actitudes Alimentarias (EAT-40) en mujeres mexicanas. Revista Mexicana de Psicología, 19, 47-56.         [ Links ]

Bardone-Cone, A. M. (2007). Self-briented and socially prescribed perfectionism dimensions and their associations with disordered eating. Behaviour Research and Therapy, 45, 1977-86.         [ Links ]

Bardone-Cone, A. M., Abramson, L. Y., Vohs, K. D., Heatherton, T. F., & Joiner, T. E., Jr. (2006). Predicting bulimic symptoms: An interactive model of self-efficacy, perfectionism, and perceived weight status. Behaviour Research and Therapy, 44, 27-42.         [ Links ]

Bergman, A. J., Nyland, J.E., & Burns, L.R. (2007). Correlates with perfectionism and the utility of a dual process model. Personality and Individual Differences, 43, 389-399.         [ Links ]

Bieling, P. J., Israeli, A. L., & Anthony, M. M. (2004). Is perfectionism good, bad, or both? Examining models of the perfectionism construct. Personality and Individual Differences, 36, 1373-1385.         [ Links ]

Bizuel, C., Sadowsky, N., & Rigaud, D. (2001). The prognosis value of initial EDI scores in anorexia nervosa patients: A prospective follow-up study of 5-10 years. European Psychiatry, 16, 232-238.         [ Links ]

Bruch, H. (1973). Eating disorders: Obesity, anorexia nervosa, and the person within. New York: Basic Books.         [ Links ]

Castro-Fornieles, J., Gual, P., Lahortiga, F., Gila, A., Casula, V., Fuhrmann, C., ... Toro, J. (2007). Self-oriented perfectionism in eating disorders. International Journal of Eating Disorders, 40, 562-568.         [ Links ]

Cheng, S. K., Chong, G. H., & Wong, C. W. (1999). Chinese Frost Multidimensional Perfectionism Scale: A validation and prediction of Self-Esteem and psychological distress. Journal of Clinical Psychology, 55, 1051-1061.         [ Links ]

Cortina, J. M. (1993). What is coefficient alpha? An examination of theory and applications. Journal of Applied Psychology, 78, 98-104.         [ Links ]

Cox, B. J., Enns, M. W., & Clara, I. P. (2002). The multidimensional structure of perfectionism in clinical distressed and college student samples. Psychological Assessment, 14, 365-373.         [ Links ]

Cronbach, Lee J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16, 297-334.         [ Links ]

Cronbach, L. J., & Shavelson, R. J. (2004). My current thoughts on coefficient alpha and successor procedures. Educational and Psychological Measurement, 64, 391-418.         [ Links ]

Fairburn, C.G., Cooper, Z., & Shafran, R. (2003). Cognitive behavior therapy for eating disorders: A "transdiagnostic" theory and treatment. Behaviour Research and Therapy, 41, 509-528.         [ Links ]

Franco, K., Mancilla-Díaz, J. M. Vázquez, R., Alvarez, G., & López, X. (2011). El papel del perfeccionismo en la insatisfacción corporal, la influencia sociocultural del modelo de delgadez y los síntomas de trastorno del comportamiento alimentario. Universitas Psychologica, 10, 829-840.         [ Links ]

Franco, K., Mancilla-Díaz, J. M., Vázquez, R., Álvarez, G., & López, X. (2010). Estructura factorial y consistencia interna de la Escala Multidimensional de Perfeccionismo. Revista Mexicana de Psicología, 27, 143-149.         [ Links ]

Franco, K., Santoyo, F., Díaz, F. J., & Mancilla-Díaz, J. M. (In press). Estructura factorial y consistencia interna de la Escala Multidimensional de Perfeccionismo en muestra clínica. Revista Argentina de Clínica Psicológica.

Franco-Paredes, K., Mancilla-Díaz, J. M., Vázquez-Arévalo, R., López-Aguilar, X., & Alvarez-Rayón, G. (2005). Perfectionism and eating disorders: A review of the literature. European Eating Disorders Review, 13, 61-70.         [ Links ]

Frost, R. O., Marten, P., Lahart, C., & Rosenblate, R. (1990). The dimensions of perfectionism. Cognitive Therapy and Research, 14, 449-468.         [ Links ]

Garner, D. M., & Garfinkel, P. E. (1979). The Eatting Attitudes Test: An index of the symptoms of anorexia nervosa. Psychological Medicine , 9, 273-279.         [ Links ]

Gelabert, E., García-Esteve, L., Martín-Santos, R., Gutiérrez, F., Torres, A., & Subira, S. (2011). Psychometric properties of the Spanish version of the Frost Multidimensional Perfectionism Scale in women. Psicothema, 23, 133-139.         [ Links ]

González, M., Ibañez, I., Rovella, A., López, M., & Padilla, L. (2013) Perfeccionismo e intolerancia a la Incertidumbre: relaciones con variables psicopatológicas. Behavioral Psychology/Psicologia Conductual, 21, 81-101.         [ Links ]

Gregory, J. R. (2001). Evaluación psicológica, historia, principios y aplicaciones. México: Manual Moderno.         [ Links ]

Harvey, B., Pallant, J., & Harvey, D. (2004). An evaluation of the factor structure of the Frost Multidimensional Perfectionism Scale. Educational and Psychological Measurement, 64, 1007-1018.         [ Links ]

Hawkins, C. C., Watt, H. M. G., & Sinclair, K. E. (2006). Psychometric properties of the Frost Multidimensional Perfectionism Scale with Australian adolescent girls: Clarification of multidimensionality and perfectionist typology. Educational and Psychological Measurement, 66, 1001-1022.         [ Links ]

Khawaja, N. G., & Armstrong, K. A. (2005). Factor structure and psychometric properties of the Frost Multidimensional Perfectionism Scale: Developing shorter versions using an Australian sample. Australian Journal of Psychology, 57, 129-138.         [ Links ]

Lilenfeld, L.R.R., Wonderlich, S., Riso, L. P., Crosby, R., & Mitchell, J. (2006). Eating disorders and personality: A methodological and empirical review. Clinical Psychology Review, 26, 299-320.         [ Links ]

Monteiro, A. P., Soares, M. J., Pereira, A. T., Carvalho, S., Marques, M., Valente, J., ... Macedo, A. (2013). Frost Multidimensional Perfectionism Scale: The Portuguese version. Revista Psiquiatría Clínica, 40, 144-149.         [ Links ]

Nunnally, J. C., & Bernstein, I. J. (1995). Teoría psicométrica. Madrid: McGraw-Hill.         [ Links ]

Oviedo, H. C., & Campo-Arias, A. (2005). Aproximación al uso del coeficiente alfa de Cronbach. Revista Colombiana de Psiquiatría, 4, 572-580.         [ Links ]

Parker, W. D. (2000). Healthy perfectionism in the gifted. Journal of Secondary Gifted Education, 11, 173-182.         [ Links ]

Parker, W. D., & Adkins, K. K. (1995). A psychometric examination of the Multidimensional Perfectionism Scale. Journal of Psychopathology and Behavioral Assessment, 17, 323-334.         [ Links ]

Purdon, C., Antony, M. M., & Swinson, P. R. (1999). Psychometric properties of the Frost Multidimensional Perfectionism Scale in clinical anxiety disorder sample. Journal of Clinical Psychology, 55, 1271-1286.         [ Links ]

Rhéaume, J., Freeston, M. H., Dugas, M. J., Letarte, H., & Ladouceur, R. (1995). Perfectionism, responsibility, and obsessive-compulsive symptoms. Behaviour Research and Therapy, 33, 785-795.         [ Links ]

Rutsztein, G., Scappatura, M. L., & Murawski, B. (2014). Perfeccionismo y baja autoestima a través del continuo de los trastornos alimentarios en adolescentes mujeres de Buenos Aires. Revista Mexicana de Trastornos Alimentarios/Mexican Journal of Eating Disorders, 5, 39-49.         [ Links ]

Sastre-Riba, S. (2012). Alta capacidad intelectual: Perfeccionismo y regulación metacognitiva. Revista de Neurología, 54, 521-529.         [ Links ]

Scappatura, M. L., Arana, F., Elizathe, L., & Rutsztein, G. (2011). Perfeccionismo adaptativo y desadaptativo en trastornos alimentarios: Un estudio de revisión. Anuario de Investigaciones, 18, 81-88.         [ Links ]

Shafran, R., Cooper, Z., & Fairburn, C. (2002). Clinical perfectionism: A cognitive-behavioral analysis. Behaviour Research and Therapy, 40, 773-791.         [ Links ]

Stöber, J. (1998). The Frost Multidimensional Perfectionism Scale revisited: More perfect with four (instead of six) dimensions. Personality and Individual Differences, 24, 481-491.         [ Links ]

Stoeber, J., & Otto, K. (2006). Positive conceptions of perfectionism: Approaches, evidence, challenges. Personality and Social Psychology Review, 10, 295-319.         [ Links ]

Stumpf, H., & Parker, W. D. (2000). A hierarchical structural analysis of perfectionism and its relation to other personality characteristics. Personality and Individual Differences, 28, 837-852.         [ Links ]

Sutandar-Pinnock, K., Woodside, B., Carter, J., Olmsted, M., & Kaplan, A. (2003). Perfectionism in anorexia nervosa: A 6-24 month follow-up study. International Journal of Eating Disorders, 33, 225-229.         [ Links ]