<?xml version="1.0" encoding="ISO-8859-1"?><article xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<front>
<journal-meta>
<journal-id>1405-6666</journal-id>
<journal-title><![CDATA[Revista mexicana de investigación educativa]]></journal-title>
<abbrev-journal-title><![CDATA[RMIE]]></abbrev-journal-title>
<issn>1405-6666</issn>
<publisher>
<publisher-name><![CDATA[Consejo Mexicano de Investigación Educativa A.C.]]></publisher-name>
</publisher>
</journal-meta>
<article-meta>
<article-id>S1405-66662018000200597</article-id>
<title-group>
<article-title xml:lang="es"><![CDATA[El uso de Many-Facet Rasch Measurement para examinar la calidad del proceso de corrección de pruebas de desempeño]]></article-title>
<article-title xml:lang="en"><![CDATA[The Use of Many-Facet Rasch Measurement to Examine the Quality of Evaluating Performance Assessments]]></article-title>
</title-group>
<contrib-group>
<contrib contrib-type="author">
<name>
<surname><![CDATA[Mendoza Ramos]]></surname>
<given-names><![CDATA[Arturo]]></given-names>
</name>
<xref ref-type="aff" rid="Aff"/>
</contrib>
</contrib-group>
<aff id="Af1">
<institution><![CDATA[,Universidad Nacional Autónoma de México Escuela Nacional de Lenguas, Lingüística y Traducción Departamento de Lingüística Aplicada]]></institution>
<addr-line><![CDATA[Ciudad de México ]]></addr-line>
<country>Mexico</country>
</aff>
<pub-date pub-type="pub">
<day>00</day>
<month>06</month>
<year>2018</year>
</pub-date>
<pub-date pub-type="epub">
<day>00</day>
<month>06</month>
<year>2018</year>
</pub-date>
<volume>23</volume>
<numero>77</numero>
<fpage>597</fpage>
<lpage>625</lpage>
<copyright-statement/>
<copyright-year/>
<self-uri xlink:href="http://www.scielo.org.mx/scielo.php?script=sci_arttext&amp;pid=S1405-66662018000200597&amp;lng=en&amp;nrm=iso"></self-uri><self-uri xlink:href="http://www.scielo.org.mx/scielo.php?script=sci_abstract&amp;pid=S1405-66662018000200597&amp;lng=en&amp;nrm=iso"></self-uri><self-uri xlink:href="http://www.scielo.org.mx/scielo.php?script=sci_pdf&amp;pid=S1405-66662018000200597&amp;lng=en&amp;nrm=iso"></self-uri><abstract abstract-type="short" xml:lang="es"><p><![CDATA[Resumen: Las pruebas de desempeño son criticadas por la supuesta falta de consistencia en los resultados que otorgan los evaluadores. Sin embargo, herramientas estadísticas como Many-Facet Rasch Measurement (MFRM) son útiles para examinar la calidad del proceso de evaluación en pruebas con múltiples facetas de variabilidad. El objetivo de este artículo es dar a conocer el funcionamiento y aportaciones de MFRM en pruebas de desempeño. El estudio se realizó con estudiantes universitarios no hispanohablantes que sustentaron una prueba escrita de Español con fines académicos. Los resultados mostraron niveles de severidad e indulgencia adecuados por parte de los evaluadores, y de dificultad en las tareas y en la rúbrica analítica empleada para evaluar los textos. Del estudio se concluye la utilidad de MFRM para examinar el proceso de corrección de pruebas de desempeño.]]></p></abstract>
<abstract abstract-type="short" xml:lang="en"><p><![CDATA[Abstract: Performance assessments are criticized for the assumed lack of consistency in the results delivered by evaluators. However, statistical tools such as the Many-Facet Rasch Measurement (MFRM) are useful for examining the quality of the assessment process in tests with multiple facets of variability. The objective of this article is to present the functioning and contributions of MFRM on performance assessments. The study was carried out with non-Spanish-speaking university students who completed a written test of Spanish with academic ends. The results showed adequate levels of severity and indulgence among the evaluators, as well as in task difficulty and the analytical rubric employed to evaluate the texts. The conclusion is that MFRM is useful for examining the process of evaluating performance assessments.]]></p></abstract>
<kwd-group>
<kwd lng="es"><![CDATA[análisis estadístico]]></kwd>
<kwd lng="es"><![CDATA[evaluación cuantitativa]]></kwd>
<kwd lng="es"><![CDATA[evaluación académica]]></kwd>
<kwd lng="es"><![CDATA[exámenes y certificación]]></kwd>
<kwd lng="en"><![CDATA[statistical analysis]]></kwd>
<kwd lng="en"><![CDATA[quantitative evaluation]]></kwd>
<kwd lng="en"><![CDATA[academic evaluation]]></kwd>
<kwd lng="en"><![CDATA[examinations and certification]]></kwd>
</kwd-group>
</article-meta>
</front><back>
<ref-list>
<ref id="B1">
<nlm-citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname><![CDATA[Attali]]></surname>
<given-names><![CDATA[Yigal]]></given-names>
</name>
<name>
<surname><![CDATA[Lewis]]></surname>
<given-names><![CDATA[Will]]></given-names>
</name>
<name>
<surname><![CDATA[Steier]]></surname>
<given-names><![CDATA[Michael]]></given-names>
</name>
</person-group>
<article-title xml:lang=""><![CDATA[Scoring with the computer: Alternative procedures for improving the reliability of holistic essay scoring]]></article-title>
<source><![CDATA[Language Testing]]></source>
<year>2012</year>
<volume>30</volume>
<numero>1</numero>
<issue>1</issue>
<page-range>125-41</page-range></nlm-citation>
</ref>
<ref id="B2">
<nlm-citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname><![CDATA[Barkaoui]]></surname>
<given-names><![CDATA[Khaled]]></given-names>
</name>
</person-group>
<article-title xml:lang=""><![CDATA[Multifaceted Rasch Analysis for Test Evaluation]]></article-title>
<person-group person-group-type="editor">
<name>
<surname><![CDATA[Kunnan]]></surname>
<given-names><![CDATA[A.]]></given-names>
</name>
</person-group>
<source><![CDATA[The Companion to Language assessment]]></source>
<year>2014</year>
<volume>3</volume>
<page-range>1301-22</page-range><publisher-loc><![CDATA[Oxford, Reino Unido ]]></publisher-loc>
<publisher-name><![CDATA[Wiley-Blackwell]]></publisher-name>
</nlm-citation>
</ref>
<ref id="B3">
<nlm-citation citation-type="book">
<person-group person-group-type="author">
<name>
<surname><![CDATA[Bond]]></surname>
<given-names><![CDATA[Trevor]]></given-names>
</name>
<name>
<surname><![CDATA[Fox]]></surname>
<given-names><![CDATA[Christine M.]]></given-names>
</name>
</person-group>
<source><![CDATA[Applying the Rasch model: Fundamental measurement in the human sciences]]></source>
<year>2007</year>
<edition>2</edition>
<publisher-loc><![CDATA[Mahwah, NJ ]]></publisher-loc>
<publisher-name><![CDATA[Erlbaum]]></publisher-name>
</nlm-citation>
</ref>
<ref id="B4">
<nlm-citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname><![CDATA[East]]></surname>
<given-names><![CDATA[Martin]]></given-names>
</name>
</person-group>
<article-title xml:lang=""><![CDATA[Evaluating the reliability of a detailed analytic scoring rubric for foreign language writing]]></article-title>
<source><![CDATA[Assessing Writing]]></source>
<year>2009</year>
<volume>14</volume>
<numero>2</numero>
<issue>2</issue>
<page-range>88-115</page-range></nlm-citation>
</ref>
<ref id="B5">
<nlm-citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname><![CDATA[Eckes]]></surname>
<given-names><![CDATA[Thomas]]></given-names>
</name>
</person-group>
<article-title xml:lang=""><![CDATA[Examining rater effects in TestDaF writing and speaking performance assessments: A many-facet Rasch analysis]]></article-title>
<source><![CDATA[Language Assessment Quarterly]]></source>
<year>2005</year>
<volume>2</volume>
<page-range>197-221</page-range></nlm-citation>
</ref>
<ref id="B6">
<nlm-citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname><![CDATA[Eckes]]></surname>
<given-names><![CDATA[Thomas]]></given-names>
</name>
</person-group>
<article-title xml:lang=""><![CDATA[Many-Facet Rasch Measurement]]></article-title>
<person-group person-group-type="editor">
<name>
<surname><![CDATA[Takala]]></surname>
<given-names><![CDATA[S.]]></given-names>
</name>
</person-group>
<source><![CDATA[Reference supplement to the manual for relating language examinations to the Common European Framework of Reference for Languages: Learning, teaching, assessment (Section H)]]></source>
<year>2009</year>
<publisher-loc><![CDATA[Strasbourg ]]></publisher-loc>
<publisher-name><![CDATA[Council of Europe]]></publisher-name>
</nlm-citation>
</ref>
<ref id="B7">
<nlm-citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname><![CDATA[Eckes]]></surname>
<given-names><![CDATA[Thomas]]></given-names>
</name>
</person-group>
<article-title xml:lang=""><![CDATA[Operational rater types in writing assessment: Linking rater cognition to rater behavior]]></article-title>
<source><![CDATA[Language Assessment Quarterly]]></source>
<year>2012</year>
<volume>9</volume>
<numero>3</numero>
<issue>3</issue>
<page-range>270-92</page-range></nlm-citation>
</ref>
<ref id="B8">
<nlm-citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname><![CDATA[Engelhard]]></surname>
<given-names><![CDATA[George]]></given-names>
</name>
</person-group>
<article-title xml:lang=""><![CDATA[Standard errors for performance standards based on bookmark judgments]]></article-title>
<source><![CDATA[Rasch Measurement Transactions]]></source>
<year>2008</year>
<volume>21</volume>
<page-range>1132-3</page-range></nlm-citation>
</ref>
<ref id="B9">
<nlm-citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname><![CDATA[Esfandiari]]></surname>
<given-names><![CDATA[Rajab]]></given-names>
</name>
<name>
<surname><![CDATA[Myford]]></surname>
<given-names><![CDATA[Carol M.]]></given-names>
</name>
</person-group>
<article-title xml:lang=""><![CDATA[Severity differences among self-assessors, peer-assessors, and teacher assessors rating EFL essays]]></article-title>
<source><![CDATA[Assessing Writing]]></source>
<year>2013</year>
<volume>18</volume>
<numero>2</numero>
<issue>2</issue>
<page-range>111-31</page-range></nlm-citation>
</ref>
<ref id="B10">
<nlm-citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname><![CDATA[Hamp-Lyons]]></surname>
<given-names><![CDATA[Liz]]></given-names>
</name>
</person-group>
<article-title xml:lang=""><![CDATA[Worrying about rating]]></article-title>
<source><![CDATA[Assessing Writing]]></source>
<year>2007</year>
<volume>12</volume>
<page-range>1-9</page-range></nlm-citation>
</ref>
<ref id="B11">
<nlm-citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname><![CDATA[Huang]]></surname>
<given-names><![CDATA[Jinyan]]></given-names>
</name>
<name>
<surname><![CDATA[Foote]]></surname>
<given-names><![CDATA[Chandra J.]]></given-names>
</name>
</person-group>
<article-title xml:lang=""><![CDATA[Grading between the lines: What really impacts professors&#8217; holistic evaluation of ESL graduate student writing?]]></article-title>
<source><![CDATA[Language Assessment Quarterly]]></source>
<year>2010</year>
<volume>7</volume>
<numero>1</numero>
<issue>1</issue>
<page-range>37-41</page-range></nlm-citation>
</ref>
<ref id="B12">
<nlm-citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname><![CDATA[Knoch]]></surname>
<given-names><![CDATA[Ute]]></given-names>
</name>
</person-group>
<article-title xml:lang=""><![CDATA[Do empirically developed rating scales function differently to conventional rating scales for academic writing?]]></article-title>
<source><![CDATA[Spaan Fellow Working Papers in Second or Foreign Language Assessment]]></source>
<year>2007</year>
<volume>5</volume>
<page-range>1-36</page-range></nlm-citation>
</ref>
<ref id="B13">
<nlm-citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname><![CDATA[Knoch]]></surname>
<given-names><![CDATA[Ute]]></given-names>
</name>
</person-group>
<article-title xml:lang=""><![CDATA[Diagnostic assessment of writing: A comparison of two rating scales]]></article-title>
<source><![CDATA[Language Testing]]></source>
<year>2009</year>
<volume>26</volume>
<numero>2</numero>
<issue>2</issue>
<page-range>275-304</page-range></nlm-citation>
</ref>
<ref id="B14">
<nlm-citation citation-type="book">
<person-group person-group-type="author">
<name>
<surname><![CDATA[Linacre]]></surname>
<given-names><![CDATA[John M.]]></given-names>
</name>
</person-group>
<source><![CDATA[A user&#8217;s guide to facets: Rasch-Model Computer Programs]]></source>
<year>2011</year>
<publisher-loc><![CDATA[Chicago ]]></publisher-loc>
<publisher-name><![CDATA[Winsteps.com]]></publisher-name>
</nlm-citation>
</ref>
<ref id="B15">
<nlm-citation citation-type="">
<person-group person-group-type="author">
<name>
<surname><![CDATA[Linacre]]></surname>
<given-names><![CDATA[John M.]]></given-names>
</name>
</person-group>
<source><![CDATA[Facets Tutorial 1. 1-32]]></source>
<year>2012</year>
</nlm-citation>
</ref>
<ref id="B16">
<nlm-citation citation-type="">
<person-group person-group-type="author">
<name>
<surname><![CDATA[Linacre]]></surname>
<given-names><![CDATA[John M.]]></given-names>
</name>
</person-group>
<source><![CDATA[Facets Tutorial 2. 1-40]]></source>
<year>2012</year>
</nlm-citation>
</ref>
<ref id="B17">
<nlm-citation citation-type="">
<person-group person-group-type="author">
<name>
<surname><![CDATA[Linacre]]></surname>
<given-names><![CDATA[John M.]]></given-names>
</name>
</person-group>
<source><![CDATA[Facets Tutorial 3]]></source>
<year>2012</year>
<page-range>1-29</page-range></nlm-citation>
</ref>
<ref id="B18">
<nlm-citation citation-type="">
<person-group person-group-type="author">
<name>
<surname><![CDATA[Linacre]]></surname>
<given-names><![CDATA[John M.]]></given-names>
</name>
</person-group>
<source><![CDATA[Facets Tutorial 3. 1-18]]></source>
<year>2012</year>
</nlm-citation>
</ref>
<ref id="B19">
<nlm-citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname><![CDATA[Linacre]]></surname>
<given-names><![CDATA[John M.]]></given-names>
</name>
</person-group>
<article-title xml:lang=""><![CDATA[A user&#8217;s guide to facets. Rasch-Model Computer Programs]]></article-title>
<source><![CDATA[Program Manual 3.71.0]]></source>
<year>2013</year>
</nlm-citation>
</ref>
<ref id="B20">
<nlm-citation citation-type="book">
<person-group person-group-type="author">
<name>
<surname><![CDATA[Linacre]]></surname>
<given-names><![CDATA[John. M.]]></given-names>
</name>
</person-group>
<source><![CDATA[Facets computer program for Many-facet Rasch Measurement, versión 3.71.4]]></source>
<year>2015</year>
<publisher-loc><![CDATA[Beaverton ]]></publisher-loc>
<publisher-name><![CDATA[Winsteps.com]]></publisher-name>
</nlm-citation>
</ref>
<ref id="B21">
<nlm-citation citation-type="book">
<person-group person-group-type="author">
<name>
<surname><![CDATA[McNamara]]></surname>
<given-names><![CDATA[Tim F.]]></given-names>
</name>
</person-group>
<source><![CDATA[Measuring second language performance]]></source>
<year>1996</year>
<publisher-loc><![CDATA[Londres, Reino Unido ]]></publisher-loc>
<publisher-name><![CDATA[Longman]]></publisher-name>
</nlm-citation>
</ref>
<ref id="B22">
<nlm-citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname><![CDATA[Mendoza]]></surname>
<given-names><![CDATA[Arturo]]></given-names>
</name>
</person-group>
<article-title xml:lang=""><![CDATA[La selección de las tareas de escritura en los exámenes de lengua extranjera destinados al ámbito académico]]></article-title>
<source><![CDATA[Revista Nebrija de Lingüística Aplicada a la Enseñanza de Lenguas]]></source>
<year>2015</year>
<numero>18</numero>
<issue>18</issue>
<page-range>106-23</page-range></nlm-citation>
</ref>
<ref id="B23">
<nlm-citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname><![CDATA[Mendoza]]></surname>
<given-names><![CDATA[Arturo]]></given-names>
</name>
<name>
<surname><![CDATA[Knoch]]></surname>
<given-names><![CDATA[Ute]]></given-names>
</name>
</person-group>
<article-title xml:lang=""><![CDATA[Examining the validity of the analytic rating scale for a Spanish test for academic purposes using the argument-based approach to validation]]></article-title>
<source><![CDATA[Assessing Writing]]></source>
<year>2018</year>
<numero>35</numero>
<issue>35</issue>
<page-range>41-55</page-range></nlm-citation>
</ref>
<ref id="B24">
<nlm-citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname><![CDATA[Myford]]></surname>
<given-names><![CDATA[Carol M.]]></given-names>
</name>
<name>
<surname><![CDATA[Wolfe]]></surname>
<given-names><![CDATA[Edward W.]]></given-names>
</name>
</person-group>
<article-title xml:lang=""><![CDATA[Detecting and measuring rater effects using Many-Facet Rasch Measurement: Part I]]></article-title>
<source><![CDATA[Journal of Applied Measurement]]></source>
<year>2003</year>
<volume>4</volume>
<page-range>386-422</page-range></nlm-citation>
</ref>
<ref id="B25">
<nlm-citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname><![CDATA[Myford]]></surname>
<given-names><![CDATA[Carol M.]]></given-names>
</name>
<name>
<surname><![CDATA[Wolfe]]></surname>
<given-names><![CDATA[Edward W.]]></given-names>
</name>
</person-group>
<article-title xml:lang=""><![CDATA[Detecting and measuring rater effects using Many-Facet Rasch Measurement: Part II]]></article-title>
<source><![CDATA[Journal of Applied Measurement]]></source>
<year>2004</year>
<volume>5</volume>
<page-range>189-227</page-range></nlm-citation>
</ref>
<ref id="B26">
<nlm-citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname><![CDATA[Prieto]]></surname>
<given-names><![CDATA[Gerardo]]></given-names>
</name>
</person-group>
<article-title xml:lang=""><![CDATA[Evaluación de la ejecución mediante el modelo Many-Facet Rasch Measurement]]></article-title>
<source><![CDATA[Psicothema]]></source>
<year>2011</year>
<volume>23</volume>
<numero>2</numero>
<issue>2</issue>
<page-range>233-8</page-range></nlm-citation>
</ref>
<ref id="B27">
<nlm-citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname><![CDATA[Prieto]]></surname>
<given-names><![CDATA[Gerardo]]></given-names>
</name>
<name>
<surname><![CDATA[Nieto]]></surname>
<given-names><![CDATA[Eloísa]]></given-names>
</name>
</person-group>
<article-title xml:lang=""><![CDATA[Analysis of rater severity on written expression exam using Many-Faceted Rasch Measurement]]></article-title>
<source><![CDATA[Psicológica]]></source>
<year>2014</year>
<volume>35</volume>
<page-range>285-397</page-range></nlm-citation>
</ref>
<ref id="B28">
<nlm-citation citation-type="book">
<person-group person-group-type="author">
<name>
<surname><![CDATA[Rasch]]></surname>
<given-names><![CDATA[George]]></given-names>
</name>
</person-group>
<source><![CDATA[Probabilistic models for some intelligence and attainment tests]]></source>
<year>1960</year>
<publisher-loc><![CDATA[Chicago ]]></publisher-loc>
<publisher-name><![CDATA[Mesa Press]]></publisher-name>
</nlm-citation>
</ref>
<ref id="B29">
<nlm-citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname><![CDATA[Rezaei]]></surname>
<given-names><![CDATA[Ali R.]]></given-names>
</name>
<name>
<surname><![CDATA[Lovorn]]></surname>
<given-names><![CDATA[Michael]]></given-names>
</name>
</person-group>
<article-title xml:lang=""><![CDATA[Reliability and validity of rubrics for assessment through writing]]></article-title>
<source><![CDATA[Assessing Writing]]></source>
<year>2010</year>
<volume>15</volume>
<numero>1</numero>
<issue>1</issue>
<page-range>18-39</page-range></nlm-citation>
</ref>
<ref id="B30">
<nlm-citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname><![CDATA[Wind]]></surname>
<given-names><![CDATA[Stefanie A.]]></given-names>
</name>
<name>
<surname><![CDATA[Engelhard]]></surname>
<given-names><![CDATA[George]]></given-names>
</name>
</person-group>
<article-title xml:lang=""><![CDATA[How invariant and accurate are domain ratings in writing assessment?]]></article-title>
<source><![CDATA[Assessing Writing]]></source>
<year>2013</year>
<volume>18</volume>
<numero>4</numero>
<issue>4</issue>
<page-range>278-99</page-range></nlm-citation>
</ref>
<ref id="B31">
<nlm-citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname><![CDATA[Wright]]></surname>
<given-names><![CDATA[Benjamin D.]]></given-names>
</name>
<name>
<surname><![CDATA[Linacre]]></surname>
<given-names><![CDATA[John M.]]></given-names>
</name>
</person-group>
<article-title xml:lang=""><![CDATA[Reasonable mean-square fit values]]></article-title>
<source><![CDATA[Rasch Measurement Transactions]]></source>
<year>1994</year>
<volume>8</volume>
<numero>3</numero>
<issue>3</issue>
<page-range>370</page-range></nlm-citation>
</ref>
</ref-list>
</back>
</article>
