¿Están sesgadas las evaluaciones de la docencia universitaria realizadas por los estudiantes?

  1. María Castro-Morera 1
  2. Enrique Navarro Asencio 1
  3. Coral González-Barbera 1
  1. 1 Universidad Complutense de Madrid
    info

    Universidad Complutense de Madrid

    Madrid, España

    ROR 02p0gd045

Revista:
Revista de educación

ISSN: 0034-8082

Año de publicación: 2024

Número: 404

Páginas: 169-200

Tipo: Artículo

DOI: 10.4438/1988-592X-RE-2024-404-621 DIALNET GOOGLE SCHOLAR lock_openAcceso abierto editor

Otras publicaciones en: Revista de educación

Resumen

Questionnaires that use students as a source of information to evaluate university teaching are a common tool in university evaluation systems. The lecturers often question their value by alluding to the possibility that students may make biased judgments, linked to teaching traits or events not related to a fair assessment of the teaching activity. The main objective of this work is to examine the relationships between the characteristics of students and lecturers and the scores on the teaching evaluation questionnaire applied to students at the Complutense University of Madrid, in order to detect possible biased patterns in the evaluation they offer of their teachers. A hierarchical linear crossclassification model was used, with two levels, taking students as the first level and the lecturers as the second. The sample of this work is composed of 143,377 surveys, completed by 33,071 students, which involved the evaluation of 7,885 teaching activities and 3,922 university teachers in the academic year of 2016- 17.The results show that the students' evaluations of their lecturers are mainly influenced by their interest in the subject, the age of the students and their lecturers and, to a lesser extent, attendance, hours of study and research quality. It should be noted that the type of undergraduate or master's degree studies, student's academic performance, and the lecturer’s job category are not related to the teaching evaluations.After this analysis of the results, we cannot deduce the existence of invalidating biases derived from the evaluation of university teaching by questionnaires answered by the students.

Referencias bibliográficas

  • ANECA (2017). Orientaciones generales para la aplicación de los criterios acreditación nacional para el acceso a los cuerpos docentes universitarios. Recuperado el 22/10/2022 de: https://acortar.link/ SMjquS
  • Basow, S. A. & Montgomery, S. (2005). Student ratings and professor selfratings of college teaching: Effects of gender and divisional affiliation. Journal of Personnel Evaluation in Education, 18, 91-106.
  • Basow, S. A., Phelan, J. E., & Capotosto, L. (2006). Gender patterns in college students’ choices of their best and worst professors. Psychology of Women Quarterly, 30(1), 25-35. https://doi.org/10.1111/j.1471- 6402.2006.00259.x
  • Beran, T., & Violato, C. (2005). Ratings of university teacher instruction: How much do student and course characteristics really matter? Assessment and Evaluation in Higher Education, 30(6), 593–601. https://doi.org/10.1080/02602930500260688
  • Berezvai, Z., Lukáts, G. D. & Molontay, R. (2021) Can professors buy better evaluation with lenient grading? The effect of grade inflation on student evaluation of teaching. Assessment Evaluation in Higher Education, 46(5), 793-808. https://doi.org/10.1080/02602938.2020.1 821866
  • Boring, A. (2017). Gender biases in student evaluations of teaching. Journal of public economics, 145, 27-41. https://doi.org/10.1016/j. jpubeco.2016.11.006
  • Boring, A, Ottoboni., K & Stark, P.B. (2016). Student evaluations of teaching (mostly) do not measure teaching effectiveness. ScienceOpen Research. 1-11. https://doi.org/10.14293/S2199-1006.1.SOR-EDU.AETBZC.v1
  • Braga, M., Paccagnella, M., & Pellizzari, M. (2014). Evaluating students’ evaluations of professors. Economics of Education Review, 41, 71-88. https://doi.org/10.1016/J.ECONEDUREV.2014.04.002
  • Carpenter, S. K., Witherby, A. E., & Tauber, S. K. (2020). On Students’ (Mis)judgments of Learning and Teaching Effectiveness. Journal of Applied Research in Memory and Cognition, 9(2), 137-151. https:// doi.org/10.1016/J.JARMAC.2019.12.009
  • Casero, A. (2008). Propuesta de un cuestionario de evaluación de la calidad docente universitaria consensuada entre alumnos y profesores. Revista de Investigación Educativa, 26(1), 25-44.
  • Casero, A. (2010). ¿Cómo es el buen profesor universitario según el alumnado? Revista Española de Pedagogía, 246, 223-242.
  • Castro, M., Navarro, E. & Blanco, A. (2020). La calidad de la docencia percibida por el alumnado y el profesorado universitarios: análisis de la dimensionalidad de un cuestionario de evaluación docente. Educación XX1, 23(2), 41-65. https://doi.org/10.5944/educXX1.25711
  • Centra, J. A., & Gaubatz, N. B. (2000). Is there gender bias in student evaluations of teaching? The Journal of Higher Education, 71, 17–33. https://doi.org/10.1080/00221546.2000.11780814
  • Clayson, D. E. (2009). Student evaluations of teaching: Are they related to what students learn? A meta-analysis and review of the literature. Journal of Marketing Education, 31(1), 16-30. https://doi. org/10.1177/0273475308324086
  • Clayson, D. E. (2018). Student evaluation of teaching and matters of reliability. Assessment Evaluation in Higher Education, 43(4), 666- 681. https://doi.org/10.1080/02602938.2017.1393495
  • Cohen, J. (1992). A power primer. Psychological Bulletin, 112(1), 155-159.
  • Cohen, P. A. (1980). Effectiveness of student-rating feedback for improving college instruction: a meta-analysis of findings. Research in Higher Education, 13(4), 321-341.
  • Cohen, P. A. (1981). Student ratings of instruction and student achievement: a meta-analysis of multisection validity studies. Review of Educational Research, 51(3), 281-309. https://doi.org/10.3102/00346543051003281
  • Cox, S. R., Rickard, M. K., & Lowery, C. M. (2021). The student evaluation of teaching: let’s be honest – who is telling the truth? Marketing Education Review, 32(1), 82-93. https://doi.org/10.1080/10528008.2 021.1922924
  • Davidovitch, N., & Soen, D. (2006). Class attendance and students’ evaluation of their college instructors. College Student Journal, 40(3), 691–703.
  • De Juanas, A. & Beltrán, J.A. (2014). Valoraciones de los estudiantes de ciencias de la educación sobre la calidad de la docencia universitaria. Educación XXI, 17(1), 59-82. https://doi.org/10.5944/ educxx1.17.1.10705
  • Esarey, J. & Valdes, N. (2020). Unbiased, reliable, and valid student evaluations can still be unfair. Assessment Evaluation in Higher Education, 45(8), 1106-1120. https://doi.org/10.1080/02602938.2020 .1724875
  • Fjortoft, N. (2005). Students’ motivation for class attendance. American Journal of Pharmaceutical Education, 69(1), 107–112.
  • García, E., Colom, X., Martínez, E., Sallarés, J. & Roca, S. (2011). La encuesta al alumnado en la evaluación de la actividad docente del profesorado. Aula abierta, 39(3), 3-14.
  • Gómez, J. C., Gómez, M., Pérez, M. C., Palazón, A. & Gómez, J. (2013). Interacción entre las expectativas académicas del alumno y la evaluación del profesor. Aula abierta, 41(2), 35-44.
  • Greimel-Fuhrmann, B., & Geyer, A. (2003). Students' evaluation of teachers and instructional quality-Analysis of relevant factors based on empirical evaluation research. Assessment Evaluation in Higher Education, 28(3), 229-238. https://doi.org/10.1080/0260293032000059595
  • Griffin, B. W. (2004). Grading leniency, grade discrepancy, and student ratings of instruction. Contemporary Educational Psychology, 29(4), 410–425. https://doi.org/10.1016/j.cedpsych.2003.11.001
  • Guinn, B., & Vincent, V. (2006). The influence of grades on teaching effectiveness ratings at a Hispanic-serving institution. Journal of Hispanic Higher Education, 5(4), 313–321. https://doi. org/10.1177/1538192706291138
  • Gump, S. E. (2007). Student evaluation of teaching effectiveness and the leniency hypothesis: A literature review. Educational Research Quarterly, 30(3), 55–68.
  • Guthrie, E. R. (1954). The evaluation of teaching: a progress report. University of Washington.
  • Hornstein, H. A. (2017). Student evaluations of teaching are an inadequate assessment tool for evaluating faculty performance. Cogent Education, 4(1), https://doi.org/10.1080/2331186X.2017.1304016
  • Johnson, R. (2000). The authority of the student evaluation questionnaire. Teaching in Higher Education, 5(4), 419–434. https://doi. org/10.1080/713699176
  • Kember, D. & Leung, D. Y. P. (2011). Disciplinary Differences in Student Ratings of Teaching Quality. Research in Higher Education, 52, 278– 299. https://doi.org/10.1007/s11162-010-9194-z
  • Kulik, J. A. (2001). Student ratings: validity, utility and controversy. New Directions for Institutional Research, 109, 9-25. https://doi. org/10.1002/ir.1
  • Lizasoain-Hernández, L., Etxeberria-Murgiondo, J., & Lukas-Mujika, J. F. (2017). A proposal for a new questionnaire for the evaluation of teachers at the University of the Basque Country. Dimensional, differential and psychometric study. RELIEVE, 23(2). https://doi. org/10.7203/relieve.23.2.10436
  • López-Cámara, A. B., González-López, I. & de León-Huertas, C. (2016). Un análisis factorial exploratorio para la construcción de un modelo de indicadores de evaluación docente universitaria. Cultura y Educación, 27(2), 337-371.
  • Lorah, J. (2018). Effect size measures for multilevel models: Definition, interpretation, and TIMSS example. Large-Scale Assessments in Education, 6(1), 1-11.
  • Marsh, H. W. (1984). Students’ evaluations of university teaching: Dimensionality, reliability, validity, potential biases and utility. Journal of Educational Psychology, 76(5), 707-754. https://doi. org/10.1037/0022-0663.76.5.707
  • Marsh, H. W. (1987). Students’ evaluations of university teaching: research findings, methodological issues and directions for future research. International Journal of Educational Research, 11(3), 253-388. https://doi.org/10.1016/0883-0355(87)90001-2
  • Marsh, H. W., & Roche, L. A. (2000). Effects of grading leniency and low workload on students’ evaluation of teaching: Popular myth, bias, validity or innocent bystanders? Journal of Educational Psychology, 92(1), 202–228. https://doi.org/10.1037/0022-0663.92.1.202
  • Mayorga, M. J., Gallardo, M. & Madrid, D. (2016). Cómo construir un cuestionario para evaluar la docencia universitaria. Revista de Ciències de l’educació, 2, 6-22. https://doi.org/10.17345/ute.2016.2.974
  • McPherson, M. A. & Jewell, R. T. (2007). Leveling the playing field: Should student evaluation scores be adjusted?. Social Science Quarterly, 88(3), 868–881. https://doi.org/10.1111/j.1540-6237.2007.00487.x
  • McPherson, M. A., Jewell, R. T., & Kim, M. (2009). What determines student evaluation scores? A random effects analysis of undergraduate economics classes. Eastern economic journal, 35(1), 37-51. https:// www.jstor.org/stable/20642462
  • Mitchell, K., & Martin, J. (2018). Gender Bias in Student Evaluations. PS: Political Science & Politics, 51(3), 648-652. https://doi.org/10.1017/ S104909651800001X
  • Mohanty, G., Gretes, J., Flowers, C., Algozzine, B., & Spooner, F. (2005). Multi- method evaluation of instruction in engineering classes. Journal of Personnel Evaluation in Higher Education, 18, 139-151. http://doi. org/10.1007/s11092-006-9006-3
  • Molero, D. & Ruíz, J. (2005). La evaluación de la docencia universitaria. Dimensiones y variables más relevantes. Revista de Investigación Educativa, 23(1), 57-84.
  • Muñoz, J. M., Ríos de Deus, M. P. & Abalde, E. (2002). Evaluación docente vs. Evaluación de la calidad. RELIEVE, 8(2).
  • Ordoñez, R. & Rodríguez, M. R. (2015). Docencia en la universidad: valoraciones de los estudiantes de la universidad de Sevilla. Bordón. Revista de Pedagogía, 67(3), 85-101. http://doi.org/10.13042/ Bordon.2015.67305
  • Paswan, A. K., & Young, J. A. (2002). Student evaluation of instructor: A nomological investigation using structural equation modeling. Journal of Marketing Education, 24(3), 193-202. https://doi. org/10.1177/0273475302238042
  • Penny, A. R. (2003). Changing the agenda for research into students views about university teaching: four shortcomings of SRT research. Teaching in Higher Education, 8(3), 399-411. https://doi. org/10.1080/13562510309396
  • Rasbasch, J., & Goldstein, H. (1994). Efficient analysis of mixed hierarchical and cross- classified random structures using a multilevel model. Journal of Educational and Behavioral Statistics, 19(4), 337– 350. https://doi.org/10.2307/1165397
  • Rivera, L. A., & Tilcsik, A. (2019). Scaling down inequality: Rating scales, gender bias, and the architecture of evaluation. American Sociological Review, 84(2), 248-274. https://doi.org/10.1177/0003122419833601
  • Snijders, T. A. B., & Bosker, R. J. (2012). Multilevel analysis: An introduction to basic and advanced multilevel modeling. Sage.
  • Spencer, K. J., & Schmelkin, L. P. (2002). Student perspectives on teaching and its evaluation. Assessment and Evaluation in Higher Education, 27(5), 397-409. https://doi.org/10.1080/0260293022000009285
  • Spooren, P. (2010). On the credibility of the judge. A cross-classified multilevel analysis on student evaluations of teaching. Studies in Educational Evaluation, 36(4), 121-131. https://doi.org/10.1016/j. stueduc.2011.02.001
  • Spooren, P.; Brockx, B. & Mortelmans, D. (2013). On the validity of student evaluation of teaching: the state of the art. Review of Educational Research, 83(4), 598-642. https://doi.org/10.3102/0034654313496870
  • Spooren, P.; Mortelmans, D. & Christiaens, W. (2014). Assessing the validity and reliability of a quick scan for student's evaluation of teaching. Results from confirmatory factor analysis and G Theory. Studies in Educational Evaluation, 43, 88-94. https://doi.org/10.1016/j. stueduc.2014.03.001
  • Spooren, P.; Vandermoere, F.; Vanderstraeten & Pepersmans, K. (2017). Exploring high impact scholarship in research on students evaluation of teaching (SET). Educational Research Review, 22, 129-141. https:// doi.org/10.1016/j.edurev.2017.09.001
  • Sprinkle, J. E. (2008). Student Perceptions of Effectiveness: An Examination of the Influence of Student Biases. College Student Journal, 42(2), 276–293.
  • Stark-Wroblewski, K., Ahlering, R. F., & Brill, F. M. (2007). Toward a more comprehensive approach to evaluating teaching effectiveness: Supplementing student evaluations of teaching with pre-post learning measures. Assessment & Evaluation in Higher Education, 32(4), 403– 415. https://doi.org/10.1080/02602930600898536
  • Sulis, I., Porcu, M. & Capursi, V. (2019). On the use of the Student Evaluation of Teaching: A longitudinal analysis combining measurement issues and implications of the exercise. Social Indicators Research, 142, 1305-1331. https://doi.org/10.1007/s11205-018-1946-8
  • Ting, K. (2000). A multilevel perspective on student ratings of instruction: Lessons from the Chinese experience. Research in Higher Education, 41, 637–661. https://doi.org/10.1023/A:1007075516271
  • Theall, M., & Franklin, J. (2001). Looking for bias in all the wrong places: A search for truth or a witch hunt in student ratings of instruction? New Directions for Institutional Research, 109, 45–56. https://doi. org/10.1002/ir.3
  • Uttl, B., White, C. A., & Gonzalez, D. W. (2017). Meta-analysis of faculty's teaching effectiveness: Student evaluation of teaching ratings and student learning are not related. Studies in Educational Evaluation, 54, 22–42. https://doi.org/10.1016/j.stueduc.2016.08.007
  • Wachtel, H. K. (1998). Student evaluation of college teaching effectiveness: A brief review. Assessment and Evaluation in Higher Education, 23(2), 191–210. https://doi.org/10.1080/0260293980230207
Fuente de los datos: Dialnet