Welcome to the Research Unit Educational Psychology with Focus on Measurement, Counseling, and Evaluation!

In our research we focus on the following questions:

(a) How can digital technology and artificial intelligence help to innovate educational measurement instruments?

(b) How should education-relevant empirical data be statistically modeled?

(c) What insights do large-scale assessments such as PISA, IGLU or TIMSS provide about educational systems?

In current projects, we deal, among other things, with the use of competence-oriented feedback generated with artificial intelligence (AI) in higher education (HIKOF-DL), with the reading comprehension of fourth graders (PIRLS), with the identification and individual support of students at risk in secondary education (CoBi, CoBi-Transfer, CoBi-Online), the establishment and evaluation of an interdisciplinary master's program on the use of AI and digital technologies in education (ALI), the implementation of AI-based feedback and assessment in the study entry phase (IMPACT), the development of a highly adaptive test design for PISA (HAT), the advancement of the Goethe Test Pro (GTP+) and the advanccement of teaching, learning and examination scenarios by digital technology and innovative learning designs (DigiTeLL).

 

In teaching we are active in the B.Sc. and M.Sc. Psychology, in the educational science parts of the teacher training programs and the M.Sc. Clinical Psychology and Psychotherapy.


Contribution at the NCME conference 2024

Our lab was represented with the following presentation by Aron Fink at the NCME conference in Philadelphia.

Fink, A., Gombert, S., Liu, T., & Frey, A., (2024, April). The integration of automated essay scoring models using hierarchical rater models. Paper presented at the Annual Meeting of the National Council on Measurement in Education (NCME), Philadelphia, USA.

The conference took place with the title "Reconceptualizing Best Practices in Measurement to Reduce Inequities". Further information on the conference can be found here.

May 29th, 2024


Contributions at the GEBF-Conference in Potsdam

Our lab was represented with the following three presentations at the eleventh GEBF-Conference in Potsdam:

  • König, C., Frey, A., & Goldhammer, F.: Leistungsemotionen im Vergleich zwischen papier- und computerbasierter Testung bei IGLU 2021. [Comparison of paper-based and computer-based testing regarding achievement emotions in PIRLS 2021]
  • Frey, A., Liu, T., Fink, A., & König, C.: Metaanalyse zu den Effekten computerisierten adaptiven Testens auf Motivation und Emotion von Testpersonen. [Meta-analysis of the effects of computerised adaptive testing on motivation and emotion of test participants]
  • Schenk, C., Naumann, P., & Frey, A.: Emotionale Effekte verschiedener Item-Review-Verfahren bei adaptiven Hochschulklausuren. [Emotional effects of different approaches for item review in adaptive university exams]

The conference took place under the motto "Understanding Education - Achieving Participation - Shaping Transfer". The (German) website of the conference with further information can be found here

March 26th, 2024


Interview with Thomas Dreisörner in "17:30 Sat.1 Live - Regionalmagazin für Rheinland-Pfalz und Hessen"

ADHS diagnosis via YouTube and TikTok or better ask a specialist?

The video "Lilly aus Gießen klärt online über ADHS auf" (Lilly from Gießen explains ADHD online), broadcast on Sat.1 on February 5th 2024, deals with the danger of self-diagnosis, in this case using the example of ADHD. Thomas Dreisörner was interviewed. As a specialist, he advises against using social media such as YouTube or TikTok for self-diagnosis. "If my car isn't working properly, I also wouldn't use TikTok to find the problem but ask a mechanic instead."

February 06th, 2024


New study on automated content analysis published in the International Journal for Artificial Intelligence in Education

   

In cooperation with the DIPF, a new article has been published within the framework of the HIKOF-DL project. The article presents a case study in which we implemented a pipeline for the automated scoring of essay content using natural language processing to provide students with automated, highly informative feedback. 689 manually coded essays were used as the training data.

Various models for automated scoring were trained (GBERT, T5 and bag-of-words baseline models). In particular, the transformer-based models showed high performance. Finally, the GBERT-based model was used for the automated evaluation of the essays, on the basis of which the automated feedback was derived.

Gombert, S., Fink, A., Giorgashvili, T. et al. (2024). From the automated assessment of student essay content to highly informative feedback: A case study. International Journal of Artificial Intelligence in Educationhttps://doi.org/10.1007/s40593-023-00387-6

January 29th, 2024


Study on new PISA design published in the Journal of Educational Measurement

   

The article authored by Andreas Frey, Christoph König and Aron Fink, proposes a new test design that is much more adaptive than the test design currently used in PISA. The results of a simulation study, which is also presented in the article, show that the new design meets content constraints better (such as the required proportions of certain item groups) and results in significantly higher measurement precision compared to the current PISA design. 

The study is a good example of how findings from basic psychometric research can be used to further develop socially relevant studies that attract public attention.

We hope that the new design will be influential regarding the further methodological development of PISA.

The study was conducted as part of the “PISA Research, Development and Innovation (RDI) project on Multistage Adaptive Testing” of the OECD and was funded by the OECD.

The reference reads as follows; the paper is available with Open Access:

Frey, A., König, C., & Fink, A. (2023). A highly adaptive testing design for PISA. Journal of Educational Measurement. Advance online publication. https://doi.org/10.1111/jedm.12382 

December 21st, 2023


End of the year 2023 with Christmas treats and suspense

This year we celebrated our end-of-year party at the headquarters, which proved to be logistically clever because the train drivers' strike began in the evening. Unfortunately, the train drivers also tore our two MCs Anelia Vanova and Nariman Utegaliyev away from us before the end of the party. Nonetheless, we all enjoyed the Christmas treats, the cheesy Christmas music and the obligatory junk gift giving together. Once again, extremely beautifully shaped, mysterious and scary gifts changed hands several times. The boss handed out the 2023 work report and looked back with us on a successful year. We also received his thanks and appreciation individually. Afterwards, we tucked into the buffet and one of the exit games that got our brains fuming. We freed ourselves from the perilous station in the eternal ice and were very silly about it.

The entire department hopes that this positive ending brings in a good year for us all and wishe our visitors to this page a Merry Christmas and a happy and prosperous New Year!


Retreat of the Professorship 2023

Our retreat this year took place over two days in the beautiful Spessart in the countryside, in a remote inn with regional full supply. Here we were able to look back and ahead in peace and quiet, set goals, revise and optimize the departmental structure, and work out the medium- and long-term plans for our professorship with very entertaining and effective tasks. In addition, our stay was dominated by unrestrained gluttony, which we could not control even with a two-hour hike. We left the wonderful place with new tasks in our pockets and a strengthened sense of unity.

October 19th, 2023


Contribution at the 17th DPPD Conference


Our research unit was represented by a contribution from Tuo Liu at the 17th Conference of the DGPS Division of Differential Psychology, Personality Psychology, and Psychological Assessment (DPPD) in Salzburg, Austria.

Liu, T., Molenaar, D., & Hildebrandt, A. (2023, September). Improving the precision of emotion identification task: A comparison of keyboard response times and mouse-tracking metrics in a joint model analysis. Talk presented at the 17. Arbeitstagung der Fachgruppe Differentielle Psychologie, Persönlichkeitspsychologie und Psychologische Diagnostik [17th Working Meeting of the Division of Differential Psychology, Personality Psychology, and Psychological Assessment] (DPPD), Universität Salzburg [University of Salzburg].

For more information about the conference, click here.

October 5th, 2023


New Publication from the HIKOF project

A new article was published in the proceedings of this year's EC-TEL conference.

The paper introduces an automated approach designed to provide comprehensive and insightful text-based feedback on the processes involved in CSCL forum discussions within an introductory teacher education course. Our primary focus is on assessing the group communication skills of individual students. This feedback is systematically generated by analyzing various discourse indicators, which are then utilized to identify the evolving roles that students assume within their groups. Consequently, we derive feedback that highlights both their strengths and areas for potential improvement.

Menzel, L., Gombert, S., Weidlich, J., Fink, A., Frey, A., Drachsler, H. (2023). Why You Should Give Your Students Automatic Process Feedback on Their Collaboration: Evidence from a Randomized Experiment. In: Viberg, O., Jivet, I., Muñoz-Merino, P., Perifanou, M., Papathoma, T. (eds) Responsive and Sustainable Educational Futures. EC-TEL 2023. Lecture Notes in Computer Science, vol 14200. Springer, Cham. https://doi.org/10.1007/978-3-031-42682-7_14

September 28th, 2023


Paper presented at the FREMO conference in Oslo, Norway

Our lab was represented with the following presentation at the FREMO conference in Oslo, Norway:

Fink, A. Gombert, S., & Frey, A.: Using Natural Language Processing to Automate the Assessment of Student Essay Content

Further information on the conference can be found here.

September 28th, 2023


Two paper presentations at the PAEPS meeting in Kiel, Germany

Our lab was represented with the following two presentations at the 19th meeting of the DGPs section Educational Psychology in Kiel, Germany:

  • Frey, A., König, C., & Goldhammer, F.: Leistungsemotionen im Vergleich zwischen papier- und computerbasierter Testung bei IGLU 2021 [Comparison of paper-based and computer-based testing regarding achievement emotions in PIRLS 2021].
  • Hahnel, C. Kröhne, U., Frey, A., Ludewig, U., & Goldhammer, F.: Unterscheidet sich die mentale Fokussiertheit beim Lesen am Computer vom Lesen auf Papier? Eine Ergänzungsstudie zu IGLU 2021. [Does mental focus in computer reading differ from reading on paper? A Supplementary Study to IGLU 2021]

Further information on the conference can be found here.

September 22nd, 2023 


Five papers at the 16th FGME conference in Constance


Our lab was represented with the following five presentations at the FGME meeting in Constance, Germany:

  • Fink, A. Gombert, S., & Frey, A.: Automatisierte Inhaltsanalyse von Essays mittels Transformern [Automated essay scoring using transformers]
  • Frey, A., Fink, A., König, C., & Liu, T.: Metaanalyse zu den Effekten des computerisierten adaptiven Testens auf Motivation und Emotion [Meta-analysis on the effects of computerized adaptive testing on motivation and emotion]
  • König, C., Fink, A., & Frey, A.: Der Einfluss hoch-adaptiven Testens auf den Ländervergleich bei PISA [The effect of highly adaptive testing on the international comparison in PISA.]
  • Liu, T. & Epskamp, S.: Adaptive Form of Network Psychometrics: a simulation study.
  • Ranger, J., Koenig, C., Domingue, B., Kuhn, J. T. & Frey, A.: A Multidimensional Partially-Compensatory Response Time Model on Basis of the Log-Normal Distribution.

Further information on the conference can be found here.

September 22nd, 2023


Information material on topic "Competency-based Testing" available online

Within the context of the project "DigiTeLL", Andreas Frey and Cosima Schenk created information material on the topic of "competency-based testing", which gives an overview of the work steps that are neccessary for the development of competency-based exams and explains the construction of multiple choice items - especially for the assessment of higher cognitive processes.

The materials can be viewed and downloaded from the DigiTeLL Website

September 21st, 2023


Presentation at earli 2023

Andreas Frey served as invited discussant of the four individual presentations of the symposium "Effects of COVID-19 pandemic on student achievement" (chairperson: Natalie Foerster, Münster University, Germany) at the earli 2023 conference (as video). 

August 29th, 2023


Nariman Utegaliyev newest member of the work unit

The work unit welcomes Nariman Utegaliyev as its newest member!

August 1st, 2023


Our work unit at the EAM2023

From 11th to 13th July the 10th European Congress of Methodology (EAM 2023) took place at Ghent University, Belgium. The unit of educational psychology, with focus on counseling, measurement, & evaluation, contributed to the congress with three presentations.

Prof. Dr. Andreas Frey gave a presentation entitled "A Highly Adaptive Testing Design for PISA". Dr Christoph König gave a talk entitled "A Similarity-Weighted Informative Prior Distribution for Bayesian Multiple Regression Models". Tuo Liu presented a poster entitled "Enhancing Predictive Cut Scores in Higher Education Enrolment with Explainable Machine Learning Algorithms".


National German Report PIRLS 2021

On May 16, 2021, the results of the PIRLS 2021 study were presented. The press conference can be viewed here. The results were described as "alarming" in the aftermath.

Among other things, it showed that a quarter of the fourth graders in Germany do not meet the international standard for reading literacy necessary for a successful transition from “learning to read” to “reading to learn”. As a result of their weak reading achievement, these children will have considerable problems in all subjects in their further school careers, insofar as they are unable to catch up. Together with Sweden and the Netherlands, Germany belongs to a small group of three participating countries with problematic development of reading literacy across the last 20 years. This development was characterized by a significant decline in mean reading achievement with an significant increase in the standard deviation.

The results were discussed extensively in the press and subsequently in the German federal states. We are proud that we had the opportunity to contribute to this important study for our society. The national report with differentiated results can be downloaded here with open access.

June 13th, 2023


Colloquium "Methods of Empirical Educational Research” 2023

Once again the annual colloquium "Methods of Empirical Educational Research" was held from the 5th to 10th of March 2023 in Hirschegg, Austria. In the course of the colloquium, 15 experts discussed contributions on current methodological developments within the scope of educational research. In addition to the scientifically stimulating contributions, there was also a birthday to celebrate this year.

On behalf of the work unit, talks were held by

  • Prof. Dr. Andreas Frey on differential item functioning in adaptive tests
  • Dr. Aron Fink on the usage of Natural Language Processing for the automatic coding of essays in higher education

We look once again back at a very stimulating time in Kleinwalsertal and give our thanks to all participants!

April 12th, 2023


New Publication in the International Encyclopedia of Education

Prof. Dr. Andreas Frey has published an article on Computerized Adaptive Testing (CAT) and Multistage Testing (MST) in the International Encyclopedia of Education.

This article provides an overview of the basic concepts of CAT and MST. It describes the basic methods used in the six building blocks for designing a modern CAT system. It then summarizes the current state of knowledge about the effects of CAT on (a) measurement efficiency, (b) the validity of test score interpretation, and (c) test-taking motivation. It is concluded that CAT is likely to play a further-growing role in future assessments as a core element in dynamic testing systems with real-time optimization processes.

The article is available here:

Frey, A. (2023). Computerized adaptive testing and multistage testing. In R. J. Tierney, F. Rizvi & K. Erkican (Eds.), International Encyclopedia of Education (4th ed., pp. 209–216). Elsevier. https://doi.org/10.1016/B978-0-12-818630-5.10028-4

March 27th, 2023


New Publication in British Journal of Mathematical and Statistical Psychology

Dr. Christoph Koenig, together with Benjamin Becker (IQB Berlin) and Dr. Esther Ulitzsch, recently published a tutorial on Bayesian hierarchical response time modelling. In this accessible tutorial, they discuss one of the most common response time models—the lognormal response time model—embedded in the hierarchical framework by van der Linden (2007). They provide detailed guidance on how to specify and estimate this model in a Bayesian hierarchical context. Furthermore, they illustrate three model extensions: (a) application to non-cognitive data incorporating the distance-difficulty hypothesis, (b) modelling conditional dependencies between response times and responses, and (c) identifying differences in response behaviour via mixture modelling. The tutorial aims to provide a better understanding of the use and utility of response time models, displays how these models can easily be adapted and extended.

The paper is published in British Journal of Mathematical and Statistical Psychology erschienen as open access, and is available here:

König, C., Becker, B., & Ulitzsch, E. (2023). Bayesian hierarchical response time modelling—A tutorial. British Journal of Mathematical and Statistical Psychology. Advance online publication. https://doi.org/10.1111/bmsp.12302

March 21st, 2023


Meta-analysis received most-cited paper award

In February 2022, Dr. Christoph Koenig and Prof. Dr. Andreas Frey showed in one of the first meta-analyses on the effect of COVID-19 related school closures in 2020 and 2021, that they affected student achievement with an average negative effect of -0.17 SD. This negative effect was confirmed in several follow-up studies and meta-analyses. The meta-analysis now received the most-cited paper award, issued by the journal Educational Measurement: Issues and Practice.

The open access publication is available here:

König, C., & Frey, A. (2022). The Impact of COVID-19-Related School Closures on Student Achievement—A Meta-Analysis. Educational Measurement: Issues and Practice, 41, 16–22. https://doi.org/10.1111/emip.12495

March 21st, 2023


Anelia Vanova newest member of the work unit

 

The work unit welcomes Anelia Vanova as its newest member!

December 1st, 2022


Dr. Päivi Hannele Taskinen, Cosima Schnek & Lara Weiß newest members of the work unit

The work unit welcomes Dr. Päivi Hannele Taskinen, Cosima Schenk & Lara Weiß as its newest members!

November 9th, 2022


New Publication in Education and Information Technologies

Aron Fink and Prof. Dr. Andreas Frey, together with Prof. Dr. Christian Spoden, have published a new paper on Determinants of higher education teachers’ intention to use technology-based exams. This paper proposes the novel technology-based exams acceptance model (TEAM) and empirically evaluates its model structure and usefulness from the perspective of higher education teachers. With TEAM, a highly predictive model for explaining the behavioral intention to use e-exams is now available. It offers a theoretical basis that can be used for the successful implementation of e-exams in higher education.

The paper is published in Education and Information Technologies erschienen as open access:
Fink, A., Spoden, C., & Frey, A. (2022). Determinants of higher education teachers’ intention to use technology-based exams. Education and Information Technologies. Advance online publication. https://doi.org/10.1007/s10639-022-11435-4  

November 22nd, 2022


New Publication in Behavior Research Methods

Dr. Christoph König and Prof. Dr. Andreas Frey, together with Prof. Dr. Christian Spoden, have published a new paper on Bayesian hierarchical modeling of Item Response Theory (IRT) models. In an extensive simulation study, the robustness of the performance of a previously optimized Bayesian hierarchical 2PL IRT model was investigated and compared to a number of alternative estimation methods and approaches. We could show that the optimized model exhibits a relatively robust performance, especially in small samples, and outperforms alternative approaches in a wide range of conditions. Thus, this simulation study contributes to the extension of applicability of IRT models to, for instance, university contexts, where small samples and suboptimal test conditions are prevalent, and IRT models are seldom used.

The paper is published as an open access publication in Behavior Research Methods, and is available here:
König, C., Spoden, C., & Frey, A. (2022). Robustness of the performance of the optimized hierarchical two-parameter logistic IRT model for small-sample item calibration. Behavior Research Methods. Advance online publication. https://doi.org/10.3758/s13428-022-02000-5

November 10th, 2022


The CoBi project family grows

The Beisheim-Stiftung is funding the project " Coronabedingte Bildungsbenachteiligungen erkennen und verringern – Online” (CoBi-Online) for two years (2022-2024). CoBi-Online is the third project of the CoBi project family (besides CoBi and CoBi-Transfer). CoBi-Online aims at transferring the learning support system developed in the CoBi project to a video-based online context and at offering it beyond Hesse. Thus, CoBi-Online will be making another important contribution to mitigating the educational disadvantages created by the Corona pandemic and to supporting students in Germany to optimize their learning. 

November 1st, 2022


Project CoBi-Transfer started

The project „Coronabedingte Bildungsbenachteiligungen erkennen und verringern - Transfer“ (CoBi-Transfer) conducted by the Goethe University Frankfurt (Prof. Dr. Andreas Frey) is funded by the Hessian Ministry of Education as part of the Loewenstark-Program in the school year 2022/2023. CoBi-Transfer allows the CoBi Learning Support System (developed in the CoBi project funded by the Beisheim Stiftung and the Goethe Corona Fonds) to be offered to students in K-12 education in the Frankfurt area and beyond. Thus, CoBi-Transfer further contributes to mitigating the educational disadvantages caused by the Corona pandemic by supporting students in optimizing their learning. 

October 15th, 2022


IACAT 2022 - A great success

On September 20-22, 2022, the Goethe University Frankfurt hosted the 8th Conference of the International Association for Computerized Adaptive Testing. The three days of the conference were packed with high quality research talks on how to enhance testing and assessment in the digital age with CAT. Over 150 researchers from 29 countries have contributed in a variety of ways, making the conference highly stimulating with a very good atmosphere for discussion and networking. Thus, it is fair to conclude that the conference was a great success! See you at the next IACAT conference in 2024!

October 1st, 2022


New Publication on Empirical Research Methods

The 6th, revised edition of the textbook "Psychology" (Schütz, Brand, & Steins-Loeber, 2022) was published on Sept. 15 2022. The book contains the chapter "Empirical Research Methods" written by Prof. Dr. Martin Schultze and Prof. Dr. Andreas Frey. The chapter provides an overview of the research methods relevant in psychological research and is the conceptual framework for the two semester lecture PsyBSc1 "Introduction to Psychology as an Empirical Science I and II" as part of our Bachelor's program in Psychology.

You can find the revised edition of the textbook here.

September 20th, 2022


Numerous contributions at the 52nd DGPs-Congress

The 52nd Congress of the German Psychological Society (DGPs) took place at the University of Hildesheim from September 10 to 15, 2022. Our team was represented with numerous contributions. Prof. Dr. Andreas Frey held a presentation entitled “Highly Adaptive Testing in PISA” (Frey, König & Fink). Aron Fink gave a talk on "Using Natural Language Processing for Automated Coding of Essays in Digitally Supported Large-Scale University Lectures" (Fink, Frey, Liu & Gombert). He also presented the KAT-HS-App in a 90-minute software demonstration (Fink, Spoden, Frey & Naumann). Dr. Christoph König also gave a presentation entitled "The Impact of COVID-19-Related School closures on Student Achievement - A Meta-Analysis" (König & Frey). In addition, Tuo Liu presented a paper entitled "To see or not to see: Self-association does not facilitate the perception of facial expressions" (Liu & Hildebrandt) as well as a software demonstration on the "Open Virtual Mirror Framework" (Grewe, Liu, Hildebrandt & Zachow).

September 19th, 2022


Summer semester end in Goethe's footsteps

The end of the semester was hot, 35 degrees. This time fearlessly on the move again, but without an overweight dog: Team Frey. Start at the Goethe House, then along the banks of the Main, into the cool city forest, have a cold drink at the Goethe Tower, preferably Hessian, and then hike through Sachsenhausen to the Ristorante with courtyard garden in the evening. There relaxation, benevolent review of research, projects and teaching in the summer semester and strengthening with fine delicacies. Satisfied, slowly evaporating, we indulged in relaxation and fooling around. Energy and confidence for new tasks!

June 20th, 2022


Image film about the project HIKOF-DL published

The HIKOF-DL project aims to support universities and companies in analyzing the potential of online teaching data sets, which have grown rapidly due to the COVID-19 pandemic, by means of artificial intelligence (AI) and transforming them into highly informative and competence-oriented feedback for students.

The project is being carried out in cooperation with the Leibniz Institute for Human Development and Educational Information and "studiumdigitale", the central e-learning facility of Goethe University. The project is funded by the Distr@l support program of the Hessian state government.

June 22nd, 2022


New publication on learning to read during the Corona pandemic

Dr. Ulrich Ludewig, Ruben Kleinkorres, Dr. Rahim Schaufelberger, Theresa Schlitter, Dr. Ramona Lorenz, Dr. Christoph König, Prof. Dr. Andreas Frey and Prof. Dr. Nele McElvany have published a research paper on the state of reading literacy after corona-related limitations in schooling. They examined the data from 4,290 fourth graders* from 111 schools in Germany. The observed significant decline in reading literacy from 2016 to 2021 can only be partially explained by changes in the student composition. The observed decline is way beyond typical changes during five years in average reading achievement as found in large-scale assessment. Thus, it is likely that this decline in average reading achievement is at least partly associated with COVID-19-related measures.

The paper was published with open access in Frontiers in Psychology: Ludewig, U., Kleinkorres, R., Schaufelberger, R., Schlitter, T., Lorenz, R., Koenig, C., Frey, A., & McElvany, N. (2022). COVID-19 pandemic and student reading achievement – Findings from a school panel study. Frontiers in Psychology, 13. 1–15.https://doi.org/10.3389/fpsyg.2022.876485

June 8th, 2022


Tuo Liu newest member of the work unit

The work unit welcomes Tuo Liu as its newest member!

June 1st, 2022


Article regarding children's mixed-rounding use in computational estimation published

In cooperation with Prof. Dr. Gerhard Büttner and Prof. Dr. Sebastian Poloczek, Dr. Svenja Hammerstein published an article regarding children's mixed-rounding use in computational estimation.

Poloczek, S., Hammerstein, S., & Büttner, G. (2022). Children’s Mixed-Rounding Strategy Use in Computational Estimation. Journal of Numerical Cognition, 8(1), 24-35. https://doi.org/10.5964/jnc.7299

Available under:
https://jnc.psychopen.eu/index.php/jnc/article/view/7299/7299.html

May 5th, 2022


Chapter on measurement of competencies published

 

Recently, the chapter "Kompetenzdiagnostik” (Measurement of competencies) was published in the second edition of the Handbuch Schulpädagogik (Harring, Rohlfs & Gläser-Zikuda, 2022). Prof. Dr. Andreas Frey and Prof. Dr. Johannes Hartig describe in this chapter what competence tests are and what goals they pursue as well as the principles of their construction, evaluation and test score interpretation.

The Handbuch Schulpädagogik is available here:
Harring, M., Rohlfs, C., & Gläser-Zikuda, M. (2022). Handbuch Schulpädagogik (2nd ed.). Waxmann. 

The chapter Kompetenzdiagnostik is available here:
Frey, A., & Hartig, J. (2022). Kompetenzdiagnostik. In M. Harring, C. Rohlfs, & M. Gläser-Zikuda (2nd ed.). Handbuch Schulpädagogik (pp. 928–937). Waxmann.

April 26th, 2022


Christian Spoden appointed professor

Our former team member Christian Spoden, was recently appointed professor for "Psychology with a focus on General Psychology, Methodology and Personality Psychology" at the Emden/Leer University of Applied Sciences (https://www.hs-emden-leer.de). We are very happy for Christian and wish him every success in his new position!

April 19th, 2022


Another article on adaptive and customised e-exams published

Prof. Dr. Christian Spoden and Hanna Köhler, together with Prof. Dr. Andreas Frey, Aron Fink and Patrick Naumann, have published an article in the Zeitschrift für Hochschulentwicklung (Journal for Higher Education Development). The article describes how competence-oriented adaptive e-exams can contribute to an increased comparability and fairness of exams. Furthermore, the article presents a trial implementation, from which recommendations for implementation in regular exam operations can be derived. Finally, possible extensions and challenges of adaptive and customised e-exams are discussed.

The article is available here:

Spoden, C., Fink, A., Frey, A., Köhler, H., & Naumann, P. (2022). Kompetenzorientierung und Fairness bei individualisierten E-Klausuren [Competence orientation and fairness in individualised e-exams]. Zeitschrift für Hochschulentwicklung, 17(1), 121–140. Available at: https://zfhe.at/index.php/zfhe/article/view/1588

April 07th, 2022


Article on competence-oriented adaptive e-exams published

Prof. Dr. Andreas Frey, Aron Fink, and Patrick Naumann, together with Prof. Dr. Christian Spoden, have written an article on competence-oriented adaptive exams in higher education. The article describes how to improve exams in higher education based on a novel combination of methods, quality standards and approaches stemming from psychometrics and educational and psychological measurement. The article refers to recent trends in these fields and illustrates how they can contribute to the digital transformation in higher education.

The article is available here:

Spoden, C., Frey, A., Fink, A., & Naumann, P. (2022). E-Klausuren für die Hochschule: Zeit der Veränderung. In-Mind. Available at: https://de.in-mind.org/article/e-klausuren-fuer-die-hochschule-zeit-der-veraenderung

March 21st, 2022


Report on the PIRLS reading literacy test on the reading literacy of fourth graders in Germany before and during the COVID-19 pandemic

Prof. Dr. Andreas Frey, together with Dr. Ulrich Ludewig, Theresa Schlitter, Dr. Ramona Lorenz, Ruben Kleinkorres, Dr. Rahim Schaufelberger, and Prof. Dr. Nele McElvany, published a report on the reading literacy of fourth graders in Germany before and during the COVID-19 pandemic.

For the report, data from 4,290 fourth graders from 111 schools in Germany who took the comprehensive PIRLS reading literacy test in 2016 and 2021 were analyzed. Compared to 2016, fourth graders in 2021 who had been affected by COVID-19-related schooling restrictions for more than one year had, on average, statistically significant and substantially lower reading literacy.

The report is available here:

Ludewig, U., Schlitter, T., Lorenz, R., Kleinkorres, R., Schaufelberger, R., Frey, A. & McElvany, N. (2022). Die COVID-19 Pandemie und Lesekompetenz von Viertklässler*innen. Ergebnisse der IFS-Schulpanelstudie 2016-2021. Dortmund: Technische Universität Dortmund, Institut für Schulentwicklungsforschung. DOI: 10.17877/DE290R-22667

An english version with the cooperation of Dr. Christoph König can be found here:

Ludewig, U., Kleinkorres, R., Schaufelberger, R., Lorenz, R., König, C., Frey, A., & McElvany, N. (2022). COVID-19 Pandemic and Student Reading Achievement –Findings from a School Panel Study. Available at: https://psyarxiv.com/hrzae/download

March 18th, 2022


Markus Spitzer newest member of the work unit

The work unit welcomes Markus Spitzer as its newest member!

March 1st, 2022


Meta-analysis on effects of COVID-19 related school closures published

In a meta-analysis, Dr. Christoph König and Prof. Dr. Andreas Frey found that COVID-19-related school closures in 2020 and 2021 had a negative effect on student achievement in German schools, with a mean effect of -0.17. The meta-analysis was published in Educational Measurement: Issues and Practice and can be accessed here.

February 8th, 2022


One of the Top 5% of all research outputs ever tracked by Altmetric

 

Dr. Svenja Hammerstein, Dr. Christoph König, Dr. Thomas Dreisörner and Prof. Dr. Andreas Frey have published a systematic review on the effects of COVID-19-related school closures on student achievement in spring 2020. The review article is in the top 5% of all research outputs ever tracked by Altmetric: https://frontiers.altmetric.com/details/114122477#score 

The study is available here: 
Hammerstein, S., König, C., Dreisörner, T., & Frey, A. (2021). Effects of COVID-19-Related School Closures on Student Achievement-A Systematic Review, Frontiers in Psychology, 12, 746289. https://doi.org/10.3389/fpsyg.2021.746289

February 1st, 2022


What I haven't learned, I have traveled [J. W. Goethe]

We ended our all around successful year 2021 with a hike around the Saalburg in the most beautiful atmospheric flurry of snow. Armed with a dog, cookies, sandwiches, hot cider and tea, we tackled the sometimes challenging, yet delightful hiking route. One and a half kilometers from the designation, a delegation had to stay behind with the limping dog. Amid the growing darkness, the boss initiated the rescue and recovery of the animal and brought everyone back safely to the designation, where we ended up rewarding ourselves with the remaining treats and the work report of the previous year. It was very amusing and we laughed a lot at ourselves.

The entire department hopes that this positive end will herald a good year for all of us and wishes our visitors of this site a Merry Christmas and a happy and prosperous New Year!

January 13th, 2022


Diana Dusny newest member of the work unit

The work unit Educational Psychology with focus on Counselling, Measurement, and Evaluation welcomes Diana Dusny as its newest member!

December 1st, 2021


Interview in Journal Frankfurt on the CoBi project

Dr. Thomas Dreisörner gave an interview in the context of the project "Identifying and reducing Corona-related educational disadvantages (CoBi)" on the results of a systematic review on the effects of COVID-19-related school closures on student achievement in spring 2020. The interview is now available in the "Journal Frankfurt": Lernen, zu Hause lernen.

The systematic review can be accessed here:
Hammerstein, S., König, C., Dreisörner, T., & Frey, A. (2021). Effects of COVID-19-Related School Closures on Student Achievement-A Systematic Review, Frontiers in Psychology, 12, 746289. https://doi.org/10.3389/fpsyg.2021.746289

Here you can find the official press release:
Studie zeigt: Distanzunterricht genauso effektiv wie Sommerferien

November 11th, 2021


Book chapter on increasing test efficiency of the Teacher Knowledge Survey Assessment Module through multidimensional adaptive testing published

Prof. Dr. Andreas Frey and Aron Fink have published a book chapter entitled "Increasing test efficiency in an international assessment of teachers' general pedagogical knowledge through multidimensional adaptive testing" as part of the book "Teaching as a Knowledge Profession - Studying Pedagogical Knowledge across Education Systems" published by the OECD.

The chapter discusses the potential of multidimensional adaptive testing (MAT) for increasing the measurement efficiency of large-scale assessments using the example of the Teacher Knowledge Survey Assessment Module, which will be part of the Teaching and Learning International Survey (TALIS) 2024.

Frey, A., & Fink, A. (2021). Increasing test efficiency in an international assessment of teachers’ general pedagogical knowledge through multidimensional adaptive testing. In H. Ulferts (ed.). Teaching as a Knowledge Profession: Studying Pedagogical Knowledge across Education Systems (pp. 123—140). OECD Publishing. https://doi.org/10.1787/e823ef6e-en. 

October 28th, 2021


New press release on the CoBi project

 

Dr. Svenja Hammerstein, Dr. Christoph König, Dr. Thomas Dreisörner und Prof. Dr. Andreas Frey have published a systematic review on the effects of COVID-19-related school closures on student achievement in spring 2020 in the project "Identifying and reducing Corona-related educational disadvantages (CoBi)", which is funded by the Beisheim Foundation and the Goethe Corona Fund. A new press release from the idw scientific information service has now been published on this. The press release can be found here: „Die Pandemie zeigt, wo Kinder und Jugendliche schulisch besonders gefährdet sind“

The study is available here for free:
Hammerstein, S., König, C., Dreisörner, T., & Frey, A. (2021). Effects of COVID-19-Related School Closures on Student Achievement-A Systematic Review, Frontiers in Psychology, 12, 746289. https://doi.org/10.3389/fpsyg.2021.746289

October 28th, 2021


Prof. Dr. Andreas Frey speaks at the „Forum Bildung“ at Frankfurt Book Fair 2021

The “Forum Bildung” took place on the Stage of the Education Media at this year's Book Fair. The challenges of educational disadvantages caused by the Corona pandemic were discussed. The occasion was the systematic review "Effects of COVID-19-Related School Closures on Student Achievements - A Systematic Review" published as a pre-print in June 2021. The associated press release from Goethe University generated an intensive media coverage and stimulated subsequent discussions – such as this one. Prof. Dr. Andreas Frey discussed the contents of the study, the current situation in Germany and abroad and the conclusions that can be drawn from it with Karin Plötz, Director of LitCam.

The video is now available here:

October 27th, 2021 (video added on November 1st, 2021)


Marion Schneckenbühl newest member of the work unit

The work unit Educational Psychology with focus on Counselling, Measurement, and Evaluation welcomes Marion Schneckenbühl as its newest member!

October 4th, 2021


Project DigiTeLL started

The project "Digital Teaching and Learning Lab - DigiTeLL" is funded by the Stiftung Innovation in der Hochschullehre and will be carried out from 2021 to 2024. The projects aims at establishing, carrying out and implementing an innovation structure for the development, integration and further development of digital teaching, learning and testing scenarios. Prof. Dr. Andreas Frey heads the DigiTeLL cross-sectional topic "Digital Examination". Hereby, the research and software regarding digital examinations developed in the former projects KAT-HS and HIKOF-DL will be made available to a wide range of users at the Goethe University and will be further developed.

September 28th, 2021


Presentations at the 15th Conference of the Subdivision “Methods and Evaluation”

From 15th to 17th September 2021, the 15th conference of the Subdivision of Methods and Evaluation (FGME) of the German Psychological Society (DGPs) took place as a virtual event. The conference was hosted by the University of Mannheim. Dr. Christoph König gave the talk “Accounting for heterogeneity when specifying informative prior distributions for Bayesian multiple regression models - A novel similarity measure”. Aron Fink gave a talk on accounting for measurement error in item parameters in computerized adaptive testing (CAT). Patrick Naumann presented a poster on emotional effects of computerized adaptive exams in higher education. Further information about the FGME conference can be found here.

September 17th, 2021


Keynote by Prof. Dr. Andreas Frey at the 32nd International Congress of Psychology (ICP)

On 19 July 2021, Prof. Dr. Andreas Frey gave a keynote at the 32nd International Congress of Psychology (ICP). The title of his presentation was "Computerized Adaptive Testing in Psychological Research and Application". The congress takes place every four years and is one of the world's largest international psychological events. It was first held in 1889.

You can find the abstract of the presentation here:

https://whova.com/embedded/session/csg_202107/1774435/

Video recording of the keynote:

July 22th, 2021


Book on psychometrically sound e-exams in higher education published

Prof. Dr. Andreas Frey and Prof. Dr. Christian Spoden have published a book that presents a concept for psychometrically sound e-exams based on the current state of scientific knowledge. The concept can be applied in all disciplines at all higher education institutions. The concept relies on the use of digital technologies to make exams more individualized, more precise, and fairer. Exams are thereby designed as criterion-referenced, computerized adaptive testing procedures. In this book, the necessary steps for the psychometrically sound e-exam-based measurement of competence are explained in detail.

The book also contains chapters by Aron Fink and Patrick Naumann, who are members of the working unit. With the publication at Pabst Science Publishers, a milestone of the KAT-HS project (Criterion-Referenced Adaptive Testing in Higher Education) is reached.

The book is available here:

Spoden, C. & Frey, A. (Hrsg.). (2021). Psychometrisch fundierte E-Klausuren für die Hochschule [Psychometrically sound e-exams for higher education]. Lengerich: Pabst Science Publishers.

July 6th, 2021

 


Commentary on the current state of higher education exams in psychology studies published

Frey, A., Spoden, C., Schultze, M. (2021). Die Zukunft der Hochschulklausuren hat bereits begonnen [The future of university exams has already begun]. Psychologische Rundschau, 72(2), 113–116. https://doi.org/10.1026/0033-3042/a000528

May 19th, 2021


Commentary on computerized adaptive exams in psychology studies published

Aron Fink and Patrick Naumann, along with Dr. Christoph König, wrote a commentary that refers to a position paper on the improvement of written exams in psychology studies. In the commentary, the advantages of computerized adaptive testing for exams in psychology studies are demonstrated. The paper was published in the journal Psychologische Rundschau.

Fink, A., Naumann, P. & König, C. (2021). Computerisierte adaptive Klausuren im Psychologiestudium [Computerized Adaptive Testing for Exams in Psychology Studies]. Psychologische Rundschau, 72(2), 125–127. https://doi.org/10.1026/0033-3042/a000532

May, 19th, 2021


Article on the development and evaluation of a framework for performance-based testing of ICT-skills published

Prof. Dr. Andreas Frey and Dr. S. Franziska C. Wenzel, together with Dr. Lena Engelhardt, Prof. Dr. Johannes Naumann, Prof. Dr. Frank Goldhammer, Prof. Dr. Holger Horz, and Dr. Katja Hartig, developed and evaluated a framework for performance-based testing of Information and Communications Technology (ICT)-skills. Specifically, the framework is used to appropriately assess competences in the use of information and communication technologies (e.g., problem solving, understanding graphics) using performance-based items. The article was recently published in the scientific journal Frontiers in Education.

Engelhardt, L., Naumann, J., Goldhammer, F., Frey, A., Horz, H., Hartig, K., & Wenzel, S. F. C. (2021). Development and Evaluation of a Framework for the Performance-Based Testing of ICT Skills. Frontiers in Education, 6, Article 668860. https://doi.org/10.3389/feduc.2021.668860

May, 19th, 2021


New Project: Highly Informative and Competence-Oriented Feedback for Digital Learning (HIKOF-DL)

The Government of the Federal State of Hessen is funding the project "Highly Informative and Competency-Oriented Feedback for Digital Learning (HIKOF-DL)" as part of the Distr@l funding program. The project is being carried out in cooperation with the DIPF and "studiumdigitale". The goal of the project is to implement two feedback systems for formative and summative feedback and to combine them into a unified concept for highly informative and action-oriented feedback for supporting the acquisition of competencies of learners. Among others, the project builds on and continues the results of KAT-HS.

You can find more HERE.

May, 07th, 2021


Software demonstration on the KAT-HS-App published

Aron Fink, Prof. Dr. Christian Spoden, Prof. Dr. Andreas Frey, and Patrick Naumann have published a software demonstration on the KAT-HS-App in the scientific journal Diagnostica. This software demonstration presents the possibilities for the construction, administration, and evaluation of criterion-referenced, computerized adaptive and nonadaptive tests with the R-based open-source KAT-HS-App, which was developed by our team. The app is available free of charge for research and teaching after registration via https://kat-hs.uni-frankfurt.de/materialien/software/ .

Fink, A., Spoden, C., Frey, A., & Naumann, P. (2021). Kriteriumsorientiertes adaptives Testen mit der KAT-HS-App [Criterion-referenced adaptive testing using the KAT-HS-App]. Diagnostica, 67(2), 110 – 114. https://doi.org/10.1026/0012-1924/a000268 

April, 26th, 2021


Beisheim Stiftung supports project on educational disadvantage due to the coronavirus

In our project "Detecting and reducing Corona-related educational disadvantages; CoBi" we received financial support through the Beisheim Stiftung for a term of two years.

Currently we work on a literature review "A systematic literature review on effects of school closures during the COVID-19 pandemic on student learning outcomes". First results are reported in a conference fee at digiGEBF21 in regard to the theme "education and corona" by Prof. Dr. Andreas Frey.

April 19th, 2021


Empirical results on the construction of psychometrically sound written university exams

Prof. Dr. Andreas Frey, Prof. Dr. Christian Spoden and Dr. Sebastian Born have written an article on the construction and administration of psychometrically sound written university exams, which has now been published. Elements of the proposed procedure are (a) the definition of the content domain of interest in relation to the learning objectives of the course, (b) the specification of an assessment framework, (c) the operationalization of the assessment framework with test items, (d) the standardized administration of the exam, (e) the scaling of gathered responses with item response theory models, and (f) the setting of grade levels with standard-setting procedures.
Empirical results obtained from six test cycles of a real university exam show that this procedure can successfully be applied in a typical university setting.

The paper is available here: https://www.psychologie-aktuell.com/fileadmin/Redaktion/Journale/ptam-2020-4/PTAM_4-2020_21900493_ebook_frey_4.pdf

Frey, A., Spoden, C., & Born, S. (2020). Construction of Psychometrically Sound Written University Exams. Psychological Test and Assessment Modeling, 65(4), 415-525.

January 15th, 2021


Publication in Behavior Research Methods

On the occasion of Behavior Research Methods’ (BRM) change of the editorial board, the editorial entitled "Into a new decade" has been published. It contains important information on the type of manuscripts whose submission to BRM as one of the world's leading methods-journals (impact factor: 4.425) is promising. Prof. Dr. Andreas Frey serves as associate editor of BRM since the beginning of 2020.

December 18th, 2020