Blog

Preliminary evaluation results of pilot game

In summer 2021, a prototype of the ProTUce serious game was tested at the ESCP Business School (campus Berlin) in two introductory Business Analytics classes with about 40 students each. In winter 2020/21, the game was also tested at the Berlin School of Economics and Law (HWR Berlin) in an introductory Operations Management (OM) class with about 40 students. For those purposes, a prototype was designed in Siemens Tecnomatix Plant Simulation (with free academic student licenses) because the frontend of the INSYSTED implementation was still work in progress.

Figures 1 and 2 indicate on a seven-point Likert scale, -3 representing ”strongly disagree” and + 3 ”strongly agree”, the results of the pilot’s evaluation. Even though some of the results are not representative because of the small number of respondents (HWR n=11, ESCP n=64), they give some first student feedback. To check for plausibility, we compare our results to evaluations in literature using the Littlefield game of Snider and Balakrishnan (2013), as well as Miyaoka (2005). The tasks in both games are similar even though our factory is simpler and our levels successively increase in difficulty.

Overall, we find ourselves very much in line with literature; the students state that the game increases their curiosity and interest in the topic and wish for more classes to include game-based simulations. Building on that, we aim at improving these results even further with our web-based implementation by providing a smoother user interface, as well as simplified installation and administration.


Figure 1: The simulation game has increased my interest and curiosity about the course topics.

Figure 2: More business school courses should include game-based simulations.

Literature:

Miyaoka, J., 2005. Making Operations Management Fun: Littlefield Technologies. INFORMS Transactions on Education , 5.

Snider, B., Balakrishnan, J., 2013. Lessons Learned from Implementing Web-Based Simulations to Teach Operations Management Concepts. INFORMS Transactions on Education 13, 152–161. doi:10.1287/ited.2013.0108. publisher: INFORMS.