Designing, Building and Evaluating Intelligent Psychomotor AIED systems (IPAIEDS@AIED2023)
Date: July 3rd, 2023 afternoon
Place: 24th International Conference on Artificial Intelligence in Education (AIED 2023) will take place in Tokyo, Japan. Info about the venue is available here. Registration at the conference is required (see instructions here).
Organizers: Olga C. Santos, Miguel Portaz, Alberto Casas-Ortiz, Jon Echeverria and Luis F. Perez-Villegas. Artificial Intelligence Department. UNED (Spain)
Publication: Santos, O.C., Portaz, M., Casas-Ortiz, A., Echeverria, J., Perez-Villegas, L.F. (2023). Designing, Building and Evaluating Intelligent Psychomotor AIED Systems (IPAIEDS@AIED2023). In: Wang, N., Rebolledo-Mendez, G., Dimitrova, V., Matsuda, N., Santos, O.C. (eds) Artificial Intelligence in Education. Posters and Late Breaking Results, Workshops and Tutorials, Industry and Innovation Tracks, Practitioners, Doctoral Consortium and Blue Sky. AIED 2023. Communications in Computer and Information Science, vol 1831. Springer, Cham. https://doi.org/10.1007/978-3-031-36336-8_14.
About: Psychomotor learning is an emerging research direction in the AIED (Artificial Intelligence in Education) field. This topic was introduced in the AIED research agenda back in 2016 (see contribution at IJAIED), when the SMDD (Sensing-Modelling-Designing-Delivering) process model to develop AIED psychomotor systems was introduced. Recently, a systematic review of the state of the art on this topic has also been published (see contribution at the Handbook of AIED). In this context, the aim of the IPAIEDS tutorial is to motivate the AIED community to research on intelligent psychomotor systems and give tools to design, build and evaluate this kind of systems. This tutorial is framed in the project ”HUMANAIDSens: HUMan-centered Assisted Intelligent Dynamic systems with SENSing technologies (TED2021-129485B-C41)” funded by MCIN/AEI/10.13039/501100011033 and the European Union ”NextGenerationEU”/PRTR.
Background
Psychomotor learning involves the integration of mental and muscular activity with the purpose of learning a motor skill. Many types of psychomotor skills
can be learned, such as playing musical instruments, dancing, driving, practicing martial arts, performing a medical surgery, or communicating with sign
language. Each one has a different set of unique characteristics that can make the learning process even more complex. To define and categorize the learning
process of psychomotor activities, several psychomotor taxonomies have been proposed (Dave (1970); Ferris and Aziz (2005); Harrow (1972); Simpson
(1972); Thomas (2004)). These taxonomies are defined in terms of progressive levels of performance during the learning process, going from observation to
the mastery of motor skills. In this context, it is expected that AIED systems can be useful to enhance the performance of motor skills in a faster and safer
way for learners and instructors.
To build procedural learning environments for personalized learning of motor skills the process model SMDD (Sensing-Modelling-Designing-Delivering)
has been proposed (Santos (2016)). This process model guides the flow of information about the movements performed when using an intelligent psychomotor AIED system along four interconnected phases:
1. Sensing the learner’s corporal movement as specific skills are acquired within the context in which this movement takes place.
2. Modelling the physical interactions, which allows comparing the learner’s movements against pre-existing templates of an accurate movement
(e.g., a template of how an expert would carry out the movement).
3. Designing the feedback to be provided to the learner (i.e., what kind of support and corrections are needed, and when and how to provide them).
4. Delivering the feedback in an effective non-intrusive way to advise the learner on how the body and limbs should move to achieve the motor learning
goal.
To identify the situation of the research of psychomotor learning in the AIED field, we have carried out a systematic review that resulted in the identification
of 12 AIED psychomotor systems. This analysis is included in the Chapter 18 of the Handbook of AIED just published (Casas-Ortiz, Echeverria, and Santos (2023)). Nonetheless, we have just ran the queries of that systematic review again on the databases, obtaining new results. This shows the field is in continuous evolution and the tutorial can be a good opportunity to present this research and updates.
In our contribution to the Handbook of AIED we present KSAS and KUMITRON psychomotor systems as case studies. These two systems are in fact included as showcases in the AIED website. The purpose of KSAS is to assist during the learning of the order in which a set of movements (known in martial arts as katas, forms, or sets) is performed. KSAS was developed as a mobile application. Based on this experience, a new system called KLS that uses Extended Reality is being developed to make the psychomotor learning experience more engaging. In turn, the purpose of KUMITRON is to teach learners how to anticipate the opponent’s movement during a martial arts combat (known as kumite) to train the combat strategy. Since kumites involve two participants, collabora- tive learning in psychomotor activities needs to be further explored (as discussed in a LBR at AIED 2023).
Initially, we selected martial arts for our research because it encompasses many of the characteristics common to other psychomotor activities like the management of strength and speed while executing the movements, visuomotor coordination of different parts of the body to respond to stimuli, participation of different agents during the learning like opponents or instructor, improvisation and anticipation against stimuli or even the use of tools accompanying the movement.
However, we are also exploring other psychomotor domains, in particular, we have started to build a psychomotor system to recommend the physical activities and movements to perform when training in basketball, either to improve the technique, to recover from an injury or even to keep active when getting older. This system is called iBAID (intelligent Basket AID).
Moreover, sign language communication is also being explored, where hand movements are involved. In particular, so far we have focused on segmenting each of the signs, which can be used to build datasets that can feed the development of a psychomotor systems to learn sign language.
Relevance to AIED community
AIED research has traditionally focused on cognitive/meta-cognitive and affective skills. However, in the last years some AIED researchers have suggested
the need to take into account the psychomotor domain in the AIED field, thus addressing the three domains defined in Bloom’s taxonomy of learning
objectives (Bloom, Engelhart, Furst, Hill, and Krathwohl (1956)).
The IJAED paper in the Special Issue of the 25th anniversary (Santos (2016)) was the first contribution that explicitly called for an expansion of AIED research into motor skill learning and asked to revisit AIED’s roots and to highlight the powerful potential marriage between AIED and existing (non-adaptive) systems for motor skill learning. This opened the path to other AIED resarchers who have also called the attention to the psychomotor domain, such as the workshop on Authoring and Tutoring Methods for Diverse Task Domains: Psychomotor, Mobile, and Medical at AIED 2018. Another related workshop is the workshop on Multimodal Artificial Intelligence in Education (MAIEd) that was run in AIED 2021 addressing two complementary approaches: i) extending computer-based learning activities with physical sensors for tracking learners’ behaviour, and ii) tracing learning activities that require levels of physical coordination.
It should also be noted that for the first time, in AIED 2023 representing and modelling psychomotor learning and the learning of motor skills are
included explicitly as research lines of the conference’s topics.
The proposal of this tutorial coincides also with the publication of the Handbook of Artificial Intelligence in Education by professors Ben du Boulay, Tanja Mitrovic and Kalina Yacef. This Handbook is aimed to present the most important topics and future trends of the field, and whose chapter 18 (authored by the organizers of this tutorial) focuses on intelligent systems for psychomotor learning, presenting, as mentioned before, a systematic review and two cases of study.
Structure of the tutorial
In order to share with the AIED community our view and research experience in developing psychomotor intelligent systems , our proposal for a half day tutorial at AIED 2023 was accepted.
The tutorial will start with a theoretical introduction of the field. In particular, we will present the motivation for the psychomotor research, the state of the art of the field and the SMDD process model to design intelligent psychomotor AIED systems.
After that, we will provide practical examples and exercises to the participants showing different proceesing approaches along the SMDD cycle. We will prepare these practical activities in Google Colab notebooks so that participants can try them during the tutorial to learn how to build a psychomotor system following the SMDD phases. We will cover both both data gathering from inertial signals and video signals for the sensing stage. These activities could be done individually or in pairs/groups.
We also plan to focus on evaluation approaches for psychomotor systems. For this, we will show participants some of the intelligent AIED psychomotor systems that we are developing to learn some motor skills (e.g., KSAS, KLS, KUMITRON and iBAID).We then will show the information collected by the systems and how it can be used to measure to the learning progress.