Mastering Mind and Movement. ACM UMAP 2024 Tutorial on Modeling Intelligent Psychomotor Systems (M3@ACM UMAP 2024)

Date: TBA (July 1-4, 2024)

Place: 32nd ACM Conference on User Modeling, Adaptation and Personalization (ACM UMAP 2024) will take place in Cagliari, Sardinia, Italy. Info about the venue is available here. Registration at the conference is required (see instructions at ACM UMAP 2024 website). 

Organizers: Miguel Portaz [1], Alberto Corbí [2], Pablo García [1], Rwitajit Majumdar [3] and Olga C. Santos [1]

[1] PhyUM Research Center. Artificial Intelligent Department, Computer Science School, UNED, Spain
[2] Research Institute for Innovation & Technology in Education (UNIR iTED), Spain
[3] Research and Educational Institute for Semiconductors and Informatics, Kumamoto University, Japan

Publication: M. Portaz, A. Corbí, P. García, R. Majumdar, O.C. Santos (2024). Mastering Mind and Movement. ACM UMAP 2024 Tutorial on Modeling Intelligent Psychomotor Systems (M3@ACM UMAP 2024). In: ACM UMAP Proceedings (details to be added when publication ready).

About: Research in the psychomotor field to provide personalization support to users based on the modeling of their interactions poses several research challenges within the UMAP community. This topic was introduced in the UMAP research agenda back in 2017, with two relevant papers: 1) a discussion on approaches and open issues regarding the modeling of users’ physical activity when learning motor skills were analyzed, revealing the lack of personalized psychomotor learning systems (see Santos and Eddy, 2017) and 2) a review showing the UMAP opportunities in the martial arts domain (see Santos, 2017).

We have been working on addressing these challenges, with our most recent contribution in the UMUAI journal (Portaz et al., 2024), where we present our modelling approach to classify participants in terms of their expertise level.

In this context, the aim of the M3@ACM UMAP 2024 tutorial is to provide the researchers of the UMAP community with methodologies, tools and techniques to model complex psychomotor behaviours that can later personalize learning support in realms like sports, physical education or for rehabilitation purposes, providing insights into data gathering from activities that involve human movement.

In the M3 tutorial we will focus on how to take advantage of a learning analytics platform to support the data engineering process applied to the psychomotor domain, thus allowing for a practical and guided experience to the tutorial participants. Participants will engage in hands-on activities, recording specific movements and learning how to capture human body keypoints and model psychomotor learning.

Funding: This tutorial is framed in the project “HUMANAIDSens: HUMan-centered Assisted Intelligent Dynamic systems with SENSing technologies (TED2021-129485B-C41)” funded by MCIN/AEI/10.13039/501100011033 and the European Union ”NextGenerationEU”/PRTR. The research and development of the LAreflecT platform is partially supported by JSPS KAKENHI Grant-in-Aid for Scientific Research (B) JP22H03902 “GOAL project: AI-supported self-directed learning lifestyle in data-rich educational ecosystem”

Background

Psychomotor learning involves the integration of mental and muscular activity with the purpose of learning a motor skill. Many types of psychomotor skills can be learned, such as playing musical instruments, dancing, driving, practicing martial arts, performing a medical surgery, or communicating with sign language. Each one has a different set of unique characteristics that can make the learning process even more complex. To define and categorize the learning process of psychomotor activities, several psychomotor taxonomies have been proposed (Dave (1970); Ferris and Aziz (2005); Harrow (1972); Simpson (1972); Thomas (2004)). These taxonomies are defined in terms of progressive levels of performance during the learning process, going from observation to the mastery of motor skills. In this context, it is expected that AIED systems can be useful to enhance the performance of motor skills in a faster and safer
way for learners and instructors.

To build procedural learning environments for personalized learning of motor skills the process model SMDD (Sensing-Modelling-Designing-Delivering)
has been proposed (Santos (2016)). This process model guides the flow of information about the movements performed when using an intelligent psychomotor UMAP system along four interconnected phases:

  1. Sensing the learner’s corporal movement as specific skills are acquired within the context in which this movement takes place.
  2. Modelling the physical interactions, which allows comparing the learner’s movements against pre-existing templates of an accurate movement
    (e.g., a template of how an expert would carry out the movement).
  3. Designing the feedback to be provided to the learner (i.e., what kind of support and corrections are needed, and when and how to provide them).
  4. Delivering the feedback in an effective non-intrusive way to advise the learner on how the body and limbs should move to achieve the motor learning goal. This relates to the adaptation and personalization to be delivered to the users.

Relevance to UMAP community

The modeling and personalization of psychomotor activity was introduced in the UMAP research agenda back in 2017 (see contribution at ACM UMAP 2017), where current approaches and open issues regarding the modeling of users’ physical activity when learning motor skills were analyzed, revealing the lack of personalized psychomotor learning systems.

A review was done on the martial arts domain (see TOR contribution at ACM UMAP 2017), showing that although psychomotor learning is crucial for many kind of tasks that involve the acquisition of motor skills, such as practicing martial arts, the technological solutions developed do not adapt and personalize the system response to their users’ needs. From the research opportunities identified in that review, the system MyShikko was proposed in ACM UMAP 2018. The modelling approach to classify the expertise of the participants in this system has just been published in this UMUAI paper (link to be added when published), where we show how raw data transformations on inertial sensor data can be used to model user expertise level when learning psychomotor skills.

Initially, we selected martial arts for our research because it encompasses many of the characteristics common to other psychomotor activities like the management of strength and speed while executing the movements, visuomotor coordination of different parts of the body to respond to stimuli, participation of different agents during the learning like opponents or instructor, improvisation and anticipation against stimuli or even the use of tools accompanying the movement, as discussed in ACM UMAP 2021. 

However, we are also exploring other psychomotor domains, in particular, we have started to build a psychomotor system to recommend the physical activities and movements to perform when training in basketball, either to improve the technique, to recover from an injury or even to keep active when getting older. This system is called iBAID (intelligent Basket AID), and is the one that will be used as basis for the tutorial activities.

Moreover, in ACM UMAP 2023 we have also explored the intrinsic human aspects, such as ethics, transparency, explainability and sustainability, that should be taken into account when building the user models that provide the personalization. In this sense, the hybrid intelligence paradigm serves as a basis for the development of human-centered environments that are collaborative, adaptive, responsible and explainable.

Structure of the tutorial

In order to share with the UMAP community our view and research experience in developing psychomotor intelligent systems, our proposal for a 90 tutorial at ACM UMAP 2024 was accepted. As seen above, this tutorial follows previous contributions on modelling psychomotor activities within the UMAP community. These efforts between UNED and UNIR researchers framed within the PhyUM Research Center have culminated in a recently accepted contribution in the UMUAI journal. In addition, this tutorial builds upon a previous one (IPAIEDS23) conducted during the AIED 2023 conference with around 25 attendees, articulated around the outcomes of a recent systematic review of the psychomotor intelligent systems. For ACM UMAP 2024, we incorporate the integration with a learning analytics platform (LAReflecT platform) to support the data engineering process, thus allowing for a more practical and guided experience to the participants with the learning analytics perspective applied to the psychomotor domain.

The tutorial welcomes both beginners and experts in user modelling and learning analytics to delve into the exciting world of intelligent psychomotor learning analysis. Through an optional hands-on pre-task and an interactive demonstration on the process of capturing, modeling, designing, and delivering psychomotor data through LAreflecT platform, you’ll gain practical insights into how multidimensional data can transform how we learn and perform psychomotor skills.

What you’ll experience:

  • Optional Pre-Task: Understand the basis of data gathering concepts on psychomotor scenarios.
  • Engaging Demonstration: Witness the power of learning analytics with live case studies across different domains.
  • Expert Insights: Gain valuable knowledge from presenters with deep expertise in the field.

What you’ll take away:

  • A clear understanding of current applications of learning analytics in psychomotor skills.
  • Practical tools and techniques to analyze and improve psychomotor performance.
  • Inspiration to explore the potential of multidimensional data in your own field.

Ready to dive in? Register today and discover the future of psychomotor learning with data analysis!