KLS

Immersive AI-Powered Martial Arts Training in Virtual Reality

The Kenpo Learning Simulator (KLS) is an Intelligent System for Psychomotor Learning (ISPL) designed to teach defensive martial arts movements using Virtual Reality (VR) and Artificial Intelligence (AI). Developed using the PsyLearn Framework, KLS demonstrates how emerging technologies can be used to create effective ISPLs to learn physical skills.

About KLS

KLS focuses on teaching the Blocking Set I from American Kenpo Karate, a sequence of foundational defensive movements. It provides a fully immersive learning environment where a virtual instructor (AIDA) demonstrates techniques that learners must replicate. Using AI-based pose estimation, rule-based analysis, multiple-choices tests, and emotion recognition, the system evaluates performance, detects errors, and delivers personalized real-time feedback.

Unlike traditional video tutorials or mobile apps, KLS allows learners to see, move, and react naturally within a 3D space, learning through embodied interaction and guided practice.

Key Features

• AI-Driven Feedback: Detects and classifies movement errors using real-time pose estimation and domain-specific rules.

• Virtual Instructor (AIDA): A lifelike AI teacher created using Unreal Engine and MetaHuman to demonstrate the movements and guide learners.

• Emotion Recognition: Estimates affective states through facial analysis (even with headset occlusion) to adapt the learning experience.

• Cognitive Integration: Includes quizzes and explanations to reinforce understanding of biomechanics and theory.

• Gamified Learning: Motivates learners through immersive VR challenges and progress tracking.

• Ethically Designed: Built to complement, not replace, human instructors, respecting privacy and learner autonomy.

KLS demonstrates how AI and XR can be harmonized with pedagogical theory to enhance psychomotor learning. It supports not only physical execution but also the cognitive and affective dimensions of Bloom’s taxonomy, helping learners understand, feel, and perform better. KLS is part of a broader effort to create modular, scalable, and open-source ISPLs that can be adapted beyond martial arts, to rehabilitation, sports training, healthcare, and education.

Built with the PsyLearn Framework

KLS is both a case study and a proof of concept for the PsyLearn Framework: a holistic, research-based methodology to create next-generation intelligent systems for psychomotor learning. Together, they lay the foundation for adaptive, ethical, and human-centered skill training in the digital age.

Code Availability

KLS is an open source application that follows the philosophy Open Science. The source code of the application is available in our GitHub website. All components are licensed under GNU AGPLv3, and media assets are shared under CC BY-NC-SA 4.0, fostering transparency, collaboration, and reproducibility.