Biomechanically informed, trust-based machine learning from social video platforms for monitoring physical training


Period: 7.23 – 8.25


Prof. Dr. Christoph Hoog Antink FB 18, KIS*MED – Künstlich intelligente Systeme der Medizin

Dr. Maziar Sharbafi FB 3, Lauflabor, Institut für Sportwissenschaft



Project description:

Social video platforms such as YouTube are a popular source for learning various skills, including physical exercise. We want to develop a system that makes use of this abundant source of information to help users perform exercises correctly. In the bigger picture, we want the machine to automatically learn what is correct exercise from the “wisdom of the crowds” contained in social video platforms.

For this, we propose a biomechanics- informed machine learning approach. By fusing computer vision, biomechanical modeling and machine learning, the system will be rooted in meaningful biomechanical parameters and hence be efficient in terms of computational demand and data required. Most importantly, it will be intrinsically explainable and thus will be able to classify exercise quality and give recommendations in a trustworthy way. In this interdisciplinary project, KIS*MED will contribute with expertise in unobtrusive sensing of motion, signal processing, and machine learning, while Lauflabor locomotion lab will contribute with expertise in biomechanical modeling, sport science, and recording of human motion.