A detective and corrective exercise assistant using computer vision and machine learning

Lynne Grewe; Dung Nu Thanh Pham; Dikshant Pravin Jain; Ankush Mahajan; Allen Shahshahani

doi:10.1117/12.2619102

8 June 2022 A detective and corrective exercise assistant using computer vision and machine learning

Lynne Grewe, Dung Nu Thanh Pham, Dikshant Pravin Jain, Ankush Mahajan, Allen Shahshahani

Proceedings Volume 12122, Signal Processing, Sensor/Information Fusion, and Target Recognition XXXI; 121220X (2022) https://doi.org/10.1117/12.2619102
Event: SPIE Defense + Commercial Sensing, 2022, Orlando, Florida, United States

Abstract

Recently, Exercise Analysis has gained strong interest in the sport industry including athletes and coaches to understand and improve performance, as well as preventing injuries caused by incorrect exercise form. This work describes a system, USquat, that utilizes computer vision and machine learning for understanding and analyzing of a particular exercise, squatting, as proof of concept for the creation of a detective and corrective exercise assistant. Squatting was chosen as it is a complicated form of exercise and is often mis performed. For USquat, a Recurrent Neural Network is designed using Convolutional Neural Networks and Long Term Short Term networks. A sizable video library dataset containing numerous “bad” forms of squatting was created and curated to train the USquat system. When evaluated on test data, USquat achieved 90% accuracy on average. On a developed Android application that uses the resulting model, upon detection of “bad” squatting forms, it offers an instructive “good” video related specifically to the user’s bad form. Results including live application outcomes are demonstrated as well as challenging video results, problems, and areas of future work. Early work on the creation of a follow-on system to USquat that automatically generates a custom video of the user performing a correct action for the purpose of learning proper activity performance. Additionally, early work on a different version of USquat that explores an attention mechanism network is discussed.

Conference Presentation

Citation Download Citation

Lynne Grewe, Dung Nu Thanh Pham, Dikshant Pravin Jain, Ankush Mahajan, and Allen Shahshahani "A detective and corrective exercise assistant using computer vision and machine learning", Proc. SPIE 12122, Signal Processing, Sensor/Information Fusion, and Target Recognition XXXI, 121220X (8 June 2022); https://doi.org/10.1117/12.2619102

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available