Background
Markerless motion capture systems have provided faster data collection with greater environmental flexibility compared to markered systems. These systems have gained significant interest from research and industry with applications in injury prevention and rehabilitation monitoring. A primary goal of motion capture systems is to produce joint angle trajectories to reconstruct observed motion, but standard markerless processes have limitations, including lack of temporal smoothness, ignoring ground contact constraints, and marker misalignment. The objective of this exploratory internal research project aims to integrate an advanced physics engine into SwRI’s Engine for Automatic Biomechanical Evaluation (ENABLE™) markerless system, addressing the drawbacks in standard markerless motion capture processes to improve the accuracy of joint angle estimation along with dynamics, contact, and control optimization capabilities.
Approach
As a preliminary step, to incorporate a physics engine into SwRI’s ENABLE™ markerless system, we process our evaluation videos to generate virtual marker predictions. Next, we modify the OpenSim human biomechanical model into a format compatible with the physics engine environment. Following, we design an inverse kinematics optimization loop that includes virtual marker offset optimization, scaling, and contact detection. We also initiate control-based trajectory optimization using iterative Linear Quadratic Regulator (iLQR) and reinforcement learning (RL) algorithms. Finally, we compare our joint angle estimation errors with existing ENABLE™ results on the evaluation videos. We show an example output of the inverse kinematics optimization loop with ground contact detection in Figure 1.
Figure 1: Squat motion from inverse kinematics optimization in MuJoCo. Contact forces are shown with red arrows, predicted markers are blue and virtual markers are magenta.
Accomplishments
We show comparable joint angle estimation error results between existing results (OpenSim) and a physics engine (MuJoCo), with slight advantages in optimized marker offsets. RMSE values indicate within 1-3 degrees for most movements, with better performance in single leg jumps (over prior work) due to improved pose estimation in the ENABLE™ system (Figure 2). However, the physics engine approach promises additional capabilities such as dynamics output (e.g., joint torques and forces) and constraints with fully integrated reinforcement learning for control optimization.
Figure 2: Average RMSE between OpenSim, MuJoCo, and MuJoCo with optimized marker offsets.