Modeling Human Motion from Video for Use in Gait Analysis
Rick Parent, Ph.D.*, Kathy Johnson, Ph.D.**, Jianxiang Chang*,
Xiaoning Fu*, Suba Varadarajan*
* CIS Dept., The Ohio State University; ** Dept. Health Informatics, UT Houston
Background. Today, human leg motion has been widely studied and simulated. Some approaches have used several detectors and multiple cameras to record the position of the leg and then used image processing to simulate the leg motion. In this work, our objective is to minimize the use of markers and cameras in order to provide a low-tech solution to gait digitization that can use standard video for follow-up analysis after a more traditional gait study has been done. In order to accomplish this, we are using a 3D model of the patient’s leg to drive the image processing.
In our scenario, a patient has an initial visit to a clinic at which time information about the patient's leg geometry and pathology are recorded. After some treatment period has elapsed (e.g. wearing a brace or post-surgical recovery), a videotape of the patient walking is taken at a remote site and sent in for analysis. We are in the process of developing techniques to enable this analysis.
Methodology. Step 1: Leg modeling. For the initial stage of the project we created surface and skeleton data for a human leg. The "Visual Man" data was used for contour extraction to obtain surface data and a digital model of the leg’s bones was obtained from the internet1.
Step 2: Image processing. The next step was to obtain gait data from a human subject without the use of markers. With the assistance of The Ohio-State University Gait Lab, video sequences of a subject's leg motion was taken. The video was recorded with a camera in the front of the subject and the other at the side of the subject. The video was then dumped to a digital disk and silhouettes of the legs were extracted from the images. A cubic B-Spline curve was used to reduce the noise. Horizontal slices were taken through each scanline of the silhouettes and the midpoint of the slice was used to produce a central axis for each image. Figure 1 shows a sample contour.
Step 3: 3-D leg motion simulation. The multiple 2D views of the leg motion, basic knowledge about the human anatomy, and velocity constraints on the motion were used to reconstruct the leg motion from the axis data2. First, for each time slice, the two 2D axes were used to construct a 3D axis by constructing the planes of project from the camera setup, and then intersecting the projections of the 2D axes from the image planes. The knee joint was then located in key frames based on the largest curvature of the contour3. A degree of reliability for the positioning was determined for each key frame. Next, the ankle was located in a similar manner. Once this was done for the set of frames, the more reliable points could be used as control points of a space-time curve to track the motion of the knee joint and ankle joint throughout the sequence, inferring the position of joints during frames in which data was missing due to occlusion. Once these joints were located in all frames, the entire skeleton could be constructed in all frames fitting the data in a manner like that used for the surface data.
Conclusions. Progress is being made rapidly in being able to do the tasks that we have set out for ourselves. Examples of the state of our work are given in the figure. We have successfully analyzed video and recreated a human gait cycle with this technique; however there are many improvements to be made. The next step is to simultaneously capture marker data and use it as a baseline comparison for our results. We have already used motion data from our gait lab to animate the initial skeleton and surface data.
References