Movement of the human body is achieved through a complex interaction of muscles, skeleton and the central nervous system. Possible impairments of one of these anatomical systems can be detected at an early stage by experts using a gait analysis. For this purpose, for example, step length, walking speed, step frequency, joint angle, forces acting on the joints, muscle activity and energy consumption are evaluated. This information can be recorded using sensors.

In cooperation with Prof. Kerstin Lüdtke (Physiotherapy, University of Lübeck) and the company Future-Shape GmbH, we are developing artificial intelligence methods to automate and objectify gait and movement analysis. To record the necessary data, we use the SensFloor, RGB cameras, depth sensors and wearables (IMUs), among other things. The SensFloor is a sensor that can be laid widely under normal flooring and measures the electrical capacities of people and objects on it. Using this information, we can then calculate the geometric and temporal information of movement patterns. IMUs are usually attached directly to the body and record data such as acceleration, rotation rate and orientation of the test participants. In addition, information such as joint angles and their positions can be extracted from the camera images. We use all this data to differentiate between pathological and healthy gait patterns, as well as to define a general movement behaviour.

From an AI methodology perspective, we use conventional Deep Learning techniques in this context, as well as algorithms we have developed ourselves that compute the spatiotemporal motion features of human gait for individual identification, without the need for traditional silhouette extraction from video sequences. For example, the extracted local motion data is coded into high-level descriptors using the so-called codebook approach based on the Gaussian mixture model and classified using a simple linear Support Vector Machine. We have already tested our pattern recognition platform on five widely used gait databases and obtained a recognition rate of up to 98%. Another approach deals with person identification and event recognition using motion data recorded by the SensFloor. In this area, we are working on real-time processing of SensFloor data using LSTM and unsupervised learning algorithms.