Using Physical Activity for User Behavior Analysis

Using Physical Activity for User Behavior Analysis - Using...

Info iconThis preview shows page 1. Sign up to view the full content.

View Full Document Right Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: Using Physical Activity for User Behavior Analysis Gerald Bieber Fraunhofer-Institute for Computer Graphics J.-Jungius-Str. 11 D-18059 Rostock, Germany Tel: (++49) 381-4024-110 Christian Peter Fraunhofer-Institute for Computer Graphics J.-Jungius-Str. 11 D-18059 Rostock, Germany Tel: (++49) 381-4024-110 ABSTRACT Physical activity is one important aspect in user behavior analysis. Abnormal movement behavior might be an indicator for an inappropriate lifestyle, insufficient social inclusion, or generally disadvantageous life conditions which might call for medical treatment. Assistive technologies can make use of information on the physical activity of e.g. residents of a nursing home or elderly patients living alone at home. In this paper, we present a mobile technology for identifying movement behavior in everyday life. A three-dimensional acceleration sensor is used to determine physical activity by domain specific feature extraction. By use of data mining techniques and a feature set extracted from everyday usage data, we achieve a high quality and robust classification of physical activity. This can be used for further user behavior analysis. Especially non-linear features like step-detection, horizontal and vertical acceleration as well as spectral analysis proved to be very powerful. A proof-of concept prototype is described which shows the applicability of the developed technologies in everyday life. in real life situations. Most people show periodical pattern of physical activity over the day which can be detected with appropriated sensor techniques. For non-stationary data acquisition, wearable devices based on acceleration sensors are well suited [18-20]. Previous work showed that the recognition of basic physical activities is possible even with just one acceleration sensor. While tests in laboratory environments showed a recognition rate of over 95 % [15], realworld results are less satisfying. The following chapter gives a brief summary of related work in the field of activity recognition. After presenting current solutions and technologies, relevant features of sensor data are described. In the next section of the paper, a proof-of-concept application is presented and finally a compilation of current concerns and future work is shown in the last section. 2. RELATED WORK Physical activity monitoring is used in therapies of many chronic diseases. Also, activity monitoring enables verification of treatments and the progress of therapies. The severity of many diseases like Multiple Sclerosis or Parkinson as well as the necessary degree of professional care for the patients can be much better assessed by use of monitoring and analysis of the physical activity of the patient [11]. Activity behavior of patients with chronic diseases is usually different to that of healthy people. For instance, most patients spend significantly less time walking and standing but more time sitting and lying when compared with sedentary healthy subjects of the same age. On the other hand it could be shown that physical activity positively influences medication and treatment therapy in diabetic patients [10]. Another research area is the inclusion of activity behavior in the field of ergonomics. Hereby physical activities are monitored for determining the workload or phases of inactivity, e.g. by use of the Ovako Working Posture Analysis System (OWAS). Usually data acquisition is done manually, supported by a mobile device [9]. Computer Vision techniques can also support activity monitoring. They offer a relatively high accuracy in tracking people and are already used in some medical environments for monitoring movements of patients in e.g. a hospital. Current systems for medical walk analysis require a fixed camera infrastructure and are obviously only suitable for activity detection in indoor scenarios. Another domain in which activity monitoring is also relevant is Virtual and Augmented Reality. Head movement tracking is used here to improve data pre-fetching, i.e. loading visualization data in advance, and hence to improve the reaction time of the system Categories and Subject Descriptors H.5.2 [Information Interfaces and Presentation]: Interfaces, I.5.2 [Pattern Recognition]: Design Methodology User General Terms Algorithms, Measurement, Performance, Design, Economics, Reliability, Human Factors. Keywords Physical Activity Monitoring, Mobile Assistance, Acceleration Sensor, Pattern Recognition, feature extraction. 1. INTRODUCTION Detecting a person’s activity is an important requirement for user behavior analysis. With today’s sensors and processing devices becoming ever smaller and at the same time more powerful, mobile monitoring of such activities becomes reasonably possible Permission to make digital or hard copies of part or all of this work or personal or classroom use is granted without fee provided that make digitalnothard copies of all or part of this work for Permission to copies are or made or distributed for profit or commercialclassroom useand that copies bear provided thatand theare personal or advantage is granted without fee this notice copies full citationor distributedpage. To copy otherwise,advantage and that not made on the first for profit or commercial to republish, to post bear this notice to redistribute to lists, requirespage. To copy copies on servers, or and the full citation on the first prior specific permission and/or post on servers or to redistribute to lists, otherwise, or republish, to a fee. PETRA'08, July 15-19, 2008, Athens,aGreece. requires prior specific permission and/or fee. Copyright 2008 ACM 978-1-60558-067-8... $5.00 PETRA 2008, July 15-19,2008, Athens, Greece. Copyright 2004 ACM 1-58113-000-0/00/0004…$5.00. [12]. These systems mainly use magnetic, ultrasonic or infrared tracking technologies, sometimes supported by acceleration sensors. Monitoring multiple body parts and analyzing their movements complementary delivers high accuracy results for activity recognition. Various setups for accessing movement data of different body parts with high accuracy have been developed in the past, mainly using strain gauges. Systems like CUELA (Fig. 1), developed by the BG-Institute for Occupational Safety and Health provides a large set of activity data but is very cumbersome and not applicable under everyday conditions. case of a car accident. Today, MEMS are sort of a synonym for the integration of mechanical elements into microelectronic structures, i.e. microchips. The sensors are very small (e.g. 3 x 3 mm2, height 0.9 mm) and are hence very suitable for mobile applications. Current sensors provide a measure range from e.g. +/-2 g to +/-10 g with a quantization of 8 bit. In recent years MEMS are also used in consumer products such as Laptops (fall detection to prevent loss of data due to hard disk failure), irons (for saving energy by switching off the device when not used), or entertainment devices like the Wii game console. Most acceleration sensors used in activity monitoring today are 3axis sensors. They provide 3 degrees of freedom in transitional movements. Furthermore, a 3-axis sensor enables a device to analyze the data independently from its current position and angle. Finally, multi-axis sensors offer a more precise view on the current situation and provide a lower noise level combined with an enhanced accuracy. Commercial systems available today are able to only detect walking related activities (walking, running, resting and horizontal position). The current focus of research is on distinguishing more activities and the quality of activity performance. Activity monitoring includes the intensity of the physical activity, partly an energy equivalent (e.g. calories, activity units etc.), and a comprehensive detection of basic activities in everyday life. At IGD we developed a mobile, three dimensional wireless motion sensing device (MoSeBo, for Motion Sensor Board) which collects movement data of the person wearing it, pre-processes the data, and makes them wirelessly available to processing applications. MoSeBo is very robust and is currently used in different studies related to monitoring bodily activity. Key features of MoSeBo are described in more detail in the following section. Figure 1: CUELA System Activity monitoring systems based on acceleration sensors are well suited for mobile applications. The MIThrill system[2], an activity monitoring system developed at the Massachusetts Institute of Technology (MIT), identifies movement behaviors by Gaussian mixtures and Markov chains. Others use Bayesian networks for activity recognition. Artificial intelligence techniques are used by e.g. Hallym University, Korea who include fuzzy logic and Kohonen self-organizing maps. Rich sensor devices are used by Intel / University of Washington or VTT Finland / Nokia, who are using up to 22 different sensors. These examples give evidence that activity detection based on acceleration sensors is possible with high accuracy. While the systems mentioned here are fairly complex using a number of sensor nodes, the system introduced later in this paper uses only a single sensor and dedicated analysis algorithms. Products measuring the physical activity of a user by multi-axis acceleration sensors are already available on the market. In 2004, King et. al. [16] described that only a few systems are available to be used as devices for measurement of physical activity. Today, a great variety of such sensors are available. However, DeVries et al [17] found in a survey of valuations of 20 items and features that pedometers, single axis acceleration sensors, and 3-axis acceleration sensors differ significantly with regard to the actual movement characteristics. Generally they found that multiple-axis acceleration sensors achieve the best results for all forms of movements and all groups of users. The more axes were available, the better the results were. An open research field is on sufficient physical activity detection with only a few or just one sensor. 2.2 Data Collection and Preprocessing The MoSeBo, a small sized, mobile measurement and preprocessing unit, was developed for monitoring physical activity over a longer period of time, at least 24 hours non-stop, with real time availability of the data, in high accuracy. Also, the data are stored on an exchangeable memory card, in addition to a common wireless connection. The MoSeBo device is about the size of a matchbox, weighs 78 grams and can be carried easily in the pocket of a trouser or shirt (see Fig. 2). We use a Bosch 3D-acceleration sensor with digital output, 8 bit quantization, low noise, low power consumption, and a high sampling rate. 2.1 ACTIVITY RECOGNITION WITH ACCELERATION SENSORS The technology behind acceleration sensors are Micro-ElectroMechanical Systems (MEMS). At the beginning, MEMS were used in the automotive domain, e.g. for triggering the airbag in Figure 2: MoSeBo V5 To guarantee a high accuracy of the sampled data, MoSeBo determines its orientation in space by use of gravitational effects. This frees the user from taking care of the sensing device, allowing him to simply put it in a pocket or on the belt, without further attention to it. MoSeBo uses a quantization of 10 bit per measurement range of +/-8g which is defined by the used MEMS. Trading off computational load, memory requirements, and accuracy, the sensor board uses a sampling rate of 32Hz. This rate is fast enough to detect the fastest human body movement which is about 1.5 Hz for the average person and 12-15 Hz maximum at special activities such as of the fingers of a musician when playing a trill on a flute or piano [25]. settings with the recognition rate dropping to 67% [22]. Hence, for real life scenarios and real-world applications more features are necessary which are tailored to the specifics of activity monitoring. In the following we discuss some of the domain specific features identified by us as valuable indicators for different activities. Including these features in our data analysis, we could increase the recognition rates for activities walking, running, cycling, and resting to 91.15 %. Step Detection and Step Variance. The key feature of the activity types walking and running is the existence of steps. While walking is a periodical movement, the spectral analysis shows significant parameters in the FFT. But also other activities like cycling are periodic activities and might result in similar parameters. In addition to the FFT we also use a time domain algorithm for step detection. This applies a dynamic threshold detection and a time series analysis. The advantage of the step detection in the time domain is the knowledge about the point in time of the occurred step. This provides the additional feature of step variance. The resulting accuracy of step detection is about 95%, depending on the walking speed. Vertical Acceleration. As mentioned above, the influence of the earth’s gravity allows permanently to calculate the current sensor orientation. With this additional information, we can distinguish further activity types, such as cycling. The activity type cycling (i.e. sitting on a bike and moving the legs) mainly influences the vertical acceleration. Even cycling in a curve with a tilted position provides vertical forces because of the centrifugal acceleration. This feature enables a good detection of cycling. Phase shifting. The long term average of the acceleration over the three axes result in a constant value aconst = ax2 + ay2 + az2. The long term average over each axis provide the orientation of the sensor in the Cartesian coordinate system. A rotation of the sensor causes a change of the axis-average. This change, which we define as phase shifting, is a feature specific to the rotation of the sensor and is very useful for transition recognition of physical activity. Cross Acceleration (horizontal acceleration). Like vertical acceleration, the acceleration in horizontal direction is a very relevant feature, for instance for detecting car driving activities. Accelerating, decelerating, and changing directions all result in characteristic patterns of cross acceleration. For instance, car driving is characterized by relatively long periods of cross acceleration (several seconds). This feature allows to distinguish e.g. sitting in front of a TV from car driving. Absent Feature. People are usually starting to use the monitoring system when they are dressed up, stopping the monitoring while they get undressed. Often, when having a nap, people also dismounted the motion sensing device. Monitoring data of existing activity monitor systems hence collect data only when the user is active. Periods of bodily inactivity don’t result in any data which effectively leads to a fairly spotty day activity profile. However our assumption is that the idle time of the sensor belongs to the recreation time of the user which indeed is relevant for e.g. tailoring a therapy. Since the acceleration data of a motion sensor which is put aside significantly differs from that of a body worn device we can distinguish e.g. a user having a nap from a user not using the device. Figure 3: Error caused by sampling rate Figure 3 shows the dependency of the sampling rate for a step detection algorithm, determined by constant walking speed of 4 km/h. The accuracy of the step detection decreased by reducing the sampling rate and showed significantly loss of quality less than 12 Hz. Concerning the sample size, in [13] it is worked out that there is no single ‚best‘ length for all activities but 2 seconds is a good average. The implementation of the MoSeBo uses 64 samples for each frame without a window overlap. 2.3 Feature Extraction The MoSeBo extracts features by spectral analysis, time domain analysis, phase shift analysis, and domain specific analysis. Taking all possible features together, a feature set of about 600 elements can be created. Since the size of the learning set increases with the numbers of features, a tradeoff has to be found between the practical size of the learning set and the number of features used. In [14] it is worked out that a number of 10-20 carefully selected features are sufficient to reliably detect and distinguish a variety of movement patterns. In MoSeBo we are currently using a vector which consists of 16 elements. Bao [18] has found that with just one sensor and 4 features a recognition rate of 80 % can be achieved for physical activities in seminaturalistic conditions. Hereby standard features are used, such as mean, energy, frequency-domain entropy, and correlation features. 2.3.1 Domain Specific Features While standard features like mean, energy, or frequency-domain entropy deliver high-accuracy data in laboratory environments (95% according to [22]), they perform less good in non-lab 2.4 Activity Classification As described above, we use a time dependent feature set, represented in a feature vector. This cyclic feature vector is then used in a classifier to determine the performed activity in real time on the device. While different approaches (SVM, Bayesian Nets and Decision Trees) all yielded good results, we finally decided for a J48 decision tree since it had the best performance on the mobile target device. 3. MOBILE APPLICATION 3.1 DiaTrace Some diseases are caused by the lack of physical activity, e.g. diabetes, adiposity or hypertension. The aim of the DiaTrace application is to provide a mobile assistance for these patients. DiaTrace uses a MoSeBo system in combination with a camera phone, both connected via Bluetooth (Fig. 4). It measures the person’s daily activity, encourages to do some sports, gives support at exercises and allows to monitor the food consumption of the patient by taking pictures of the meals using the built-in camera. The collected data and pictures can be viewed on the mobile phone (Fig. 5) as well as sent to a central server via GSM where they can be viewed and analyzed by e.g. the therapist or the patient himself (Fig 6, 7). Figure 5: Current Activity:Running (left) and activity history (right) The first tests of DiaTrace confirmed that a 24h support by mobile assistants is possible and acceptable by users. Only some women complained that they had difficulties to adequately fix the sensor on their body because they usually wear skirts without belt or pockets. Another interesting finding was that some participants in the test increased their physical activity just because of the feeling that they were monitored. They more often rode their bike, walked, and watched less TV. After a week’s time subjects got used to the system and started to use it as an autonomous diary system. They could see on the phone at which time they returned from a party or how long they stayed at different locations. The system has even been used by one test subject for travel expense reports because of the easy identification of start or ending of business trips. The following figure 6 shows the start of a day with a run, followed by a working day with occasional walks, a cycling period at about 5 pm and a TV session from 8 to 10:30. Figure 7 displays the according shares of activities per entire day. Figure 4: DiaTrace, Sensor and Phone With it being as easy to use as a bathroom scale DiaTrace is an innovative digital assistant, giving advise, and informing on current and average activity intensity plus qualitative data of the type of activity (e.g. cycling, car, bus riding, sitting, walking etc.). In addition, it reminds the user to get active, or congratulates when the daily activity goal has been achieved. Hereby DiaTrace reacts as a non-obtrusive interface with the user getting situation related feedback in real-time. The camera phone performs the activity classification and provides multimediafeedback to the wearer according to patient specific settings. Figure 6: Activities over a day, taken with DiaTrace 6. REFERENCES [1] Chávez E, Ide R., Kirste T. 1999. Interactive applications of personal situation-aware assistants. Computers & Graphics, 23(6):903-915. [2] DeVaul R., Sung M, Gips J., Pentland A. (2003) MIThril 2003: Applications and Architecture. Proc. 7th IEEE International Symposium on Wearable Computers, White Planes, NY, USA, 21-23 October 2003. The IEEE. [3] Clarkson B.,2002, Life Patterns: structure from wearable sensors, PhD Thesis, MIT Media Lab [4] Laerhoven K., Cakmakci O., 2000, What shall we teach our pants, proceeding wearable computers, IEEE, Atlanta USA, 2000 Figure 7: Activity shares over the day [5] Bieber G., Diener H., 2005, StepMan - New kind of Music Interaction, HCII 2005, 11th International Conference on Human-Computer-Interaction, Las Vegas, U.S.A., July 2228 [6] Bieber G., 2006, Generic Architecture for Personal Assistance Architecture for Mobile Environments, IEEE Proceedings 1st. SMAP 2006, Athens, Greece [7] Azuma, R. and Bishop, G. 1995. A frequency-domain analysis of head-motion prediction. In Proceedings of the 22nd Annual Conference on Computer Graphics and interactive Techniques S. G. Mair and R. Cook, Eds. SIGGRAPH '95. ACM, New York, NY, 401-408. DOI= [8] Hayashi Saeko, Noda Yoshiyuki, Kitagawa Hideo, Terashima Kazuhiko, 2006, Behavior Analysis of Passenger and Comfort Traveling by Omni-Directional Mobile Wheelchair, Journal Title;Nippon Kikai Gakkai Robotikusu, Mekatoronikusu Koenkai Koen Ronbunshu (CD-ROM), Journal Code:L0318B, Accession number 06A0594017, Vol.2006, page 2P2-E18, Japan [9] Janik H., Bieber G., Gabrecht S., Urban B., 2004, Erfassung der muskuloskelettalen Belastung an Arbeitsplätzen mittels Mobile Computing, proceedings of Mobiles Computing in der Medizin, MoCoMed Eds: Kirn, Anhalt, Heine, p.3-13, Shaker Verlag Stuttgart, Germany, ISBN: 3-8322-2714-8 [10] Martin S, Schneider B, Heinemann L, Lodwig V, Kurth HJ, Kolb H, Scherbaum WA, 2005, Self-monitoring of blood glucose in type 2 diabetes and long-term outcome. Diabetes 54 Suppl. 1, A75 [11] Daumer M., Thaler K., Kruis E., Feneberg W., Staude G., Scholz M., 2006, "Steps towards a miniaturized, robust and autonomous measurement device for the long-term monitoring of the activity of patients – ActiBelt". Biosignalverarbeitung, University Potsdam, 13.-14.07.2006 [12] Lumsden, J., Brewster, S., 2003, A paradigm shift: alternative interaction techniques for use with mobile & wearable devices. In: CASCON ’03: Proceedings of the 2003 conference of the Centre for Advanced Studies on Collaborative research, IBM Press, p. 197–210 [13] Schiele, T. Huynh; B. 2005: Analyzing Features for Activity Recognition, European Conference/Symposium on Ambient Intelligence, Joint sOc-EUSAI conference, Grenoble 4. CONCLUSIONS AND FUTURE WORK Physical activity detection in everyday life is very complex. By use of relevant features of acceleration data, mobile physical activity monitoring and assistance systems will become more powerful. In this paper, we presented specific features, which allow for high-quality activity classification in real-life settings. Especially non-linear features like a robust step-detection and the diversion of horizontal and vertical oriented acceleration can be easily extracted even on mobile devices with limited computing power. With DiaTrace we presented a new application to monitor a person’s physical activity and to assist in changing the lifestyle and eating habits. Currently, DiaTrace is used in a pilot study with overweight children aged 7-17 years in a specialized clinic. Hereby a longterm evaluation of body weight progress and phone based motivation tools (e.g. multimedia messages, activity high score list etc.) are planned. In the future, build in acceleration sensors will be used for every day activity monitoring. We already transferred the developed algorithm of activity recognition and adjusted the parameters to the very low sensor performance (quantization and sampling rate) of the mobile phone. The algorithm showed very good results on activity recognition. However, the movements while using the phone as a calling device might affect the activity detection which has to be worked on in the future. We also envision the setup of a physical activity database for a homogeneous appraisal of results of activity recognition. Furthermore we are working on a combination of physical activity monitoring with emotion sensing devices like EREC [24], which would allow for an even better personalized, sensitive assistance. 5. ACKNOWLEDGMENTS Our thanks to Tom Steinfeldt for his support throughout the tests and data evaluation phase. [14] Hein A. 2007, Echtzeitfähige Merkmalsgewinnung von Beschleunigungswerten und Klassifikation von zyklischen Bewegungen, diploma thesis, mobile multimedia information systems Group, Universität of Rostock, Germany [15] Lester J., Choudhury T., Borriello G., Consolvo S., Landay J., Everitt K., Smith I., 2005, Sensing and Modelling Activities to Support Physical Fitness, Ubicomp Workshop: Monitoring, Measuring and Motivating Exercise: Ubiquitous Computing to Support Fitness [16] King G., Torres N., Potter C., Brooks T., Coleman K., 2005, Comparison of Activity Monitors to Estimate Energy Cost of Treadmill Exercise, Medicine & Science in Sports & Exercise. 36(7):1244-1251 [17] De Vriesa S., Bakkera I., Hopman-Rocka M., Hirasinga R., van Mechelenb W., 2006, Clinimetric review of motion sensors in children and adolescents, Journal of Clinical Epidemiology, Volume 59, Issue 7, July 2006, Pages 670680, doi:10.1016/j.jclinepi.2005.11.020 [18] L. Bao, 2003. Physical Activity Recognition from Acceleration Data Under Semi-Naturalistic Conditions, M.Eng. thesis, Electrical Engineering and Computer Science, Massachusetts Institute of Technology [19] Najafi, B. Aminian, K. Paraschiv-Ionescu, A. Loew, F. Bula, C.J. Robert, P., 2003, Ambulatory system for human motion analysis using a kinematic sensor: monitoring of daily physical activity in the elderly. IEEE Transactions on Biomedical Engineering Volume: 50, Issue: 6, 711- 723. ISSN: 0018-9294 [20] Roozbeh Jafari R., Li W., Bajcsy R., Glaser S. and Sastry S. 2007. Physical Activity Monitoring for Assisted Living at Home. In: IFMBE Proceedings 4th International Workshop on Wearable and Implantable Body Sensor Networks, pp.213-219. Springer Berlin Heidelberg, ISBN 978-3-54070993-0 [21] Pirttikangas S., Fujinami K., Nakajima T., 2006, Feature Selection and Activity Recognition from Wearable Sensors,Lecture Notes in Computer Science, Volume 4239/2006, p. 5^6-527, Springer, DOI 10.1007/11890348_39 [22] Pärkkä, J., Ermes M., Korpipaä P., Mäntyjärvi J., Peltola J., Korhonen I., 2006, Activity Classification Using Realistic Data From Wearable Sensors, IEEE Transaction on Information Technology in Biomedicine, Vol 10, No. 1, Jan 2006 [23] Weka (Waikato Environment for Knowledge Analysis), Waikato University,, last access 04.2008 [24] Peter, C., Ebert E., Beikirch, H., 2005. A Wearable MultiSensor System for Mobile Acquisition of Emotion-Related Physiological Data. Proceedings of the 1st International Conference on Affective Computing and Intelligent Interaction, Beijing 2005. Springer Verlag Berlin, Heidelberg, New York, pp. 691-698. ISBN: 3-540-29621-2 [25] Lange H., 2001, Allgemeine Musiklehre und Musikalische Ornamentik, Franz Steiner Verlag, ISBN-13: 9783515056786 ...
View Full Document

Ask a homework question - tutors are online