Mitja Luštrek Jožef Stefan Institute Department of Intelligent Systems Slovenia Tutorial at the University of Bremen November 2012 Outline Fall detection with accelerometers Fall detection with location sensors ID: 778157
Download The PPT/PDF document "Additional tasks related to activity re..." is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1
Additional tasks related to activity recogntion
Mitja Luštrek
Jožef Stefan InstituteDepartment of Intelligent SystemsSlovenia
Tutorial at the University of Bremen, November 2012
Slide2OutlineFall detection with accelerometersFall detection with location sensors
Fall detection with both sensor typesHuman energy expenditure estimation
Slide3OutlineFall detection with accelerometers
Fall detection with location sensorsFall detection with both sensor typesHuman energy expenditure estimation
Slide4Acceleration threshold
1–1.5 g
IF acceleration > threshold
THEN
fall
Slide5OrientationIF a
cceleration > threshold AND AFTERWARDS not upright for 10 seconds
THEN fallNot upright = not –30ᵒ < ϕ < 30ᵒ for 90% of the timeϕ can be adapted to the person
Slide6Activity and movementIF
acceleration > threshold AND AFTERWARDS recognized (lying
OR on all fours OR sitting on the ground) for 10 seconds AND user immobile THEN fallImmobile = s < thresholdCan skip some activities or movement or switch conjunction/disjunction
Slide7OutlineFall detection with accelerometers
Fall detection with location sensorsFall detection with both sensor typesHuman energy expenditure estimation
Slide8Fall detection
Sensors
PreprocessingActivity recogntion
Detection of unusual behavior
Fall detection
User interface
Alarm
Warning
Initialization
False alarm
Machine learning
Expert rules
Merging
Slide9Machine learningAttributes:Percentages of activities in the last 5, 7.5 and 10 seconds (fall is detected after 10 seconds of lying)
The time of last recognized fallingIs the user in bed
Slide10Machine learningAttributes:Percentages of activities in the last 5, 7.5 and 10 seconds (fall is detected after 10 seconds of lying)
The time of last recognized fallingIs the user in bedTwo classifiers:
SVM and C4.5 decision treesFall if both return fall
Slide11Expert rulesIF recognized
falling AND AFTERWARDS recognized
lying/sitting on the ground outside bed AND user immobile THEN fall
Slide12Expert rulesIF recognized
falling AND AFTERWARDS recognized
lying/sitting on the ground outside bed AND user immobile THEN fallIF recognized falling AND AFTERWARDS recognized lying/sitting on the ground outside bed for a longer time THEN fall
Slide13Expert rulesIF recognized
falling AND AFTERWARDS recognized
lying/sitting on the ground outside bed AND user immobile THEN fallIF recognized falling AND AFTERWARDS recognized lying/sitting on the ground outside bed for a longer time THEN fallIF recognized lying/sitting on the ground outside bed for an even longer time
AND
user
immobile
THEN
fall
Slide14Expert rulesIF recognized
falling AND AFTERWARDS recognized
lying/sitting on the ground outside bed AND user immobile THEN fallIF recognized falling AND AFTERWARDS recognized lying/sitting on the ground outside bed for a longer time THEN fallIF recognized lying/sitting on the ground outside bed for an even longer time
AND
user
immobile
THEN
fall
IF
recognized
lying/sitting on the ground outside bed
for a very long time
THEN
fall
Slide15MergingFall in two cases:
Machine learning and expert rules return fallOne of the modules returns fall continuously for 3 seconds
Slide16OutlineFall detection with accelerometersFall detection with location sensors
Fall detection with both sensor typesHuman energy expenditure estimation
Slide17Accelerometers + location sensorsIF recognized (lying
OR on all fours OR sitting on the ground) for 10 seconds AND location = floor
AND user immobile THEN fallActivity recognition (mainly) from accelerometersMobility from accelerometersLocation from location sensorsAgain variations possible
Slide18Experimental results – activity recogntion
Slide19Experimental results – fall detection
Slide20OutlineFall detection with accelerometersFall detection with location sensors
Fall detection with both sensor typesHuman energy expenditure estimation
Slide21IntroductionMotivation:
Chiron project – monitoring of congestive heart failure patientsThe patient’s energy expenditure (= intensity of movement) provides context for heart activity
Slide22IntroductionMotivation:
Chiron project – monitoring of congestive heart failure patientsThe patient’s energy expenditure (= intensity of movement) provides context for heart activity Method:
Two wearable accelerometers → accelerationAcceleration → activityAcceleration + activity → energy expenditureMachine learning
Slide23Direct calorimetryMeasures the total heat output of a person
Separate “dry” and “wet” measurements
Slide24Indirect calorimetry
Measure inhaled, exhaled O2, CO2Compute the amount of O2 consumed and CO
2 produced by metabolismCompute energy consumption from these amounts
Slide25Doubly labeled waterWater with deuterium and oxygen-18 administered
Oxygen-18 lost through breathing out CO2Both elements lost through water in urine, sweat ...The difference can be used to compute the amount CO2
produced by metabolismThis amount proportional to the energy consumed
Slide26DiaryWrite down the activitiesUse reference energy consumption value for each activity
Compendium of Physical Activities with values for over 500 common activities
Slide27Comparison of the approachesDirect calorimetry
Most reliable, laboratory conditions
Slide28Comparison of the approachesDirect calorimetry
Most reliable, laboratory conditionsIndirect calorimetry
Quite reliable, field conditions, mask needed
Slide29Comparison of the approachesDirect calorimetry
Most reliable, laboratory conditionsIndirect calorimetry
Quite reliable, field conditions, mask neededDoubly labeled water Quite reliable, field conditions, long-term (days)
Slide30Comparison of the approachesDirect calorimetry
Most reliable, laboratory conditionsIndirect calorimetry
Quite reliable, field conditions, mask neededDoubly labeled water Quite reliable, field conditions, long-term (days)Diary Simple, unreliable
Slide31Comparison of the approachesDirect calorimetry
Most reliable, laboratory conditionsIndirect calorimetry
Quite reliable, field conditions, mask neededDoubly labeled water Quite reliable, field conditions, long-term (days)Diary Simple, unreliable
Wearable accelerometers
Slide32Hardware
Co-located with ECG
One placement
to be selected
Slide33Hardware
Co-located with ECG
One placement
to be selected
Shimmer sensor nodes
3-axial accelerometer @ 50 Hz
Bluetooth and 802.15.4 radio
Microcontroller
Custom firmware
Slide34Hardware
Co-located with ECG
One placement
to be selected
Shimmer sensor nodes
3-axial accelerometer @ 50 Hz
Bluetooth and 802.15.4 radio
Microcontroller
Custom firmware
Android smartphone
Bluetooth
Slide35Training/test data
ActivityLying
SittingStandingWalkingRunningCyclingScrubbing the floorSweeping...
Slide36Training/test data
ActivityEnergy expenditure
Lying1.0 METSitting1.0 METStanding1.2 METWalking3.3 METRunning11.0 MET
Cycling
8.0 MET
Scrubbing the floor
3.0 MET
Sweeping
4.0 MET
...
1 MET =
energy
expended
at rest
Recorded
by five
volunteers
Slide37Machine learning procedure
a
tat+1at+2...
Acceleration data
Sliding window (2 s)
Slide38Machine learning procedure
a
tat+1at+2...
Acceleration data
Sliding window (2 s)
f
1
f
2
f
3
...
Activity
Training
Machine learning
AR Classifier
Slide39Machine learning procedure
a
tat+1at+2...
Acceleration data
f
1
f
2
f
3
...
Use/testing
Activity
Sliding window (2 s)
AR Classifier
Slide40Machine learning procedure
a
tat+1at+2...
Acceleration data
Activity
AR Classifier
Slide41Machine learning procedure
a
tat+1at+2...
Acceleration data
Sliding window (10 s)
Activity
AR Classifier
Slide42Machine learning procedure
a
tat+1at+2...
Acceleration data
Sliding window (10 s)
f’
1
f’
2
f’
3
...
Activity
EE
Training
Machine learning (regression)
EEE Classifier
Activity
AR Classifier
Slide43Machine learning procedure
a
tat+1at+2...
Acceleration data
Sliding window (10 s)
f’
1
f’
2
f’
3
...
Activity
Use/testing
EEE Classifier
Activity
AR Classifier
EE
Slide44Machine learning procedure
a
tat+1at+2...
Acceleration data
EE
Energy expenditure
Slide45Features for activity recognitionAverage accelerationVariance in acceleration
Maximum and minimum accelerationAccelerometer orientationSpeed of change between min. and max.Correlations between accelerometer axes
Frequency domain features (FFT)
Slide46Features for energy expenditure est.The most prevalent activity The average length of the acceleration vector
The area under the absolute acceleration along the x, y, and z axes and the area under the length of the acceleration vector
Slide47Features for energy expenditure est.The sum of the areas under the absolute acceleration along the x, y, and z axes
The area under the gravity-subtracted (high-pass filtered) acceleration along the x, y, and z axesThe change in the velocity of the a
ccelerometer along the x, y and z axes
Slide48Features for energy expenditure est.The number of times the movement of the length of the acceleration vector changes direction, i.e., stops increasing and starts decreasing or vice versa
, the sum of the values at which the changes occur
Slide49Features for energy expenditure est.The integral of the change in the kinetic energy due to translation along the x, y and z axes over the window
Slide50Sensor placement and algorithm
Linear regression
Support vector regressionRegression treeModel treeNeural networkChest + ankle5.093.291.412.18
1.65
Chest
+ thigh
6.75
3.68
1.58
2.38
1.66
Chest + wrist
6.75
3.94
1.30
4.95
1.39
Mean absolute error in MET
Slide51Sensor placement and algorithm
Linear regression
Support vector regressionRegression treeModel treeNeural networkChest + ankle5.093.291.412.18
1.65
Chest
+ thigh
6.75
3.68
1.58
2.38
1.66
Chest + wrist
6.75
3.94
1.30
4.95
1.39
Mean absolute error in MET
Lowest error, poor extrapolation, interpolation
Second lowest error, better flexibility
Slide52Estimated vs. true energy
Average
error:1.39 MET
Slide53Estimated vs. true energy
Low intensity
ModerateintensityRunning, cyclingAverageerror:1.39 MET
Slide54Estimated vs. true energy
Low intensity
Moderateintensity
Running, cycling
Average
error:
1.39 MET
Slide55Multiple classifiers
Activity
AR Classifier
Slide56Multiple classifiers
Activity
AR Classifier
General
EEE Classifier
EE
Cycling
EEE Classifier
Running
EEE Classifier
Activity = cycling
Activity = running
Activity = other
Slide57Estimated vs. true energy, multiple cl.
Low intensity
ModerateintensityRunning, cyclingAverageerror:0.91 MET
Slide58ConclusionEnergy expenditure estimation with wearable accelerometers using machine learning
Study of sensor placements and algorithmsMultiple classifiers: error 1.39 → 0.91 MET
Slide59ConclusionEnergy expenditure estimation with wearable accelerometers using machine learning
Study of sensor placements and algorithmsMultiple classifiers: error 1.39 → 0.91 METCardiologists judged suitable to monitor congestive heart failure patientsOther medical and sports applications possible