/
Additional tasks  related to activity recogntion Additional tasks  related to activity recogntion

Additional tasks related to activity recogntion - PowerPoint Presentation

sterialo
sterialo . @sterialo
Follow
344 views
Uploaded On 2020-06-15

Additional tasks related to activity recogntion - PPT Presentation

Mitja Luštrek Jožef Stefan Institute Department of Intelligent Systems Slovenia Tutorial at the University of Bremen November 2012 Outline Fall detection with accelerometers Fall detection with location sensors ID: 778157

activity energy detection recognized energy activity recognized detection fall acceleration machine expenditure learning lying classifier ground reliable met sitting

Share:

Link:

Embed:

Download Presentation from below link

Download The PPT/PDF document "Additional tasks related to activity re..." is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.


Presentation Transcript

Slide1

Additional tasks related to activity recogntion

Mitja Luštrek

Jožef Stefan InstituteDepartment of Intelligent SystemsSlovenia

Tutorial at the University of Bremen, November 2012

Slide2

OutlineFall detection with accelerometersFall detection with location sensors

Fall detection with both sensor typesHuman energy expenditure estimation

Slide3

OutlineFall detection with accelerometers

Fall detection with location sensorsFall detection with both sensor typesHuman energy expenditure estimation

Slide4

Acceleration threshold

1–1.5 g

IF acceleration > threshold

THEN

fall

Slide5

OrientationIF a

cceleration > threshold AND AFTERWARDS not upright for 10 seconds

THEN fallNot upright = not –30ᵒ < ϕ < 30ᵒ for 90% of the timeϕ can be adapted to the person

Slide6

Activity and movementIF

acceleration > threshold AND AFTERWARDS recognized (lying

OR on all fours OR sitting on the ground) for 10 seconds AND user immobile THEN fallImmobile = s < thresholdCan skip some activities or movement or switch conjunction/disjunction

Slide7

OutlineFall detection with accelerometers

Fall detection with location sensorsFall detection with both sensor typesHuman energy expenditure estimation

Slide8

Fall detection

Sensors

PreprocessingActivity recogntion

Detection of unusual behavior

Fall detection

User interface

Alarm

Warning

Initialization

False alarm

Machine learning

Expert rules

Merging

Slide9

Machine learningAttributes:Percentages of activities in the last 5, 7.5 and 10 seconds (fall is detected after 10 seconds of lying)

The time of last recognized fallingIs the user in bed

Slide10

Machine learningAttributes:Percentages of activities in the last 5, 7.5 and 10 seconds (fall is detected after 10 seconds of lying)

The time of last recognized fallingIs the user in bedTwo classifiers:

SVM and C4.5 decision treesFall if both return fall

Slide11

Expert rulesIF recognized

falling AND AFTERWARDS recognized

lying/sitting on the ground outside bed AND user immobile THEN fall

Slide12

Expert rulesIF recognized

falling AND AFTERWARDS recognized

lying/sitting on the ground outside bed AND user immobile THEN fallIF recognized falling AND AFTERWARDS recognized lying/sitting on the ground outside bed for a longer time THEN fall

Slide13

Expert rulesIF recognized

falling AND AFTERWARDS recognized

lying/sitting on the ground outside bed AND user immobile THEN fallIF recognized falling AND AFTERWARDS recognized lying/sitting on the ground outside bed for a longer time THEN fallIF recognized lying/sitting on the ground outside bed for an even longer time

AND

user

immobile

THEN

fall

Slide14

Expert rulesIF recognized

falling AND AFTERWARDS recognized

lying/sitting on the ground outside bed AND user immobile THEN fallIF recognized falling AND AFTERWARDS recognized lying/sitting on the ground outside bed for a longer time THEN fallIF recognized lying/sitting on the ground outside bed for an even longer time

AND

user

immobile

THEN

fall

IF

recognized

lying/sitting on the ground outside bed

for a very long time

THEN

fall

Slide15

MergingFall in two cases:

Machine learning and expert rules return fallOne of the modules returns fall continuously for 3 seconds

Slide16

OutlineFall detection with accelerometersFall detection with location sensors

Fall detection with both sensor typesHuman energy expenditure estimation

Slide17

Accelerometers + location sensorsIF recognized (lying

OR on all fours OR sitting on the ground) for 10 seconds AND location = floor

AND user immobile THEN fallActivity recognition (mainly) from accelerometersMobility from accelerometersLocation from location sensorsAgain variations possible

Slide18

Experimental results – activity recogntion

Slide19

Experimental results – fall detection

Slide20

OutlineFall detection with accelerometersFall detection with location sensors

Fall detection with both sensor typesHuman energy expenditure estimation

Slide21

IntroductionMotivation:

Chiron project – monitoring of congestive heart failure patientsThe patient’s energy expenditure (= intensity of movement) provides context for heart activity

Slide22

IntroductionMotivation:

Chiron project – monitoring of congestive heart failure patientsThe patient’s energy expenditure (= intensity of movement) provides context for heart activity Method:

Two wearable accelerometers → accelerationAcceleration → activityAcceleration + activity → energy expenditureMachine learning

Slide23

Direct calorimetryMeasures the total heat output of a person

Separate “dry” and “wet” measurements

Slide24

Indirect calorimetry

Measure inhaled, exhaled O2, CO2Compute the amount of O2 consumed and CO

2 produced by metabolismCompute energy consumption from these amounts

Slide25

Doubly labeled waterWater with deuterium and oxygen-18 administered

Oxygen-18 lost through breathing out CO2Both elements lost through water in urine, sweat ...The difference can be used to compute the amount CO2

produced by metabolismThis amount proportional to the energy consumed

Slide26

DiaryWrite down the activitiesUse reference energy consumption value for each activity

Compendium of Physical Activities with values for over 500 common activities

Slide27

Comparison of the approachesDirect calorimetry

Most reliable, laboratory conditions

Slide28

Comparison of the approachesDirect calorimetry

Most reliable, laboratory conditionsIndirect calorimetry

Quite reliable, field conditions, mask needed

Slide29

Comparison of the approachesDirect calorimetry

Most reliable, laboratory conditionsIndirect calorimetry

Quite reliable, field conditions, mask neededDoubly labeled water Quite reliable, field conditions, long-term (days)

Slide30

Comparison of the approachesDirect calorimetry

Most reliable, laboratory conditionsIndirect calorimetry

Quite reliable, field conditions, mask neededDoubly labeled water Quite reliable, field conditions, long-term (days)Diary Simple, unreliable

Slide31

Comparison of the approachesDirect calorimetry

Most reliable, laboratory conditionsIndirect calorimetry

Quite reliable, field conditions, mask neededDoubly labeled water Quite reliable, field conditions, long-term (days)Diary Simple, unreliable

Wearable accelerometers

Slide32

Hardware

Co-located with ECG

One placement

to be selected

Slide33

Hardware

Co-located with ECG

One placement

to be selected

Shimmer sensor nodes

3-axial accelerometer @ 50 Hz

Bluetooth and 802.15.4 radio

Microcontroller

Custom firmware

Slide34

Hardware

Co-located with ECG

One placement

to be selected

Shimmer sensor nodes

3-axial accelerometer @ 50 Hz

Bluetooth and 802.15.4 radio

Microcontroller

Custom firmware

Android smartphone

Bluetooth

Slide35

Training/test data

ActivityLying

SittingStandingWalkingRunningCyclingScrubbing the floorSweeping...

Slide36

Training/test data

ActivityEnergy expenditure

Lying1.0 METSitting1.0 METStanding1.2 METWalking3.3 METRunning11.0 MET

Cycling

8.0 MET

Scrubbing the floor

3.0 MET

Sweeping

4.0 MET

...

1 MET =

energy

expended

at rest

Recorded

by five

volunteers

Slide37

Machine learning procedure

a

tat+1at+2...

Acceleration data

Sliding window (2 s)

Slide38

Machine learning procedure

a

tat+1at+2...

Acceleration data

Sliding window (2 s)

f

1

f

2

f

3

...

Activity

Training

Machine learning

AR Classifier

Slide39

Machine learning procedure

a

tat+1at+2...

Acceleration data

f

1

f

2

f

3

...

Use/testing

Activity

Sliding window (2 s)

AR Classifier

Slide40

Machine learning procedure

a

tat+1at+2...

Acceleration data

Activity

AR Classifier

Slide41

Machine learning procedure

a

tat+1at+2...

Acceleration data

Sliding window (10 s)

Activity

AR Classifier

Slide42

Machine learning procedure

a

tat+1at+2...

Acceleration data

Sliding window (10 s)

f’

1

f’

2

f’

3

...

Activity

EE

Training

Machine learning (regression)

EEE Classifier

Activity

AR Classifier

Slide43

Machine learning procedure

a

tat+1at+2...

Acceleration data

Sliding window (10 s)

f’

1

f’

2

f’

3

...

Activity

Use/testing

EEE Classifier

Activity

AR Classifier

EE

Slide44

Machine learning procedure

a

tat+1at+2...

Acceleration data

EE

Energy expenditure

Slide45

Features for activity recognitionAverage accelerationVariance in acceleration

Maximum and minimum accelerationAccelerometer orientationSpeed of change between min. and max.Correlations between accelerometer axes

Frequency domain features (FFT)

Slide46

Features for energy expenditure est.The most prevalent activity The average length of the acceleration vector

The area under the absolute acceleration along the x, y, and z axes and the area under the length of the acceleration vector

Slide47

Features for energy expenditure est.The sum of the areas under the absolute acceleration along the x, y, and z axes

The area under the gravity-subtracted (high-pass filtered) acceleration along the x, y, and z axesThe change in the velocity of the a

ccelerometer along the x, y and z axes

Slide48

Features for energy expenditure est.The number of times the movement of the length of the acceleration vector changes direction, i.e., stops increasing and starts decreasing or vice versa

, the sum of the values at which the changes occur

Slide49

Features for energy expenditure est.The integral of the change in the kinetic energy due to translation along the x, y and z axes over the window

Slide50

Sensor placement and algorithm

Linear regression

Support vector regressionRegression treeModel treeNeural networkChest + ankle5.093.291.412.18

1.65

Chest

+ thigh

6.75

3.68

1.58

2.38

1.66

Chest + wrist

6.75

3.94

1.30

4.95

1.39

Mean absolute error in MET

Slide51

Sensor placement and algorithm

Linear regression

Support vector regressionRegression treeModel treeNeural networkChest + ankle5.093.291.412.18

1.65

Chest

+ thigh

6.75

3.68

1.58

2.38

1.66

Chest + wrist

6.75

3.94

1.30

4.95

1.39

Mean absolute error in MET

Lowest error, poor extrapolation, interpolation

Second lowest error, better flexibility

Slide52

Estimated vs. true energy

Average

error:1.39 MET

Slide53

Estimated vs. true energy

Low intensity

ModerateintensityRunning, cyclingAverageerror:1.39 MET

Slide54

Estimated vs. true energy

Low intensity

Moderateintensity

Running, cycling

Average

error:

1.39 MET

Slide55

Multiple classifiers

Activity

AR Classifier

Slide56

Multiple classifiers

Activity

AR Classifier

General

EEE Classifier

EE

Cycling

EEE Classifier

Running

EEE Classifier

Activity = cycling

Activity = running

Activity = other

Slide57

Estimated vs. true energy, multiple cl.

Low intensity

ModerateintensityRunning, cyclingAverageerror:0.91 MET

Slide58

ConclusionEnergy expenditure estimation with wearable accelerometers using machine learning

Study of sensor placements and algorithmsMultiple classifiers: error 1.39 → 0.91 MET

Slide59

ConclusionEnergy expenditure estimation with wearable accelerometers using machine learning

Study of sensor placements and algorithmsMultiple classifiers: error 1.39 → 0.91 METCardiologists judged suitable to monitor congestive heart failure patientsOther medical and sports applications possible