Minsoo Choy 1 Daehoon Kim 1 JaeGil Lee 1 Heeyoung Kim 2 Hiroshi Motoda 3 1 Graduate School of Knowledge Service Engineering KAIST 2 Department of Industrial and Systems Engineering KAIST ID: 546221
Download Presentation The PPT/PDF document "Looking Back on the Current Day: Interru..." is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1
Looking Back on the Current Day: Interruptibility Prediction Using Daily Behavioral Features
Minsoo Choy
1
, Daehoon Kim
1
, Jae-Gil Lee
1
, Heeyoung Kim
2
, Hiroshi Motoda
3
1
Graduate
School of Knowledge Service Engineering,
KAIST
2
Department
of Industrial and Systems Engineering, KAIST
3
Institute
of Scientific and Industrial Research, Osaka UniversitySlide2
01. Introduction
Contents
03. Methodology
- Daily Feature Extraction
02. Interruptibility Dataset
04. Evaluation
05. ConclusionSlide3
The degree of how opportune it is to interrupt a person [Nicky et al. 2004]Interruptibility | Definition
2016-09-16
Looking Back on the Current Day: Interruptibility Prediction Using Daily Behavioral Features
3
Seek attention
Call
Ask a question
…
or
Interruptibility
predictionSlide4
Interruptibility | Prediction in Use
2016-09-16
Looking Back on the Current Day: Interruptibility Prediction Using Daily Behavioral Features
4
Questioner
Experts
Question
Interruptible
NOT Interruptible
Question
Answer
AnswerSlide5
Context-Aware Prediction
2016-09-16
Looking Back on the Current Day: Interruptibility Prediction Using Daily Behavioral Features
5
Temporal
info
(Date / Hour)Charging infoScreen info
Battery infoMovement info(Acceleration force)
Illumination infoLocation info
…
..
Connectivity
info
(Cellular / Wifi)
App Usage
Mood
Interruptibility
…
..
PredictionSlide6
Key Idea62016-09-16
Current
p
oint
15 Min
Start of today
Today
window
Immediate-Past
window
Previous Studies
Current
Point
Imm-Past
Window
Proposed Method
Today Window
Daily Features!
Looking Back on the Current Day: Interruptibility Prediction Using Daily Behavioral FeaturesSlide7
Motivation72016-09-16Self-Regulation / Conservation of energyPeople consciously manage their own thoughts and behaviors
The amount of human activity
per day
is in fact limited and conserved [Kazuo 2014]ProlongationMood
could last all day or longer [Christopher et al. 2005]
Intentional unavailability (“I’m in a bad mood now! ”)
[Afternoon][Night]
[
Morning][Night]
Looking Back on the Current Day: Interruptibility Prediction Using Daily Behavioral FeaturesSlide8
2016-09-16Looking Back on the Current Day: Interruptibility Prediction Using Daily Behavioral Features8Research Questions
RQ1
Current
+
Imm
-Past
RQ2
Current
Current
+
Imm
-Past
+
Today
RQ3
Current
+
Imm
-Past
+
Today
RQ4
Today
Yesterday
Day before
Yesterday
Before
15 Min
Imm
-Past
Window
Day-before-
Yesterday
Window
Yesterday
Window
Today
Window
Current
point
<
Current
+
Imm
-Past
<
Current
+
Imm
-Past
+
Today
Current
+
Imm-Past
+
Today
+
Yesterday
Current
+
Imm-Past
+
Today
+
Yesterday
+
D-b-Yesterday
Current
+
Imm-Past
+
Yesterday
Current
+
Imm-Past
+
D-b-Yesterday
Slide9
We systematically extract daily features in support of interruptibility predictionWe collect smartphone usage data from 25 participants for four weeks as a field studyWe improve accuracy by up to 16% compared with the baseline and by up to 7% compared with the state-of-the-art method2016-09-16
Looking Back on the Current Day: Interruptibility Prediction Using Daily Behavioral Features
9
ContributionSlide10
01. Introduction
Contents
03. Methodology
- Daily Feature Extraction
02. Interruptibility Dataset
04. Evaluation
05. ConclusionSlide11
2016-09-16Looking Back on the Current Day: Interruptibility Prediction Using Daily Behavioral Features11Data Sets
KAIST
25
users (from KAIST)February ~ March 201524
attributes (via our logging app)4,103
interruptibility labelsDevice AnalyzerLargest collection of smartphone usage data907 users (out of 9,641 users)Snapshot as of November 201526 attributes (out of more than 50 attributes)1,646,066
incoming calls (70% of all)Missed & Quickly (< 10 sec) quit Not InterruptibleOtherwise
Interruptible9 a.m. 2 p.m.
10 p.m.
“Are you interruptible?”Slide12
2016-09-16Looking Back on the Current Day: Interruptibility Prediction Using Daily Behavioral Features12Attributes
Numeric
CPU usage
Battery (Level, T
emperature)Ambient light
Signal strength (Cellular, WiFi)Accelerometer (x, y, z)Binary [True/ False]ScreenHeadset modeAirplane modeCellular modeWIFI mode
Nominal
Ringtone mode
Application (Name, Category)LocationWiFi SSID
Charging mode
Call event
Message event
Temporal
Hour (of day)
Timeslot
Day (of week)
(Day, Hour) pairSlide13
2016-09-16Looking Back on the Current Day: Interruptibility Prediction Using Daily Behavioral Features13
Statistics
|Number of Class Labels
[KAIST]
[Device Analyzer]Slide14
2016-09-16Looking Back on the Current Day: Interruptibility Prediction Using Daily Behavioral Features14Statistics | Proportion of Class Label Values
The two label values tend to be
balanced across all timeslots
in both data sets
[KAIST]
[Device Analyzer]Slide15
01. Introduction
Contents
03. Methodology
- Daily Feature Extraction
02. Interruptibility Dataset
04. Evaluation
05. ConclusionSlide16
Overview162016-09-16
Past
Today
Window
Immediate-Past
Window
(e.g., 15 Min.)
Current Point
Feature Extraction
All
Features
Selected
Features
Feature Selection
Training / Prediction
Looking Back on the Current Day: Interruptibility Prediction Using Daily Behavioral FeaturesSlide17
The basic features are those extracted from the current point2016-09-16Looking Back on the Current Day: Interruptibility Prediction Using Daily Behavioral Features17
Feature Extraction
| Basic Feature
Current
Start of Today
Before 15 Min
Imm-Past Window
Today WindowSlide18
Temporal Window
Temporal Window
Temporal Window
2016-09-16
Looking Back on the Current Day: Interruptibility Prediction Using Daily Behavioral Features
18
Feature Extraction
| Extended Feature
T
he
extended features
are defined separately for each type
0
1
val
3
val
2
val
1
t
duration
t
t
Numeric
Mean
Standard Deviation
Binary
Duration
(0
→
1) Numbers
Nominal
Duration
(
0
→
1
) Numbers
Current
Start of Today
Before 15 Min
Imm-Past Window
Today WindowSlide19
2016-09-16Looking Back on the Current Day: Interruptibility Prediction Using Daily Behavioral Features19Feature Extraction | Extended Feature
Current
Start of Today
Before 15 Min
Imm-Past Window
Today Window
A day is divided into
six equi-width timeslots
The
DWT (Discrete Wavelet Transform)
is applied to
numeric attributes
in order to capture a general trend (i.e., temporal shape)
Morning
Lunch
Afternoon
Dinner
Night
Dawn
6 a.m. 9 a.m. 12 p.m. 3 p.m. 6 p.m. 9 p.m. 12 a.m.Slide20
Motivation2016-09-16Looking Back on the Current Day: Interruptibility Prediction Using Daily Behavioral Features
20
Feature Extraction
| DWT
Al
most the same mean & standard deviation
Different shape (i.e., movement)Morning
Different interruptibilityDinner
2015Feb
14
2015Feb
24
DWTSlide21
Creating (Haar) DWT features2016-09-16Looking Back on the Current Day: Interruptibility Prediction Using Daily Behavioral Features21
Feature Extraction
| DWT
=
Time
Frequency
Input
Using
as the
DWT features
…
+
+
+
+
+
+
+
Slide22
2016-09-16Looking Back on the Current Day: Interruptibility Prediction Using Daily Behavioral Features22Feature Extraction | DWT
Creating (Haar) DWT features
Using the
first 32 coefficients (i.e.,
)
out of 256 coefficients Slide23
Feature SelectionCorrelation-based feature selection (CFS)Selecting a subset of features that are highly correlated with the class while having low intercorrelationTrainingPersonalized classification model
2016-09-16
Looking Back on the Current Day: Interruptibility Prediction Using Daily Behavioral Features
23
Feature Selection & Training
Morning
Lunch
Afternoon
DinnerNight
…
Morning
Lunch
Afternoon
Dinner
NightSlide24
01. Introduction
Contents
03. Methodology
- Daily Feature Extraction
02. Interruptibility Dataset
04. Evaluation
05. ConclusionSlide25
Classification AlgorithmsNaïve BayesSupport Vector MachineRandom ForestC4.5 Decision TreeEvaluation TechniqueKAIST: 5-fold cross validationDevice Analyzer: 10-fold cross validationEvaluation MetricsAccuracy
Kappa
2016-09-16
Looking Back on the Current Day: Interruptibility Prediction Using Daily Behavioral Features
25
Evaluation SettingSlide26
Day-before-Yesterday
Yesterday
Today
Imm-Past
Current
CURR
✓IPAST
✓
✓DAY[0]
✓
✓
✓
DAY[-1:0]
✓
✓
✓
✓
DAY[-2:0]
✓
✓
✓
✓
✓
DAY[-1]
✓
✓
✓
DAY[-2]
✓
✓
✓
2016-09-16
Looking Back on the Current Day: Interruptibility Prediction Using Daily Behavioral Features
26
Configurations & Research Questions
Conf.
Window
Research Questions
Representation
Days of Prediction
RQ1, RQ2
CURR
< IPAST < DAY[0]
Seven
days (All Days)
RQ3
DAY[0]
DAY[-1:0]
DAY[-2:0]
Wed, Thu, Fri
RQ4
DAY[0]
DAY[-1]
DAY[-2]
Wed, Thu, Fri
Research Questions
Representation
Days of Prediction
RQ1, RQ2
CURR
< IPAST < DAY[0]
Seven
days (All Days)
RQ3
Wed, Thu, Fri
RQ4
Wed, Thu, Fri
Baseline
State-of-the-art
Proposed MethodSlide27
2016-09-16Looking Back on the Current Day: Interruptibility Prediction Using Daily Behavioral Features27RQ1 & RQ2: Daily Features
Accuracy and kappa to address RQ1 and RQ2
Not Enough Data
F
or each
Day of Week
(a) KAIST data set(b) Device Analyzer data set
(a) KAIST data set(b) Device Analyzer data setF
or each Timeslot
We can increase the accuracy by leveraging the
daily featuresSlide28
2016-09-16Looking Back on the Current Day: Interruptibility Prediction Using Daily Behavioral Features28RQ1 & RQ2: Daily Features
Top-15 discriminative features in DAY[0]
(a) KAIST data set (for
dinner
)
(b) Device Analyzer data set (for night
)
A number of features were extracted from the
today window
The
behaviors in the previous several hours
affect the current interruptibilitySlide29
2016-09-16Looking Back on the Current Day: Interruptibility Prediction Using Daily Behavioral Features29RQ3: Temporal Window Length
(a) KAIST data set
(b) Device Analyzer data set
NOT
statistically
significant
NOT
statisticallysignificantLooking further back beyond the current day is not very helpful for increasing the prediction accuracy
when the data on the current day does existAccuracy to address RQ3Slide30
2016-09-16Looking Back on the Current Day: Interruptibility Prediction Using Daily Behavioral Features30RQ4: Daily Routineness
NOT
statistically
significant
NOT
statisticallysignificant(a) KAIST data set
(b) Device Analyzer data set
Data from the previous day can be a good substitute when the model suffers from the lack of the data from the current
dayAccuracy to address RQ4Slide31
01. Introduction
Contents
03. Methodology
- Daily Feature Extraction
02. Interruptibility Dataset
04. Evaluation
05. ConclusionSlide32
2016-09-16Looking Back on the Current Day: Interruptibility Prediction Using Daily Behavioral Features32Conclusion
We proposed a
daily feature
extraction methodology for interruptibility predictionWe found that:
Accuracy improvement was attributed to the fact that daily features
were included in the prediction for many users (RQ1, RQ2)Looking further back beyond the current day did not improve accuracy (RQ3)A day’s behavior was replaceable with another day’s behavior (RQ4)Our methodology will improve a design of
intelligent notification systems (e.g., real-time mobile Q&A service)Slide33
Thank you