Understanding Black-box Predictions via Influence Functions Pang Wei Koh (Stanford), Percy Liang (Stanford) ICML 2017 Best Paper Award Background The explainability of ML models Best-performing models: complicated, black-box What is now
"Understanding Black-box Predictions via Influence" is the property of its rightful owner. Permission is granted to
download and print the materials on this website for personal, non-commercial use only, and to display it
on your personal computer provided you do not modify the materials and that you retain all copyright
notices contained in the materials. By downloading content from our website, you accept the terms of this
agreement.
Presentation Transcript
01
Understanding Black-box Predictions via Influence Functions Pang Wei Koh (Stanford), Percy Liang (Stanford)
ICML 2017 Best Paper Award<br>
02
Background The explainability of ML models
Best-performing models: complicated, black-box
What is now often being studied?
How a fixed model leads to particular predictions,
i.e., what predictions will a model make and
why a model predicts in this way.<br>
03
Goal To explain where the model come from.
Without a single training point (V1) or if it were changed slightly (V2),
what would happen to the model? Original Model Model
V1 Model
V2<br>
04
Difficulties Unable to really perturb the data and retrain the model.
Too many parameters and the cost is too high.
How to estimate the effect of a training point
without retraining the model?
We introduce INFLUENCE FUNCTION<br>
05
Approach<br>
06
Upweighting a training point What if we remove a particular point z?
But we cannot retrain the model!<br>
07
Upweighting a training point Alternative method
Compute the parameter change with a single point’s slight upweight.
The removal of a point is equal to<br>
08
Upweighting a training point Thus the influence of upweighting z is
Therefore,<br>
09
Upweighting a training point What do we need to know?
Only the information of the original model
and the particular point!
NO MORE RETRAIN!!!<br>
10
Perturbing a training input What if we change a particular point z?
Almost the same mathematic methods so it’s not that important.<br>
11
Speed up IF calculation Hessian-vector products (HVPs)
Calculate a product as one entity To calculate H matrix<br>
12
Speed up IF calculation Ways to get HVPs efficiently
1. Conjugate Gradients (CG)
Convert the HVP calculation to a quadratic optimization
2. Stochastic Estimation
Each iteration we use a single point instead of CG’s all data point. n: point number
p: parameter number
t: sample number
r: repeat number<br>
13
Validation and Extensions Accuracy
Compare the estimation made by IF to the retrain result.
Accuracy is high.<br>
14
Validation and Extensions Non-convexity
Use a damping term to convert a non-convex optimization
into a convex one.<br>
15
Validation and Extensions Non-differentiable losses
Use a smooth function to replace the functions without good properties.<br>
16
Case Studies Understanding model behavior
Different models “care” different training points.<br>
17
Case Studies Adversarial training examples
Find the most effective training point and use it to make a fake example.<br>
18
Case Studies Debugging domain mismatch
Fixing mislabeled examples<br>
19
Comments Strengths
Introduce IF, a measurement tool in a detailed, mathematical manner.
Also tell us how to really and efficiently calculate IF.
Various applications have been mentioned.
Weaknesses
Only care about one particular training point, neglecting the collaborative effects made by multiple points.
Some applications seem not very convincing.
Their source code is in a soooooo bad style.<br>
