ACFE amp IIA Fraud Conference 2017 March 7 2017 Edmonton AB CAS Analytics amp Data Provisioning Team v01 Jil Tanguay BSc Spec CFI CRMA Manager Claims Assurance Services Alberta Blue Cross ID: 804216
Download The PPT/PDF document "Maximizing Investigation ROI with Integr..." is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1
Maximizing Investigation ROI with Integrated Analytics
ACFE & IIA Fraud Conference 2017March 7, 2017Edmonton, AB
CAS Analytics & Data Provisioning Team
v01
Jil Tanguay, BSc (Spec), CFI, CRMA ManagerClaims Assurance ServicesAlberta Blue Crossjtanguay@ab.bluecross.ca
Darryl Humphrey, PhD, PMP
Senior Data Scientist
Claims Assurance Services
Alberta Blue Cross
dhumphrey@ab.bluecross.ca
Slide2Provider and member claiming behavior is affected by many factors.
2
CAS ADP Team
Member and Provider Claiming Patterns are Dynamic
Economic Conditions
Plan Design
Policies and Processes
Compliance Verification
Fraud Analytics
Industry Realities
Slide3There are many paths to developing an integrated set of analytics-enabled business processes.
Business knowledge and a willingness to learn are more important than the tool setBe prepared to examine all your processes to determine where analytics can support decision makingSeek guidance from experienced practitioners, internal and external
Ask yourself if the analyses are:
RelevantReliable
ResponsibleCAS ADP Team3
Slide4Enhanced risk mitigation: new organizational structure, additional capabilities, revised project portfolio.
Analytics &
Data Provisioning
Insight &
FocusTriage
Ongoing
Evaluation
of Priorities
Portfolio
Maintenance
Resourcing
Agility
Compliance
Verification
Efficient &
Extensible
Remedies
Recoveries &
Contract
Status
CAS ADP Team
External
Sources
Tips
Secret Shopper
Social Media
4
Slide5CAS’ in-house team takes a structured, iterative approach to developing and executing analysis plans.
CAS ADP Team5
Data Wrangler: Extensive ABC claims data knowledge
Statistical Analysts (3):
PhD in Analytic Mathematics specializing in deep learning algorithmsBSc in Mathematics & BusinessBSc in Computer Science
Manager:
Certified investigator with 25 years experience with ABC
Sr. Data Scientist:
PhD in behavioral science with advanced statistical training and extensive business consulting experience
Business Understanding
Data
Understanding
Data
Preparation
Modeling
Evaluation
Deployment
Slide6Analyzing equivalent of 80,000,000 claim lines monthly encompassing 17,000 providers and 1.6 million members.
6CAS ADP Team
Nine (9) practice areas across health, dental, and pharmacy benefits
70 measures of claiming behavior
Six (6) algorithmsLook for converging results
Slide7Outliers for each individual variable provide a first look at claiming patterns.
7
The information is in the variance
CAS ADP Team
Example output from pharmacy analyses.
Z Score
Slide8Multivariate techniques identify outliers in the n-dimensional space.
Example output from pharmacy analyses.8
CAS ADP Team
4.18
Percentile of Total Paid Drug claims
4.18
Slide9Focusing on the dollars associated with anomalous claims for each provider gives greater clarity.
9Proportion of Total $ that are at Risk
4.18
CAS ADP Team
Slide10Clustering algorithm sharpens the focus on the
riskiest providers.10
4.18
Providers that cluster together have similar claiming patterns.
24
5
54
n=34
Small clusters with high RD scores are of most interest.
Proportion of Total $
that are at Risk
CAS ADP Team
Slide11CAS ADP Team
11
Reviewing the cluster characteristics gives insight into what claiming patterns are driving the outlier scores.
Mean Z Scores
Cluster
#
Prvr
Avg DrugRD
Var1
Var2
Var3
Var4
Var5
Var6
Var7
Var8
Var9
4
5
24
2.0
29.7
-0.5
-0.1
0.1
-0.4
-0.2
-0.2
-0.5
2
24
14
-0.1
0.8
10.3
1.7
1.7
-0.3
0.3
0.6
0.7
3
34
84
-1.3
0.8
-0.4
36.8
8.4
8.9
4.3
0.5
1.1
5
54
46
-0.4
0.9
0.9
17.1
3.4
1.9
2.5
0.9
0.5
…
1
682
2.15
0.2
0.0
0.2
-0.3
0.0
-0.2
-0.1
-0.2
0.2
Slide12CAS ADP Team
12
Claim-specific risk is estimated for the variables highlighted in the K-means and MCD analyses.
Risk
MA(i,j) = (e-(MA(i,j)/MaxMA(
i
))
* (1-d
i
(j)/r))-e
-1
)/(1-e
-1
)
Limited investigation resources are targeted on the specific claims most likely to be an issue.
Slide13Machine learning (ML) = architectures for building algorithms that learn.
CAS ADP Team13
mA
SVM
Random Forest
NN
Neural Network
CNN
DBN
Deep Learning
RBM
K-NN
RNN
Machine learning
Slide14CAS ADP Team
Artificial neural networks are composed of multiple nodes which imitate neurons of the human brain.
14
Neural networks are well-suited to detection tasks.
Neurons are connected by links and they interact with each other. Each link is associated with a weightArtificial neural networks learn by modifying the weights in response to feedback
Deep learning = lots of hidden layers
Most often used for images
Slide15Eye movement research indicates that we recognize objects by extracting features.
CAS ADP Team
15
Slide16The series of layers between input & output do
feature extraction and processing in stages, just as our brains do.
CAS ADP Team
16
Learning
Variables
Slide17Random Forest algorithm classifies observations based on the majority vote of many decision trees.
Risk classification
…
1200
obs
7
vars
Sample
with
replacement
Sample
with
replacement
Sample
with
replacement
17
CAS ADP Team
Slide18Random forest provides clear separation of the riskiest providers and information on variable importance.
CAS ADP Team18
80
th
percentile in Drug RD scores (#5)
60
th
percentile in Drug RD scores (#4 & #5)
Mean Decrease in Accuracy
Dimension 1
Dimension 2
Slide19Many data sets contain nonlinear relationships which can reduce the effectiveness of some detection methods.
Datasets that are linearly separable with some noise work out great
0
x
0
x
0
x
2
x
CAS ADP Team
Some data sets aren’t linear in their initial state
The data can be mapped to a higher-dimensional space
19
Slide20Map feature space to one of higher dimensionality where the training set is separable.
Φ
:
x
→
φ
(
x
)
CAS ADP Team
20
Slide2121
Support Vector Machines find the optimal surface that separates the groups.Maximizes the distance between the hyperplane and the “
difficult points” close to decision boundaryIf there are no points near the decision surface, then there will be fewer false positives and false negatives
Support vectors are the observations near the decision boundary that contribute to determining the boundary.Implies that only support vectors matter; other training examples are ignorable
Ch. 15
CAS ADP Team
Slide22RD Quintile
RD QuintileRandom Forest Confusion Matrix
22
CAS ADP TeamQuintile
Accuracy
0-0.20
0.80
0.20-0.40
0.67
0.40-0.60
0.50
0.60-0.80
0.69
0.80 -1
0.91
Quintile
Accuracy
0-0.20
0.85
0.20-0.40
0.54
0.40-0.60
0.54
0.60-0.80
0.67
0.80 -1
0.94
SVM Confusion Matrix
Slide23Interpreting analytics results requires context.
23CAS ADP Team
The
Beast
Slide24Enhanced risk mitigation: new organizational structure, additional capabilities, revised project portfolio.
Analytics &
Data Provisioning
Insight &
FocusTriage
Ongoing
Evaluation
of Priorities
Portfolio
Maintenance
Resourcing
Agility
Compliance
Verification
Efficient &
Extensible
Remedies
Recoveries &
Contract
Status
CAS ADP Team
External
Sources
Tips
Secret Shopper
Social Media
Slide25Analytics is one input used to match-cost-to-investigate with the anticipated ROI.
CAS ADP Team
25
Slide26Knowledge of your business coupled with informed analytics yields higher ROI for investigations.
Focuses investigation resources where there is an indication of riskRandom selection is a waste of resourcesCAS ADP Team
Sampling Method
Fraud Identified
Random
15.0%
Analytics Alone
43.8%
Analytics & Network Analysis
78.8%
Tailor the projects to the nature and magnitude of the risk
Investigators are more engaged when they have greater confidence their efforts will have a positive outcome
Take steps to manage risk introduced by plan design, culture, economic forces
Analytics are a tool; keep them sharp and use appropriately
Slide27Jil Tanguay, BSc
(Spec), CFI, CRMA ManagerClaims Assurance ServicesAlberta Blue Crossjtanguay@ab.bluecross.ca
Darryl Humphrey, PhD, PMP
Senior Data Scientist Claims Assurance ServicesAlberta Blue Cross
dhumphrey@ab.bluecross.caCAS ADP TeamNazanin Tahmasebi, PhDYemi Dare-Ode, BSc
Wesley Wood,
Bsc