People Matching October 22 2014 Jay Colbert Indianapolis Domestic Violence Network Phase1 Developed Domestic Violence in the Criminal Justice System report in 20132014 Phase 2 Updating report ID: 576268
Download Presentation The PPT/PDF document "Domestic Violence Network" is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1
Domestic Violence NetworkPeople Matching
October 22, 2014
Jay Colbert, IndianapolisSlide2
Domestic Violence NetworkPhase1: Developed “Domestic Violence in the Criminal Justice System” report in 2013-2014Phase 2: Updating reportThere are many different sources of DV data and the question was “how many unique people are involved?”Slide3
Data Sources
Notes
1
Domestic Violence Shelter
2
No names given
3Special IMPD domestic violence program
Source
Table
Date Range
Records
Julian
Center
1
Julian Outreach
2008
- 2013
32,540
Indianapolis
Metro PD
IMPD
Victims
2
2009 -
2013
341,558
IMPD Perpetrators
2009 - 2013
520,148
(38,780 linked to other DV Data)
Baker
One
3
2013
2,715
Marion County Prosecutor’s Office
MCPO Cases
2008
-
2013
32,885
MCPO Victims
2008
-
2013
36,133
Indiana
Supreme Court
Protective Orders (Protected Person)
2009-2013
123,630
Protective Orders (Respondent)
2009-2013
123,630
Total (with names)
871,681
Marion County DV Total (with names)
177,545Slide4
Phase 1 people matchingIn-house solution using (mainly) python scripts.Looked at combination of name, race, gender, age.Gave pretty good final results, but…It took 10 days to run.Slide5
Phase 2 people matchingResearched multiple commercial software solutions geared towards data deduplication.Some enterprise solutions as much as $30,000 and up.Some desktop solutions as cheap as $500. Slide6
Phase 2 name matchingPurchased MatchIT software.$4,000 initial, $2,000 annual renewalDiscounted since we are a nonprofitFlexibility on matching algorithms.We match on name and date of birth first.Then we move on to matching on name, race, gender, and year of birth (not all sources give us exact DOB)
We
previously used age
, but that becomes problematic with increasing number of years of data.Slide7
Example
DVNMATCH
FIRST NAME
LAST NAME
RACE
GENDER
BIRTH_DATE
BIRTH_YEAR
Tammy
Copelan
Black
Female
11/25/1983
1983
Tammy
Copland
Black
Female
11/25/1983
1983
Tamara
Copeland
Black
Female
11/25/1983
1983
Tamy
CopelandBlackFemale11/25/19831983TammyCopelandBlackFemale11/25/19831983Tammy CopelandBlackFemale11/25/19831983TammieCopelandUnknownFemale11/25/19831983TammyCopelandBlackFemale1983TammyCopelandBlackFemale1983TammyCopelandBlackFemale
These records identified as same person.
Names have been changed to protect the innocent (and the guilty).Slide8
Old Method
Old Results
POLISID
FIRST_NAME
LAST_NAME
RACE
GENDER
DOB
BIRTH_YEAR
6486
Eric
Jones
White
Male
10/1/1980
1980
6486
ERIC
JONES
White
Male
10/1/1980
1980
6486
ERIC
JONES
White
Male10/15/198219826486ERICJONESWhiteMale1/15/198519856486ERICJONESWhiteMale8/20/198119816486
EricJones
White
Male
19826486EricJonesWhiteMale10/30/198219826486EricJonesWhiteMale12/17/1985198581210EricJonesWhiteMale7/3/1988198881210EricJonesWhiteMale6/10/1989198981210ERICJONESWhiteMale2/5/1986198681210ERICJONESWhiteMale2/5/1986198681210EricJonesUnknownMale7/3/1988198881210ERICJONESWhiteMale7/3/19881988174231ERICJONESWhiteMale2/5/19861986174231ERICJONESWhiteMale2/5/19861986
Old Method had a hard time seeing people with different birthdates within a few years of one another as different people.Slide9
New Method
New Results
MATCH_REF
FORENAMES
SURNAME
RACE
GENDER
BIRTH_DATE
BIRTH_YEAR
0008050
ERIC
JONES
White
Male
1/15/1985
1985
0008050
Eric
Jones
White
Male
1/15/1985
1985
0008050
Eric
Jones
White
Male1/15/1985 19850039351EricJonesWhiteWhite2/5/1986 19860039351ERICJONESWhiteMale2/5/1986 19860039351
ERICJONES
White
Male
2/5/1986 19860039351ERICJONESWhiteMale2/5/1986 19860039351ERICJONESWhiteMale2/5/1986 19860039351ERICJONESWhiteMale2/5/1986 19860039351ERICJONESWhiteMale2/5/1986 19860141208EricJonesWhiteMale10/15/1982 19820141208ERICJONESWhiteMale10/15/1982 19820254712ERICJONESWhiteMale10/1/1980 19800254712ERICJONESWhiteMale10/1/1980 19800310884EricJonesWhiteMale7/3/1988 19880310884ERICJONESWhiteMale7/3/1988 1988
0310884
Eric
JonesWhiteMale7/3/1988 19880310884EricJonesUnknownMale7/3/1988 19880351920EricJonesWhiteMale19820427720EricJonesWhiteMale6/10/1989 19890465350ERICJONESWhiteMale8/20/1981 1981
New Method much better since it looks at exact dates of birthSlide10
New Method
New Results
SURNAME
FORENAMES
RACE
GENDER
BIRTH_DATE
BIRTH_YEAR
STEPHENS CLARENCE
Black
Male
12/3/1969
1969
Stephens
Clarence
Black
Male
1969
Stephens
Clarence
Black
Male
1969
Stephens
Clarence
Black
Male
12/3/1969 1969STEPHENSCLARENCEBlackMale12/3/1969 1969STEPHENSCLARENCEBlackMale12/3/1969 1969STEPHENSCLARENCE
BlackMale
12/3/1969
1969
STEPHENSCLARENCEBlackMale12/3/1969 1969STEPHENSCLARENCEBlackMale12/3/1969 1969STEPHENSCLARENCEBlackMale12/3/1969 1969StephensClarenceBlackMale12/31/1969 1969STEPHENSCLARENCE LBlackMale12/3/1969 1969Even gets it right when both names are accidently in one field or there is a typo in the birthdateSlide11
Many other ways to matchNameDate of Birth/AgeRace, GenderAddressPhone numbersAnything else you can think ofSlide12
What did it boil down to?The total 871,681 records boiled down to 400,736The 177,545 Marion County DV people boiled down to 92,908.Slide13
Example Output
Data Year
Julian
Outreach
Marion County Prosecutor’s Office
Protective Orders
IMPD Domestic Violence Officer Information Sheets
Total Unduplicated Count
2009
4,499
5,471
3,888
12,944
2010
5,457
6,532
3,802
13,966
2011
5,081
5,899
3,178
12,155
2012
5,3545,9583,058 12,04920136,5605,8902,6532,54311,559 Unique VictimsSlide14
Example OutputSlide15
Example Output