Supportive Services for Veterans and Families - PowerPoint Presentation

Slide1

Supportive Services for Veterans and Families

Developing an

Excellent

Data Quality (DQ) Plan Slide2

What do we mean by Data Quality?

Does

your data reflect reality?

How accurately are you communicating the picture of homelessness in your community to others? How accurately are you able to measure your progress towards achieving your goal of ending veteran homelessness?

2Slide3

Elements of a Data Quality PlanTimeliness

Completeness-

All Clients Served

Data Quality BenchmarksAccuracyConsistencyMonitoringIncentives & Enforcement

3Slide4

Importance of Data Quality

Identifies the risk

factors for Veteran

homelessnessIdentifies the needs of homeless VeteransIdentifies what works in ending homelessness and promoting housing stabilityKnow if you’re reaching target populationInform your outreach approachKnow where changes are needed Measure your progress against goals and benchmarks set

4Slide5

Data Quality & SSVF programSSVF Legislation requires use of HMIS for

client-level data collection

HMIS reports provide outcome data to VA

Paints the picture of veteran homelessness, informs national resource allocation. 5Slide6

SSVF Data RequirementsYou must ask for every required data element for every clientHousehold members are clients

You must enter the data into HMIS

accurately and completely

You must make every effort to enter and update client records in HMIS within 24 hours of data collectionYou must make sure that your data is exported from HMIS and uploaded to the VA’s Repository every monthReference: Data Quality Policy & Thresholds Document

6Slide7

Data Quality (DQ) Standards

7Slide8

1) TimelinessIs

the client information, including intake data, program entry dates, services provided, and program exit dates entered into the HMIS within a reasonable period of time?

8Slide9

Timeliness ExampleStandard:

Client information is entered within 24 hours of intake. Monitoring plan: Hard copy intake forms are date stamped when the client information is first collected. The dates on the forms

are then compared to

actual

HMIS

data entry dates by a program manager to

determine

if the time between initial collection

and

entry into HMIS is 24

hours

or less.

9Slide10

2a) Completeness: All Clients Served

Are all of the clients receiving services being entered into the HMIS?

Are

all of the appropriate data elements being collected and entered into the HMIS? 10Slide11

Completeness ExampleStandard- Maine

:

The program

shall enter data on 100% of the clients they serve in accordance with the data set outlined in the program type’s definition of record. No anonymous client records are allowed. All clients must be exited from system within 24 hours of exit from program. Don’t Know, Unknown, or Refused cannot be recorded in the HMIS because the question was not asked, the intake worker did not record the answer on the intake/assessment sheets, or the data entry worker does not have access to the information

.

Policies on obtaining SSN, estimating DOB, Children & Veteran Status

11Slide12

Completeness- Sample Monitoring PlanMonitoring Plan:

All

programs must review, as part of their monthly data quality process, the list of current clients to

ensure that client exits are recorded properly in the HMIS.Run DQ Report weekly or bi-weekly to identify missing/refused/don’t know responses in advance of the upload. Find missing data or re-train case managers where needed.Compare SSVF Dashboard Report #’s to HMIS #’s

12Slide13

2b) Completeness: Establishing Data Quality Benchmarks/Thresholds

To

ensure completeness of the data in these data element sets, all programs are required to meet the Data Quality Benchmark Rates as outlined by

SSVF. Unlike other HMIS applications, SSVF sets data quality thresholds for you. Standard: Data Quality Policy & Thresholds documentMonitoring Plan: Monthly uploads – Repository automatically classifies errors - Notify or Reject.

13Slide14

DQ Threshold: NOTIFYAt

the

Notify

level, users are alerted to potential data quality issues, but the Repository will accept the upload. For example, if any client record is missing a last name, the user will be notified. Grantees are strongly encouraged to correct issues at the Notify level since missing data at these levels is a problem.14Slide15

DQ Threshold: REJECT

In order for the Repository to process the data set, the export must be compliant with HUD’s XML or CSV standards

If not compliant

(too many client records are missing essential data elements), the data set will be rejected by the Repository. You’ll receive a message either from us (if XML upload) or automatically generated from the Repository (if CSV upload)If a data set is rejected because of missing data, SSVF program staff will have to update client records in HMIS and then upload corrected data into Repository again

15Slide16

3) Accuracy

Does

the HMIS data accurately and consistently match information recorded on paper intake forms and in client files?

Determined by:Truthfulness by the clientAccuracy of data collected by staffAccuracy of data entered into the system by staff

16Slide17

Accuracy- ExampleStandard:

100%

of data entered into an HMIS must reflect what clients are reporting. Monitoring Plan: SSVF program will, on a biweekly/monthly basis, run DQ report from HMIS to identify any required data elements with missing or unknown/refused responses. Data for active clients should be reviewed monthly.

17Slide18

4) Consistency

To ensure a common interpretation of questions, answers, and process for data

entry, including which HMIS fields require completion.

18Slide19

Consistency- Example

Standard- Vermont:

Forms and Documentation:

Universal Data Elements (UDE’s) are collected on initial intake. Use CoC’s uniform intake, assessment and service documentation templates. Establish definitions of data points, interpretation of questions, answers and required fields. Staff Training:

New intake staff must complete training on both data collection and HMIS software prior to conducting assessments and using HMIS. Follow-up training in 3 months after gaining access to system.

Regular training of data entry staff- must attend annual HMIS training, test and document proficiency on data elements and collection.

Program Flow

: Standardized data entry screens and processes. Where appropriate, use software validations to force data entry and/or provide prompts to

assist in data entry of valid data. Data flow will be reviewed at least annually.

Feedback loop

:

Regularly check paper to computer data. Run DQ report to identify bed utilization, missing/null data, percentage of unknown/don’t know/refused data. Compare paper records to identify issues.

19Slide20

5) MonitoringTo ensure that the standards on the extent

and quality of data entered are

met

To identify DQ issues that affect timeliness, completeness and accuracy of the data. Resolve expediently.Specify frequency of monitoring activities.20Slide21

Monitoring Plan- Example

Example: Maine

Establish internal data quality assurance procedure.

Designate HMIS DQ Lead. This person should contact case managers whose data elements do not meet the benchmarks and assist in identifying solutions for correcting the data. Assistance provided by the HMIS Lead include:

Assist

in generating data quality reports showing instances of client level null/missing and/or unknown/don’t know/refused;

Assist

in identifying potential changes to workflow to better accommodate data collection and data entry for the program;

In depth analysis of technical issues that may be causing reporting errors on data quality;

Providing training on data entry;

Providing training on data element definition and meaning;

Clarification on exceptions to data quality standards.

HMIS DQ Lead should not correct data, rather will help identify problems and teach case managers how to correct.

HMIS DQ Lead should document individual staff DQ issues on monthly

basis.

Program Manager should be final reviewer for data submitted for reports.

21Slide22

Finding & Correcting HMIS Errors

Most HMIS applications have data quality reports that can help identify records with missing

data.

Exported CSV files open in Excel and every record has a Personal Identification Number that uniquely identifies a client.If your HMIS exports XML, you may be able to use the XML->CSV Parsing Tool.Resolving issues may take some time – upload to the Repository as early as possible each month to allow extra time to make corrections and resubmit data. Ask for help if you need it!

Reference

:

SSVF Data Quality Policy & Thresholds

22Slide23

Common Errors

Name

First and Last name not same; Suffix properly formatted; No numerals in name fields; Suffixes not in last name field, First name is not “Husband,” “Wife,” “Man,” “Woman,” “Boy,” “Girl,” “Child”, “Baby,” , etc.

Social Security NumberSSN has all numbers and no dashes; 9 digits when quality code indicates complete; Less than 9 digits when code indicates partial; All digits not same (333333333); all numbers not sequential (123456789)Date of BirthEarlier than current date; Earlier than program entry date; Later than 90 years from present; Not minor in adult

shelter

Ethnicity/Race

Primary and secondary race not the same

Gender

Men not pregnant; No Male referred from woman’s shelter/Woman referred from men’s shelter

Veteran Status

Client under 18 not veteran; All veterans in veteran shelter; Those receiving veteran’s pension marked as veteran

23Slide24

Common ErrorsHousing Status at Program Entry

Response will determine whether client will be categorized as RRH or HP in the APR

Income

Incomes should be reasonable, no extra digits, misplaced decimalsDisabling ConditionThose receiving SSDI for themselves are marked as having a disability; Those indicating substance abuse, mental health, physical disability, developmental disability, HIV/AIDS marked as having disabilityResidence prior to program entry

Self-report not contradicted by other HMIS data

Zip Code of Last Permanent Address/Quality Code

Zip code complete if quality code marked a complete; Zip code five or nine characters; Zip code valid (If list of zips available); Zip code has only numbers

Program Entry Date/ Program Exit Date

All clients have a program entry date.

;

Program Entry Date later than Birth Date; Program Entry Date prior to Exit Date

;

Length of program enrollment outliers are reasonable considering program type

Household ID

Single person not in family shelter; Family not in individual shelter

24Slide25

Useful Data Quality Reports

Null/Missing and Unknown/Don’t Know/Refused Reports on Universal Data Elements

Universal

Data Elements by Program Type – Benchmark for % Null/Missing and Unknown/Don’t Know/Refused Program Data Elements by Program Type – Benchmark for % Null/Missing and Unknown/Don’t Know/Refused Universal Data Elements by Client ID Report Length of Stay Report by Client ID

Intake

and Exit Data Entry Date Timeliness Report

Bed

Utilization Tool

25Slide26

6) Incentives & Enforcement

Positively reinforce adherence

to the data

standards and achievement of excellent data qualitySet protocol and consequences for making corrections to data in a timely manner.26Slide27

Incentives & Enforcement- Example

Standards

:

Annual recognition awards given to SSVF HMIS case managers who have substantially improved DQ and to those who consistently meet or exceed DQ standards. CoC publicly posts among its members the monthly DQ ranking for each program27Slide28

Questions to Ask as You Develop Your DQ Plan

Does your

CoC

have a set of HMIS data standards? If so, take a look at them…If they don’t meet the standards stated above, you can raise the bar and set a higher standard!Does your local HMIS have additional requirements not mentioned here? If so, do they conflict with SSVF requirements? Contact Regional Coordinator to request TA if needed.

28Slide29

Get Started Developing Your Excellent

Plan

:

Establish specific guidelines for all above standards. Put in writing. Borrow from other successful programs! Train SSVF staff on each data collection and entry standards, roles and responsibilities. Test data collection workflow and tools (before you begin to serve clients!)Designate

staff roles & responsibilities for creating, implementing and monitoring plan.

Meet

weekly w/ SSVF staff

to

address questions, issues and to ensure consistency.

29