/
Psychometric Properties of the Psychometric Properties of the

Psychometric Properties of the - PowerPoint Presentation

stefany-barnette
stefany-barnette . @stefany-barnette
Follow
405 views
Uploaded On 2015-11-15

Psychometric Properties of the - PPT Presentation

Consumer Assessment of Healthcare Providers and Systems CAHPS Clinician and Group Adult Visit Survey September 11 2012 Naomi Dyer PhD Joann Sorra PhD Scott Smith PhD Westat Paul Cleary PhD Yale ID: 194647

care cahps doctor items cahps care items doctor site practice survey reliability visit individual analysis access composites office helpful

Share:

Link:

Embed:

Download Presentation from below link

Download Presentation The PPT/PDF document "Psychometric Properties of the" is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.


Presentation Transcript

Slide1

Psychometric Properties of the

Consumer

Assessment of Healthcare Providers and Systems (CAHPS) Clinician and Group Adult Visit Survey

September 11, 2012

Naomi Dyer, PhD,

Joann Sorra

, PhD, Scott Smith, PhD, Westat

Paul Cleary, PhD, Yale

Ron Hays, PhD, RANDSlide2

CAHPS is a multiyear initiative of AHRQ to support and promote the assessment of consumers’ experiences with health care

Numerous CAHPS surveys have been created for different health care settings such as :

Health Plan

Surgical Care

Home Health CareHospitalIn-center HemodialysisNursing HomeClinician & Group

BackgroundSlide3

Westat, under contract with AHRQ, houses the database for Clinician and Group CAHPS (CG-CAHPS) and reports on these data

BackgroundSlide4

There are several CG-CAHPS Surveys:

CG-CAHPS 12-month Survey (Adult and Child)

CG-CAHPS

12

month Survey with Patient-Centered Medical HomeCG-CAHPS Visit Survey (Adult and Child)CG-CAHPS Database is the newest component of the CAHPS DatabasesThe CG-CAHPS Adult Visit Survey database received the most data in 2011

BackgroundSlide5

The CG-CAHPS

Visit

surveys were created to allow patients to report on and evaluate their experiences during their most recent outpatient medical visitThe 12 month versions ask about their experiences in

the

previous yearSurveys and related material available at:http://cahps.ahrq.gov/clinician_group/

BackgroundSlide6

Goal: Develop a survey

to assess patients' experiences with medical groups and clinicians.

Development of the Visit Survey began in 2008Field testing in 2009 to compare with the 12-month survey versionResults led to the Access to Care items in the survey reverting to a 12-month reference period rather than visit-specific

Survey DevelopmentSlide7

CG-CAHPS Composites

3 composites

1. Access to Care (5 items)2. Doctor Communication (6 items)3. Courteous/Helpful Staff (2 items) 2 overall rating items1. Overall Doctor Rating (11-point scale; 0 = “Worst doctor possible”,

10 =

“Best doctor possible”)2. Recommend the Doctor’s Office to family and friends (3-point scale; “No” to “Yes, definitely”)Various socio-demographics including overall health, age, gender, and educationSlide8

Access to Care Composite

All 5 items reference a 12-month period

4-point scale (1 = Never, 2 = Sometimes, 3 = Usually, 4 =Always

)Slide9

Access to Care Composite

In the last 12 months,

How often did you get an appointment for care you needed right away as soon as you needed?

How

often did you get an appointment for routine care as soon as you needed?When you phoned this doctor’s office during regular office hours, how often did you get an answer to your medical question that same day? When you phoned this doctor’s office after regular office hours, how often did you get an answer to your medical question as soon as you needed? How often did you see this doctor within 15 minutes of your appointment time? Slide10

Doctor Communication Composite

All six items reference

the most recent office visit3-point scale (1 = Yes, definitely; 2 = Yes, somewhat; 3 = No)

During your most recent visit, did this doctor…

Explain things in a way that was easy to understand?Listen carefully to you?Give you easy to understand information about your health questions or concerns? Seem to know the important information about your medical history?Show respect for what you had to say

?Slide11

Courteous/Helpful Staff Composite

Both items reference the office visit

3-point scale (1 = Yes, definitely; 2 = Yes, somewhat; 3 = No)

During your most recent visit

, were clerks and receptionists at this doctor’s office…As helpful as you thought they should be? Treat you with courtesy and respect? Slide12

CG-CAHPS Analysis Dataset

Adult Visit Survey Data from the CG-CAHPS Comparative Database103,442 responses

469 practice sitesThere were 5 screener questions that determined if the composite item was to be answered by respondents 4 screener questions for Access to Care1 screen question Doctor CommunicationOnly

non-missing data for the composite items were included in the analysisSlide13

CG-CAHPS Analysis Dataset

93% of respondents did not phone their doctor after regular hours Because of the high percentage of skips, “How

often did you get an appointment for care you needed right away as soon as you needed?” was dropped from analysis Remaining 4 Access to Care composite items had responses between 46% to 98% of respondents Final CG-CAHPS Analysis Dataset450 practice sites21,318 responsesSlide14

Characteristics of Analysis Dataset

All respondents received paper/mail surveys

89% of sites were Family Practice or Internal Medicine69% of sites were owned by a hospital or integrated delivery system67% of respondents were female;

81

% were 45 years or olderSlide15

Psychometric Analyses and Criterion

Individual and Multilevel Confirmatory Factor Analysis (CFA

)Factor loadings above 0.40Acceptable model fit indices CFI > 0.95 RMSEA < 0.06 SRMR < 0.08 (Individual, Between- and Within-)Individual Internal Consistency Reliability AnalysisCronbach’s α ≥ 0.70 Slide16

Psychometric Analyses and Criterion

Practice Site Reliability AnalysisReliability ≥ 0.70

Examined reliability by practice size categories1 clinician2 to 3 clinicians4-9 clinicians10-13 clinicians14-19 clinicians20 or more clinicians

Correlation analysis among the composites and global ratings

Individual and practice site levelsSlide17

Individual Level CFA Results

All items within composites had individual factor loadings above 0.40 with average loadings ofAccess to Care: 0.68

Doctor Communication: 0.76Courteous/Helpful Staff: 0.86All model fit indices met criteriaCFI: 0.97RMSEA: 0.05SRMR: 0.04Slide18

Multilevel CFA Results

All items at practice site level had factor loadings above 0.40

Between site factor loadings range: 0.59 to 0.99Within site factor loadings rage: 0.45 to 0.99All model fit indices met criteria except between-practice site SRMRCFI: 0.97RMSEA: 0.03Between SRMR: 0.10

Within SRMR: 0.05Slide19

Internal Consistency Reliability

All items at individual level had Cronbach’s alpha above 0.70Access to Care

α = 0.77Doctor Communication α = 0.89Courteous/Helpful Staff α = 0.85Slide20

Practice Site Reliability

Practice site reliability estimate were acceptable for all site with at least 4 cliniciansFor

a site with 1 clinicianOnly Access to Care had reliability above 0.70For a site with 2-3 cliniciansAccess to Care and Courteous/Helpful Staff had reliability above 0.70The average number of respondents for 1 clinician and 2-3 clinicians was less than 100

Smaller sites need more respondents per practice to increase reliability to acceptable levelsSlide21

Individual Correlations

Composites and

Global Ratings12341. Access to Care 2. Doctor Communication0.35

3. Courteous/Helpful

Staff0.290.254. Overall Doctor Rating0.390.520.225. Recommend Doctor’s Office0.340.520.29

0.47

All correlations were significant

(

p < 0.05)

Strongest correlation was between Doctor Communication and Overall Global Rating ItemsSlide22

Practice Site Level Correlations

All correlations were significant (p < 0.05)

Composites and Global Ratings1234

1. Access

to Care 2. Doctor Communication0.453. Courteous/Helpful Staff0.570.414. Overall Doctor Rating0.42

0.750.345. Recommend Doctor’s Office

0.52

0.76

0.43

0.76

Strongest correlation was between Doctor Communication and Recommend Doctor’s OfficeSlide23

Summary

Overall, both the individual level and multilevel confirmatory factor analysis results provided support for the survey’s three composites

The CG-CAHPS composites have acceptable individual-level internal consistency reliabilityThe reliability remains acceptable across sites with four to twenty or more clinicians Slide24

Email:

naomidyer@westat.com