Outcome Measures Session 4: Collaborative Learning - PowerPoint Presentation

Slide1

Outcome Measures

Session 4: Collaborative Learning

Project

Slide2

Today We Will

…

Describe outcome measures relevant to healthy weight program (HWP) evaluations

Identify valid, reliable, and generalizable tools and/or methods to collect outcome measuresPrioritize outcome measures to identify core measures to include across HWPs

Slide3

Anthropometric and Behavioral Outcomes

Bill

Stratbucker

, MD, MS

Associate Professor

Michigan State University

Slide4

General Considerations

Identify outcomes of interestBalance feasibility and validityUnderstand / educate stakeholdersSet expectationsConsider non-anthropometric outcomesUse intermediate outcomesEncourage accountability of measurements

Slide5

Anthropometrics - Examples

HeightWeightWaist circumferenceBody composition (body fat %, lean mass)

Slide6

Anthropometrics - Reliability

Training of staffConsider the settingPortability of equipmentErroneous measurement (measuring twice to reduce error)

Avoid inconsistent methods of measurementConsider when to measure

Slide7

Anthropometrics – Logistics

Cost of equipmentTimeFrequencyOveremphasis of weight (using kilograms)Privacy and acceptability of measurements (e.g., waist circumference)Caregiver(s), sibling(s) measured?

Slide8

POWER Site Guidance

The validity of human measurements is crucial to the comparisons we will make between patients and between sites. It is essential to obtain consistent and accurate measurements and documentation. Rather than prescribe human measurement procedures, POWER offers the following guidance: (National Health and Nutrition Examination Survey (NHANES) Anthropometry Procedures Manual):

Height: Depending on the overall body conformation of the individual, all of the four contact points— head, shoulders, buttocks, and heels—may not touch the stadiometer backboard. In such instances, the patient should be as vertical as possible.

Weight: Patients are asked to remove shoes or any extra layers of clothing (sweaters, 2

nd shirt, belt). Pockets should be emptied (e.g., cell phone, wallet, keys) and any heavy jewelry should be removed.

Slide9

Measuring Anthropometric Change Over Time

HeightWeightBMIBMIz% of the 95th percentileWeight gain velocity

Waist circumference

Slide10

No trajectory change BMI%ile increases

BMIz increases%95th%ile increasesTrajectory alteredBMI%ile maintained

BMIz decreased%95

th%ile maintainedBMI maintainedBMI%ile decreasedBMIz decreased

%95th%ile decreasedBMI reduced

Potential Outcomes

Slide11

Baseline Anthropometry

Variable

Mean

Standard

Deviation

BMI (kg/m

2

)

34.3

6.8

BMI percentile

98.7

1.3

BMI

z-score

2.37

0.36

Waist circumference (cm)

108.0

18.9

Percent body fat (%)

37.2

9.5

Slide12

Z-Scores

Mean BMI z-score = 2.37

Slide13

Variable

Baseline

Follow Up

BMI (kg/m

2)34.733.8

BMI z-score

2.39

2.27

BMI percentile

98.7

98.2

Percent

body fat (%)

36.9

33.3

71% of participants reduced their BMI percentile (i.e., reduced BMI trajectory)

Program Results: BMI Centiles

Slide14

Variable

Baseline

Follow Up

Change

BMI (kg/m2)

35.8

34.2

-1.6

BMI z-score

2.42

2.26

-0.16

BMI percentile

98.8

98.0

-0.8

51% of participants reduced their BMI

Program Results: BMI Reduction

Slide15

Clinically Meaningful Change

% who reached a threshold of improvement in BMIz or %95th%ile0.2 reduction BMIz (USPSTF)5% reduction in %95th

%ile (POWER)

Slide16

Physiologic/Metabolic - Examples

Heart rateBlood pressureLipidsGlucoseHgbA1c

Slide17

Physiologic/Metabolic

Training of staffEquipment needsRelationship with ordering medical providerSharing of dataLab testing vs. point-of-careNon-fasting vs. fasting

Slide18

Physiologic/Metabolic

Measures of fitness (12-minute walk/run, FitnessGram Pacer, Heart rate recovery, etc.)Measures of strength (grip strength)Measures of motor skill developmentSpirometryAll are effort dependent

Slide19

Variable

%

Elevated

Baseline

Follow UpChange

Systolic

BP (>120 mmHg)

30%

128.4

126.5

-1.9

Diastolic BP

(>80 mmHg)

10%

84.3

76.9

-7.4

Blood pressure changes in patients with elevated risk

Program Results: Biomarkers

Slide20

Variable

Baseline

Follow Up

Change

Total cholesterol (mg/dL)160.2

156.7

-3.5

HDL (mg/

dL

)

41.0

44.5

3.5*

LDL (mg/

dL

)

95.8

87.4

-8.4*

Triglycerides (mg/

dL

)

111.5

114.0

2.5

Glucose (mg/

dL

)

90.9

87.6

-3.3

Mean changes in blood lipids and blood glucose* Statistically significant change (p<0.05)

Program Results: Biomarkers

Slide21

Blood lipid changes in patients with elevated risk

Program Results: Biomarkers

Slide22

Blood lipid changes in patients with elevated risk

Mean

HDL/LDL ratio

among those with elevated risk changed from

0.28 to

0.35

Program Results: Biomarkers

Slide23

22% of patients had elevated blood glucose (>100 mg/dL)

On average, blood glucose levels were reduced to normal (20 mg/

dL reduction).

Program Results:

Prediabetes

Slide24

Measuring Physiologic/Metabolic Change Over Time

Mean reduction in blood pressureMean reduction in Tchol, LDL, TGMean increase in HDLMean change of “at risk group”% in abnormal range that normalized

Slide25

Lifestyle Behavior Change

NutritionPhysical activitySleepSelf-report questionnaireAccelerometer/pedometerLogging

Slide26

Lifestyle Behaviors Assessment

FNPA, Family Nutrition and Physical Activity AssessmentFeeding practices surveyParenting style

Sources: Development and Preliminary Validation of a Family Nutrition and Physical Activity Assessment, Ihmels, et. al,

Int J Behav Nutr Phys Act

, 2009. The Feeding Practices and Structure Questionnaire: Construction and Initial Validation in a Sample of Australian First-time Mothers and Their 2-year olds, Jansen, E., et. al,

International Journal of

Behavioral

Nutrition and Physical Activity,

2014.

Slide27

Lifestyle Behaviors Assessment

3-day, 7-day food logFood frequency questionnaire24-hour recall (self report, guided)Specific questions from larger survey (e.g., Youth Risk Behavior Survey)Example: Days per week children are engaged in moderate-to-vigorous physical activity

Slide28

Slide29

Changes in Sedentary Behaviors

*

*

Program Results: Behaviors

Slide30

Variable

Baseline

Follow Up

Change

Physical activity (min/d)

18.0

38.3

20.3*

HR

sub-max (

bpm

)

131.32

126.43

-4.89

VO2

max (mL/kg/min)

26.34

27.69

1.35*

Changes in Physical Activity & Fitness

Program Results: Behaviors

Slide31

Associated Health ConditionsAsthma

Sleep apneaNon-alcoholic steatohepatitisPrediabetes

Slide32

Associated Health ConditionsDefinitions

DocumentationManaged vs. resolved

Slide33

Measures Summary

Easy-to-more difficult, feasible-to-less feasibleHeight, weight (multiple outcome variations)Lifestyle behaviors (questions or full surveys)Fitness (resting heart rate,12 min walk/run, heart rate recovery)

Slide34

Measures Summary

Easy-to-more difficult, feasible-to-less feasibleBlood pressureWaist circumference Body composition (skin fold, bioelectrical impedance analysis)Metabolic labs (point-of-care, non-fasting)

Slide35

Measures Summary

Easy-to-more difficult, feasible-to-less feasibleStrengthMotor skill developmentSpirometryObjective measures of physical activity/nutrition (accelerometer, dietary recall interviews, photo food diary)

Associated health conditions

Slide36

Psychosocial Outcomes

Elissa Jelalian, PhD

Professor

Brown University

Slide37

General Considerations

What domains of child or caregiver behavior/feelings/attitudes do you expect will be impacted by your intervention?Who is best suited to report on the outcome?

Child < 7 years – caregiver proxyChild >

7 years – child report and/or caregiver proxy, depending on domain of interest

Slide38

General domains: physical, social, emotional, and school functioning

Several studies document that children and teens with overweight/obesity report significant impairments in daily functioning relative to healthy weight peers

Among youth with overweight/obesity,

HRQoL decreases as BMI increasesImprovements in

HRQoL are associated with participation in weight control treatment

Health-Related Quality of Life (

HRQoL

)

Slide39

General Measure: PedsQL

TM23 itemsMultiple domains:

PhysicalEmotional

Social supportSchool functioningAvailable formats: Child self-report: ages 5–7; 8–12; 13–18Parent proxy: ages 2–4, 5–7, 8–12, 13–18

https://www.pedsql.org/about_pedsql.html

Slide40

“Sizing Me Up”

Self-report measuresChildren 5–13 yearsDomains: Emotional, physical, social avoidance, positive social attributes and teasing / marginalization

“Sizing Them Up”

Parent-report measure Youth 5–18 yearsDomains:

Emotional, physical, teasing / marginalization, positive attributes, mealtime challenges and school

Obesity-specific Measures of

HRQoL

The Impact of Weight on Quality of Life-Kids (IWQOL-Kids)

Youth 11

–

19 years

Caregiver proxy and adolescent self-report versions

Physical comfort, body esteem, social life, and family relations

Validated; shown to be responsive to weight change

Slide41

Self-Esteem

Broadly defined as thoughts and feelings about oneselfSingle vs. multi-dimensional construct (e.g., academic, physical, social)Fairly consistent literature suggesting decreased self-concept in children with overweight/obesity compared to healthy weightParticular vulnerabilities - physical appearance and social functioningStronger relationship among adolescents, girls, and youth who experience teasing and peer victimization

Slide42

Self-Esteem

Majority of studies document increases in self-esteem or components of self-esteem following participation in a pediatric weight control programMixed results as to whether these improvements are related to decreases in weight statusSelf-esteem related to physical appearance seems to show the most consistent improvement following intervention

Slide43

Assessing Self-Esteem

Rosenberg Self-Esteem Scale:Unidimensional measure10-item scale that measures global self-worth by assessing both positive and negative feelings about the self Available at no cost

https://socy.umd.edu/about-us/using-rosenberg-self-esteem-scale

Slide44

Self-perception Profile for Children/ Self-perception Profile for Adolescents Both measures include multiple domains of competence – e.g., physical appearance, social competence, as well as global self-worth

Can administer subscales of interest rather than the entire questionnaireNeed to monitor administration due to item formatDimensional Measures of Self-Esteem

https://portfolio.du.edu/SusanHarter/page/44210

Slide45

Intervention-related Improvements in Self-esteem

Slide46

Weight-related Teasing

Commonly endorsed among youth with overweight/obesityPerceptions of Teasing Scale (POTS): 11-items total

General weight teasing (6 items; e.g., people made jokes about you being heavy)General competency (5 items; e.g., “People laughed at you because you didn’t understand something”)

Frequency and associated distress

Slide47

Mood

Children’s Depression Inventory 2 (CDI):Children and adolescents aged 7–17 yearsSelf-report and parent report measures

28 items; short form = 12 itemsGold standard, but costly and requires some credentialing

PROMIS® (Patient-Reported Outcomes Measurement Information System) - person-centered measures that evaluate physical, mental, and social health in adults and childrenPROMIS Pediatric Item Bank v2.0:

14-item measureCan be accessed at no cost

Slide48

Weight and Shape Concerns/Body Image

Definition of weight and shape concerns – the extent to which weight influences overall feelings about oneselfMay be more appropriate to assess in older children and adolescentsMcKnight Risk Factor Survey: Lengthy measure, but several potentially relevant scales/items: weight teasing; body appearance

“In the past year, how happy have you been with the way your body looks?”; “In the past year, how much has your weight made a difference to how you feel about yourself?”

Body Dissatisfaction Scale: Asks teens to rate their satisfaction with nine body parts on a Likert scale, from ‘extremely satisfied’ to ‘extremely dissatisfied; appropriate for adolescentsFigural Drawings: Comparison between current and ideal

Slide49

“Distal” OutcomesSocial anxiety

Family functioningSocial support

Slide50

Cautions and Considerations

Screening versus observation of change resulting from interventionExtreme weight control behaviorsLoss of control/binge eatingImplications of assessment for follow-upAvoid measures that provide diagnostic informationAvoid measures that assess specific risk – e.g., suicidal ideation, unless there is a clear plan for responding in real time

Slide51

AccessibilityPhysical environmentMaterials and resources

Participant- interventionist communicationUtility of treatment – e.g., topics, effectivenessConvenienceCostDuration

Overall satisfactionOther domains – e.g., challenges, recommendations for improvement

Treatment Satisfaction

Slide52

Cost Analyses

Slide53

General Considerations

Cost-effectiveness analysis: Does an intervention provide value relative to an existing intervention

Value = cost/relative to health outcomeCost and health outcomesBudget impact analysis:

Assesses whether the valued intervention is affordablePayer perspectiveFocus is on cost

Slide54

Costs of Program Delivery: Intervention

Facilitator/coach time spentTrainingDelivering interventionParticipant contacts outside of intervention deliveryMaterial preparation

Multiply hours by hourly wageSpaceCost of materials

Slide55

Costs of Program Delivery: Participants

Travel timeTransportation timeOpportunity costs for lost wages – if program time is displacing work timeParticipant time can be estimated based on an average hourly wage rate of US adults from the Bureau of Labor Statistics

Slide56

Cost EffectivenessCalculate cost of intervention per unit change in weight status measure for child and caregiver

Slide57

RecommendationsBrief assessments that are available at no-cost and are likely to show improvement

HRQoLKey self-esteem domains – physical appearance; global self-worthSatisfaction with intervention to document program strengths and inform quality improvement

Slide58

Resources

http://www.healthmeasures.net/explore-measurement-systems/promis/intro-to-promis/list-of-pediatric-measureshttps://www.herc.research.va.gov/include/page.asp?id=budget-impact-analysis

Slide59

Discussion Questions

What outcome measures does your program evaluate and why?

What

outcome measures do you believe should be prioritized for HWP evaluation?

What tools and/or methods do you use to evaluate your program’s

outcom

e

measures outlined in Q1?

How does your program use outcome evaluation data to enhance outcomes?