IMPROVING ADULT ADHD ASSESMENT 2017 ACHA Conference Austin, Texas - PowerPoint Presentation

IMPROVING ADULT ADHD ASSESMENT 2017 ACHA Conference Austin, Texas Paul Marshall, Ph.D., ABCN/ABPP Department of Psychiatry, Hennepin County Medical Center James Hoelzle, Ph.D. Department of Psychology, Marquette University

Increase in Attention Deficit Hyperactivity Disorder (ADHD) Prescription Use IMS Health The number of ADHD medication prescriptions for patients ages 20-39 increased 288% from 5.6 million to almost 16 million between 2007-2012. Sales of stimulant medication increased over 400% from $1.7 billion to almost $9 billion between 2002-2012. The New York Times (12/16/13)

Reasons to Seek ADHD DiagnosisMedications improve academic performance (Connor, 2006; DeSantis, Webb,& Nora, 2008) ADHD illness identity provides rationale for failure (Suhr & Wei, 2015 ) Academic accommodations (Harrison, Edwards, & Parker, 2007) ADHD medications are a relatively inexpensive, prescription based alternative to cocaine and methamphetamine ( McCabe et al. 2005, Teter et . al, 2003). 7.5% of a large sample of university students reported using stimulant medications without a prescription within the last 30 days (Weyandt, et al. , 2009).

Learning about ADHD Symptoms Symptoms are widely publicized on the internet (Conti, 2004) Entering “ADHD symptoms” in an internet browser pulled up 3.7 million websites, most containing full DSM diagnostic criteria (Pazol & Griggins, 2012) Internet sites provide tips on how to convince medical providers you have ADHD (Molina & Sibley, 2014) 28% of 698 college students reported knowing someone who pretended to have ADHD in a survey (Santos, Degail , Storch , Hummer, & Osborn, 2014).

  Primary Care Physicians Concerns about Diagnosing Adult ADHD 34% of 400 PCPs surveyed felt they were “very or extremely knowledgeable” about adult ADHD 13% felt they had received “very or extremely thorough” clinical training in making this diagnosis 44% thought the diagnostic criteria were not clear 72% indicated it was easier to diagnose ADHD in children than adults 75% rated the quality and accuracy of current ADHD diagnostic measures as either “poor” or “fair ” ( Adler, Shaw, Stitt, Maya, & Morrill, 2009 )

Challenges in the Assessment and Diagnosis of Adult ADHD Non-specificity of adult ADHD symptomsReliability and accuracy of patient’s and informant’s reports of ADHD symptomsIdentifying symptoms most appropriate and discriminative between those with and without adult ADHD Identifying appropriate symptom thresholds for frequency and severity of ADHD symptoms

Challenges in the Assessment and Diagnosis of Adult ADHDDetermination of functional impairmentIntegration of multiple sources of assessment information Patient misrepresentation of ADHD symptoms

Diagnostic Ambiguity of Adult ADHD Clinical Interviews102 of 428 young adults (24%) presenting for ADHD assessment diagnosed with ADHD diagnosis based on interview, behavior rating scales, cognitive tests, and validity tests45% had an indeterminate interview39% had an interview consistent with ADHD16% had an interview inconsistent with their having adult ADHD

Faking ADHD Successfully 115 of 428 young adults (27%) exaggerated or feigned symptoms associated with ADHDPotentially incorrect diagnosis rates depend upon which assessment methods are employed71% clinical interview alone 65% clinical interview and completion of behavioral rating scales 57% clinical interview, completion of behavioral rating scales, and impaired performance on a continuous performance test

Three groups: (1) ADHD, (2) simulate ADHD, and (3) control subjects A structured interview was conducted (MINI ADHD Module) Six measures of selective attention, sustained attention, working memory, and cognitive processing speed were administered 2 ADHD behavior rating scales completed 2 Performance Validity Tests (FIT; WMT) complete

44% Simulators diagnosed with ADHD 11% Simulators classified as “normal”

ADHD Academic Accommodation Assessment Practices87% of psychologists used patient interviews, 76 % used ADHD behavior rating scales 57% used cognitive/linguistic processing testing, 45% used continuous performance testing 19% used executive function behavior rating scales 3% used measures to identify invalid symptom presentation Nelson, Whipple, Lindstrom, & Foels (2014)

Discriminant Validity of ADHD Behavior Rating Scales in Patients Presenting for ADHD EvaluationBarkley Current Symptoms Scale -Self Report Form had a sensitivity of 85% and specificity of 40% (Soderstrom et al., 2013)Adult ADHD Self Report Scale (ASRS) had a sensitivity of 90% and specificity of 35% (Soderstrom et al., 2013) CAARS ADHD Index score (t score> 65) had a sensitivity of 64% and specificity of 86% in a postsecondary population (Harrison et al., 2016)

Discriminant Validity of ADHD Behavior Rating Scales in Patients Presenting for ADHD EvaluationBAARS-IV self-report current inattention symptoms ratings had a sensitivity of 89% and specificity of 30% BAARS-IV self-report childhood inattention symptoms ratings had a sensitivity of 65% and specificity of 40% Regression model including both BAARS-IV self-report and parent ratings of current and childhood symptoms had a sensitivity of 89% and specificity of 63% ( Dvorsky et al., 2016)

Discriminant Validity of alternative rating scales in Patients Presenting for ADHD EvaluationBarkley Deficits in Executive Function Scales had a positive predictive power of .94 and negative predictive power of .87 in discriminating adults diagnosed with ADHD vs. a normative control group (Barkley, 2011) Emotional lability scale had a sensitivity of .85 and a specificity of .81 in distinguishing adults diagnosed with ADHD from a control group ( Skirrow et al., 2013)

Discriminant Validity of Individual Cognitive TestsMost individual cognitive tests have poor sensitivity and are not effective in ruling ADHD in CPTs are not sufficiently sensitive to be effective in ruling ADHD in ( Riccio & Reynolds, 2001 ) CPT’s might have sufficient specificity to be useful in ruling ADHD out

Discriminant Validity of Cognitive Tests CombinedDifferentiating ADHD patients and control subjectsImpaired performance on any 2 of 6 executive function tests had a sensitivity of 69% and specificity of 96% (Lovejoy, et al. 1999)Discriminant function analysis based on a CPT, 5 executive function tests, and 1 working memory test had a sensitivity of 59% and specificity 81% (Rapport et al., 2001) Differentiating ADHD and psychiatric patients Discriminant function analysis based on 12 attention measures had a sensitivity of 97% and specificity of 40% (Katz et al., 1998)

Diagnostic Utility of Neuropsychological Tests, ADHD Behavior Rating Scales, and Interviews CombinedPettersson, Soderstrom, & Nilsson 2015 Evaluation included an ADHD interview (the DIVA), behavior rating scale (the ASRS screener), and a battery of eight cognitive tests A regression model included the DIVA, ASRS, reaction time variability, PASAT, & the Conners CPT commission errors and reaction time variability had a sensitivity of .90 and specificity of .81 Measures in regression model also used to make initial ADHD diagnosis

An alternative (larger) concernThe failure to consider engagement and response distortion may impact clinical decision makingFailing to consider these issues also ultimately impacts our understanding of ADHD and the utility of measuresHow might the failure to consider task engagement alter research findings?

“Results suggest that the co-occurance of ADHD and depressive symptoms in adults is associated with additional neurocognitive impairment.” “While adolescents with only ADHD have lower reaction time variability, those with comorbid depression have poorer working memory performance.”

An illustration: Erroneous conclusions regarding comorbid conditionsPrimary objective:Examine differences in neuropsychological functioning between individuals diagnosed with ADHD and/or mood disorders before and after participants with suspect effort are removed from analysesParticipants: 330 adults were evaluated and assigned to one of four groups ADHD n = 28 Mood disorder n = 178 ADHD/Mood disorder n = 81 No diagnosis n = 43

Participants & MethodsADHD diagnosis required:An interview that supported a diagnosis or was indeterminate Completion of current and childhood self-report measures in a manner that was consistent with diagnostic criteriaGeneral mood disorder diagnosis required one of the following: Primary or secondary ICD diagnostic code associated with depression, anxiety, or an adjustment disorder Current use of an antidepressant A BDI or BAI score suggesting at least moderate symptoms

Participants & MethodsParticipants:Mean age 26.81 (8.02)Mean estimated FSIQ 117.42 (13.77)Methods:Patients completed performance validity, working memory, attention, verbal memory, and executive functioning measures Group differences were explored before and after excluding individuals putting forth suspect effort

Select neurocognitive performances: All included ADHD ( n = 28) Mood ( n = 178) ADHD/Mood ( n = 81) No Dx ( n = 43) WAIS-IV Estimated FSIQ 117.14 (12.41) 120.04 (13.13) 1 112.35 (14.89) 1 116.26 (12.55) WAIS-IV Letter Number Seq 55.79 (9.62) 54.51 (10.26) 1 50.20 (10.63) 1 51.26 (7.99) WAIS-IV Digit Span Backward 55.02 (7.61) 1, 3 47.94 (8.91) 3 46.38 (11.21) 1, 2 51.55 (8.32) 2 CVLT-II Learning Trials 1-5 48.46 (8.38) 50.36 (10.34) 46.93 (12.70) 50.95 (9.30) Sentence Span 44.48 (11.89) 142.90 (13.07)37.99 (13.83)1, 246.21 (12.68)2WMS-IV Spatial Addition53.18 (9.21)54.48 (10.22)149.16 (12.24)152.73 (8.30)DKEFS Design Fluency Filled/ Empty58.07 (7.14)56.07 (8.49)53.97 (10.23)57.99 (8.26)DKEFS C/W Interference: Inhib. vs Color Naming52.11 (7.35)52.11 (7.71)50.53 (9.28)54.27 (5.41)DKEFS Tower Test Achievement54.96 (7.41)53.26 (8.64)52.22 (8.9652.69 (7.54)PASAT-100 Total Correct47.08 (9.95)45.94 (9.87)44.72 (10.27)46.78 (10.19)TOVA Comission Errors50.39 (9.09)147.37 (14.36)238.23 (16.51)1, 2, 350.48 (8.90)3TOVA Omission Errors34.07 (19.40)38.58 (16.99)127.65 (18.75)1, 238.94 (17.27)2TOVA Reaction Time Variability34.11 (16.43)36.36 (17.44)127.51 (17.18)135.50 (16.46) Note. Superscript denotes statistically significant mean differences between groups.

Suspected performance invalidity excluded ADHD (-7) Mood (-34) ADHD/Mood (-39) No Dx (-9) WAIS-IV Estimated FSIQ 120.33 (12.60) 122.08 (12.07) 119.83 (11.57) 118.65 (11.52) WAIS-IV Letter Number Seq 57.52 (9.94) 56.10 (9.64) 56.34 (9.95) 52.29 (8.12) WAIS-IV Digit Span Backward 55.21 (8.65) a 49.33 (8.60) a 52.19 (11.20) 50.76 (9.17) CVLT-II Learning Trials 1-5 49.38 (8.81) 52.17 (9.45) 53.38 (11.85) 52.82 (7.96) Sentence Span 46.95 (10.32) 45.51 (12.33) 45.93 (12.95) 47.79 (12.81) WMS-IV Spatial Addition 55.10 (7.33) 56.88 (8.74) 54.56 (9.42) 47.79 (12.81) DKEFS Design Fluency Filled/ Empty 59.05 (7.28) 57.19 (7.73) 58.31 (8.59) 59.25 (8.09)DKEFS C/W Interference: Inhib. vs Color Naming53.81 (6.06)52.40 (7.43)52.24 (7.91)54.63 (5.61)DKEFS Tower Test Achievement56.10 (7.99)54.16 (8.56)53.83 (9.36)53.96 (7.06)PASAT-100 Total Correct49.35 (9.98)47.08 (9.87)47.38 (10.81)48.50 (9.82)TOVA Comission Errors50.52 (9.58)49.14 (13.22)45.83 (12.81)49.46 (9.36)TOVA Omission Errors38.69 (18.99)43.17 (13.68)36.89 (17.85)41.60 (15.13)TOVA Reaction Time Variability39.33 (14.11)40.87 (15.25)37.61 (16.77)38.88 (14.55)Note. Superscript denotes statistically significant mean differences between groups.

Results & DiscussionWhen all patients are considered, individuals with comorbid ADHD and emotional symptoms performed more poorly on numerous measures compared to those with only ADHD or emotional difficultiesCohen’s d ranged from .40 to .91After excluding 91 participants with suspect effort, there was no evidence that the cumulative effect of ADHD and emotional symptoms resulted in diminished cognitive performance relative to either condition independently Key point: Understanding whether individuals are sufficiently engaged is tremendously important

How do we evaluate effort

HCMC Department of Psychiatry adult ADHD study participants268 consecutive adult ADHD assessment referrals July 2005 - February 201074% of sample 17-30 years old 16% high school graduates, 40% 1-3 years college, 23% college graduates

Suspect Effort Diagnostic Criteria (1) Failure on 2 separate SVT measures (e.g., the B test and the Dot Counting test) or failure on 1 SVT measure and (2) an unusually impaired performance on 1 cognitive test (e.g., the TOVA reaction time variability measure). (1) Failure on 1 SVT measure or an unusually impaired performance on 1 cognitive test and (2) invalid completion of behavior rating scales indicated by either CAT-A infrequency scale score or significant discrepancy between patient rating of distractibility, restlessness, and excessive talking on Barkley scales and the psychometrist’s behavioral observations. Of the 268 patients, 59 (22%) made a suspect effort based on their meeting Slick et al. (1999) criteria.

Sensitivity, specificity, positive and negative predictive accuracy rates for PVT and SVT measures 15% Base Rate 30% Base Rate Cutoff Sens. (%) Spec. (%) PPA (%) NPA (%) PPA (%) NPA (%) WMT Consistency 82.5% 63.64 95.24 70.23 93.69 85.14 85.94 WMT Pass/Fail 82.5% 63.64 90.48 54.12 93.38 74.13 85.31 b Test E-score 70+ 46.55 93.43 55.79 90.84 75.40 80.32 b Test Commission Errors 2+ 34.48 90.41 38.82 88.66 60.64 76.30 b Test Total Time 550+ 34.48 92.93 46.25 88.93 67.6476.80b Test d Error2+32.7290.4137.5888.3959.3975.82Dot Counting E-score14+33.8995.1054.9789.0774.7777.05TOVA Omission Errors>2562.8691.5456.7393.3276.1085.19TOVA RT Variability> 18054.2991.5453.11 91.90 73.34 82.37 Conner's Omission T Score < 20 56.52 87.00 43.41 91.90 65.08 82.36 CAT-A Self 3+ 58.33 89.39 49.24 92.40 70.20 83.35 CAT-A Self 4+ 36.11 96.97 67.77 89.58 83.63 77.98 BAASFR vs. Psychometrist Ratings 2/3 34.62 86.29 30.83 88.21 51.97 75.49

Probabilities of suspect effort derived from chaining the likelihood ratios for failures on 2 PVT measures Post-test Probability 15% Post-test Probability 30% WMT/b Test E-score   0.949   0.979 WMT/TOVA Reaction Time Variability 0.944 0.976 WMT/TOVA Omission Errors 0.952 0.979 WMT/CAT-A Infrequency Scale – Self 0.936 0.972 WMT/BAASFR vs. Psychometrist’s Ratings 0.870 0.942 b Test E-score/TOVA Reaction Time Variability 0.889 0.951 b Test E-score/TOVA Omission Errors 0.903 0.958 b Test E-score/CAT-A Infrequency Scale – Self 0.873 0.943 b Test E-score/BAASFR vs. Psychometrist’s Ratings 0.759 0.885 TOVA Reaction Time Variability/CAT-A Infrequency Scale - Self 0.862 0.938 TOVA Reaction Time Variability/BAASFR vs. Psychometrist’s Ratings 0.741 0.874 TOVA Omission Errors/CAT-A Infrequency Scale – Self 0.878 0.946 TOVA Omission Errors/BAASFR vs. Psychometrist’s Ratings 0.768 0.889 CAT-A Infrequency Scale – Self/BAASFR vs. Psychometrist’s 0.710 0.856

ConclusionsAn increasing number of adults are seeking ADHD evaluations for various reasonsDiagnosing adults with ADHD presents a number of challenges because symptoms are non-specific and it is difficult to establish an onset of difficultiesSecondary gain issues need to be recognized and also present unique challenges

ConclusionsAdults can easily learn about ADHD symptoms and present themselves during a clinical interview and complete behavior rating scales in a manner that increases the likelihood that they will receive a diagnosisIt is not uncommon for individuals to engage in response distortion or put forth suboptimal effort during an adult ADHD evaluation It is clear that a large number of college students (potentially 20-30%) are presenting in an invalid manner It is essential to recognize that patients may have ADHD and want to prove they have it

ConclusionsEncouragingly, numerous well-validated measures exist that help clinicians more objectively determine whether an individual is presenting in a credible manner This research supports the clinical utility of the WMT and B test The embedded CAARS-II Infrequency Scale and the CAT-A Infrequency Scale appear to be effective at detecting symptom exaggeration It is imperative that clinicians understand how these measures work so that they can be fairly and appropriately administered and interpreted

ConclusionsFailing to consider engagement and effort may lead to erroneous conclusions and inappropriate treatmentA majority of students putting forth suspect effort would likely be diagnosed with ADHD based on a clinical interview, a clinical interview and behavior ratings scales, and cognitive testing if invalid symptom presentation is not considered Most ADHD evaluations are not sufficiently thorough, there is a dire need for a brief but comprehensive ADHD assessment battery Failing to consider engagement and effort in research has the potential to confound our understanding of ADHD, which will ultimately impact clinical practice in a negative way

A Proposed Adult ADHD Diagnostic Battery       Includes 3 widely recognized components of ADHD assessment: a clinical interview, ADHD behavior rating scales, and some cognitive testing assesses the 3 widely recognized major cognitive phenotypes of ADHD: i.e ., sustained attention, response inhibition, and working memory ( Pinto et al., 2015)   cognitive tests are commercially available, have appropriate norms, and have demonstrated significant discriminative validity includes both performance validity tests and ADHD behavior rating scale SVT measures  

  A Proposed Adult ADHD Diagnostic Battery The Structured Clinical Interview for DSM-IV (SCID) Module for Attention-Deficit/Hyperactivity Disorder ( Gorlin , Dalrymple, Chelminski , & Zimmerman, 2016) A semi-structured ADHD clinical interview was developed and validated as part of the Rhode Island Methods to Improve Diagnostic Assessment and Services Project ( Brown University) takes approximately 20-25 minutes to administer

A Proposed Adult ADHD Diagnostic Battery The CAARS ( Conners , Erhardt , & Sparrow, 1999) completed not only by the patient but an informant, ideally a parent this scale has demonstrated better sensitivity and specificity than the BAARS-IV, ASRS, and BAADS in research to date the unmodified CAARS has a scale to identify invalid symptom presentation (Cook et al., 2016)

A Proposed Adult ADHD Diagnostic Battery The Test of Variables of Attention (TOVA) ( Greenberg, Kindschi , Dupuy , & Corman , 1996 ) CPT assesses sustained attention (via reaction time variability and omission errors) and measures response inhibition (via commission errors ) 22 minutes long unlike other CPT tasks, it contains indices to detect inadequate effort with good sensitivity (63%) and specificity (92%) (Marshall et al., 2010 )

A Proposed Adult ADHD Diagnostic Battery Salthouse Listening Span Test ( Salthouse , 1994) a challenging verbal working memory test that places greater demands on the central executive (i.e., storage and manipulation of information) scores: number of last words recalled correctly has much greater discriminant validity than other commonly used measures of verbal working memory (Nikolas, Marshall, & Hoelzle , under review)

A Proposed Adult ADHD Diagnostic Battery Salthouse Listening Span Test ( Salthouse , 1994) a challenging verbal working memory test that places greater demands on the central executive (i.e., storage and manipulation of information) scores: number of last words recalled correctly has much greater discriminant validity than other commonly used measures of verbal working memory (Nikolas, Marshall, & Hoelzle , under review)

A Proposed Adult ADHD Diagnostic Battery b test (Boone, Lu, & Herzberg, 2002), a PVT designed to detect inadequate effort on tests of visual sustained attention excellent face validity best sensitivity (47%) and specificity (93%) of any attention PVT (Marshall et al., 2010 )

A Proposed Adult ADHD Diagnostic Batteryentire assessment battery should take approximately 1 hour and 45 minutes for the patient to complete administration and subsequent scoring of the various assessment measures done by a psychometrician or assistant should take no more than 2 hours administration of the clinical interview as well as review and interpretation of the assessment results will take the clinician less than 1 hour and 30 minutes

A Proposed Adult ADHD Diagnostic Batteryentire ADHD evaluation should take approximately 5 hours and 15 minutes to complete 6-8 hours required to do an ADHD assessment consisting of a review of medical records, a diagnostic interview, neuropsychological testing, and a patient feedback session ( Pazol & Griggins , 2012 )

A Proposed Adult ADHD Diagnostic Batteryinitial cost of purchasing assessment manuals, tests, and software would be approximately $ 1,510 subsequent cost of the assessment measures would be approximately $30 per administration