TheOutcomeofPsychotherapy:Yesterday,Today,andTomorrowScottD.Miller,Mar - PDF document

TheOutcomeofPsychotherapy:Yesterday,Today,andTomorrowScottD.Miller,MarkA.Hubble,DarylL.Chow,andJasonA.SeidelInternationalCenterforClinicalExcellenceIn1963,thefirstissueofthejournalappeared.RespondingtofindingsreportedinapreviouspublicationbyEysenck(1952),Struppwroteofthe“staggeringresearchproblems”(p.2)confrontingthefieldandthenecessityofconducting“properlyplannedanexecutedexperimentalstudies”toresolvequestionsabouttheprocessandoutcomeofpsychotherapy.Today,boththeefficacyandeffectivenessofpsychotherapyhasbeenwellestablished.Despitetheconsistentfindingssubstan- ScottD.Miller,MarkA.Hubble,DarylL.Chow,andJasonA.Seidel,InternationalCenterforClinicalExcellence.CorrespondenceconcerningthisarticleshouldbeaddressedtoScottD.Miller,P.O.Box180147,Chicago,IL60618.E-mail:©2013AmericanPsychologicalAssociation2013,Vol.50,No.1,88–970033-3204/13/$12.00DOI: includingtheAPApresident,arecallingfortheabandonmentoftheDSMandtransitiontotheWorldHealthOrganization’sInter-nationalClassificationofDiseases(Bradshaw,2012Clay,2012TheprincipledisagreementbetweenStruppandEysenckre-cordedinthefirstvolumeofhasbeenresolved.Notonlyistheefficacyofpsychotherapywellestablished,butsoisitseffectivenessinrealworldclinicalsettings(AmericanPsycholog-icalAssociation,2012Duncan,Miller,Wampold,&Hubble,Wampold,2001).Despitetheconsistentfindingssubstanti-atingthefield’sworth,asignificantquestionremainsunanswered:Howdoespsychotherapywork?InStrupp’swords(1963,p.2),thefieldwould“notbesatisfiedwithstudiesoftherapeuticoutcomesuntil(it)succeed(ed)inbecomingmoreexplicitabouttheinde-pendentvariable”—inparticular,thecontributionsmadebytheclient,thetherapist,thetreatmentmethod,andcommercebetweentheparticipants.Here,debatecontinuestodividetheprofession.Gatheredononesidearethosewhohavelongarguedthatpsychotherapyisanalogoustomedicine.Fromthispointofview,psychologicallyinformedinterventionsworkinmuchthesamewaypenicillintreatsinfection.Thehallmarkoftheirpositionisthateffectivetreatmentsmustcontainspecificingredientsremedialtotheconditionbeingtreated.Forthisgroup,randomizedclinicaltrials(RCTs)aretheprincipalmeansofinvestigation,thefindingsofwhichareusedtogeneratetreatmentguidelines,manuals,andlistsof“empiricallysupported”or“validated”therapies(e.g.,Barlow,2004Chambless&Hollon,1998).Theycontendthatforpsychotherapytoadvanceasascience,psychologistsmustopera-tionalizefalsifiablehypothesesusingspecificmethods(discreteindependentvariables),testthosehypotheses,andteachstudentsthosemethodsthatstanduptorigorandreplication(Zuriff,1985).Thecriticalargumentsupportingthisap-proachisthatdifferenttherapiesaredifferentiallyeffective,andspecifictherapiesaremoreeffectivethannonspecifictreatment-as-usual(TAU).Exponentsfortheothersideinsistthatanysuggestionpsycho-therapyiscomparablewithamedicalinterventionisgrosslyinac-curate(Frank&Frank,1999Miller,Duncan,&Hubble,2004Insteadoffocusingonspecificmethods,theyinsistthatmecha-nismscommontoallapproaches,nomatterthetheoryortech-nique,areresponsibleforchange.Inadditiontotheinstillationofhope,provisionofatherapeuticrationale,andstrategiesforachievingchange,thetherapeuticrelationshipismostoftencitedasone,ifnotthemost,potenttranstheoreticalingredientofpsy-chotherapy(Bachelor&Horvath,1999Grencavage&Norcross,Norcross,2010).Threeconverginglinesofresearcharecitedinsupportofthesenonspecificfactorsasthemostsignificantindependentvariablesresponsibleforclientchange:(1)theab-senceofdifferentialeffectivenesswhenspecificapproachesaredirectlycomparedandwhenresearcherallegianceandotherbias-ingvariablesarecontrolled(Wampold,2001);(2)dismantlingstudiesthatshowthecontributionofspecifictechniquestotreat-mentoutcomeisnegligible(Duncanetal.,2010);and(3)researchshowingconsistentlygreatervarianceinoutcomesbetweenpsy-chotherapistsinagivenstudythanbetweenthetypesoftherapytheyarepracticing(Benish,Imel,&Wampold,2008Beutleretal.,2004Crits-Christoph&Mintz,1991Crits-Christophetal.,Imel,Wampold,Miller,&Fleming,2008Kim,Wampold,&Bolt,2006Luborskyetal.,1986Lutz,Leon,Martinovich,Lyons,&Stiles,2007Okiishi,Lambert,Eggett,Nielsen,Dayton,&Vermeersch,2006Shapiro,Firth-Cozens,&Stiles,1989Wampold&Bolt,2006Wampold,Mondin,Moody,&Ahn,Thefailuretoreachagreementabouthowpsychotherapyworksisnotwithoutconsequence.Tobegin,howwilltheoutcomeofpsychotherapyeverimproveifthetwomajorexplanatorypara-digmsareincontinuousdisputeandthecausalvariablesdefyconsensus?Onthatscore,meta-analyticevidenceshowsoutcomehaschangedlittleoverthepast40yearsdespiteoverwhelmingsupportofpsychotherapyandadramaticincreaseinthenumberofdiagnosesandtreatmentapproaches(cf.,APA,2012Smith&Glass,1977Wampold,Mondin,Moody,&Ahn,1997Mondin,Moody,etal.,1997Thepolarizationamongresearchersandinabilitytoanswerbasicquestionsabouttheinternalworkingsofpsychotherapyalsounderminethestandingoftheprofessionwithintheworldofhealthcare,especiallyamongconsumers.Nationwidesurveysofpotentialusersofpsychotherapyfindthataclearmajority(77%)doubtitsefficacy(APA,2004TherapyinAmerica,2004).More-over,although90%ofpeoplereporttheywouldprefertotalkabouttheirproblemsratherthantakemedication,useofpsycho-tropicdrugshascontinuedtorise,whereasvisitstopsychothera-pistshavesteadilydeclined(Duncan,Miller,Wampold,&Hubble,Somecontendthatthethreattothefield’ssurvivalissogravetheprofession’sinterestwouldbestbeservedbysettingthescientificissuesasideandactingasthoughthemedicalmodelapplies(Nathan,1997).“MovingaggressivelyinthedirectionofdevelopingandimplementingempiricallyvalidatedtreatmentWilson(1995)argues,“wouldseemimperativeinse-curingtheplaceofpsychologicaltherapyinfuturehealthcarepolicy”(p.163).Doingotherwise,itisclaimed,risksexclusion.Suchassertionsareentirelyunderstandable.Economicpressuresonpractitionersarepowerfulandreal.Withoutadoubt,debatedoesnotputfoodonthetable.Forallthat,anequallypassionatecallcomesfromtheotherside.“Themedicalizationofpsychotherapy,”Wampold(2001,p.protests,“mightwelldestroytalktherapyasabeneficialtreat-mentofpsychologicalandsocialproblems.”Onthefaceofit,thepremisehasmerit.Therapyisafluid,dynamicprocess,onein-volvingacomplexandnuancedseriesofinterchanges.Forcingclinicianstoadopt“truncatedandprescriptive”treatmentsmaywellstriptherapyoftheveryinterpersonalprocessescriticaltoitsToresolvethepredicamentinwhichtheprofessionremainsmired,threepossiblesolutionsareimmediatelyapparent.First,bothsidescancontinuetoconductmoreofthesametypeofresearchinthehopethatnewfindingswillemergevindicatingone,whileforcingtheothertocapitulate.Second,endtheproblembylegislativefiat.Ineffect,owingtothepressingfinancialandpoliticalconsiderations,declareawinner,ofnecessityplacingexpedienceabovescience.Third,findamiddleway.Inthissce-nario,thetwowarringcampsfinallymovetothecenter,integrat-ingtheirbeliefsandbestpractices.Onreview,eachoftheseapproachesisempiricallyplausible.Itisthecasethoughthat,ifhavingnotalreadyfailed,theyseemdestinedtodoso.Takingeachofthethreesolutionsinorder,thehopethatwiththerightresearchdesignorlineofinvestigation,aclearvictorwillcomeforthis—toputitbluntly—akintoanTHEOUTCOMEOFPSYCHOTHERAPY alchemist’soptimism.After50years,andamassiveexpenditureoftime,effort,andmoney,hadonesideortheotherbeenright,leadwouldhavebeentransformedintoempiricalgoldlongagoDuncanetal.,2010).Numerousreplications,meta-analyses,andcritiquessupportingbothsideshavebeenhailedashightruthononeside,andsomuchsoundandfuryontheother.Fewhavebeensufficientlyswayedtogiveuptheirclaimsorviewoftheevidence.Thesecondsolutionofdefiningpracticebystatuteiswellunderway.In2009,CooperandAratani(CooperandAratani,)foundthat90%ofstateswereimplementingstrategiestosupporttheuseof“evidence-basedpractices”(EBPs).Withfewexceptions,sucheffortshaveequatedEBPwithlistsofspecifictreatmentsforspecificdisorders(e.g.,Addiction&MentalHealthServices,2011).Inturn,reimbursementhasbeenmadecontingentonanadherencetoofficiallysanctionedtherapies.Atpresent,onelooksinvainforevidencethatthesepolicieshaveendeddivisionsamongresearchersandcliniciansregardingwhatconstitutesa“bestpractice,”improvedeitheroutcomeoraccesstocare(hanske&Franczak,2010),bolsteredconsumerconfidence,orsecuredfinancialstabilityforclinicians.Asforthelatter,inthesameperiod,psychologists’incomeshavebeenindecline(Monitor,2010Cummings&O’Donohue,2008Finally,whatofthehopeforfindingamiddleway?Ifthesuccessofanintegrativemovementcouldbemeasuredbythenumberofbooksandarticlespublished,professionalmeetingsheld,orrhetoricaleloquenceoftheadvocates,thenitwouldbereasonabletoconcludeanewageofcooperationandunityhasalreadyarrived.Ofcourse,thishasnothappened,atall.Farfromunifyingtheprofession,anentirenewmovementhascomeonthescene,burdenedbyitsowndisagreementsaboutwhatintegrationactuallymeansand,atstreetlevel,howtoputitintopracticeMilleretal.,2004Norcross,1997).Outsideofthelaboratoryandthehallsofacademia,theoriesandtechniquesareusedidiosyn-craticallyratherthansystematically,accumulatedratherthaninte-gratedonanylevelbutthatoftheindividualclinician.Likeitornot,thatistherealityontheground.TheWayOutAfter50years,andlittlesuccessindecidinghowpsychotherapyworks,wereturntoStrupp’s(1963)proposition.Oncemore,“Itseemstomethatweshallnotbesatisfiedwithstudiesofthera-peuticoutcomesuntilwesucceedinbecomingmoreexplicitabouttheindependentvariable”(p.2).Handsdown,forallconcerned,theindependentvariableofconsuminginteresthasbeenpsycho-therapy—thetreatmentphilosophy,theoreticalconstructionsre-gardingetiologyandcure,andassociatedproceduresandtech-niques.Ofslightlylesserinteresthavebeentherecipientsofcare;inparticular,theirdiagnosisorpathology,personalityformationandmalformations,lifesituation,socioeconomicstatus,environ-mentalsupportsandstressorsand,inmorerecentyears,genderandethnicity.AlthoughidentifiedbyStrupp(1963),farlessattentionhasbeenpaidtothecontributionofthetherapist(Beutleretal.,2004etal.,2006Wampold,2010).Doing,performing,anddeliveringhasconsistentlyovershadowedthedoer,performer,anddeliverer.Lookingpastthetherapist’scontributionhasbeenandcontinuestobeanegregiouserror.Availableevidencedocumentsthatthetherapistisoneofthemostrobustpredictorsofoutcomeamongfactorsstudied.Indeed,thevarianceofoutcomesattributabletotherapists(5%–9%)islargerthanthevariabilityamongtreatments(0%–1%),thealliance(5%),andthesuperiorityofanempiricallysupportedtreatmenttoaplacebotreatment(0%–4%)(Duncanetal.,2010Lutzetal.,2007Wampold,2005Beginningin1997,Garfieldandothernotableresearchers,includingStrupp(Strupp&Anderson,1997Luborsky,McClel-lan,Woody,O’Brien,&Auerbach,1985Luborsky,Mclellan,Diguer,Woody,&Seligman,1997Okiishi,Lambert,Nielsen,&Ogles,2003),broughtthetherapistbacktothetable,inanem-phaticcritiqueoftheprofession’sfocusontreatmentmodelsandtechniques.Notsurprisingly,forthosewhobelievethatpsycho-therapyisanalogoustomedicine,therapistdifferencesareconsid-ereda“nuisancevariable,”noisetobefilteredoutviastrictadherencetothetreatmentprotocol.Ontheotherside,thetherapistisnotonlyaninterventionist,butalsoanintrinsicpartoftheintervention;notjustthedeliverymechanism,butanimportantpartofwhatisdelivered.Effectiveness,itisbelieved,resultsfromacombinationoftherapists’“desirablepersonalrequisites”(field,1997,p.41)andtheirabilitytousewhatevermethodsempowerthecoreconditionssharedbyallhealingpractices(cf.,Duncan,2010).Simplyput,onecannotremovetheeffectofthetherapistwithoutunderminingthetherapy.Strupp(1963)foresawthevariabilitybetweentherapistsbeforethecollectionoftheevidencethatconfirmedit:“Letusstay,however,withthemethodoftreatmentandconsiderfurtheritsrelationtooutcomes.Forthispurposeletusdisregard(whatinrealitycannotbedisregarded)therapistvariablesandsocioenvi-ronmentalfactors”(p.2).AlthoughEysenck(1964)theneedforclarityandprecisioninmethodsandmeasurement,Strupp(1963)grappledwiththeimportanceofthecontextualnuancesunfortunatelyreflectedin“crude...quasidocumentationwhichhashopelesslybefoggedtheissue”(p.2).Fortunately,alargebodyofresearchoutsideofpsychotherapynowprovidesanewclearerdirectionthattakesintoaccountboththeneedforclearmeasurementandtheimportanceofcontextualinfluencesonmethodologythatdrivebetteroutcomes(Ericsson,2009bEricsson,Charness,Feltovich,&Hoffman,).Thesefindingsarelessconcernedwiththeparticularsofagivenareaofperformancethanhowmasteryofanyhumanen-deavorisacquired.Acrossavarietyoffields,includingsports,music,medicine,mathematics,teaching,computerprogramming,andmore,thesubjectofthesestudieshasbeentheindividualperformer,andthequestionofinteresthasbeen,WhyaresomebetterthanothersInsharpcontrasttothefieldofpsychotherapy—withitsrivalparadigms,competingschools,anddisparateconclusions—inves-tigationsrevealasingleunderlyingtraitsharedbytopperformers:deepdomain-specificknowledge.Inshort,thebestknowmore,perceivemore,andremembermorethantheiraveragecounter-parts.Thesameresearchidentifiesauniversalsetofprocessesthatbothaccountforhowdomain-specificknowledgeisacquiredandfurnishstep-by-stepdirectionsanyonecanfollowtoimprovetheirperformancewithinaparticulardiscipline(Ericssonetal.,2006Insummary,nomatterone’sallegiance,thehopehasbeenthatknowinghowpsychotherapyworkswouldgiverisetoauniver-sallyacceptedstandardofcarewhich,inturn,wouldyieldmoreeffectiveandefficienttreatment.However,iftheoutcomeofpsychotherapyisinthehandsofthepersonwhodeliversit,thenMILLER,HUBBLE,CHOW,ANDSEIDEL attemptstoreachaccordregardingtheessentialnature,qualities,orcharacteristicsoftheenterprisearemuchlessimportantthanknowinghowthebestaccomplishwhattheydo.Lookingtothefuture,theapplicationofresearchmethodsandfindingsfromthefieldofexpertiseandexpertperformancepro-videsthewayoutofthefield’scurrentbalkanizationandstale-mate.Suchresearchisalreadyunderway,andtheinitialresultsareinformativeandprovocative(Miller&Hubble,2011Hubble,&Duncan,2007Miller,Hubble,Duncan,&Wampold,The“RoadBestTraveled”:ImprovingOutcomesOneTherapistataTimeAfundamentalfindingoftheresearchonsuperiorperformanceisthattalentisnotafunctionofgenetics,degreesearned,title,privilege,orexperience.Inshort,talentismade.Itresultsfromaprocessofanaltogetherdifferentnature,beyondtraditionalpro-fessionalpreparationandthemereinvestmentoftime.Informedbyfindingsreportedbyresearchers(Ericsson,1996Ericsson,2009bEricssonetal.,2006Ericsson,Krampe,&Tesch-Romer,1993)andwriters(Colvin,2008Coyle,2009Shenk,2010Syed,2010)onthesubjectofexpertise,Milleretal.identifiedthreecomponentscriticalforsuperiorperfor-mance.Workingintandemtocreatea“cycleofexcellence,”theseinclude:(1)determiningabaselinelevelofeffectiveness;(2)obtainingsystematic,ongoing,formalfeedback;and(3)engagingindeliberatepractice.Eachisdiscussedinturn.Tobethebestrequiresknowinghowonefaresinagivenpracticedomain.Interestinglyenough,theexactmethodsbywhichtopperformersdeterminetheirbaselinearehighlyvariable,defy-inganysimpleattemptatclassificationandreplication(Milleretal.,2007).Whatcanbesaidwithcertaintyisthatthebestareconstantlycomparingwhattheydototheirown“personalbest,”theperformanceofothers,andexistingstandardsorbaselinesEricsson,2006).Fortunately,intherealmofpsychotherapy,nu-merouswell-establishedoutcomemeasuresareavailabletoclini-ciansforassessingtheirbaseline(cf.,Froyd&Lambert,1989Ogles,Lambert,&Masters,1996).Additionally,computerizeddatabasesexistthatallowtherapiststomakerealtimecomparisonsoftheirresultswithnationalandinternationalnorms(Miller,Duncan,Sorrell,&Brown,2005).ItisalsoworthnotingthatsincethetimeofthedebatebetweenStrupp(1963Eysenck(1964),severalmethodshaveemergedforoperationalizingandstandardizingtheconceptsofclinicalim-provementandtreatmentfailure(cf.,Hedges&Olkin,1985Jacobson&Truax,1991Ogles,Lambert,&Fields,2002).Al-thougheachconceptualizationandmeasurementschemehasbothbenefitsanddrawbacks,thesetechniquesshowaconsiderableimprovementbeyondthe“befogged”understandingsandinterpre-tationsof50yearsago(Strupp,1963Nevertheless,thoughmeasuresandnormsarenowwidelyavail-able,surveysindicatethatfewcliniciansactuallyusethemintheirday-to-daywork(Phelps,Eisman,&Kohout,1998).Indeed,thecollectionofoutcomedataofanysortisrare.Curiously,despitethelowuse,Bickmanandassociates(Bickmanetal.,2000)foundintheirownsurveythatalargepercentageoftherapistsholdinterestinreceivingregularreportsofclientprogress.Later,fieldandOgles(2004)conductedasurveywithanationalsampleoflicensedpsychologiststoinvestigatethisdiscontinuity.Asbe-fore,cliniciansexpressedinterestinhavingreliableoutcomein-formation.Amongthereasonsgivenbythosechoosingnottouseoutcomemeasures,thetoptwowere,“practical(e.g.,costandtime)andphilosophical(e.g.,relevance)barriers”(p.485).Fullyawareoftherealitiesofclinicalpractice,andinanefforttoovercometheobstaclestoroutineoutcomemeasurement,andDuncan(2000)developed,tested,anddisseminatedtwobrief,four-itemmeasures(Duncanetal.,2003Miller,Duncan,Brown,Sparks,&Claud,2003Thefirst,theOutcomeRatingScale(ORS),assessesclientprogressand,whenaggregated,canbeusedtodetermineatherapist’soveralleffectiveness.Thesecond,theSessionRatingScale(SRS),measuresthequalityofthetherapeu-ticrelationship,akeyelementofeffectivetherapy(Bachelor&Horvath,1999Norcross,2010).Writtenandoralformsareavail-ableatnocostandhavebeentranslatedinto20differentlan-guages.Bothscalestakelessthanaminutetocompleteandscore.Owingtotheirbrevityandsimplicity,adoptionandusageratesamongtherapistshasbeenshowntobedramaticallyhigher(89%)ascomparedwithotherassessmenttools([20%–25%]Duncan,Brown,Sorrell,&Chalk,2006Milleretal.,2003Thesecondelementinfosteringsuperiorperformanceisobtain-ingfeedback.Howard,Moras,Brill,Martinovich,andLutz(1996)wereamongthefirsttosuggestthatformalroutinemeasurementofclientprogresscouldbeusedforoptimizingtreatment.In2001,Lambertandcolleagues(Lambertetal.,2001)reportedresultsdemonstratingthatprovidingfeedbacktocliniciansaboutclientprogressdoubledtherateofclinicallysignificantandreliablechange,decreaseddeteriorationby33%,andreducedtheoverallnumberoftreatmentsessions.Overthepastdecade,researchhascontinuedandaccelerated.Forexample,studiesinvolvingtheORSandSRShaveshownthatexposuretofeedbackasmuchastriplestherateofreliablechangewhilecuttingdeteriorationratesinhalf(Anker,Duncan,&Sparks,2009Lambert&Shimokawa,Reese,Norsworthy,&Rowlands,2009Reese,Toland, TheORSwasdevelopedfollowingthefirstauthor’slonguseoftheOutcomeQuestionnaire45(OQ),atooldevelopedbyhisprofessor,MichaelJ.Lambert,Ph.D.AtaworkshopMillerwasteachingonroutineoutcomemeasurementinIsrael,hementionedthetimethemeasuretooktoadministeraswellasthedifficultymanyofhisclientsexperiencedcom-pletingthetoolowingtoitsrequiredliteracylevel.Apsychologistinattendance,HaimOmer,Ph.D.,suggestedbypassingthelanguage-dependentitemsandusingavisualanaloguescaletocapturethemajordomainsassessedbythelongertool.Miller’sexperiencewiththeBissectionTestSchenkenberg,Bradford,&Ajax,1980)duringhisneu-ropsychologyinternshipandsubsequentworkonthedevelopmentofscalingquestionsattheBriefFamilyTherapyCenterBergandMiller,MillerandBerg,1995)ledhimtosuggesttohiscolleague,BarryDuncan,Psy.D.,thatameasurebecreatedwithfourlines,each10centimetersinlength,representingthefourdomainsofclientfunctioningassessedbytheOQ45(Miller,2010a).AsimilarprocessledtothecreationoftheSRS(Miller,2010b).Onceagain,amentorandsupervisor,LynnJohnson,Ph.D.,developeda10-itemlikertscaleforassessingthequalityofthetherapeuticinteraction(includingalliance[Johnson,1995]).Theauthorhadusedthescalebutwantedasimpler,brieferscaletofitwiththedemandsofaninnercityclinic.ThemeasurewasshortenedandconvertedintoavisualanaloguescalecapturingthemajorelementsofagoodtherapeuticallianceasoriginallydefinedbyBordin(1979).TogetherwithBarryDuncan,Psy.D.,andothers,measuresforchildren,youngchildren,andgroupswereaddedandtestedforreliability,validity,andfeasibility.THEOUTCOMEOFPSYCHOTHERAPY Slone,&Norsworthy,2010).AccordingtoLambert(2010),“itistime(forclinicians)toroutinelytrackclientoutcome”(p.260).Lambert’sproprietary,outcomemanagementsystem,hasbeenapprovedasevidence-basedbytheSubstanceandMentalHealthServicesAdministrationNationalRegistryofEvidence-basedPro-gramsandPractices(SAMHSANREPP).TheORSandSRS,interpretivealgorithms,andnormativedatabase,collectivelyknownas“FeedbackInformedTreatment”(FIT),arecurrentlyunderreviewbySAMHSA.In2012,moreover,theInternationalCenterforClinicalExcellence(ICCE)releasedaseriesofsix“how-to”manualsforimplementingroutineoutcomemeasure-mentinindividualandagencysettings(Bertolino&Miller,2012TheprocesssummarizedinthemanualsconformstotheAmericanPsychologicalAssociation’s(APA)definitionofevidence-basedpractice.Ofnote,thedefinitioncombines“theintegrationofthebestavailableresearch”withclinicalexpertisein“themonitoringofpa-tientprogress(andofchangesinthepatient’scircumstances—e.g.,jobloss,majorillness)thatmaysuggesttheneedtoadjustthetreatment(e.g.,problemsinthetherapeuticrelationshiporintheimplementationofthegoalsofthetreatment)”(APAPresidentialTaskForceonEvidence-BasedPractice,2006,pp.273,276–277).Aspowerfulaneffectasfeedbackexertsonoutcome,itisnotenoughforthedevelopmentofexpertise.Astheliteratureonsuperiorperformanceshowsinotherfields,moreisneededtoenableclinicianstolearnfromtheinformationprovided.DeJong,vanSluis,Nugter,Heiser,andSpinhoven(2012)found,forin-stance,thatnotalltherapistsbenefitfromfeedback.Inaddition,Lambertreportsthatpractitionersdonotgetbetteratdetectingwhentheyareofftrackortheircasesareatriskfordropoutordeterioration,despitebeingexposedto“feedbackonhalftheircasesforover3years”(Milleretal.,2004,p.16).Ineffect,feedbackfunctionslikeaGPS,pointingoutwhenthedriverisofftrackandevensuggestingalternateroutes,whilenotnecessarilyimprovingoverallnavigationskillsorknowledgeoftheterritoryand,attimes,beingcompletelyignored.Learningfromfeedbackrequiresanadditionalstep:engagingindeliberatepractice(Ericsson,1996Ericsson,2006Ericsson,Krampe,&Tesch-Romer,1993).Deliberatepracticemeanssettingasidetimeforreflectingonfeedbackre-ceived,identifyingwhereone’sperformancefallsshort,seekingguidancefromrecognizedexperts,andthendeveloping,rehears-ing,executing,andevaluatingaplanforimprovement.Researchindicatesthateliteperformersacrossmanydifferentdomainsde-votethesameamountoftimetothisprocess,onaverage,everyday.Inastudyofviolinists,forexample,Ericssonetal.(1993)foundthatthetopperformershaddevotedtwotimesasmanyhours(10,000)todeliberatepracticeasthenextbestplayersand10timesasmanyastheaveragemusician.Inadditiontohelpingrefineandextendspecificskills,engaginginprolongedperiodsofreflection,planning,andpracticeengendersthedevelopmentofmechanismsenablingtopperformerstousetheirknowledgeinmoreefficient,nuanced,andnovelwaysthantheirmoreaveragecounterparts(Ericsson&Stasewski,1989Turningtopsychotherapy,researchontheallianceisillustra-tive.Studieshaveconsistentlyfoundamoderate,yetrobust,cor-relationbetweenthequalityofthetherapeuticrelationshipandoutcome(Baldwin,Wampold,&Imel,2007Horvath,DelRe,Fluckiger,&Symonds,2011).Atthesametime,neithertraininginthealliancenorexperienceconductingtherapyhasprovenpartic-ularlypredictiveofclinicianeffectiveness(Horvath,2001son,Ogles,Patterson,Lambert,andVermeersch,2009).Inat-temptingto“untanglethealliance–outcomecorrelation,”etal.(2007)examinedagroupof81cliniciansandfoundthat97%ofthedifferenceinoutcomebetweenthepractitionerswasattrib-utabletotherapistvariabilityinthealliance.Bycontrast,clientvariabilitywasunrelatedtooutcome.Theresultsshowthatsometherapistsareconsistentlybetteratestablishingandmaintaininghelpfulrelationshipsthanothers.Evidencethatthedifferenceisattributabletotheirpossessionofdeeperdomain-specificknowl-edgecanbefoundinarelatedstudybyAndersonetal.(2009)Inbrief,Andersonetal.(2009)examinedtherapisteffectsusingasampleof25providerstreatingclientsinauniversitycounselingcenter.Theclinicianswereaskedtorespondtoaseriesofvideosimulationstotestfor“facilitativeinterpersonalskills”(FIS).Eachsimulationpresentedadifficultclinicalsituation,complicatedbyaclient’sanger,dependency,passivity,confusion,orneedtocontroltheinteraction.Differencesinclientoutcomesbetweentherapistswerefoundtobeunrelatedtotherapistgender,theoreticalorien-tation,professionalexperience,andoverallsocialskills.Instead,thebestresultswereobtainedbythosewhoexhibiteddeeper,broader,moreaccessible,interpersonallynuancedknowledgeasmeasuredontheFIStask.Nomattertheclient’spresentingprob-lemorstyleofrelating,topperformerswereabletorespondcollaborativelyandempathically,andfarlesslikelytomakere-marksorcommentsthatdistancedoroffendedaclient.Acquiringsuchunderstanding,perception,andsensitivityisacommongoalforclinicians.Researchershavefoundthat“healinginvolvement”—apractitioner’sexperienceofengaging,affirming,beinghighlyempathic,stayingflexible,anddealingconstructivelywithdifficultiesencounteredinthetherapeuticinteraction—isthepinnacleoftherapists’aspirations(Orlinsky&Ronnestad,2005Andyet,thestudybyAndersonetal.(2009)suggeststhatenduphavingsuchknowledgewhileothers,ofequalexperienceandsocialability,donot.TworesearchprojectsareunderwaybymembersoftheICCEcommunity.Oneisarandomizedclinicaltrialofdeliberateprac-ticeappliedtotrainingtherapists—alongitudinalstudybeingconductedattheUniversityofNorthCarolinaWilmingtonSchoolofSocialWork.Uponentrytothe2-yearprogram,beginningstudentsarebeinggivenabatteryofassessments,including(a)theFISinventory,avideo-interactivetooldesignedtomeasurealli-ancebuilding,(b)theValuesinActionInventoryofStrengths(VIA-IS),whichmeasurescharacterstrengths,and(c)ademo-graphicquestionnaire.Duringtheirfirstyear,allstudentsreceivethetraditionaltrainingcurriculum.Inyeartwo,studentsareran-domlysplitintotwogroups,withgrouponecontinuingthetradi-tionaltraining,andtheother,experimentalgroup,receivingthetraditionaltrainingplusaprogramofdeliberatepracticeaimedatimprovingtrainees’skillsinallianceformationandmaintenance(i.e.,ongoingmeasurement,feedback,andpracticeopportunitiesundervaryingconditions).Thehypothesisofthestudyisthathoursspentindeliberatepracticeactivitieswillbemorepredictiveofoutcomethanparticipationintraditionaltraining,cliniciancharacterstrengths,andotherdemographicvariables.ItishopedthatthisRCTwilladdress,inpart,Strupp’s(1963)regardingthe“varianceintroducedbythepersonofthetherapistpracticingthem—hisdegreeofexpertness,hispersonality,andattitudes”(pp.1–2).Resultsarenotyetavailable.MILLER,HUBBLE,CHOW,ANDSEIDEL Thesecondresearchprojectexaminestherelationshipbetweenoutcomeandpractitionerdemographicvariables,workpractices,participationinprofessionaldevelopmentactivities,beliefsregard-inglearningandpersonalappraisalsoftherapeuticeffectiveness.Althoughpreliminary,resultsfromthisstudyareinlinewithearlierresearchonthefactorsthataccountforexpertise.SimilartoAndersonetal.(2009)andothers(Wampold&Brown,2005therapistgender,qualifications,professionaldiscipline,yearsofexperience,andtimespentconductingtherapyareunrelatedtooutcomeortherapiststandingwithinthestudysample.SimilartofindingsreportedbyWalfish,McAlister,O’Donnell,andLambert,therapistself-appraisalisnotareliablemeasureofeffec-tiveness.Thefindingsalsoprovidepreliminarysupportforthekeyroledeliberatepracticeplaysinthedevelopmentofexpertiseamonghighlyeffectiveclinicians;specifically,theamountoftimetherapistsreportedspendingengagedinsolitaryactivitiesintendedtoimprovetheirskillswasrelatedtooutcome(Chow,Miller,Kane,&Thorton,n.d).Inall,theevidenceathandindicatesthatthefindingsfromtheexpertiseliteraturelikelyapplytothedomainofpsychotherapy.Furthermore,thethreeactivities—knowingone’sbaseline,obtain-ingfeedback,andengagingindeliberatepractice—likelyprovidethemeansforachievingthegainsinoutcomethathaveforsolongeludedthefield.Iftheresultsreportedhereholduptofurtherinvestigation,itwouldsuggestthatashiftinfocusisrequired.Insteadoftryingtoimproveoutcomesmerelythroughthestudyofpsychotherapiesingeneral(i.e.,premises,models,andassociatedprocedures),thefutureoftheprofessionmaybebetterservedbyworkingtoimprovetheoutcomeofeachandeverytherapist.SummaryConclusionsThequestionthatgaverisetotheexchangebetweenEysenck(1964)intheinauguralissueofthisjournalhasbeensettledbytheaccumulationoffivedecadesofevidence,includingacorrectionofwhatEysenckcriticizedasalackof“asetofreasonablecriteriawhichhaveacertaindegreeofreliabilityandobjectivity”(p.99).Theefficacyandeffectivenessofpsychother-apyarewellestablished,basedon“standardsstatedandfollow-upscarriedout”(Eysenck,1964,p.99),andbenefitingfromcontinualrefinementsofwhatconstituteseffectiveness,whetherinthebe-havioraltermspreferredbyEysenckortheintrapsychicjudgmentsofclientspreferredbyStrupp(Eid&Larsen,2008).Thesecondquestionofhowitworks—inparticular,theindependentvariableofimportance—farfrommovingtheprofessionforward,hasfrag-mentedthefieldleavingoutcomesunchangedforjustasmanydecades.Inpointoffact,nomatterhowthecurativeelementsofpsychotherapyhavebeenconstruedortaught,betheyspecifictechnicaloperations,transtheoreticalhealingfactors,orsomecom-binationthereof,thefieldhasnotcreatednewgenerationsofsuperiorclinicians.Thewayoutasproposedinthisarticlenecessitatessettingasidehistoricalperspectives,traditions,andevenbiases—andembrac-ingadifferentviewofpsychotherapy.AsNorcross(1999)observed,the“ideologicalcoldwarmayhavebeenanecessarydevelopmentalstate,(but)itsdayshavecomeandpassed”(p.xvii).Indeed,onceattentionisturnedtotheperformanceoftheindividualpractitioner,astheweightoftheresearchonexpertiseisdirecting,thenitwouldmakeeminentsensetoregardtherapeu-ticpracticeascraft.Acraftisdefinedas“acollectionoflearnedskillsaccompaniedbyexperiencedjudgment”(Moore,1994;p.1).Consistentwithboththeresearchonpsychotherapyandtheliteratureontheacquisitionofexpertise,noparticularpersonalqualitiesortalentsarerequiredforentry(Ericsson,Krampe,&Tesch-Romer,1993Anyone,withamodicumofinstruction,canlearnhowtodothebasictasksandachieveoutcomescommensuratewithprofession-alsalreadypracticing(Atkins&Christensen,2001Nyman,Nafz-iger,&Smith,2010).Noamountoftheory,coursework,continu-ingeducation,oron-the-jobexperiencewillleadtothedevelopmentofthe“experiencedjudgment”requiredforsuperiorperformance.Forthat,itappearsthatpractitionersmustbeen-gagedintheprocessoutlinedabove—inessence,continuouslyreachingforobjectivesjustbeyondtheircurrentability(Hubble,&Duncan,2007Theimplicationsforthefutureofresearch,professionalprepa-rationanddevelopment,licensureandcertificationarenothinglessthanmajor.Fromacraftperspective,professionaltrainingwouldemphasizethedevelopmentofevidence-basedtherapistsatleastasmuchas,ifnotmorethan,thedisseminationoftheevidencebaseforspecifictherapies,whatStrupp(1963)called“thepersonofthetherapistpracticingthem”(p.1).Inpractice,thiscouldtranslateintoeasingadmissioncriteriasothatalargernumberofcandidatesmayentertrainingprograms.Prospectivematriculantsintograd-uateprogramsfocusedonproducingthebestcliniciansthatpsy-chologyhastooffermightlearnthatgraduationdependsnotonlyonlearningaboutpsychotherapybutalsoonbeingcapableofreliablyproducingpositiveresults.Tothatend,traineeswouldbeexposedtoclientsearlyintheirtraining,routinelymeasured,andgivenampleopportunitytopracticebasicskills(e.g.,allianceformation)undervaryingconditions(e.g.,Andersonetal.,2009Inaddition,educatorsmayimprovethereadinessoftheirin-cominggraduatestudentsbyexperimentingwithundergraduatepsychologycurriculaorientedtoelementsofclinicalqualitybe-yondthelearningoffactsandmethods,perhapsincludingoppor-tunitiesforclinicalvolunteerexperiences(e.g.,crisishotlines,safehouses,residentialtreatment)forthosewhoexpressinterestinclinicaltrainingandwhowanttobeginassessingtheirperfor-manceasbuddingcliniciansandlearningthedisciplineofcontin-uallyassessingandfindingwaystoimprovetheirclinicalout-Similarly,licensuretopracticepsychotherapyorqualitycerti-ficationscouldbegranted,inpart,onachievingandmaintainingabaselinelevelofperformanceequaltoestablishedoutcomebench-marks.Postgraduatetrainingwouldalsochange.AsTaylor,andWear(2009)pointout,“Ifcontinuingeducationisanaturalexpressionofaprofession’songoingevolution,thenpro-fessionalpsychologycanbeviewedassufferingasignificantdevelopmentaldelay”(p.617).Althoughmoststates,forexample,mandateanumberofcontinuingeducationhourstomaintainlicensuretopracticeindependently,theprocessislargelyself-regulated.Withafewnotableexceptions(e.g.,ethics),practitio-nersselecttheeventstheyattend.Directmeasuresoflearningareuncommon,andperformancemeasuresfortheparticipantscom-pletelyabsent.Noprocessisinplaceforidentifyingskillorknowledgedeficitsinneedofremediation,andnoconcreteplanisrequiredforcontinualprofessionaldevelopmentortheassessmentTHEOUTCOMEOFPSYCHOTHERAPY ofwhethersuchaplanresultsinanychangeinclinicaloutcomes.Fromanexpertiseperspective,thecurrentsystemisatbestinef-fectiveand,atworst,perilous.Itreinforcesclinicians’welldocu-mentedpropensitytoinflatetheireffectivenessandseethemselvesasdevelopingprofessionallywhen,infact,theyarenot(Walfishetal.,2012Orlinsky&Ronnestad,2005).Consideringthepotentiallag(likelyayearormoreformanyfull-timepsychotherapists)betweenclinicaltrainingandtheaccumulationofsufficientdatatodeterminewhethersuchtraininghasbeensuccessful,itisespe-ciallyimportantthattheseeffortsaresystematicallytrackedandcliniciandatapooledtogetherdevelopbettermethodsforassessingandimprovingtheimpactoftheseactivities.Withregardtoresearch,theapplicationoffindingsfromthefieldofexpertisetopsychotherapyisinitsinfancy.Asaresult,thepotentialareasforinvestigationarenumerous.Forexample,avail-ableevidencemakesclearthatsuperiorperformancedoesnotoccurinavacuum.Thebestflourishinsupportivecommunities—whathasbeentermed,“culturesofexcellence”or“communitiesofpractice”(Miller&Hubble,2011).Althoughsomeaspects(e.g.,error-centriclearningenvironment,opportunitiesforreflectionanddeliberatepracticebuiltintodailyworkflow)areknown,moreresearchisneededtoidentifythecharacteristicsofsettingsthatproveoptimalforthedevelopmentandmaintenanceofexpertAnotherpotentiallypromisinglineofresearchwouldexplorethepracticepatternsoftopperformingtherapists.AstudybyNajavitsandStrupp(1994)found,forinstance,thateffectivetherapistsreportmakingmoremistakesandbeingmoreself-criticalthantheirlesseffectivecounterparts.Otherresearchshowsthatclinicians’experienceofdifficultiesinpracticeaccountsformosttherapistvarianceinallianceratings(Nissen-Lie,Monsen,&Ronnestad,2010).Resultssuchastheseimmediatelysuggestthepossibilityofstudiesexploringmethodsforhelpingpractitionersdevelopanopen,evenwelcoming,attitudetowarderrors.InDecember2009,theICCEwaslaunched().Similartosermo.comforphysicians,thesiteprovidesafree,international,web-basedcommunityforcli-niciansandresearchersdedicatedtoexcellenceinbehavioralhealth.Memberscanchoosetoparticipateinanyofthe100-plusforums,createtheirowndiscussiongroups,immersethemselvesinalibraryofdocumentsandhow-tovideos,accessoutcometools,andmostimportant,requestandreceiveperformance-orientedfeedbackfromtheirpeers.Thefollowingyearataskforcewithintheorganizationcreatedandpublishedadocumentdetailingfour“corecompetencies”forapplyingthefindingsfromtheexpertiseliteraturetothepracticeofpsychotherapy(Miller,Maeschalck,Axsen,&Seidel,2011).ThefirstcorecompetencyisintheresearchfoundationsofFIT,in-cludingfamiliaritywithresearchonthetherapeuticalliance;be-havioralhealthcareoutcomes;expertperformanceanditsappli-cationtoclinicalpractice;andthepropertiesofvalid,reliable,andfeasibleallianceandoutcomemeasures.ThesecondcompetencyisinFITimplementation:integratingconsumer-reportedoutcomeandalliancedataintoclinicalwork;collaboratingwithconsumersaboutcollectingfeedbackregardingallianceandoutcome;andensuringthatthecourseandoutcomeofbehavioralhealthcareservicesareinformedbyconsumerpreferences.Thethirdcompe-tency,measurementandreporting,focusesonmeasuringanddocumentingthetherapeuticallianceandoutcomeofclinicalser-vicesonanongoingbasiswithconsumers,andonprovidingdetailsinreportingoutcomessufficienttoassesstheaccuracyandgeneralizabilityoftheresults.Thefourthcompetencyiscontinu-ousprofessionalimprovement:determiningone’sbaselinelevelofperformance;comparingone’sbaselinelevelofperformancetothebestavailablenorms,standards,orbenchmarks;developingandexecutingaplanforimprovingbaselineperformance;andseekingperformanceexcellencebydevelopingandexecutingaplanofdeliberatepracticeforimprovingperformancetolevelssuperiortonationalnorms,standards,andbenchmarks.Researchersareal-readyusingthesitetoformulateresearchquestions,solicitpartic-ipantsforstudiesonexpertiseinpsychotherapy,andusingsoft-waretoinvestigateinterestingoutcomepatternsaswellastheconversationaldatageneratedbycliniciansinteractingonthesite.StruppandEysenckbeganapointeddebate50yearsagoonmattersofconsequencefacingthefield.Theirpointedexchangerevealedimportantweaknessesinneedofredress.Some,suchasthegeneralefficacyofpsychotherapy,havebeensuccessfullyaddressed.Others,includinghowitworksandcanworkbetter,continuetodividethefield.Beyondthat,psychotherapyasawhole,andindividualpractitionersinparticular,faceanumberofstarkchallengesinthefuture,nottheleastofwhichisremainingcompetitive.TheauthorsbelievethatfocusingonwhatmakesforagreatperformancecurrentlyholdsthemostpromiseformeetingthesechallengesandadvancingtheunderstandingandpracticeofAddictionandMentalHealthServices.(2011).AMHapprovedpracticesandprocess.RetrievedfromAmericanPsychologicalAssociation.(2004).Communicatingthevalueofpsychologytothepublic.Washington,DC:AmericanPsychologicalAmericanPsychologicalAssociation.(2012,August9).Resolutionontherecognitionofpsychotherapyeffectiveness.AmericanPsychologicalAs-sociation.RetrievedfromAmericanPsychologicalAssociationPresidentialTaskForceonEvidence-BasedPractice.(2006).Evidence-basedpracticeinpsychology.canPsychologist,61,Anderson,T.,Ogles,B.,Patterson,C.,Lambert,M.,&Vermeersch,D.(2009).Therapisteffects:Facilitativeinterpersonalskillsasapredictoroftherapistsuccess.JournalofClinicalPsychology,65,Anker,M.G.,Duncan,B.L.,&Sparks,J.A.(2009).Usingclientfeedbacktoimprovecoupletherapyoutcomes:Arandomizedclinicaltrialinanaturalisticsetting.JournalofConsulting&ClinicalPsychology,77,APAMonitor.(2010,April).Psychologysalariesdecline.MonitoronPsychology,41,Atkins,D.C.,&Christiansen,A.(2001).Isprofessionaltrainingworththebother?AreviewoftheimpactofpsychotherapytrainingonclientAustralianPsychologist,36,Bachelor,A.,&Horvath,A.(1999).Thetherapeuticrelationship.InM.A.Hubble,B.L.Duncan,&S.D.Miller(Eds.),Theheartandsoulofchange:Whatworksintherapy(pp.133–178).Washington,DC:Amer-icanPsychologicalAssociation.Baldwin,S.,Wampold,B.,&Imel,Z.(2007).Untanglingthealliance-outcomecorrelation:ExploringtherelativeimportanceoftherapistandMILLER,HUBBLE,CHOW,ANDSEIDEL patientvariabilityinthealliance.JournalofConsultingandClinicalPsychology,75,842–852.Barlow,D.H.(2004).Psychologicaltreatments.AmericanPsychologist,869–878.Benish,S.G.,Imel,Z.,&Wampold,B.(2008).Therelativeefficacyofbonafidepsychotherapiesfortreatingposttraumaticstressdisorder:Ameta-analysisofdirectcomparisons.ClinicalPsychologyReview,28,746–758.Berg,I.K.,&Miller,S.D.(1992).Workingwiththeproblemdrinker:Asolution-focusedapproach.NewYork:Norton.Bertolino,B.,&Miller,S.D.(Eds.)(2012).ICCEmanualsonfeedback-informedtreatment(Vol.1–6).Chicago,IL:ICCEPress.Beutler,L.E.,Malik,M.,Alimohamed,S.,Harwood,T.M.,Talebi,H.,Noble,S.,&Wong,E.(2004).Therapistvariables.InM.J.LambertBerginandGarfield’shandbookofpsychotherapyandbehavior(5thed.,pp.227–306).NewYork:Wiley.Bickman,L.,Rosof-Williams,J.,Salzer,M.S.,Summerfelt,W.,Noser,K.,Wilson,S.J.,&Karver,M.S.(2000).Whatinformationdocliniciansvalueformonitoringadolescentclientprogressandoutcomes?sionalPsychology:ResearchandPractice,31,70–74.Bohanske,R.,&Franczak,M.(2010).Transformingpublicbehavioralhealthcare:Acaseexampleofconsumer-directedservices,recovery,andthecommonfactors.InB.Duncan,S.Miller,B.Wampold,&M.Hubble(Eds.),Theheartandsoulofchange:Deliveringwhatworksin(2nded.,pp.299–322).Washington,DC:APAPress.Bordin,E.S.(1979).Thegeneralizabilityofthepsychoanalyticconceptoftheworkingalliance.Psychotherapy:Theory,Research,andPractice,Bradshaw,J.(2012,September).PetitionseekstodumpDSMandadopt.NationalPsychologist.RetrievedfromBrown,G.S.,&Minami,T.(2010).Outcomesmanagement,reimburse-ment,andthefutureofpsychotherapy.InB.Duncan,S.Miller,B.Wampold,&M.Hubble(Eds.),Theheartandsoulofchange:Deliv-eringwhatworksintherapy(2nded.,pp.267–297).Washington,DC:APAPress.Chambless,D.L.,&Hollon,S.(1998).DefiningempiricallysupportedJournalofConsultingandClinicalPsychology,66,Chow,D.,Miller,S.D.,Kane,R.,&Thornton,J.(n.d.).Thestudyofsupershrinks:Developmentanddeliberatepracticesofhighlyeffectivepsychotherapists.Manuscriptinpreparation.Clay,R.A.(2012).Improvingdisorderclassification,worldwide.onPsychology,43,Colvin,G.(2008).Talentisoverrated:Whatreallyseparatesworld-classperformersfromeverybodyelse.NewYork:Penguin.Cooper,J.L.,&Aratani,Y.(2009).Thestatusofstates’policiestosupportevidence-basedpracticesinchildren’smentalhealth.PsychiatricSer-vices,60,Coyle,D.(2009).Thetalentcode:Greatnessisn’tborn.It’sgrown.Here’s.NewYork:BantamDell.Crits-Christoph,P.,Baranackie,K.,Kurcias,J.,Beck,A.T.,Carroll,K.,Perry,K.Zitrin,C.(1991).Meta-analysisoftherapisteffectsinpsychotherapyoutcomestudies.PsychotherapyResearch,1,Crits-Christoph,P.,&Mintz,J.(1991).Implicationsoftherapisteffectsforthedesignandanalysisofcomparativestudiesofpsychotherapies.JournalofConsultingandClinicalPsychology,59,20–26.Cummings,N.,&O’Donohue,W.(2008).ElevenblundersthatcripplepsychotherapyinAmerica:Aremedialunblundering.NewYork:Rout-deJong,K.,vanSluis,P.,Nugter,M.A.,Heiser,W.J.,&Spinhoven,P.(2012).Understandingthedifferentialimpactofoutcomemonitoring:Therapistvariablesthatmoderatefeedbackeffectsinarandomizedclinicaltrial.PsychotherapyResearch,22,464–474.Duncan,B.(2010).Onbecomingabettertherapist.Washington,DC:APADuncan,B.L.,Miller,S.D.,Reynolds,L.,Sparks,J.,Claud,D.,Brown,J.,&Johnson,L.D.(2003).Thesessionratingscale:Preliminarypsycho-metricpropertiesofa“working”alliancescale.JournalofBriefTher-apy,3,Duncan,B.L.,Miller,S.D.,Wampold,B.E.,&Hubble,M.A.(Eds.).Theheartandsoulofchange:Deliveringwhatworksintherapy(2nded.)Washington,DC:APAPress.Eid,M.,&Larsen,R.J.(2008).Thescienceofsubjectivewell-being.NewYork:Guilford.Ericsson,K.A.(1996).Theacquisitionofexpertperformance:Anintro-ductiontosomeoftheissues.InK.A.Ericsson(Ed.),Theroadtoexcellence:Theacquisitionofexpertperformanceintheartsandsci-ences,sports,andgames(pp.1–50).Mahwah,NJ:LawrenceErlbaumEricsson,K.A.(2006).TheInfluenceofexperienceanddeliberatepracticeonthedevelopmentofsuperiorexpertperformance.InK.A.Ericsson,N.Charness,P.J.Feltovich&R.R.Hoffman(Eds.),TheCambridgehandbookofexpertiseandexpertperformance(pp.683–703).Cam-bridge:CambridgeUniversityPress.Ericsson,K.A.(2009a).Enhancingthedevelopmentofprofessionalper-formance:Implicationsfromthestudyofdeliberatepractice.Inopmentofprofessionalexpertise:Towardmeasurementofexpertper-formanceanddesignofoptimallearningenvironments(pp.405–431).NewYork:CambridgeUniversityPress.Ericsson,K.A.(Ed.).(2009b).Developmentofprofessionalexpertise:Towardmeasurementofexpertperformanceanddesignofoptimallearningenvironments.NewYork:CambridgeUniversityPress.Ericsson,K.A.,Charness,N.,Feltovich,P.J.,&Hoffman,R.R.(2006).TheCambridgehandbookofexpertiseandexpertperformance.Cam-bridge:CambridgeUniversityPress.Ericsson,K.A.,Krampe,R.T.,&Tesch-Romer,C.(1993).Theroleofdeliberatepracticeintheacquisitionofexpertperformance.icalReview,100,363–406.Ericsson,K.A.,&Staszewski,J.(1989).Skilledmemoryandexpertise:Mechanismsofexceptionalperformance.InD.Klahr&K.KotovskyComplexinformationprocessing:TheImpactofHerbertA.(pp.265–268).Hillsdale,NJ:LawrenceErlbaum.Eysenck,H.J.(1952).Theeffectsofpsychotherapy:Anevaluation.JournalofConsultingPsychology,16,319–324.Eysenck,H.(1964).Theoutcomeprobleminpsychotherapy:Areply.Psychotherapy:Theory,Research&Practice,1,Frank,J.D.(1992).Historicaldevelopmentsinresearchcenters:TheJohnsHopkinsPsychotherapyResearchProject.InD.K.Freedheim(Ed.),historyofpsychotherapy:Acenturyofchange(pp.392–396).Washing-ton,DC:APAPress.Frank,J.D.,&Frank,J.B.(1991).Persuasionandhealing:Acomparativestudyofpsychotherapy.Baltimore,MD:JohnsHopkinsUniversityFroyd,J.,&Lambert,M.(1989,May).A5-yearsurveyofoutcomemeasuresinpsychotherapyresearch.PaperpresentedattheWesternPsychologicalAssociationConference,Reno,NV.Gambrill,E.(1990).Criticalthinkinginclinicalpractice.SanFrancisco,CA:Jossey-Bass.THEOUTCOMEOFPSYCHOTHERAPY Garfield,S.L.(1997).Thetherapistasaneglectedvariableinpsychother-apyresearch.ClinicalPsychology:Science&Practice,4,40–43.Grencavage,L.M.,&Norcross,J.C.(1990).Wherearethecommonalitiesamongthetherapeuticcommonfactors?ProfessionalPsychology:Re-searchandPractice,21,Hatfield,D.R.,&Ogles,B.M.(2004).Theuseofoutcomemeasuresbypsychologistsinclinicalpractice.ProfessionalPsychology:ResearchandPractice,35,485–491.Hedges,L.V.,&Olkin,I.(1985).Statisticalmethodsformeta-analysisSanDiego,CA:AcademicPress.Hilgard,E.(1987).PsychologyinAmerican:Ahistoricalsurvey.NewYork:HBJ.Horvath,A.(2001).Thealliance.Psychotherapy:Theory/Research/Practice/Training,38,Horvath,A.O.,DelRe,A.,Fluckiger,C.,&Symonds,D.(2011).Allianceinindividualpsychotherapy.Psychotherapy,48,9–16.Howard,K.I.,Moras,K.,Brill,P.L.,Martinovich,Z.,&Lutz,W.(1996).Efficacy,effectiveness,andpatientprogress.AmericanPsychologist,51,1059–1064.Imel,Z.E.,Wampold,B.E.,Miller,S.D.,&Fleming,R.R.(2008).Distinctionswithoutadifference:Directcomparisonsofpsychothera-piesforalcoholusedisorders.PsychologyofAddictiveBehaviors,22,Jacobson,N.S.,&Truax,P.(1991).Clinicalsignificance:Astatisticalapproachtodefiningmeaningfulchangeinpsychotherapyresearch.JournalofConsultingandClinicalPsychology,59,Johnson,L.D.(1995).Psychotherapyintheageofaccountability.NewYork:Norton.Kim,D.-M.,Wampold,B.E.,&Bolt,D.M.(2006).Therapisteffectsinpsychotherapy:Arandom-effectsmodelingoftheNationalInstituteofMentalHealthTreatmentofDepressionCollaborativeResearchPro-gramdata.PsychotherapyResearch,16,Lambert,M.J.(2010).Yes,itistimeforclinicianstoroutinelymonitortreatmentoutcome.InB.Duncan,S.Miller,B.Wampold,&M.HubbleTheheartandsoulofchange:Deliveringwhatworksintreat-(pp.239–266).Washington,DC:APAPress.Lambert,M.J.(2012).Helpingclinicianstouseandlearnfromresearch-basedsystems:TheOQ-analyst.Psychotherapy(Chicago,Ill),Training,109–114.Lambert,M.J.,&Shimokawa,K.(2011).Collectingclientfeedback.Psychotherapy,48,Lambert,M.J.,Whipple,J.L.,Smart,D.W.,Vermeersch,D.A.,Nielsen,S.L.,&Hawkins,E.J.(2001).Theeffectsofprovidingtherapistswithfeedbackonpatientprogressduringpsychotherapy:Areoutcomesen-PsychotherapyResearch,11,49–68.Luborsky,L.(1954).Selectingpsychiatricresidents:SurveyoftheTopekaBulletinoftheMenningerClinic,18,Luborsky,L.,Crits-Christoph,P.,Mclellan,A.T.,Woody,G.,Piper,W.,Liberman,B.,Imber,S.,&Pilkonis,P.(1986).Dotherapistsvarymuchintheirsuccess?Findingsfromfouroutcomestudies.AmericanJournalofOrthopsychiatry,56,Luborsky,L.,McClellan,A.T.,Diguer,L.,Woody,G.,&Seligman,D.A.(1997).Thepsychotherapistmatters:Comparisonofoutcomesacrosstwenty-twotherapistsandsevenpatientsamples.ClinicalPsychology:ScienceandPractice,4,53–65.Luborsky,L.,McClellan,A.T.,Woody,G.E.,O’Brien,C.P.,&Auer-bach,A.(1985).Therapistsuccessanditsdeterminants.ArchivesofGeneralPsychiatry,42,602–611.Lutz,W.,Leon,S.C.,Martinovich,Z.,Lyons,J.S.,&Stiles,W.B.(2007).Therapisteffectsinoutpatientpsychotherapy:Athree-levelgrowthcurveapproach.JournalofCounselingPsychology,54,Miller,S.D.(2010a).Findingfeasiblemeasuresforpractice-basedevi-dence.TopPerformanceBlog.RetrievedfromMiller,S.D.(2010b).Feedback,friends,andoutcomeinbehavioralhealth.TopPerformanceBlog.RetrievedfromMiller,S.D.,&Berg,I.K.(1995).Themiraclemethod:Aradicallynewapproachtoproblemdrinking.NewYork:Norton.Miller,S.D.,&Duncan,B.L.(2000).Theoutcomeandsessionrating.Chicago,IL:InternationalCenterforClinicalExcellence.Re-trievedfromMiller,S.D.,Duncan,B.L.,Brown,J.,Sorrell,R.,&Chalk,M.B.(2006).Usingformalclientfeedbacktoimproveretentionandoutcome:Makingongoingreal-timeassessmentfeasible.JournalofBriefTherapy,5,Miller,S.D.,Duncan,B.L.,Brown,J.,Sparks,J.,&Claud,D.(2003).Theoutcomeratingscale:Apreliminarystudyofthereliability,validity,andfeasibilityofabriefvisualanalogmeasure.JournalofBriefTherapy,2,Miller,S.D.,Duncan,B.L.,&Hubble,M.A.(1997).EscapefromBabel:Towardaunifyinglanguageforpsychotherapypractice.NewYork:Miller,S.D.,Duncan,B.L.,&Hubble,M.A.(2004).Beyondintegration:Thetriumphofoutcomeoverprocessinclinicalpractice.inAustralia,10,Miller,S.D.,Duncan,B.L.,Sorrell,R.,&Brown,J.(2005).Thepartnersforchangeoutcomemanagementsystem.JournalofClinicalPsychol-ogy,61,199–208.Miller,S.D.,&Hubble,M.(2011).Theroadtomastery.Networker,35(2),22–60.Miller,S.D.,Hubble,M.A.,&Duncan,B.L.(2007).Supershrinks.PsychotherapyNetworker,31,26–35,56.Miller,S.D.,Hubble,M.A.,Duncan,B.L.,&Wampold,B.(2010).Deliveringwhatworks.InB.L.Duncan,S.D.Miller,B.E.Wampold,&M.A.Hubble(Eds.),Theheartandsoulofchange:Deliveringwhatworksintherapy(pp.421–429).Washington,DC:APAPress.Miller,S.D.,Maeschalck,C.,Axsen,R.,&Seidel,J.(2011).Theinter-nationalcenterforclinicalexcellencecorecompetencies.Retri-evedfromMoore,D.S.(1994).Thecraftofteaching.Addressattheawardceremonyfordistinguishedcollegeoruniversityteachingofmathematics.SanFrancisco,CA.RetrievedfromNajavits,L.,&Strupp,H.(1994).Differencesintheeffectivenessofpsychodynamictherapies:Aprocess-outcomestudy.Psychotherapy,31,114–123.Nathan,P.E.(1997).Fiddlingwhilepsychologyburns?RegisterReport,1,4–5,10.Neimeyer,G.,Taylor,J.,&Wear,D.(2009).Continuingeducationinpsychology:Outcomes,evaluation,andmandates.ProfessionalPsychol-ogy:ResearchandPractice,40,617–624.Nissen-Lie,H.A.,Monsen,J.T.,&Ronnestad,M.H.(2010).Therapistpredictorsofearlypatient-ratedworkingalliance:Amultilevelap-PsychotherapyResearch,20,627–646.MILLER,HUBBLE,CHOW,ANDSEIDEL Norcross,J.C.(1997).Emergingbreakthroughsinpsychotherapyintegra-tion:Threepredictionsandonefantasy.Psychotherapy:Theory,Re-search,Practice,Training,34,86–90.Norcross,J.(1999).Foreword.InM.A.Hubble,B.L.Duncan,&S.D.Miller(Eds.).Theheartandsoulofchange(pp.xvii–xix).Norcross,J.C.(2010).Thetherapeuticrelationship.InB.L.Duncan,S.D.Miller,B.E.Wampold&M.A.Hubble(Eds.),Theheartandsoulofchange:Deliveringwhatworksintherapy(2nded.,pp.113–142).Washington,DC:AmericanPsychologicalAssociation.Nyman,S.,Nafziger,M.,&Smith,T.(2010).Clientoutcomesacrosscounselortraininglevelwithinamultitieredsupervisionmodel.ofCounseling&Development,88,204–209.Ogles,B.M.,Lambert,M.J.,&Fields,S.(2002).Essentialsofoutcome.NewYork:JohnWiley&Sons.Ogles,B.,Lambert,M.,&Masters,K.(1996).Assessingoutcomeinclinicalpractice.NeedhamHeights,MA:Allyn&Bacon.Okiishi,J.C.,Lambert,M.J.,Eggett,D.,Nielsen,S.L.,Dayton,D.D.,&Vermeersch,D.A.(2006).Ananalysisoftherapisttreatmenteffects:Towardprovidingfeedbacktoindividualtherapistsontheirpatients’psychotherapyoutcome.JournalofClinicalPsychology,62,Okiishi,J.C.,Lambert,M.J.,Nielsen,S.L.,&Ogles,B.M.(2003).Waitingforsupershrink:Anempiricalanalysisoftherapisteffects.ClinicalPsychology&Psychotherapy,10,Orlinsky,D.E.,&Ronnestad,M.H.(2005).Howpsychotherapistsde-velop:Astudyoftherapeuticworkandprofessionalgrowth.Washing-ton,DC:AmericanPsychologicalAssociation.Phelps,R.,Eisman,E.,&Kohout,J.(1998).Psychologicalpracticeandmanagedcare:ResultsoftheCAPPpractitionersurvey.Psychology:ResearchandPractice,29,Reese,R.J.,Norsworthy,L.A.,&Rowlands,S.R.(2009).Doesacontinuousfeedbacksystemimprovepsychotherapyoutcome?therapy:Theory,Research,Practice,Training,46,418–431.Reese,R.J.,Toland,M.D.,Slone,N.C.,&Norsworthy,L.A.(2010).Effectofclientfeedbackoncouplepsychotherapyoutcomes.therapy:Theory,Research,Practice,TrainingDecember,47,616–630.Rosenzweig,S.(1954).Atransvaluationofpsychotherapy:AreplytoHansTheJournalofAbnormalandSocialPsychology,49,298–304.Schenkenberg,T.,Bradford,D.,&Ajax,E.(1980).Linebisectionandunilatervisualneglectinpatientswithneurologicalimpairment.rology,30,509–517.Shapiro,D.A.,Firth-Cozens,J.,&Stiles,W.B.(1989).Thequestionoftherapists’differentialeffectiveness:ASheffieldPsychotherapyProjectBritishJournalofPsychiatry,154,Shenk,D.(2010).Thegeniusinallofus:Whyeverythingyou’vebeentoldaboutgenetics,talent,andIQiswrong.NewYork:RandomHouse.Smith,M.L.,&Glass,G.V.(1977).Meta-analysisofpsychotherapyoutcomestudies.AmericanPsychologist,32,Strupp,H.(1963).Theoutcomeprobleminpsychotherapyrevisited.chotherapy:Theory,Research&Practice,1,Strupp,H.(1964).Theoutcomeprobleminpsychotherapy:Arejoinder.Psychotherapy:Theory,Research&Practice,1,Strupp,H.,&Anderson,T.(1997).Onthelimitationsoftherapymanuals.ClinicalPsychology:ScienceandPractice,4,76–82.Syed,M.(2010).Bounce:Mozart,Federer,Picasso,Beckham,andthescienceofsuccess.NewYork:HarperCollins.TherapyinAmerica.(2004).AsurveyconductedbyHarrisInteractiveonbehalfofPsychologyTodayandPacificareBehavioralHealth.Re-trievedfromVandenBos,G.R.,Cummings,N.,&DeLeon,P.H.(1992).Acenturyofpsychotherapy:Economicandenvironmentalinfluences.InD.K.Freed-heim(Ed.),Ahistoryofpsychotherapy:Acenturyofchange.Washing-ton,DC:APAPress.Walfish,S.,McAlister,B.,O’Donnell,P.,&Lambert,M.J.(2012).Aninvestigationofself-assessmentbiasinmentalhealthproviders.logicalReports,110,639–644.Wampold,B.E.(2001).Thegreatpsychotherapydebate:Models,meth-ods,andfindings.Mahwah,NJ:Erlbaum.Wampold,B.E.(2005).Establishingspecificityinpsychotherapyscien-tifically:Designandevidenceissues.ClinicalPsychology:Science&PracticeSummer,12,194–197.Wampold,B.E.(2010).Theresearchevidenceforthecommonfactormodels:Ahistoricallysituatedperspective.InB.L.Duncan,S.D.Miller,B.E.Wampold&M.A.Hubble(Eds.),Theheartandshouldofchange:Deliveringwhatworksintherapy(2nded.,pp.49–82).Wash-ington,DC:AmericanPsychologicalAssociation.Wampold,B.E.,&Bolt,D.M.(2006).Therapisteffects:Cleverwaystomakethem(andeverythingelse)disappear.PsychotherapyResearch,184–187.Wampold,B.E.,&Brown,G.S.(2005).Estimatingvariabilityinout-comesattributabletotherapists:Anaturalisticstudyofoutcomesinmanagedcare.JournalofConsultingandClinicalPsychology,73,914–923.Wampold,B.E.,Mondin,G.W.,Moody,M.,&Ahn,H.-n.(1997).Theflatearthasametaphorfortheevidenceforuniformefficacyofbonafidepsychotherapies:ReplytoCrits-Christoph(1997),andHowardetal.PsychologicalBulletin,122,226–230.Wampold,B.E.,Mondin,G.W.,Moody,M.,Stich,F.,Benson,K.,&Ahn,H.-n.(1997).Ameta-analysisofoutcomestudiescomparingbonafidepsychotherapies:Empirically,“allmusthaveprizes.”Bulletin,122,Wilson,G.T.(1995).Empiricallyvalidatedtreatmentsasabasisforclinicalpractice:Problemsandprospects.InS.C.Hayes,V.M.Follette,R.M.Dawes,&K.E.Grady(Eds.),Scientificstandardsofpsycholog-icalpractice:Issuesandrecommendations(pp.163–196).Reno,NV:ContextPress.Zuriff,G.E.(1985).Behaviorism:Aconceptualreconstruction.NewYork:ColumbiaUniversityPress.ReceivedOctober16,2012AcceptedOctober17,2012THEOUTCOMEOFPSYCHOTHERAPY