This paper presents - PDF document

This paper presents preliminary findings and is being distributed to economists and other interested readers solely to stimulate discussion and elicit comments. The views expressed in this paper are those of the authors and do not necessarily reflect the position of the Feder al Reserve Bank of New York or the Federal Reserve System. Any errors or omissions are the responsibility of the authors. Federal Reserve Bank of New York Staff Reports What Predicts U.S. Recessions? Weiling Liu Emanuel Moench Staff Report No. 691 September 2014 What Predicts U.S. Recessions? Weiling Liu and Emanuel Moench Federal Reserve Bank of New York Staff Reports , no. 691 September 2014 JEL classification: C52, C53, E32, E37 Abstract We reassess the predictability of U.S. recessions at horizons from three months to two years ahead for a large number of previously proposed leading - indicator variables. We employ an e ffi cient probit estimator for partially missing data and assess relative model performance based on the receiver operating characteristic (ROC) curve. While the Treasury term spread has the highest predictive power at horizons four to six quarters ahead, a dding lagged observations of the term spread significantly improves the predictability of recessions at shorter horizons. Moreover, balances in b roker - d ealer margin accounts signi fi cantly improve the precision of recession predictions , especially at horizo ns further out than one year . Key words: recession predictability, ROC, term spread, leading indicators, efficient probit estimator _________________ Liu : Harvard Business School (e - mail: wliu@hbs.edu ). Moench : Federal Reserve Bank of New York (e - mail: Emanuel.moench@ny.frb.org ). The views expressed in this paper are those of the authors and do not necessarily reflect the position of the Federal Reserve Bank of New York or the Federal Reserve System. Tomakeuseofthen�rnon-missingobservations,assume:W0i=X0iC+u0i;(4)whereCisa(kxl)matrixofparametersanduiMVN(0;).Then,combining(3)and(4),oneobtains:Yi=X0i(Bx+CBw)+eyi;(5)where,conditionalonXi,(eyi;Wi)aremultivariatenormallydistributed.Theassumptionofconditionaljointnormalityisanalyticallyconvenientandallowsforecientestimation.ConnieandO'Neill(2011)showthattheirproposedestimatorisrobusttovariousdepar-turesfromtheparametricassumptionsin(4).RatherthanexplicitlyrestatingtheestimatoranditsasymptoticvariancederivedbyCon-nieandO'Neill(2011),wesimplysummarizethevariousestimationsteps:11.RunanOLSregressionofXonWforthesamplewithrcompleteobservations.2.RunastandardprobitofYonXandWforthesamplewithrcompleteobservations.3.RunaprobitofYonXforjustthesamplewithn�rmissingobservations4.Calculatethecoecientsandstandarderrorsfortheprobitmissestimatorusingasinputstheestimationoutputsfromsteps(1)-(3).Itisimportanttopointoutthatinadditiontotheassumptionofconditionalmultinor-mality,theecientprobitestimatorrequiresthatthemissingdataforWaremissingatrandom(MAR).Inotherwords,thereasonforthedata'sabsenceshouldnotberelatedtoanomittedvariablethatiscorrelatedwithrecessions,suchasthestateofthebusinesscycle.Sinceourmissingdataareonlymissingatthebeginningofthedatasetduetolimitationsofourdatabase,theMARassumptionisnaturallysatised. 1Forfurtherdetails,werefertheinterestedreadertothepaperbyConnieandO'Neill(2011).7 whereRECisabinaryvariablewhichtakesonvaluesofoneinrecessionsandzeroinexpansions,Xtisan1vectorofpredictorvariablesobservedinperiodt,anddenotesthecumulativedensityfunctionofthestandardnormaldistribution.Letting=(0;01)0;theprobitmodelmaximizestheloglikelihoodfunctionln`()=TXt=1[RECt+kln(0+01Xt)+(1�RECt+k)ln(1�(0+01Xt))](2)Hence,giventimeseriesobservationsforthepredictorvariablesXandtheresponsevariableREC,onecannumericallysolveforthemaximumlikelihoodestimates.Someofthepredictorvariableswewillconsiderinourempiricalanalysisarenotobservedoverthefullsampleperiod.Wethereforeneedtoadjusttheprobitmodeltoallowformissingobservations.Onecommonlyusedmethodofhandlingmissingdataistodisregardthedatesonwhichanyvariablesaremissing,butthismethodinecientlydiscardspotentiallyusefuldata.Instead,weemploytheecient\probitmiss"estimator,recentlyproposedbyConnieandO'Neill(2011),whichallowsustoincorporateallrelevantdata.BuildingoofChesher(1984),ConnieandO'Neill'smodelassumesthatthereexistsoneunderlyingunobservable,continuouslatentvariableYiandanobservedbinaryvariableZiwhichfollowstherelationship:Zi=1ifYi�0Zi=0ifYi0:Theregressorsaregroupedintotwocategories,denotedinvectorform:Xi(complete)andWi(incomplete).ThereareknumberofX's,andlnumberofW's.WeobservethecompletesampleofobservationsfXi;Wi;Zigfori=1,2,...r.Thisleaves(n�r)observationsonwhichfXi;Zigalonearemeasured.Theyfollowtherelationship:Yi=X0iBx+W0iBw+"i:(3)6 sionperiods.Section3providesadescriptionofthevariousrecessionindicatorsusedinouranalysis.Section4summarizesthein-sampleandout-of-samplerecessionpredictionresults.Finally,Section5providesadiscussionoftheempiricalndings.2MethodologyInthissection,webrie ydescribetheempiricalmethodsusedinthepaper.Westartbyrevisitingthestandardprobitmodelwhichweusetoestimatetherecessionprobabilitiesasfunctionsofobservablepredictorvariables.Wethenbrie ydiscussanextensionwhichallowsfortheinclusionofpartiallyunobservedpredictorvariables.Finally,wedescribetheAUROCmeasureandrelatedstatisticaltestswhichweemploytodiscriminatebetweenmodels.2.1PredictingRecessionsThestateofthebusinesscycleisabinaryvariable,takingonthevalueofoneduringarecessionandzeroduringanexpansion.Ontheotherhand,mostleadingindicatorsarecontinuousvariables.Inordertoaccountforthis,acommontooltopredictrecessionsistheprobitmodel(seeEstrellaandHardouvelis(1991),EstrellaandMishkin(1996),EstrellaandTrubin(2006),Wright(2006))whichallowsamappingfromasetofcontinuousexplanatoryvariablesintoabinarydependentvariable.Whileothermethodsareavailableforpredictingbinaryresponsevariables,werestrictourselvestothispopularclassofmodelsforitssim-plicityandeaseofuse.ThemodelischaracterizedbythesimpleequationP(RECt+k=1)=(0+01Xt);(1)5 pendentmodel,amodelwithautocorrelatederrors,andcombinationsoftheseextensions.Theyconcludethatthemoresophisticatedmodelscapturethepredictiveinstabilityoftheyieldcurvebetterbyallowingforbreakpoints.WhilealloftheabovecitedpapershavestudiedthepredictivepowerofthetermspreadforoutputgrowthandrecessionsintheU.S.,someauthorshavedocumentedsimilarlystrongpredictivepowerofgovernmentbondyieldspreadsinothercountries.Forexample,Duarte,Venetis,andPaya(2005)ndthatyieldspreadspredictrecessionsintheEuropeanMon-etaryUnion.Moreover,examiningboththeU.S.andGermany,Nyberg(2010)concludesthatthedomestictermspreadremainsthebestrecessionpredictor.Recently,RudebuschandWilliams(2009)havefoundthatthetermspreadconsistentlyoutperformsevenprofessionalforecastorsinpredictingrecessions.Thisissurprisingastheseforecastershaveawealthofinformationandmanyotherindicatorsavailabletothem.CroushoreandMarsten(2014)conrmthatRudebuschandWilliams'ndingsarerobustacrossseveraldimensionsincludingthesamplechoice,theuseofrollingregressionwindows,andvariousmeasuresofrealoutput.Moreover,Lahiri,Monokroussos,andZhao(2013)reportthattheresultremainsvalidevenafterfurtheraugmentingthemodelwithfactorsextractedfromalargemacroeconomicdataset.Thesepapers'ndingshighlightthesingularimportanceoftheTreasurytermspreadasapredictorofrecessionsandjustifyouruseofthisindicatorasthebenchmarkpredictorvariable.Methodologically,ourpaperborrowsfromBergeandJorda(2011),whousetheAUROCtobothvalidatetheNBER'sbusinesscyclechronologyaswellasinvestigatewhichleadingindicesworkbestasaclassicationmechanismforrecessions.Theyndnosupportforsta-tisticallysignicantimprovementsoftheparametricmodelsovertheNBERdates.Hence,theirresultsalsosupportouruseoftheNBERbusinesscyclechronologyasreferencefortherecessionclassicationabilityofthevariousprobitmodelsthatweconsider.Ourpaperisorganizedasfollows.Section2discussestheempiricalmethodologyusedtopredictrecessionprobabilitiesandevaluatetheclassicationoffuturerecessionandexpan-4 topredictrecessions.Thissuggeststhatattheseshorterhorizonsthereispredictiveinfor-mationnotonlyinthecontemporaneoussteepnessoftheTreasuryyieldcurve,butalsointhelaggedtermstructureslope.Thenegativesignonthecoecientoflaggedspreadhastwoimplications:persistenceandchange.First,ifspreadswerenegativesix-monthsago,thenthereisahigherprobabilityofrecessioninthefuture.Second,giventhesamestartingvalueofspreadsix-monthsago,asharperdropinthespreadsincethenleadstoahigherprobabilityofrecessioninthefuture.InadditiontothecontemporaneousandlaggedTrea-surytermspreads,anumberofothervariablesalsocontainpredictiveinformationaboutfuturerecessionsathorizonslessthanoneyearahead.Inparticular,theannualreturnontheS&P500stockmarketindex,theMichigansurveyofconsumerexpectations,andagainthemargindebitatNYSEbrokersanddealerssignicantlyincreasethepredictivepoweroftheprobitmodelwhenaddedtotheTreasurytermspread.Ourpaperisrelatedtoalargeliteratureonpredictingrealoutputgrowthandrecessionsusingnancialandmacroeconomicleadingindicators.EstrellaandHardouvelis(1991)rstpopularizedtheTreasurytermspreadasapredictoroffutureoutputgrowthandrecessions.TheyfoundthatithasgreaterpredictivepowerthantheLeadingIndicatorIndexandoutper-formssurveyforecastsbothin-andout-of-sample.EstrellaandMishkin(1996)andEstrellaandMishkin(1998)consideredtheout-of-sampleperformanceofarangeofmacroeconomicandnancialvariablesbothone-at-a-timeandincombination.Theirndingssuggestthatintheshortrun,stockreturnsareavaluableleadingindicator.However,athorizonsofoneyearaheadormore,theTreasurytermspreadisstillthesinglebestperformingpredictor.Dueker(1997)revisitedthetermspreadasaleadingindicatorwithinthecontextoftheprobitmodelstudiedinourpaper.Conrmingearlierresults,hefoundthetermspreadtobethesinglebestrecessionpredictorwhencomparedtootherleadingeconomicindicatorsandnancialvariables,andshowedthatthisndingisrobusttoaugmentationoftheprobitmodelwithlaggeddependentvariablesandMarkovswitching.ChauvetandPotter(2005)examinefurtherextensionsoftheyieldcurveprobitmodel,includingabusinesscyclede-3 Khandani,Kim,andLo(2010),JordaandTaylor(2011),JordaandTaylor(2012)).TheROCcurveiscomputedinseveralsteps.First,foragivengridofcutovaluesoftheim-pliedrecessionprobability,onecalculatesthepercentageoftruepositivesandfalsepositivesforclassifyingallperiodsinthesample.Onethenplotsthepercentageoftrueandfalsepositivesagainstoneanotherfortheentiregridtocreatethereceiveroperatingcurve.OnemethodofcomparingthepredictiveabilityofclassiersacrossaspectrumofcutovaluesistointegratetheareaundertheROCcurve,creatingtheAUROC.AmodelwhichdeliversaperfectclassicationofalltimeperiodsintorecessionandexpansionwouldonlyhavetruepositivesandnofalsepositivesandanAUROCequaltoone.Incontrast,amodelwhichistheequivalentofarandomguesswouldhaveonaverageanequalnumberoftrueandfalsepositives,whichcorrespondstoanAUROCequalto0.5.HanleyandMcNeil(1983)deriveat-testforthehypothesisthatthepredictiveabilityoftwodierentclassiersareequalbyusingtheirAUROC's.Weusetheirtestinordertodiscriminatebetweenthepredictiveabilityofdierentrecessionindicatorsconsideredintheliterature.Ourmainndingscanbesummarizedasfollows.TheTreasurytermspreadpredictsbestathorizonsofoneyearandmore.Thatsaid,someindicatorsaddtothepredictiveabilityofthetermspreadatthesehorizons.Inparticular,margindebitatNYSEbrokersanddeal-ers,ameasureofleverageinthenancialsector,signicantlyimprovesthein-sampleandout-of-samplepredictivepoweroftheprobitmodelwhenconsideredjointlywiththetermspreadattheselongerhorizons.Thishighlightstheimportanceofnancialintermediarybalancesheetconditionsinthetransmissionofeconomicshocks(see,forexample,AdrianandShin(2010)andAdrian,Moench,andShin(2010)).WhiletheimportanceofnancialintermediaryleverageforthepricingofriskhasbeenempiricallydocumentedbyAdrian,Etula,andMuir(2012)andAdrian,Moench,andShin(2013),tothebestofourknowledge,itsusefulnessforthepredictabilityofrecessionshasnotpreviouslybeenstudied.Athorizonsshorterthanoneyearahead,wendthataddingsix-monthlaggedobservationsoftheTreasurytermspreadsignicantlyimprovesthepredictivepoweroftheprobitmodel2 1IntroductionAccuratelypredictingbusinesscycleturningpoints,andinparticularimpendingeconomicrecessions,isofgreatimportancetohouseholds,businesses,investorsandpolicymakersalike.Priorresearchhasdocumentedthatavarietyofeconomicandnancialvariablescontainpre-dictiveinformationaboutfuturerecessionsintheUnitedStates.Mostprominently,EstrellaandHardouvelis(1991)andEstrellaandMishkin(1998)havedocumentedthattheslopeofthetermstructureofTreasuryyieldshasstrongpredictivepowerforUSoutputgrowthandUSrecessionsathorizonsuptoeightquartersintothefuture.Othervariablesthathavebeenconsideredasleadingrecessionindicatorsincludestockprices(EstrellaandMishkin(1998)),theindexofLeadingEconomicIndicators(StockandWatson(1989),BergeandJorda(2011)),creditmarketactivity(Levanon,Manini,Ozyildirim,Schaitkin,andTanchua(2011)),aswellasvariousemploymentandinterestratemeasures(Ng(2014)).Inthispaper,wereassessthepredictabilityofUSrecessionssince1959usingawidevarietyofleadingindicatorvariablesthathavebeenconsideredintheacademicandpractitionerliterature.Consistentwithmostofthepriorliterature,weusethebusinesscycledatingchronologyprovidedbytheNationalBureauofEconomicResearch(NBER)asthebench-markseriesofbusinesscycleturningpoints.WhiletheNBERrecessionindicatorisabinaryvariable,mostleadingindicatorshavecontinuousdistributions.Thus,muchoftheempiricalliteraturehasusedthenonlinearprobitmodeltomapchangesinpredictorvariablesintorecessionforecasts,andwefollowthistradition.Theprobabilityofarecessionimpliedbytheprobitmodelisrarelyexactlyzeroorone.Thus,acutoisusuallyadoptedsuchthatapredictedprobabilityabovethecutoisclassiedasarecession.Inordertoobjectivelyevaluatethemodel'sabilitytocategorizefuturetimeperiodsintorecessionsversusexpansionsoveranentirespectrumofdierentcutos,oneneedstocomplementtheprobitmodelwithaclassicationscheme.Aclassicationschemethathaslongbeenusedinthestatisticsliteraturebuthasonlyrecentlyfounditswayintoeconomicresearchisthereceiveroperatingcharacteristic(ROC)curve(see,forexample,1 Levanon,G.,J.-C.Manini,A.Ozyildirim,B.Schaitkin,andJ.Tanchua(2011):\UsingaleadingcreditindextopredictturningpointsintheU.S.businesscycle,"Eco-nomicsProgramWorkingPapers11-05,TheConferenceBoard,EconomicsProgram.Moore,G.H.,andJ.Shiskin(1967):IndicatorsofBusinessExpansionsandContrac-tions,NBERBooks.NationalBureauofEconomicResearch.Ng,S.(2014):\Viewpoint:Boostingrecessions,"CanadianJournalofEconomics,47(1),1{34.Nyberg,H.(2010):\Dynamicprobitmodelsandnancialvariablesinrecessionforecast-ing,"JournalofForecasting,29(1-2),215{230.Peterson,W.W.,andT.G.Birdsall(1953):\TheTheoryofSignalDetectability:PartI.TheGeneralTheory,"TechnicalReport13,ElectronicDefenseGroup.Rudebusch,G.D.,andJ.C.Williams(2009):\Forecastingrecessions:Thepuzzleoftheenduringpoweroftheyieldcurve,"JournalofBusiness&EconomicStatistics,27(4),492{503.Schrimpf,A.,andQ.Wang(2010):\Areappraisaloftheleadingindicatorpropertiesoftheyieldcurveunderstructuralinstability,"InternationalJournalofForecasting,26(4),836{857.Stock,J.H.,andM.W.Watson(1989):\Newindexesofcoincidentandleadingeconomicindicators,"inNBERMacroeconomicsAnnual1989,Volume4,NBERChapters,pp.351{409.NationalBureauofEconomicResearch,Inc.Wright,J.H.(2006):\Theyieldcurveandpredictingrecessions,"Discussionpaper,BoardofGovernorsoftheFederalReserveSystem.32 Dueker,M.J.(1997):\StrengtheningthecasefortheyieldcurveasapredictorofU.S.recessions,"St.LouisFedReview,(Mar),41{51.Estrella,A.,andG.A.Hardouvelis(1991):\Thetermstructureasapredictorofrealeconomicactivity,"TheJournalofFinance,46(2),555{576.Estrella,A.,andF.S.Mishkin(1996):\TheyieldcurveasapredictorofU.S.reces-sions,"CurrentIssuesinEconomicsandFinance,2(7),1{6. (1998):\PredictingU.S.recessions:Financialvariablesasleadingindicators,"TheReviewofEconomicsandStatistics,80(1),45{61.Estrella,A.,andM.R.Trubin(2006):\Theyieldcurveasaleadingindicator:Somepracticalissues,"CurrentIssuesinEconomicsandFinance,12(5),1{7.Hanley,J.A.,andB.J.McNeil(1982):\Themeaninganduseoftheareaunderareceiveroperatingcharacteristic(ROC)curve,"Radiology,143(1),29{36. (1983):\Amethodofcomparingtheareasunderreceiveroperatingcharacteristiccurvesderivedfromthesamecases,"Radiology,148(3),839{843.Jorda,O.,andA.M.Taylor(2011):\Performanceevaluationofzeronet-investmentstrategies,"Discussionpaper,NationalBureauofEconomicResearch. (2012):\Thecarrytradeandfundamentals:NothingtofearbutFEERitself,"JournalofInternationalEconomics,88(1),74{90.Khandani,A.E.,A.J.Kim,andA.W.Lo(2010):\Consumercreditriskmodelsviamachine-learningalgorithms,"JournalofBanking&Finance,34(11),2767{2787.Lahiri,K.,G.Monokroussos,andY.Zhao(2013):\TheyieldspreadpuzzleandtheinformationcontentofSPFforecasts,"EconomicsLetters,118(1),219{221.31 ReferencesAdrian,T.,E.Etula,andT.Muir(2012):\Financialintermediariesandthecross-sectionofassetreturns,"JournalofFinance,forthcoming.Adrian,T.,E.Moench,andH.S.Shin(2010):\Macroriskpremiumandintermediarybalancesheetquantities,"IMFEconomicReview,58(1),179{207. (2013):\Leverageassetpricing,"StaReports625,FederalReserveBankofNewYork.Adrian,T.,andH.S.Shin(2010):\Liquidityandleverage,"JournalofFinancialIn-termediation,19(3),418{437.Berge,T.J.,andO.Jorda(2011):\Evaluatingtheclassicationofeconomicactivityintorecessionsandexpansions,"AmericanEconomicJournal:Macroeconomics,3(2),246{277.Chauvet,M.,andS.Potter(2005):\Forecastingrecessionsusingtheyieldcurve,"JournalofForecasting,24(2),77{103.Chesher,A.(1984):\Improvingtheeciencyofprobitestimators,"TheReviewofEco-nomicsandStatistics,66,523{527.Conniffe,D.,andD.O'Neill(2011):\EcientProbitEstimationwithPartiallyMiss-ingCovariates,"AdvancesinEconometrics,27,209{245.Croushore,D.,andK.Marsten(2014):\Thecontinuingpoweroftheyieldspreadinforecastingrecessions,"WorkingPapers14-5,FederalReserveBankofPhiladelphia.Duarte,A.,I.A.Venetis,andI.Paya(2005):\Predictingrealgrowthandtheprob-abilityofrecessionintheEuroareausingtheyieldspread,"InternationalJournalofForecasting,21(2),261{277.30 Figure6:24mOut-of-sampleProbabilitiesandROCCurves.Theguresaboveshow,atthe24-monthforecasthorizon,theprobabilityofrecessionandthecorrespondingROCcurveforthespread-onlymodel(blueline),thespreadandlaggedspreadmodel(greenline),andoneadditionalmodelwithbestperformanceasdeterminedbytheAUROC(redline).ThemodelisestimatedovertheperiodofJanuary1959toAugust1985,andthecalculationofforecastaccuracyisestimatedovertheperiodofSeptember1985toDecember2011.Intherstchart,weshowtheprobabilityofrecessionasadecimaloveroursampleperiod,withtheactualrecessionperiodsshadedingrey.FollowingthischartisaROCcurvethatplots,foreachmodel,thetradeobetweenfalsepositiverates(x-axis)andtruepositiverates(y-axis).29 Figure5:12mand18mOut-of-sampleProbabilitiesandROCCurves.Formoreinformation,seeFigure6below.28 Figure4:3mand6mOut-of-sampleProbabilitiesandROCCurves.Formoreinformation,seeFigure6below.27 Figure3:24mIn-sampleProbabilitiesandROCCurves.Theguresaboveshow,atthe24-monthforecasthorizon,theprobabilityofrecessionandthecorrespondingROCcurveforthespread-onlymodel(blueline),thespreadandlaggedspreadmodel(greenline),andoneadditionalmodelwithbestperformanceasdeterminedbytheAUROC(redline).EachmodelisestimatedusingmonthlydatafromJanuary1959toDecember2011.Intherstchar,weshowtheprobabilityofrecessionexpressedasadecimaloveroursampleperiod,withtheactualrecessionperiodsshadedingrey.FollowingthischartisaROCcurvethatplots,foreachmodel,thetradeobetweenfalsepositiverates(x-axis)andtruepositiverates(y-axis).26 Figure2:12mand18mIn-sampleProbabilitiesandROCCurves.Formoredetails,seeFigure3below.25 Figure1:3mand6mIn-sampleProbabilitiesandROCCurves.Formoreinforma-tion,seeFigure3below.24 Table3:Out-of-samplesummaryofAUROCs.Thetablebelowshows,foreachforecasthorizon,theresultingareaoftheROCcurvefortheout-of-samplemodelsusingspread-onlyandspread-with-lagged-spreadmodels.Theremainingthreemodelsshowthetopperformingvariableswhenaddedtoaspreadandlaggedspreadmodel.Twosamplet-statisticscomparingcurrentmodeltothemodelwithspreadonly(\T-test1")andtothemodelwithspreadandlaggedspread(\T-test2")arereportedinthelasttwocolumns.TheestimationsampleisfromJanuary1959toAugust1985andtheforecastingsamplefromSeptember1985toDecember2011.***,**,and*denotesignicanceatthe1,5,and10%condencelevels,respectively. ModelAUROCT-test1T-test2 PanelA:3monthsaheadSpread(t)only0.562||Spread(t)+spread(t-6)0.7654.341***|S&P500,1y%chg0.96312.830***7.743***Michiganconsumersurvey0.94112.706***7.320***Debitmargins(BD)0.93311.920***6.458*** PanelB:6monthsaheadSpread(t)only0.674||Spread(t)+spread(t-6)0.7943.219***|S&P500,1y%chg0.9008.069***4.658***Debitmargins(BD)0.8626.826***3.407***5yr-FFspread0.8596.168***2.667*** PanelC:12monthsaheadSpread(t)only0.858||Spread(t)+spread(t-6)0.8831.152|5yr-FFspread0.9022.785***1.3651yr-FFspread0.8972.669***1.102NAPMcomprice0.8971.5550.610 PanelD:18monthsaheadSpread(t)only0.906||Spread(t)+spread(t-6)0.881-1.014|Debitmargins(BD)0.9341.4232.615***2yr-FFspread0.897-0.6611.18830yr-FFspread0.894-1.0220.907 PanelE:24monthsaheadSpread(t)only0.853||Spread(t)+spread(t-6)0.808-1.214|1yr-FFspread0.846-0.6131.762*2yr-FFspread0.827-0.9951.227Baa-Aaaspread0.801-1.906-0.298 23 Table2:In-samplesummaryofAUROCs.Thetablebelowshows,foreachforecasthorizon,theresultingareaoftheROCcurveforthein-samplemodelsusingspread-onlyandspread-with-lagged-spreadmodels.Theremainingthreemodelsshowthetopperformingvariableswhenaddedtoaspreadandlaggedspreadmodel.Twosamplet-statisticscompar-ingcurrentmodeltothemodelwithspreadonly(\T-test1")andtothemodelwithspreadandlaggedspread(\T-test2")arereportedinthelasttwocolumns.ThesampleperiodisJanuary1959toDecember2011.***,**,and*denotesignicanceatthe1,5,and10%condencelevels,respectively. ModelAUROCT-test1T-test2 PanelA:3monthsaheadSpread(t)only0.672||Spread(t)+spread(t-6)0.8485.518***|S&P500,1y%chg0.9478.679***4.137***Michiganconsumersurvey0.9297.959***3.523***Debitmargins(BD)0.9277.927***3.335*** PanelB:6monthsaheadSpread(t)only0.775||Spread(t)+spread(t-6)0.8673.484***|5yr-FFspread0.9245.239***2.754***Michiganconsumersurvey0.9195.220***2.582***Buildingpermits0.9175.137***2.461** PanelC:12monthsaheadSpread(t)only0.865||Spread(t)+spread(t-6)0.8821.038|5yr-FFspread0.9022.077**1.5771yr-FFspread0.8981.862*1.281NAPMcomprice0.8961.6190.848 PanelD:18monthsaheadSpread(t)only0.811||Spread(t)+spread(t-6)0.811-0.005|Debitmargins(BD)0.8622.054**2.056**NAPMcomprice0.8341.1831.187ISMneworderindex0.8260.8110.796 PanelE:24monthsaheadSpread(t)only0.693||Spread(t)+spread(t-6)0.6940.039|Debitmargins(BD)0.7862.849***2.829***Neworders,non-defense0.7712.519**2.497**NAPMcomprice0.7682.290**2.316** 22 Table1:SummaryofKeyVariablesThistablereportsallofthepredictorvariablesconsideredinouranalysis.Foreachindicator,wereporttheseries'name,thetransformationweperformedbeforeusingitinouranalyses,thedatasource,andthetimespanforwhichtheseriesisavailable.Transformationcodes1-6correspondtolevels,monthlylogdierence,annuallogdierence,annualdierence,6-monthsmovingaveragesmoother,and12-monthsmovingaveragesmoother,respectively.ThedatasourcesUSECON,BCI,andALFREDrefertotheU.S.EconomicsStatisticsdatabaseinHaverAnalytics,theBusinessCycleIndicatorsdatabaseinHaverAnalytics,andtheonlineArchivaLFederalReserveEconomicDatabaseattheSt.LouisFed,respectively.*denotesmacroeconomicindicatorsforwhichwereal-timedataextendingpast1985arenotavailableandwhicharethusexcludedfromourout-of-sampleanalysis. SeriesNameCodeSourceTimeSpan 10y-3mspd1USECONJan1959-Dec201110yrate1USECONJan1959-Dec20113mrate1USECONJan1959-Dec2011S&P500,1y%change1USECONJan1959-Dec2011S&P500,3y%change1USECONJan1959-Dec2011Leadingcreditindex1BCIJan1959-Dec2011Michiganconsumersurvey1BCIJan1978-Dec2011Debitmargins(BD)1BCIJan1960-Dec2011Bearlessbull4BCIJul1987-Dec2011LIBOR3month1USECONJan1963-Dec2011Baa-Aaaspread1USECONJan1959-Dec2011Aaa-FFspread1USECONJan1959-Dec2011Baa-FFspread1USECONJan1959-Dec20113mo-FFspread1USECONJan1959-Dec20116m-FFspread1USECONJan1959-Dec20111yr-FFspread1USECONJan1959-Dec20112yr-FFspread1USECONJun1976-Dec20115yr-FFspread1USECONJan1959-Dec201110yr-FFspread1USECONJan1959-Dec201130yr-FFspread1USECONMar1977-Dec2011Exrate:Japan2USECONJan1959-Dec2011Avgwklyhrs(manufacturing)1ALFREDJan1959-Dec2011Avginitialclaims*3BCIJan1959-Dec2011Neworders,goods,materials*3BCIJan1959-Dec2011Neworders,non-defense*3BCIJan1959-Dec2011ISMneworderindex*1BCIJan1959-Dec2011Buildingpermits*3BCIJan1959-Dec2011Emp:total5ALFREDJan1959-Sep2010Emp:govt5ALFREDJan1959-Dec2011Emp:mfg6ALFREDJan1959-Dec2011Emp:mining5ALFREDJan1959-Dec2011NAPMcomprice1ALFREDJan1959-Dec2011NAPMvendordel*1USECONJan1959-Dec2011NAPMinvent*1USECONJan1959-Dec2011 21 5SummaryofFindingsandConcludingRemarksInsum,ourresultsimplythefollowingmaintakeaways.First,consistentwiththepastliterature,wendthatourabilitytoimproveuponthespread-onlymodeldropsatlongerhorizonsoftwelvemonthsorgreater.However,addingthe5-yearTreasury-fedfundsratespreadsignicantlyimprovesboththein-andout-of-sampleforecastsatthetwelve-monthsaheadhorizon.Moreover,ourresultsindicatethataddingthelaggedtermspreadandmar-gindebitsatbroker-dealerssignicantlyimproveboth18-and24-monthsaheadin-sampleforecasts.Second,thereisvaluableinformationnotonlyinthecontemporaneousTreasurytermspreadbutalsoinitsdynamics.Morespecically,onecandrasticallyincreasetherecessionpredic-tionabilityforout-of-sampleforecastsbyaddinglaggedobservationsoftheTreasurytermspreadatshortforecasthorizons.Infact,addingsix-monthslaggedobservationsoftheTrea-surytermspreadessentiallymovesthemodelfromonethatislittlebetterthanarandomguesstoaveryaccurateone.Atlongerhorizons,theforecastingabilityisgenerallyworseacrossmodelspecications.Forinstance,intheout-of-sampleforecastanalysesathorizonslongerthantwelvemonths,thepredictiveabilitydecreaseswhenthelaggedspreadisaddedtotheprobitmodel.Third,wendthatmargindebitatbroker-dealersisausefulleadingindicator.Tothebestofourknowledge,thishasnotbeenappreciatedinthepreviousacademicliterature.Modelswhichaddthemargindebitvariableconsistentlyrankamongthetopthreemodelsforthethree-,18-,and24-monthsaheadin-sampleestimations,alwayssignicantlyoutperformingthespread-onlymodel.Inaddition,wendthatmodelswiththisvariablerankamongthetopthreeforthethee-,six-,and18-monthaheadhorizonsfortheout-of-sampleestimations.Asmargindebitatbroker-dealersistypicallyconsideredtobeameasureofleverageinthenancialsystem,itsimportanceinpredictingrecessionshighlightstheroleofnancialintermediarybalancesheetmanagementinthetransmissionofeconomicshocks.20 lagofthetermspreadslightlyincreasestheAUROCto0.88butthedierenceisnotfoundtobestatisticallysignicant.Consistentwiththein-sampleanalysis,wendthatthebestthreeadditionalindicatorsrankedbydecreasingimportancearethe5-yearTreasuryyield-fedfundsratespread,the1-yearTreasuryyield-fedfundsratespread,andtheNAPMcommoditypriceindex.Thesemodels'AUROC'sarealsoverysimilartotheirin-samplecounterparts,and,again,wendthatwecansignicantlyimproveuponthespread-onlymodelbyaddingthe5-or1-yearTreasuryyield-fedfundsratespreads.Atthe18-monthsaheadhorizon,thespread-onlymodelperformsquitewellwithanAUROCof0.91,evenbetterthanatthe12-monthsaheadhorizon.Thisisalsoremarkablybetterthanitsin-samplecounterpart,whichonlyhasanAUROCof0.81.Whileothermodelsperformbetterthanthespread-onlymodel,wendtheirimprovementtobestatisticallyinsignicantatconventionallevels.Unliketheshorterforecasthorizons,wendthataddingthelaggedspreadactuallydecreasestheAUROCto0.88.However,addingmargindebitatbroker-dealers,theISMnewordersindex,oraverageinitialclaimsslightlyimprovestheAUROCto0.93,0.91,and0.90respectively.Thatsaid,onlythebroker-dealervariableisfoundtosignicantlyimprovethepredictivepowerofthemodelwithrespecttothespreadandlaggedspreadmodel.However,whencomparedtothespread-onlymodeltheimprove-mentisstatisticallyinsignicant.Finally,atthe24-monthsaheadhorizon,wendthatthespread-onlymodelhasanimpres-siveAUROCof0.85.Infact,noothermodelperformsbetter.AddingthelaggedspreaddecreasestheAUROCto0.81.ThebestmodelwithanadditionalpredictoristheNAPMinventoriesindex,whichdeliversanAUROCof0.85.Comparingtothein-sampleanalysis,wendthattheout-of-sampleforecastsperformevenbetterthantheirin-samplecounter-partsatthe24-monthsaheadhorizon.Infact,thebestmodelinthein-sampleanalysishasanAUROCof0.79,whichislowerthanthecorrespondingout-of-sampleestimation.19 line),thespreadandlaggedspreadmodel(greenline),andthebestperformingmodelwhichaugmentsthespreadandlaggedspreadmodelwithanadditionalpredictorvariable(redline).Atthethree-monthsaheadhorizon,weseethatthespread-onlymodelislittlebetterthanarandomguess,withanAUROCofonly0.56.Thisperformanceisnotablyworsethanthein-samplemodel,whichhasanAUROCof0.67.Onepossibleinterpretationofthisndingisashiftinthepredictivepowerofthetermspreadforrecessionsaroundtheendofourtrainingsamplein1985.Thisisconsistentwithpriorevidenceforstructuralchangeinthepredictiverelationshipbetweenthetermspreadandfutureoutputgrowth(see,forexample,SchrimpfandWang(2010)).Thatsaid,addingasix-monthlagofthetermspreadimprovestheAUROCdramaticallyto0.77withanaccompanyingt-statisticof4.34.Inlinewiththein-sampleanalysis,wendthattheannualreturnontheS&P500index,theMichigansur-veyofconsumersentiment,andmargindebitatbroker-dealersarethethreebestperformingadditionalvariables.Theyalsosignicantlyimproveuponthespreadandlaggedspreadmodel.Thebestmodel,usingtheannualreturnontheS&P500index,hasanear-perfectpredictiveabilityandanAUROCof0.96.Next,atthesix-monthsaheadhorizon,wendthatthespread-onlymodelperformsworsethanitsin-samplecounterpartwithanAUROCof0.67.Again,addingthesix-monthslaggedspreadsignicantlyimprovesthemodel'spredictiveability,raisingitsAUROCto0.79.Sim-ilartothethree-monthsaheadhorizon,wendthattheannualreturnontheS&P500indexandmargindebitatbroker-dealersareusefulleadingindicators.Moreover,inlinewiththein-sampleanalysisatthishorizon,theremainingselectedindicatoristhe5-yearTreasuryyield-fedfundsratespread.WhilethebestadditionalindicatoristheannualreturnontheS&P500index,withasizableAUROCof0.90,allofthetopthreemodelswithadditionalindicatorssignicantlyoutperformthespreadandlaggedspreadmodel.Atthe12-monthsaheadhorizon,thespread-onlymodelhasanAUROCof0.86.Thisisverysimilartoitsin-samplecounterpart,whichhasanAUROCof0.87.Addingasix-month18 spread-onlymodelhasanAUROCof0.69,whichmakesitcomparabletoitscounterpartatthethree-monthsaheadhorizon(0.67).Addingthelaggedspreadimprovesthepredictionabilitybutonlymarginallyso.Ontheotherhand,wealsondthattheadditionofmar-gindebitatbroker-dealers,newnon-defenseorders,ortheNAPMcommoditypriceindexsignicantlyimprovesthetwobaselinemodels.AllthreeperformaboutequallywellwithAUROC'srangingfrom0.79to0.77.Whilethesearenotstrongpredictiveabilities,theyarehigherthansimilarmodelsestimatedjustusingcomponentsoftheLEI,whichareconsideredthebenchmarkleadingindicators(seeBergeandJorda(2011)).Importantly,theprobitmissestimatorbyConnieandO'Neill(2011)allowsustondstrongandsignicantforecastingvalueinvariableswhichhavemissingobservationsandwhichmayhavebeenexcludedfromouranalysisotherwise.Morespecically,thebroker-dealermargindebitvariableandtheMichigansurveyofconsumerexpectationsbothstartlaterthanJan-uary1959whichmarksthebeginningofoursample,yetwendthattheyareoftenusefulinimprovingthein-samplerecessionforecastingability.4.2Out-of-sampleAnalysisTheresultsofourbaselineandbestperformingout-of-sampleprobitregressionsaresum-marizedinTable3.Asbefore,thetableisorganizedinvepanelscorrespondingtothevedierentforecasthorizons.EachpanelshowstheAUROCforthespread-onlymodel,thespreadandlagged-spreadmodel,andthethreebestmodelsobtainedbyaddingathirdpredictorvariabletothespreadandlaggedspreadbaselinemodel.Foreachmodel,wereportitsAUROCaswellasitst-statisticwhencomparedtobothbaselinemodels.Figures1-3summarizethendingsinTable3visually.Again,theplotsaregroupedbypairsdependingontheforecasthorizon,wheretherstgraphshowsthepredictedprobabilityofrecessionovertimeandthesecondgraphshowsthecorrespondingROCgraphscalculatedbycomparingthepredictedprobabilitywiththeNBERbusinesscyclechronology.Ineachofthelattergraphs,thethreelinesrepresenttheestimatesfromthespread-onlymodel(blue17 icantlyimprovestherecessionclassicationabilityoftheprobitmodel,raisingtheAUROCfrom0.78to0.87.Inaddition,theMichiganconsumersurveycontinuestobeoneofthetopadditionalindicators.Theothertwoarethe5-yearTreasuryyield-fedfundsratespreadandbuildingpermits.Again,wendthatonecansignicantlyimproveuponamodelwithspreadandlaggedspreadbyaddinganyofthesethreeadditionalindicators.Hence,thereispredictiveinformationintheseothervariablesbeyondthatcapturedbytheTreasurytermspread.Atthetwelve-monthsaheadhorizon,thespread-onlymodelperformsremarkablybetterthanattheshorterhorizonswithanAUROCof0.87.Whileaddingthelaggedspreadimprovesthepredictiveabilitysomewhat,itdoesnotdososignicantly.Similartothesix-monthshorizon,wendthatthemodelwiththe5-yearTreasury-FFspreadperformsthebest,im-provinguponthespread-onlymodelwithanAUROCof0.90andaonlyat-statisticof2.08.Thetwonextbestmodelsusethe1-yearTreasury-FFspreadandtheNAPMcommoditypriceindex,respectively.Turningtothe18-monthsaheadforecasthorizon,weseethatthepredictiveabilityofthespread-onlymodelwithanAUROCof0.81isslightlyworsethanatthetwelve-monthsaheadhorizonbutbetterthanthethree-andsix-monthsaheadhorizons.Giventheseresultswendthat,notsurprisingly,addingasix-monthlaggedspreadactuallymarginallyhurtsthepredictiveabilityofthemodelinsteadofimprovingit.Similartothetwelve-monthsaheadhorizon,theNAPMcommoditypriceindexisamongthetopthreeadditionalpredictorvariables.Itisoutperformedonlybythemodelusingmargindebitatbroker-dealersasanadditionalpredictor.Infact,brokerdealermargindebitistheonlypredictorvariablethatsignicantlyincreasestheAUROCwhenaddedtothetwobaselinemodels.Thisisintriguinggiventhatthepreviousliteratureisgenerallyinconsensusthatatthetwelve-andeighteen-monthsaheadhorizon,thespread-onlymodelperformsthebest.Finally,atthe24-monthsaheadhorizon,weseethatpredictiveabilityisgenerallymuchlowerforallmodelsalthoughstillquiteabitbetterthanasimplerandomguessmodel.The16 4.1In-sampleAnalysisTheresultsofourin-sampleprobitregressionsaresummarizedinTable2.Thetableisbro-kendownintopanelsAthroughE,correspondingto3-,6-,12-,18-,and24-monthsaheadforecasthorizons,respectively.Eachpanelreportsresultsfromthespread-onlymodel,aspreadandsix-monthlaggedspreadmodel,andthreeadditionalmodels.Thesethreeaddi-tionalmodelsusethetermspread,thelaggedtermspread,andoneofthreebest-performingadditionalindicatorsasdeterminedbytheAUROCmetric.Foreachmodel,wereportitsAUROCaswellasitst-statisticwhencomparedtothetwobaselinemodels:spread-onlyandspreadaugmentedwithsix-monthlaggedspread.Figures1-3summarizethendingsinTable2visually.Inthesegures,theplotsarepairedbyforecasthorizon.Thetopgraphshowsthepredictedprobabilityofrecessionovertime,withactualrecessionsshownasshadedgreyareas.Thesecondgraphshowsthecorrespond-ingROCcurvescalculatedasdescribedinSection2.Ineachgraph,thethreelinesshownrepresenttheestimatesfromthespread-onlymodel(blueline),thespreadandlaggedspreadmodel(greenline),andoneadditionalmodelwithbestperformanceasdeterminedbytheAUROC(redline).Atthethree-monthsaheadhorizon,wendthatwecansignicantlyimprovethespread-onlymodelbysimplyaddingasix-monthlagofthetermspread.Infact,doingsoincreasestheAUROCfrom0.67,whichisonlyslightlybetterthanarandomguess,to0.85,whichisquiteaccurate.ThetwoAUROC'saresignicantlydierentatthe1%level,withat-statisticof5.5.Furthermore,wecanimprovethespreadandlagged-spreadmodelbyincludingoneofmanyadditionalindicators.ThebestoneistheannualreturnontheS&P500index,whichhasanear-perfectAUROCof0.95andat-statisticof4.1whencomparedtothespreadandlaggedspreadmodel.Theothertwobest-performingadditionalindicatorsaretheMichiganconsumercondencesurveyanddebitbalancesatmarginaccountsatbrokerdealers,whichhaveAUROC'sof0.929and0.927respectively.Atthesix-monthsaheadhorizon,weagainseethataddingasix-monthlaggedspreadsignif-15 dertheseriesstationary.Thesetransformationsdirectlyfollowthoseusedinthepaperscitedabove.Sincemacroeconomicvariablesareoftenrevisedaftertheirreleasedatesandtheserevisionsarenotavailabletotheeconomistforout-of-sampleforecasting,weusereal-time,ortherst-release,indicatorswheneverpossible.Whenavailable,thereal-timedataiscollectedfromtheALFREDdatabase.Forseveralmacroeconomicvariables,real-timedatadonotexistbefore1985.ThesevariablesaremarkedwithanasteriskinTable1andareexcludedfromourout-of-sampleanalysis.4ResultsInthissection,wedescribeourresultsfromcomparingvariousprobitmodelspecicationsatdierenthorizons.Wedivideouranalysisintoin-sampleversusout-of-sampleforecasts.In-sampleforecastsareestimatedovertheentiresamplefromJanuary1959toDecember2011.Out-of-sampleforecastsareestimatedusingthersthalfofthesample,fromJanuary1959toAugust1985,andthenevaluatedusingthesecondhalfofthesample,fromSeptember1985toDecember2011.Inbothexercises,weconsiderforecastsatthe3-,6-,12-,18-,and24-monthsaheadhorizons.Ateachhorizon,webeginbyestimatingabaselineprobitmodelusingjustthetermspreadasexplanatoryvariable(EstrellaandHardouvelis(1991)).Then,weaugmentthebenchmarkmodelwiththesix-monthlaggedtermspreadasanadditionalexplanatoryvariable.Wedosoinordertoassesswhetheritisthecontemporaneouslevelorthechangeinthespreadthatisimportantforpredictingrecessions.Moreover,thisallowsustotestwhethertheaddedleadingindicatorscontainpredictiveinformationoverandabovethatcapturedinthetermspread.Finally,oneatatime,weaddeachofthevariablesshowninTable1tothespreadandlaggedspreadmodels.WeusetheAUROCtoevaluatetheperformanceofeachmodelandalsotocomparetheirforecastingabilitiestothetwobaselinemodelsusingthet-statisticpresentedinSection2.14 fensecapitalgoodsexcludingaircraft(\Neworders,non-defense");buildingpermits,newprivatehousingunits(\Buildingpermits");therateofthe10-yearTreasurynotelessfederalfunds(\10yr-FFspread");averageconsumerexpectations(\Michiganconsumersurvey");andtheLeadingCreditIndex.ThelatterwasintroducedbytheConferenceBoardtosupplementtheLeadingEconomicIndexandre ectpotentialstructuralchangesinthechangingcreditandnancialmarkets.ItscomponentswereselectedinthespiritofnancialintermediationmodelsaslaidoutbyAdrianandShin(2010)andaretestedfortheirabilitytosignalbusinesscyclechangesusingaMarkovSwitchingmodel(seeLevanon,Manini,Ozyildirim,Schaitkin,andTanchua(2011)).WeincludeeachoftheLCI'scomponentsoeredatamonthlyfrequencyorgreatergoingbacktoatleast1985:theLIBOR3-monthless3-monthTreasurybillyieldspread(\LIBOR3month"),balancesinBroker-Dealermarginaccounts(\Debitmargins(BD)"),andtheAAIISentimentSurvey'sMarketSurveyofthespreadbetweenbearishandbullishsentiments(\Bearlessbull").Finally,weaddanumberofpredictorvariablesfollowingtherecentndingsofNg(2014).Inherpaper,Ngexaminesacomprehensivelistof132dierentrealandnancialindica-torsandassessestheirrelevanceforpredictingrecessions.Weincludeallofthe14uniquevariablesfoundtobemostimportantusingcross-validationboostingtechniquesaswellasthe4additionaluniquevariablesfoundthrougharollingwindowexercise.ThislistincludesthespreadsbetweentheyieldsofseveralconstantmaturityTreasurieswiththefedfunds(FF)rate(\3m/6m/1yr/2yr/5yr/30yr-FFspread");employmenthoursfortotalnon-farm,government,manufacturing,andmining(\Emp:total/govt/mfg/mining");NAMPindexofconsumercommodityprices(\NAPMcomprice");NAPMvendordeliveries(\NAPMven-dordel");NAPMinventoriesindex(\NAPMinvent");andtheUS$-Yenexchangerate(\Exrate:Japan").Table1providesalistofallthepredictorvariablesthatweconsider,theiravailabletimespan,thedatabasefromwhichtheyoriginate,andthetransformationsthatweusedtoren-13 USbusinesscycles.Moreover,asdiscussedintheintroduction,BergeandJorda(2011)ndnoimprovementofanumberofsophisticatedparametricmodelsovertherecessionclassi-cationabilityoftheNBER.Forin-samplerecessionprediction,oursamplerunsfrom1959to2011andcoversatotalofsevenrecessions,whichrangeindurationfromsixto18months.Wealsoconductout-of-sampleforecasts,usingtheperiodofJanuary1959toAugust1985(whichcoversatotaloffourrecessions)asourtrainingsample,andtheperiodfromSeptember1985throughDecem-ber2011(coveringthreerecessions)asourforecastingsample.OurexplanatoryvariablesaretakenfromHaverAnalytics,withtheexceptionofreal-timemacroeconmicindicators,whichcomefromtheArchivaLFederalReserveEconomicDatabase(ALFRED).FollowingEstrellaandHardouvelis(1991),EstrellaandMishkin(1998),andothers,weusethetermspread,preciselydenedasthedierencebetweentheten-yearandthethree-monthTreasuryyield(\10y-3mspd"),asourbenchmarkpredictorvariable.Wethenassesswhetheradditionallagsofthetermspreadaswellasothercandidatepredictorvariablesaddpredictivepowertothebenchmarkmodel.Basedonpriorresearch,weconsiderthefollowinglistofadditionalpredictorvariables.First,followingEstrellaandMishkin(1998),weaddseveralnancialindicatorsincludingreturnsontheS&P500commonstockpriceindex(\S&P500,1y%change",\S&P500,3y%change")andtheinterestratesofthe3-month(\3mrate")and10-yearTreasuries(\10yrate").Sec-ond,weincludeeachcomponentoftheConferenceBoard'sLeadingEconomicIndex(LEI)(seeLevanon,Manini,Ozyildirim,Schaitkin,andTanchua(2011)).Theseindicatorshavebeenselectedfortheirabilitytosignalpeaksandtroughsinthebusinesscycle,andtheaggregateindexhasbeenshowntodropaheadofrecessionsandrisebeforeexpansions.Theindividualfactorsconsistof:averageweeklyhoursofmanufacturing(\Avgwklyhrs(manufacturing)");averageweeklyinitialclaimsforunemploymentinsurance(\Avginitialclaims");manufacturer'sneworders,goods,andmaterials(\Neworders,goods,materials");theISMindexofneworders(\ISMnewordersindex");manufacturers'neworders,nonde-12 whereQ1=AUROC (2�AUROC),andQ2=2AUROC2 (1+AUROC),seeagainHanleyandMcNeil(1982).HanleyandMcNeil(1983)extendthisestimatorfurtherbydevelopingat-statisticforcom-paringAUROCsacrossmultiplemodels,takingintoaccountthecorrelationbetweenthetwoareasbeingcompared.Thet-statisticisgivenby:t=AUROC1�AUROC2 p 21+22�2r12:(11)Here,AUROC1andAUROC2aretheareasunderthecurveformodels1and2whicharebeingcompared.Similarly,21and22refertothevariancesoftheAUROCsformodel1andmodel2,respectively.Finally,risthecorrelationbetweenthetwoAUROCs.Toobtainr,oneneedstocomputetwointermediateparametersrEandrR,whicharethecorrelationsfortheexpansionaryobservationsandrecessionaryobservations,respectively,acrossthetwomodels.ThesecorrelationscanbecalculatedusingeitherPearsonproduct-momentcorrelationortheKendalltaurankcorrelationcoecient.Inourpaper,wechoosetousethelatter.SeeHanleyandMcNeil(1983)andJordaandTaylor(2011)formoredetailsontheteststatisticanditsimplementation.3DataWeusemonthlyU.S.dataforthesampleperiodJanuary1959toDecember2011.Thedependentvariableisabinaryrecessionindicatorwhichtakesonthevalueofoneduringarecessionandzeroduringanexpansion,bothasdenedbytheNBERbusinesscycledatingcommittee.Thecommitteemeetsperiodicallytojudgewhetherapeakortroughineco-nomicactivityhasoccurred,takingintoaccountavarietyofeconomicactivityindicators,includingrealGDPmeasuredontheproductandincomesides,economy-wideemployment,realincome,aswellasindicatorscoveringrealpartsoftheeconomy,suchasretailsalesandindustrialproduction.2TheNBER'sdatingrulesarewidelyregardedasthebenchmarkfor 2Seehttp://www.nber.org/cycles/recessions.html.11 6.Afteracoordinateiscreatedforeachthreshold,plotthecoordinatesacrossallthresh-oldswherethefalsepositiverateisonthex-axisandthetruepositiverateisonthey-axis.ConnectthesecoordinatestotraceouttheROCcurve.Insummary,theROCcurvepinpointsthepercentoffalsenegativesonewouldhavetotradeforoneadditionalpercentoftruepositives.Amodelwith100%accuracywoulddrawaROCcurvehuggingthetopleftcorner.Amodelwhichistheequivalentofarandomguesswouldfollowa45%diagonalthatrunsfromthebottom-leftcornertothetop-rightcorner.Byconstruction,ifwedenedXtintermsofexpansions(i.e.letXtequaloneduringexpansionsandzerootherwise)insteadofrecessions,thenewcurvewouldlooksymmetrictotheoldcurveabouta45degreelinefromthebottom-rightcornertothetop-leftcorner.Theareaunderthecurve,bygeometry,wouldthenremainexactlythesameasbefore.Duetoitseaseofapplicationandintuitivevisualinterpretation,theareaundertheROCcurve(AUROC)isapopularmeasureofclassicationabilityforagivenmodel.InourempiricalanalysisinSection4,wewillthereforecomparetherecessionclassicationabilityofvariousdierentprobitmodelsusingtheirimpliedAUROC's.AsdiscussedinBergeandJorda(2011),asimplenonparametricestimateoftheAUROCisgivenby\AUROC=1 nRnEnEXi=1nRXj=1I(Zi�Xj)+1 2I(Zi=Xj);(10)whereI(
)istheindicatorfunction,XaretheobservationsclassiedtobearecessionaryperiodandZaretheobservationsclassiedtobeanexpansionaryperiod.nRandnEarethetruenumbersofrecessionaryandexpansionaryperiods,respectively.Wecanassessthestatisticalsignicanceofamodel-impliedAUROCusingtheasymptoticstandarderrorderivedbyHanleyandMcNeil(1982).Thevarianceisgivenby:2=1 nRnEAUROC(1�AUROC)+(nR�1)(Q1�AUROC2)+(nE�1)(Q2�AUROC2)1=2;10 probabilityofrecession,givenbytheprobitmodel,where0Pt1:2.Deneevenlyspacedthresholds(denotedC)alongtheinterval[0,1].Alargernum-berofthresholdsleadstoasmootherROCcurvewithmorepoints.Forexample,apotentialsetwith50thresholdswouldbe:Ci=f0,0.05,...0.95,1g.3.Foreachgiventhreshold,Ci,recordthemodel'spredictedcategories.Morespeci-cally,denethepredictedcategorizationofXt,or^Xt,inthefollowingway:^Xt=8�&#x]TJ ;� -2;
.52;&#x Td ;&#x[000;:1;ifPtCi0;ifPtCi(7)4.ComparingthetrueXttopredictedcategorizations^Xt,calculatethepercentageoftruepositives(PTP)andpercentageoffalsepositives(PFP).Morespecically,theycanbedenedusingthesumoftwoindicatorvariables:PTP=1 nRNXt=1Itpt;whereItpt=8�&#x]TJ ;� -2;
.52;&#x Td ;&#x[000;:1;ifXt=1and^Xt=10;otherwise(8)PFP=1 nENXt=1Ifpt;whereIfpt=8�&#x]TJ ;� -2;
.51; Td;&#x [00;:1;ifXt=0and^Xt=10;otherwise(9)andwherenRisthenumberoftimesthetrueXtwasinarecessionandnEisthenumberoftimesthetrueXtwasnotinarecession,suchthatnR+nE=N,whereNisthetotalnumberofobservationsinoursample.5.ForeachCi,createasetofcoordinates:(PFPi,PTPi).9 2.2ModelSelectionPreviousresearchhasusedvariousdierentmetricstoevaluatethetofrecessionpredictionmodels.Forexample,MooreandShiskin(1967)presentanexplicitscoringsystemforbusi-nesscycleindicators,focusingonthelengthofleadbeforebusinesscyclesturns,smoothnessoftheseries,clarityofcyclicalmovements,andrelationshiptogeneralbusinessactivity,amongothercriterion.EstrellaandMishkin(1996)andEstrellaandMishkin(1998)usethepseudoR-squaredtoevaluatethetsofprobitmodels.Finally,Wright(2006)employstheBICcriteriontomeasurethetofhismodelin-sampleandrootmeansquaredforecasterrorstoevaluatethetofhisout-of-sampleforecasts.However,alloftheseevaluationmeasuresfocusonmodeltandnotspecicallyclassi-cationability,whichistheobjectofinterestinourapplication.BergeandJorda(2011)haverecentlyusedtheReceiverOperatingCharacteristic(ROC)curvetoassessthereces-sionclassicationabilityofvariousleadingindicators.TheROCcurveisausefulmeasure,becauseitpreciselycapturestheabilityofeachmodeltoaccuratelycategorizerecessionsandexpansions.Inparticular,byusingtheareaundertheROCcurve(AUROC),onecanevaluatethecategorizationabilityofthemodeloveranentirespectrumofdierentcut-osfordeterminingarecession,insteadofevaluatingpredictivepoweratanyonearbitrarythreshold.Inaseminalpaper,PetersonandBirdsall(1953)rstdevelopedthebasicROCmethod-ology.Theprocedurehasbeenwidelyusedinstatisticsandotherelds,butithasonlyrecentlyfounditswayintotheeconomicsliterature(see,forexample,Khandani,Kim,andLo(2010),JordaandTaylor(2011),andJordaandTaylor(2012)).Appliedtothecontextofpredictingrecessions,itcanbesummarizedasfollows:1.LetXt=8�&#x]TJ ;� -2;
.51; Td;&#x [00;:1;ifinrecession0;otherwise(6)denotethetrue,observedstateoftheeconomy.LetPtbethepredictionofXt,orthe8