InstituteforResearchonPovertyDiscussionPaperno.1024-93UsingSurveyParti - PDF document

InstituteforResearchonPovertyDiscussionPaperno.1024-93UsingSurveyParticipantstoEstimatetheImpactofNonparticipationI-FenLinDepartmentofSociologyUniversityofWisconsin-MadisonNoraCateSchaefferDepartmentofSociologyInstituteforResearchonPovertyUniversityofWisconsin-MadisonNovember1993ThisresearchwassupportedbygrantsfromtheNationalInstituteofChildHealthandHumanDevelopment(NICHD)(HD-24571);theOfficeoftheAssistantSecretaryforPlanningandEvaluation,U.S.DepartmentofHealthandHumanServices(91ASPE236A);andtheUniversityofWisconsinGraduateSchool.ComputingsupportcamefromtheCenterforDemographyandEcology,whichreceivescoresupportfromtheCenterforPopulationResearchofNICHD(HD-5876).Theopinionsexpressedinthepaperareourownanddonotnecessarilyreflectthoseofthesponsoringagencies. Theauthorsevaluatetheeffectivenessoftwomodelsoftenusedtomeasuretheextentofnonparticipationbiasinsurveyestimates.Thefirstmodelestablishesa"continuumofresistance"tobeingsurveyed,placingpeoplewhowereinterviewedafteronephonecallononeendandnonparticipantsontheother.Thesecondassumesthatthereare"classes"ofnonparticipantsandthatsimilarclassescanbefoundamongparticipants;itidentifiesgroupsofparticipantsthoughttobelikenonparticipantsandusesthemas"proxies"toestimatethecharacteristicsofnonparticipants.TheauthorsusethesemodelstoexaminehowaccuratelytheyestimatethecharacteristicsofnonparticipantsandtheimpactofnonparticipationonsurveyestimatesofmeansofchildsupportawardsandpaymentsinWisconsin.Theyfindthatneithermodeldetectsthetrueextentofnonparticipationbias. UsingSurveyParticipantstoEstimatetheImpactofNonparticipationWhethernonparticipationbiasesestimatesofmeansandotherparametersfromsamplesurveysdependsonhowgreatlynonparticipantsdifferfromparticipantsandonthenumberofnonparticipantsrelativetothesamplesize.Buttheeffectofnonparticipationonsamplesurveyestimatesisdifficulttodescribeinanyparticularcasebecauseresearchersrarelyhaveinformationaboutnonparticipantsand,inaddition,methodstoadjustfornonparticipationquicklybecomecomplex(e.g.,Madow,Olkin,andRubin1983).Forthesereasons,therehasbeenrecurringinterestindevisingasimple,adhocmethodforestimatinghowparticipantsandnonparticipantsdifferfromeachotherandthenusingthisinformationtoobtainatleastroughestimatesofthesizeanddirectionofnonparticipationbiasinsurveyestimatesofmeansandproportions.Twooftheadhocmethodsthathavebeensuggestedusedifferentunderlyingmodelsoftherelationshipbetweenparticipantsandnonparticipants.Onemethodassumesthatthereisa"continuumofresistance."Thismodelplacespersonswhowereinterviewedonthefirstcontactononeendofthecontinuumandnonparticipantsontheother(Filion1976;FitzgeraldandFuller1982).Asecondmethodassumesthatthereareunderlying"classes"ofnonparticipantsandthatsimilarclassescanbefoundamongsurveyparticipants.Themodelidentifiesgroupsofparticipantsthoughttobelikenonparticipantsandusesthemas"proxies"toestimatethecharacteristicsofnonparticipants(O'Neil1979;Stinchcombe,Jones,andSheatsley1981;Smith1984).Webrieflyreviewtheassumptionsunderlyingthesetwoapproachesandthenevaluatetheirlikelysuccessusingdatafromasourceexternaltoasamplesurvey. TWOMODELSRELATINGPARTICIPANTSANDNONPARTICIPANTSMethodsbasedonthe"continuum-of-resistance"modelassumethatpeoplewhoneedmorecallsorcontactstocompleteaninterviewaremorelikenonparticipantsthanarepeoplewhoareinterviewedwithfewercallsorcontacts.Ajustificationofthismodelisthatpeoplewhorequiremorecontactstobeinterviewedwouldhavebeennonparticipantsifdatacollectionhadbeenstoppedearlier.Inapplicationsofthismodel,researchershavetreatednonparticipantsasahomogeneousgroupandusedsurveyestimatesbasedonthosewhoareinterviewedinthefinalcontactstoinferthecharacteristicsofnonparticipants.Researchershaveusuallychosenthetotalnumberofcallsorcontactsasanindicatorofthelevelofeffortrequiredtoobtainaninterview(Hawkins1975;Platek,Singh,andTremblay1978;DunkelbergandDay1973;FitzgeraldandFuller1982;ThomsenandSiring1983;Traugott1987).Unlikethecontinuum-of-resistancemodel,methodsbasedona"classes"modeldividenonparticipantsintoseveralclassesandassumethattheseclassesarealsorepresentedamongparticipants.Usingtheclassesmodel,researchersmightsimplydivideparticipantsintotwogroups,suchascooperativerespondents(oramenables)andtemporaryrefusers(orresistors)--thosewhoatfirstrefusedtobeinterviewedbutlaterwerepersuadedtoparticipate(O'Neil1979;Smith1984).Byassumingthatfinalrefusersandtemporaryrefusersaresimilar,onecanthenestimatetheimpactofrefusalsonsurveyestimatesofmeansandproportions.Thismethodcanbeextended,forexample,bydifferentiatingpersonswhocannotbecontactedduringthesurveyfromrefusers,andthenusingrespondentswhoneededmorecallsbeforeacontacttoestimatethecharacteristicsofpersonswhoarenotcontacted(Stinchcombeetal.1981).Ifsuchassumptionsarevalid,themethodanditsextensionscouldbeusedtoobtainaroughestimateofnonparticipationbiasforawiderangeofvariables,suchasincomeorattitudesforwhichinformationaboutnonparticipantsisrarelyavailable. Pastapplicationsoftheseadhocmethods,basedoneitherthecontinuum-of-resistanceorclassesmodel,varyinhowtheytreatthediversereasonsfornonparticipation,butdonottreatthesereasonsverysystematically.Ingeneral,nonparticipationmayresultbecauseselectedpersonsarenotlocated,arelocatedbutarenotcontactedforaninterview,orarecontactedbutareunavailableforaninterviewbecauseofillnessorforotherreasons.Selectedpersonsmayalsorefusetobeinterviewed,orinterviewersmayfailtocontactthembeforethefieldperiodofasurveyisover.Whichofthesesourcesofnonparticipationarerelevantinaparticularcasedependsonthestudy'sdesignandonthefieldprocedures(Groves1983,p.62).Forexample,inanareaprobabilitydesignwithrandomselectionofarespondentwithinahousehold,householdsarerandomlyselected,andoneeligiblehouseholdmemberisthenchosenforthesample.Inthisdesignthehouseholdiseasilylocated,butselectinganeligiblerespondentmaybeimpossibleifinterviewerscannotcontactanyhouseholdmember.Inarandom-digitdialing(RDD)design,contactingapotentialsampleunitisanissue.Anunansweredtelephonemayringinahouseholdorinabusinessorbeunassigned.Inalistsampleorpanelstudydesign,nonparticipationmayalsoresultfromfailingtolocatethesamplemembersbecauseofalackofinformationabouttheperson'scurrentaddressortelephonenumber.Becausetheseadhocmethodsforestimatingtheimpactofnonparticipationrequireindicatorsderivedfromfeaturesofastudydesignorfieldprocedures--suchasindicatorsofthelevelofeffortrequiredtoobtainaninterview--implementingthemethodrequiresconsideringaspecificdesign,anditispossiblethatthesuccessofanapplicationdependsonhowanindicatorischosenandimplemented.Thestudydesignaffectstheplausibilityofamodelofacontinuum-of-resistanceintwodifferentways.Oneissueariseswhenselectinganindicatorofthelevelofeffortrequiredtoobtainaninterview,particularlywhenthestudydesignmakesseveralreasonsfornonparticipationrelevant.Dependingonthestudydesign,whetherornotacontinuumofresistanceappearsmaydependonhowonemeasuresresistanceorthelevelofeffortrequiredtoobtainaninterview.Forinstance,some researchershavefoundthat,comparedwiththeagedistributionintheirfinalsamples,olderpeoplewerelikelytobeoverrepresentedandyoungerpeopleunderrepresentedinthefirstfewcalls(Hawkins1975,p.468;FitzgeraldandFuller1982,p.13;DunkelbergandDay1973,p.162).Thisoccursbecauseelderlypeopleinhouseholdsampleswereinitiallyeasiertocontact,butoveralltheyweremorelikelytorefusethanwereyoungerpeople;ontheotherhand,althoughittookmoreefforttocontactyoungerpeople,oncecontacted,theyweremorelikelytocooperatethanwereelderlypeople(FitzgeraldandFuller1982,p.12).Thus,usingthetotalnumberofcallsastheindicatorofresistancemayconfoundtheefforttolocateandtheefforttopersuadeapotentialrespondentinhouseholdsamplesthatrequirehouseholdstobecontactedbeforearespondentisselected.Second,thecontinuum-of-resistancemodelrequirestheassumptionthatthereisamonotonicrelationshipbetweenavariableandtheeffortrequiredtoobtainaninterview.However,thisassumptionmayormaynotbetrueinaparticularstudy.Giventhevarietyofvariablesinwhichresearchersareinterested,andtheheterogeneityofnonparticipantsatanygiventime,onecouldneverbesurewhichvariableshadameaningfulrelationshipwiththelevelofeffortrequiredtoobtainaninterview.Forexample,ThomsenandSiring(1983)inaNorwegianFertilityStudyfoundthatthenumberofcontactsmadebeforeaninterviewwasobtainedwasinverselyproportionaltotheaveragenumberoflivebirths(p.37),presumablybecausethenumberofchildrenisassociatedwiththelikelihoodsomeoneisathome.ButEllis,Endo,andArmer(1970,pp.106±107)foundthatmeansforthevariablestheyexaminedwerenotmonotonicallyrelatedtolevelofeffort,eventhoughtheydidfindthat,overall,respondentswhoreturnedquestionnaireswithregularfollow-upeffortsdifferedfromthosewhoneededspecialefforts.Unlikethemodelofacontinuum-of-resistance,theclassesmodeldoesexplicitlyconsiderhowstudydesignaffectsnonparticipation,bydividingnonparticipantsintodifferentgroups,suchasthoserefuseorwhoaredifficulttocontactorlocate.Althoughtheremaybeconsiderableinterviewer variationinclassifyingtypesofrefusal,researcherscanstillrelativelyeasilyidentifyrespondentswhorefusedtheinterviewbutlaterwereconvertedtouseinestimatingmeansforfinalrefusers.Butidentifyingothercategoriesofparticipantswhoarelikenonparticipantsrequiresindicatorsofthelevelofeffortneededtoobtaintheinterview,whicharealsoproblematicforthismodel.Researchersmaylackapreciseindicatortoidentifygroups,suchasthosewhoaredifficulttolocateortocontact,foruseinestimatingthecharacteristicsofnonparticipantswhoareneverlocatedorcontacted.Furthermore,oncegroupsofparticipantsthatcanbeusedtoobtainestimatesfornonparticipantshavebeenidentified,thegroupsmaybesosmallthattestsofwhetherthegroupsdifferfromeachotherlackstatisticalpower.When,forexample,researchersmakethedifferencebetweenthosewhoareeasytocontactandthosewhoaredifficulttocontactlarge(andtrytominimizevariationwithineachgroup),theymaybeleftwithfewhard-to-contactparticipants.Methodsbasedoneitherthecontinuum-of-resistanceorclassesmodelsareessentiallyadhocwaystoestimatetheimpactofnonparticipationonsurveyestimatesofmeansandproportions.Thesemethodshavereceivedcontinuingattention,buttherehavebeenfewsystematicattemptstoevaluateeithermethodusingexternaldata.Inthisstudy,weexaminetheusefulnessofbothapproachesusingdatathatincludevaluesforbothparticipantsandnonparticipants.Becauseoursampleisfromalistofcourtcases,wemodifypreviousapplicationsofthesemodelsslightly.Forthecontinuum-of-resistancemethod,wedistinguishtheeffortrequiredtolocateparticipantsandtheeffortrequiredtointerviewalocatedrespondent.Fortheclassesmethod,wedifferentiatehard-to-locatefromeasy-to-locaterespondents.BecauseStinchcombeandhiscolleagues(1981)offerthemostdevelopeddiscussionofclasses-basedmethods,wedrawontheirworkforanextendedexaminationofthistypeofmethodandconsiderhowresponseerrorsmayaffectconclusionsaboutnonparticipationbias.Becausebothmothersandfathersfromthecourtcaseswereinterviewed,wehavetwoseparate,althoughnotindependent,instancesinwhichtoexaminetheusefulnessofthemodels. THESAMPLEANDDATATheanalysisusesthreesourcesofdata:theCourtRecordDatabase(CRD),theParentSurvey(PS),anddocumentationoffieldproceduresfromsurveycallingsheets(CS).TheCRDrecordsofficialinformationfromcourtrecordsforasampleofdivorcecaseswithchildsupport±eligiblechildrenenteringthecourtsbetween1984and1986intwentyWisconsincounties.Thecourtrecordsprovideauniqueopportunityforvalidatingparents'self-reportsaboutchildsupport,becauseaccordingtostatelaw,court-orderedchildsupportmustbetransferredbetweenparentsthroughtheClerkofCourts.AsubsampleofresidentandnonresidentparentsfromthesamecourtcasesintheCRDwasselectedfortelephoneinterviewsinthePSin1987.ThePSobtainedreportsabouttheamountofchildsupportowedandpaidin1986,andalsoaskedfortheamountofsupportthatwasnotpaidthroughthecourtasrequired.WesubtractthesedirectpaymentsfromthePSself-reportsinthisanalysis,sothattheamountsrecordedbytheCRDandthePSareascomparableaspossible(seeSchaefferinpress).TheCSprovidesinformationaboutalllocatingandinterviewingcallstoalltelephonenumbersandtheresultofeachcall.TheCSincludesthefinaldispositionforeachcase,thenumberofcallsmadeforeachcase,thenumberofdifferenttelephonenumberscalled,andthenumberoftimeseachparentrefused.Thisanalysisexaminesthe1386divorcecasesinwhichthemotherwastheresidentparentinmostof1986.Thedependentvariablesinthisanalysisaretheaverageamountsofchildsupportowedandpaid.(AppendixApresentstheexactwordingofthequestionsaboutchildsupport.)Wepresentseparateanalysesforresidentmothers(whoareowedandpaidchildsupport)andnonresidentfathers(whooweandpaychildsupport).ThecombinationoftheCRDandPSallowsustoexaminewhetherparticipantsandnonparticipantsdifferwithrespecttoanexternalcriterion,theCRD.TheavailabilityofthePSandCRDdoesintroducesomecomplications,however.AcomparisonofthePSandCRDindicatesthat thePSincludessubstantialresponseerror(e.g.,Schaeffer,Seltzer,andKlawitter1991;Schaefferinpress).Toeliminatetheeffectsofresponseerror,anevaluationoftheseadhocmethodscouldrelyontheCRD,whichprovidesacommonstandardtouseinexaminingdifferencesamonggroupsofrespondents.Acommonstandardisparticularlyimportantbecauseresponseerrorsmaydifferformothersandfathers(e.g.,Schaeffer,Seltzer,andKlawitter1991).Butinvestigatorsrelyingonadhocmethodsforexaminingnonparticipationbiastypicallyhaveonlyself-reportsavailabletothem,andthisfactsuggestspresentingthedatathataninvestigatorwouldnormallyhaveavailable,thatis,thePS.Tokeepthepresentationcompactgiventheseconflictingconsiderations,inmostoftheanalyseswepresentresultsfromeithertheCRDorthePSandcommentonhowtheresultsdifferfortheotherdataThetwoprincipalcausesofnonparticipationinthePSwereproblemsinlocatingparentssampledfromtheCRDandrefusals.Theprocessescontributingtoeachtypeofnonparticipationareheterogeneousandprobablydiffersomewhatforresidentmothersandnonresidentfathers.Inordertolocateparents,weobtainedinformationfromtheoriginalcourtcaseanddirectoryassistance,andfromtheDepartmentofRevenue,theDepartmentofTransportation,theAidtoFamilieswithDependentChildren(AFDC)agency,andtheUnemploymentCompensationfiles.Amailingwassenttothosewhocouldnotbereachedbytelephone.Attemptsweremadetopersuadeallwhorefusedtheinterview,andtointerviewthosewhohadmovedfromthestate.(SeeBartfeld[1991]fordetailsofthefieldprocedures.)TheseresourcesandproceduresprobablyfavoredfindingparentswhoremainedinWisconsin(probablymorelikelytobemothers),maintainedtheirownresidence(probablymorelikelytobefathers),anddidnotchangenames(morelikelytobefathers).Becausenonresidentparentswhodonotpaythechildsupporttheyowearesubjecttocriminalprosecution,itwasprobablymoredifficulttoconfirmthelocationoffathersinthesample(whoselocationmayhavebeenconcealedbywhoeveransweredthetelephone),andsomesampledfathershadareasontorefusethat didnotapplytothesampledmothers.Inapreviousstudyofthesedata,Schaefferetal.(1991)foundthatnonparticipantswerelesslikelytohaveawardsortopaysupportthanparticipants,andthattheaverageawardsandaverageamountpaidwerelowerforpersonswhowerenotinterviewedthanforparticipants.Itseemsthatmeasuresrelatedtochildsupportmaybecorrelatedwithparents'RESULTS:THECONTINUUM-OF-RESISTANCEMETHODInexaminingwhetherornotthereisacontinuumofresistance,researchershaveusedthetotalnumberofcallsorcontactsmadeforeachsamplememberasanindicatoroftheeffortrequiredtointerviewtherespondentand,hence,asanindicatoroftherespondent'sresistance(Hawkins1975;FitzgeraldandFuller1982;DunkelbergandDay1973;Traugott1987;ThomsenandSiring1983).However,dependingonthestudydesign,thetotalnumberofcallsmadetocompleteaninterviewmayconfounddifferenttypesofcalls.Forsomeparticipants,alargeproportionofcallsmaybeusedtocontacttheperson;oncecontacted,therespondentmaybeeasytointerview.Totakethestudydesignintoaccount,weexaminedtwoindicatorsofthetotallevelofeffortrequiredtointerviewrespondents,separatelyforresidentmothersandnonresidentfathers(datanotshown):thetotalnumberofcalls(bothlocatingandcontactingcalls),andthenumberofcallstothelasttelephonenumberatwhichwetriedtocontactaparent(forrespondents,thisisthetelephonenumberatwhichtheinterviewwascompleted).Therelationshipbetweentheconditionalprobabilityofbeinginterviewed(giventhattherespondentwasnotinterviewedonthepriorcall)andthelevelofeffortmaysuggestwhichindicatorbetterreflectsthedifficultyofobtaininganinterview.Forbothresidentmothersandnonresidentfathers,theprobabilityofbeinginterviewedgenerallydeclinesasthetotalnumberofcallsorthenumberofcallstothelasttelephonenumberincreases(datanotshown),althoughthedeclineisnotsteady(seeLin[1990]fordataonmothers).However,thenumberofcalls atthelasttelephonenumberhasaclearerinterpretationasanindicatoroftheeffortrequiredtointerviewalocatedrespondent,andasimilarindicatorcouldbedevisedformoststudydesigns.Ifthefactorsaffectinghoweasyitistolocateaparentdifferfromfactorsthataffecthoweasyitistocontactorpersuadeaparent,combiningthetwotypesofcallswouldbeinappropriate.Thus,ifthereisaregularrelationshipbetweensurveyvariablesandthelevelofeffortrequiredtoobtainaninterview,inthisstudydesign,sucharelationshipshouldbestrongerwhenusingthenumberofcallsatthelasttelephonenumber.Figure1andFigure2presentthemeandollarsofchildsupportowedandpaidbythenumberofcallstothetelephonenumberatwhichtheparentwasinterviewed,separatelyforresidentmothersandnonresidentfathers.ThisanalysiscoulduseeithertheCRDorthePS.ButonlytheCRDallowsnonparticipantstobeincludedonthecontinuumofresistanceandprovidesacriteriontouseinevaluatingapparentdifferencesand,hence,theusefulnessoftheapproach.AndsoweusetheCRDandcommentondifferencesbetweentheseandresultsobtainedusingthePS.Themeansfordollarsofchildsupportowedandpaidarecalculatedfromgroupeddataaftertencallsbecausethenumberofcasesreceivingtenormorecallsissmall.(SeeAppendixBforfrequencydistributions.)Meansforallnonparticipantsandforrefusersseparatelyareenteredatthefarrightofthecontinuum.Thefirstquestioninevaluatingthisapproachiswhetherthereappearstobeacontinuumofresistanceamongparticipants.Roughlyspeaking,motherswhorequiredfewercallsatthelasttelephonenumberhavesomewhatloweramountsofchildsupportowedandpaid,andmotherswhoreceivedmorecallshavehigheramountsofsupportowedorpaid.Atintermediatenumbersofcalls,however,thepatternisirregular;therelationshipismuchflatterandmoreirregularforfathers.ResultsaresimilarusingthePS(datanotshown);overall,thecontrastbetweenthosewithfewandthosewithmanycallsappearsstrongestforfathersusingthePSandformothersusingtheCRD. Figure1here Figure2here Thesecondquestioniswhethernonparticipantscanbeplacedattheendofanycontinuumthatappears.Twogroupsofnonrespondentsareattheendofthecontinuuminthefigures.Onegroupincludesallnonparticipants.Anothergrouphasrefusersonly.Dottedlinesinthefiguresshowthatthemeansofchildsupportowedandpaidforrefusersonlyareclosertothemeansforallnonrespondentsthantothemeansforhard-to-locaterespondents.Forthegroupofallnonparticipants,themeansofchildsupportowedandpaidaremoresimilartothemeansforrespondentswhoneedfewercallsthantothemeansforrespondentswhoneedmorecalls.Thispatternappearsforbothmothersandfathers,anddoesnotsupportthecontinuum-of-resistancemodel.Butifthemeansforrefusersonly(29percentofnonparticipatingmothersand31percentofnonparticipatingfathers)areexamined,themodelappearssomewhatmoreplausibleformothers,althoughnotforfathers.If,followingotherswhohaveexaminedwhetherthereisacontinuumofresistance,wecalculatecumulativemeansbyincludingallrespondentsinterviewedbyagivencall(notshown),themeanchangesverylittleafterfivetosevencalls.Thisisbecausethenumberofrespondentsaddedaftereachcontactistoosmalltogreatlyaffectthecumulativemean,notbecauseofthecharacteristicsoftherespondentswhorequiremoreefforttointerview.Insummary,theprobabilityofcompletinganinterviewgenerallydeclinesasourindicatorofeffort,thenumberofcallsatthefinaltelephonenumber,increases.Whenweexaminetheaverageamountsofsupportowedandpaid,calculateddirectlyforeachgroupofrespondentsreceivingagivennumberofcalls,thereappearstobesomerelationshipwithlevelofeffortformothers,butnotforfathers.Formothers,thepossiblerelationshipreceivessomeadditionalsupportifrefusersareconsideredseparatelyfromothernonparticipants,butnototherwise.Theresultsformotherssuggestthatwhetherornotacontinuumofresistanceappearsplausiblemaybesensitivetoheterogeneityamongnonparticipants,whichispartiallyafunctionofstudydesignandfieldprocedures.Thecomparisonbetweenmothersandfathersalsoindicatesthatsucharelationshipcannotbetakenfor granted.Thedifferencebetweenthetwogroupsmaybeachanceresultorafunctionoftheslightlylowerparticipationrateoffathers,whichmayinteractwithfathers'differentreasonsforrefusingRESULTS:THECLASSESMETHODStinchcombeetal.(1981)concludedthat,fortheirsample,nonparticipationbiaswasmorelikelytobecausedbythosewhorefusedthanbythosewhowerenotcontacted.Thus,theyestimatednonparticipationbiasbyassigningsampleestimatesofmeanscalculatedamongrespondentswhohadneverrefusedtopotentialrespondentsamongthosewhowerenevercontacted.Theyassignedsimilarestimatesbasedonrespondentswhotemporarilyrefusedtofinalrefusersandpotentialrefusersamongthosewhowerenevercontacted.Toapplyandextendthisapproachtothepresentstudydesignrequiresafewmodifications,mostimportantlytotakeintoaccountthatourprincipalfieldissueislocatingnotcontactingrespondents.Todifferentiaterespondentswhowereeasytolocatefromthosewhowerehardtolocate,weexaminedtheconditionalprobabilitiesoflocatingaparent(i.e.,theprobabilityoflocatingaparentonthesecondcallgiventhatshewasnotlocatedonthefirstcall)bythenumberofdifferenttelephonenumbersthatwerecalledbeforetheparentwasultimatelylocated.Thisprobabilitydeclinesasthenumberofdifferenttelephonenumbersincreases,buttherelationshipissomewhatdifferentformothersandfathers.Formothers,theprobabilitydeclinesfrom.87atonetelephonenumberto.80atthreetelephonenumbers,andthenstaysconstant.Forfathers,theprobabilityisroughlyconstant(approximately.79)uptothreetelephonenumbersandthendropsoffsharply(toapproximately.63).Inordertoexaminewhetherrespondentsforwhommoretelephonenumberswerecalledbeforetheywerelocateddifferedfromotherrespondents,weexaminedtherelationshipbetweenthisindicatorandthemeansandstandarddeviationsofchildsupportawards andpayments.HerewepresentresultsfromthePS,becauseinvestigatorswouldhavetorelyonsuchself-reportsinmakingdecisionsabouthowtoapplytheclassesmodel.Figure3showsthatformothers,themeanamountofchildsupportowedandpaidreportedinthePSdeclines,generallysmoothly,withthenumberofdifferenttelephonenumberscalled:themeanamountofchildsupportoweddeclines$1328fromthoseforwhomwecalledonetelephonenumbertothoseforwhomwetriedfiveormoretelephonenumbers(from$3876to$2548),andthemeanchildsupportpaiddeclines$1281(from$3069to$1788).Anegativerelationshipwiththenumberofdifferenttelephonenumberscalledalsoexistsforthestandarddeviationsofthesetwovariables,suggestingthathard-to-locatemothersaremorehomogeneousthaneasy-to-locatemothers.However,asFigure4shows,theseregularpatternsdonotappearforfathers.AlthoughthemeanamountofchildsupportowedinthePSalsodeclines$445fromthoseforwhomwecalledonetelephonenumbertothoseforwhomwecalledfiveormoretelephonenumbers(from$4018to$3573),andthemeanamountofchildsupportpaiddeclines$271(from$3656to$3385),therelationshipisflatterthanthatformothers.Inaddition,fatherswhorequiredmoreefforttolocatedonotappeartobecomeincreasinglyhomogeneousoneithervariable.TheresultsareverysimilarusingtheCRD(datanotAgain,itappearsplausiblethattheprocessesmakingparentsdifficulttolocateorleadingparentstorefusediffersomewhatformothersandfathers.Examiningthepatternofmeansbythenumberoftelephonenumberscalledsuggeststhat--atleastformothers--classifyingrespondentsforwhomuptothreetelephonenumberswerecalledaseasy-to-locatemightbereasonable.Forfathers,thepatternofmeansandstandarddeviationsdoesnotsuggestacleardecision;butthedeclineintheprobabilityoflocatingarespondentafterthreenumberswerecalledsuggeststhatthesamecriterionmightbeusedforfathers.Thisthresholdconsidersthatonemustattempttomaximizethedifferencebetweeneasy-anddifficult-to-locate Figure3here Figure4here respondentswithoutmakingthelattergrouptoosmallforausefulcomparison.Buttheadhoccharacterofthisdecisionisapparentandinevitable.Table1presentstheeightfinalcasedispositionsusedinthisapplicationoftheclassesmodelforthe1386mothersandfathersinthesample.Wedistinguishmoretypesofnonparticipantsthanpreviousapplicationsofthisapproachtoaidinevaluatingwhetherfinerdistinctionsmightbeusefulandtofacilitatecomparisonsbetweenourresultsandthoseobtainedfromotherdesigns.Respondentsforwhomfourormoretelephonenumberswerecalledandwhowereinterviewedareclassifiedas"interviewed,hardtolocate."Respondentswhogavea"soft"refusal,forexamplebysaying"Iamverybusyrightnow,"butwhowereinterviewedinlatercallbacksaredistinguishedfromthosewhogaveahardtemporaryrefusal,forexamplebysaying"Idonotwanttoparticipateinthesurvey."Wealsodistinguishunlocatedparentsforwhomourlocatingeffortsfoundatleastonetelephonenumberandunlocatedparentsforwhomnoneofthecourtrecords,directoryassistance,orthestateagencyfilesweexaminedwereabletosupplyinformationleadingtoatelephonenumber.Thereisasmallresidual"other"categorythatweomitinlateranalyses.TheresultsinTable1suggestthattheaveragelevelofeffortabsorbedbyacasevariesbyfinaloutcome.Forbothmothersandfathers,theaveragenumberofcallsrequiredtocompleteaninterviewatthefinaltelephonenumberisroughlysimilarforthosewhoareeasyanddifficulttolocate.Theaveragenumberoftelephonecallstothosewhoeverrefuseisapproximatelytwicethatofthemeanforthosewhoneverrefused.Themeannumberoftelephonenumbersthatweretriedbeforetherespondentwaslocatedissimilarforallgroupsexceptthosewhoweredifficulttolocate--agroupthatisdistinctfromthosewhowereeasytolocatebydesign.Inordertoevaluatetheclassesmodel,wefirstexamine,inTable2,whethertheCRDindicatesthattheremaybesystematicdifferencesamongthegroupsofparticipantsandnonparticipantswhenresponseerrorsareignored.Theorderingofthecategoriesissuggestedby TABLE1IndicatorsofLevelofEffort,byFinalDisposition NumberofCallsatFinalNumberofFinalDispositionTelephoneNumberTelephoneNumbersN Divorcedresidentmothers Finalrefusal9.521.46122(9.42)(.88)Hardtemporaryrefusal11.351.5437(11.28)(1.12)Softtemporaryrefusal11.141.4935(8.75)(.74)Interviewed,easytolocate4.941.38848(4.98)(.60)Interviewed,hardtolocate4.624.9239(3.74)(1.61)Unlocated,atleastonenumberfound3.691.70187(4.86)(1.04)Unlocated,nonumbersfound----107Other10.641.4511(11.38)(.52)Totalsample5.591.551386(6.30)(.99)N12791279Divorcednonresidentfathers Finalrefusal13.891.66179(15.41)(.81)Hardtemporaryrefusal13.961.6245(11.07)(.98)Softtemporaryrefusal12.671.9146(13.85)(1.28)Interviewed,easytolocate6.841.54651(7.36)(.69)Interviewed,hardtolocate5.774.9435(6.73)(1.21)Unlocated,atleastonenumberfound5.501.86274(9.68)(1.21)Unlocated,nonumbersfound----133Other9.261.5723(13.53)(.59)Totalsample8.041.741386(10.35)(1.05)N12531253 :Cellentriesaremeansandstandarddeviations(inparentheses)fortheindicatorsoflevelofeffort."Other"categoryincludescasesthatwerenotcompletedbecausetherespondentbrokeofftheinterview,didnotspeakEnglish,wasreportedoverseas,orwaslocatedbutnotinterviewedbeforethefieldperiodended.Casesforwhichfourormoretelephonenumberswereobtainedbeforetherespondentwaslocatedareclassifiedas"hardtolocate." TABLE2ChildSupportVariablesfromtheCourtRecordData,byFinalDisposition MeanDollarsMeanMeanDollarsMeanofChildDollarsofChildofChildDollarsofChildFinalDispositionSupportOwedSupportOwed(Log)SupportPaidSupportPaid(Log) Divorcedresidentmothers Finalrefusal48117.3633555.26(7445)(2.42)(6691)(3.87)Hardtemporaryrefusal46147.3437795.95(4829)(2.76)(5389)(3.76)Softtemporaryrefusal37867.1329975.36(3042)(2.76)(3113)(3.99)Interviewed,easytolocate33627.2524795.22(3605)(2.28)(4015)(3.72)Interviewed,hardtolocate25986.6816494.15(2176)(2.76)(2377)(3.84)Unlocated,atleastone24256.3313863.65numberfound(2614)(3.03)(2289)(3.81)Unlocated,nonumbers28436.3318973.97found(3544)(3.08)(3385)(3.93)Totalsample33487.0523874.91(4026)(2.53)(4117)(3.82)N1353135313751375Divorcednonresidentfathers Finalrefusal34517.3425205.00(3007)(2.23)(3171)(3.92)Hardtemporaryrefusal38887.2834355.60(5029)(2.45)(5525)(3.87)Softtemporaryrefusal28437.0622474.48(2294)(2.38)(2831)(4.05)Interviewed,easytolocate36957.3628075.48(4561)(2.21)(4847)(3.67)Interviewed,hardtolocate40437.5923425.52(4955)(2.03)(3153)(3.62)Unlocated,atleastone25596.4215604.11numberfound(2945)(2.96)(2807)(3.77)Unlocated,nonumbers28896.1015003.29found(3983)(3.29)(3281)(3.80)Totalsample33457.0423814.90(4030)(2.55)(4121)(3.82)N1342134213631363 :Cellentriesaremeansandstandarddeviations(inparentheses)ofthecolumnvariables.Casesforwhichfourormoretelephonenumberswereobtainedbeforetherespondentwaslocatedareclassifiedas"hardtolocate."Thereferenceperiodforallvariablesis1986.Totalsampleexcludescaseswiththefinaldisposition"other." previousresearchwhichfindsthatrefusersmayhavehigherincomesthansurveyparticipants(DuncanandMathiowetz1985,p.42)andpreviousanalysesofthesedata(Schaefferetal.1991).Becausethedistributionsofthesevariablesarehighlypositivelyskewed,wealsopresentmeansfornaturallogsoftheamountofsupportowedandpaid.Formothers,thereisaroughorderingofthemeanamountsofchildsupportowedandpaid.Fromlowesttohighestthisorderis:thosewhowereunlocatedorwhowereinterviewedbutdifficulttolocate,thosewhowereinterviewedandeasytolocate,andthosewhogavetemporaryorfinalrefusals.Thisorderingisnotpresentforfathers,however,exceptthatthemeansforthosewhowerenotlocatedarelowerthanthosefortheothergroups.Thegroupofparentswhowerenotlocatedandforwhomnotelephonenumbertouseinlocatingcouldbefoundismoreheterogeneousthanthegroupofunlocatedparentsforwhomatleastonetelephonenumberwasfound,onbothvariablesforbothmothersandfathers.Table3presentscomparablestatisticsfortheself-reportsinthePS.Mothers'self-reportsshowthepatternobservedintheCRD.Forfathers,itappearsthatonlythehardtemporaryrefusersmightbeadistinctgroup.Fatherswhowereeasyanddifficulttolocateappearquitesimilartoeachother,astheresultsinFigure4suggestedwouldbethecase.Overall,Table2suggeststhatinthissample,thosewhowerenotlocatedareprobablydifferentfromparticipantsandthatrefusersmaybeinsomecases.Butitislessclearwhetherthedifferentgroupsofrespondentsaredifferentfromeachother.Table3suggeststhatwhen,asformothers,thelevelofeffortandrefusalstatusappearsomewhatsuccessfulinidentifyingdifferentclassesofparticipants(usingtheCRDasacriterion),thesedifferencesmayalsoappearinthesample.ComparingTable2andTable3,itappearsthattheaverageamountsofsupportowedandpaidareoverstatedbythePSsampleasawhole.Inpartthisisduetothesubstantialnumberofunlocatedrespondentswithlower-than-averageamountsofsupportowedandpaid.Inaddition,onaverage,bothmothersandfathersappeartooverreporttheamountofchildsupportowedandpaid TABLE3ChildSupportVariablesfromtheParentSurvey,byFinalDisposition MeanDollarsMeanMeanDollarsMeanofChildDollarsofChildofChildDollarsofChildFinalDispositionSupportOwedSupportOwed(Log)SupportPaidSupportPaid(Log) Intervieweddivorcedresidentmothers Hardtemporaryrefusal48297.2438095.58(4680)(3.02)(5094)(3.97)Softtemporaryrefusal40627.1535115.64(3631)(2.84)(4046)(3.95)Interviewed,easytolocate36877.1128545.81(3919)(2.73)(3979)(3.55)Interviewed,hardtolocate29376.5423745.23(2254)(3.20)(2518)(3.85)Totalsample37147.0928975.77(3891)(2.77)(3988)(3.59)N901901841841Intervieweddivorcednonresidentfathers Hardtemporaryrefusal50937.6151677.36(8813)(2.28)(9274)(2.69)Softtemporaryrefusal35286.5130095.63(3002)(3.40)(3053)(3.83)Interviewed,easytolocate39697.0535786.48(4645)(2.89)(4715)(3.31)Interviewed,hardtolocate38657.0135336.44(4137)(2.97)(4311)(3.40)Totalsample40007.0536226.47(4861)(2.90)(4953)(3.32)N755755721721 :Cellentriesaremeansandstandarddeviations(inparentheses)ofthecolumnvariables.Casesforwhichfourormoretelephonenumberswereobtainedbeforetherespondentwaslocatedareclassifiedas"hardtolocate."Dollaramountsthatparentsreportedtheyexchangeddirectlyareexcluded.Thereferenceperiodforallvariablesis1986.Totalsampleexcludescaseswiththefinaldisposition"other." (seeSchaefferetal.1991).Theadjustmentfornonparticipationbiasthatisthegoalofthemethodbeingexaminedherecannotcorrectfortheseresponseerrors,andthereisnoobviouswaytoadaptthemodeltotaketheseerrorsintoaccount.Althoughthosewhohaveusedvariantsoftheclassesmodelhavenottestedwhetherthegroupstheydistinguishdiffersignificantlyfromeachother,thisisausefulstepinevaluatingtheclassesmodel.WeusetheBonferronimethodofmultiplecomparisonsforthistestbecauseseveralcomparisonsaremade,andsoasimultaneousestimationofafamilyconfidenceintervalisnecessary(Neter,Wasserman,andKutner1989,pp.163±165).Also,notallpairwisecomparisonsareinterestingtous.Weareparticularlyinterestedinninecomparisons(seeTable4).Thismethodofmultiplecomparisonsincreasesthedifficultyofrejectingthenullhypothesis,evenwhentheobserveddifferencebetweentwogroupmeansislargebecauseitusesapooledvarianceandbecausethenumberofcasesinsomeofthegroupsissmall.However,theseconditionswillalwaysbetruewhenmakingcomparisonsrequiredbythemodelofclasses.Althoughwecouldadjusttheclassificationstoincreasethenumberofhard-to-locaterespondents,thiscouldlosewhatdistinctivenessthecategoryhas.Intestingwhetherthedifferencebetweentwogroupmeansmaybesignificant,weusethe10percentlevelofsignificance,becauseofthesmallsizeofsomeofthegroupsandtheadhoccharacterofthismethod.TheresultsinTable4addressbothwhethertheCRDsuggeststhegroups"actually"differ(PanelA)andwhetherornotanydifferencesappearwhenresponseerrorisaddedtoothersourcesoferror(PanelB).Themodelofclassesassumesthatsomeofthesecomparisonswillfailandotherssucceed,butsupportforthemodelismixed.First,theCRDdatasuggestthatthetwogroupsofnonparticipatingmothers--unlocatedandrefusingmothers--differfromeachother.Althoughtheresultsarelessconsistentforfathers,whenlogsoftheamountsofsupportowedandpaidaretested(notshown),thegroupsdiffersignificantlyforbothvariables(p)Italsoappearsthatnoneof TABLE4T-valuesforComparisonsofGroupMeansfortheCourtRecordDataandParentSurveyVariables Mothers Fathers MeanDollarsMeanDollarsMeanDollarsMeanDollarsofChildofChildofChildofChildComparisonsofGroupMeansSupportOwedSupportPaidSupportOwedSupportPaid A.CourtRecordData Finalrefusal.unlocatedparents4.991.962.65Hardtemporaryrefusal.softtemporaryrefusal.86.811.231.39Finalrefusal.temporaryrefusal1.02-.05.17-.61Unlocated,nonumberfoundunlocated,atleastonenumberfound.851.03.77-.14Unlocatedparents.interviewed,hardtolocate.05-.01-1.86-1.12Interviewed,easytolocateunlocatedparents2.60Interviewed,easytolocateinterviewed,hardtolocate1.161.24-.50.65Finalrefusal.interviewed,easytolocate3.752.21-.72-.83Temporaryrefusal.interviewed,easytolocate1.681.81-.73.07B.ParentSurvey Hardtemporaryrefusal.softtemporaryrefusal.74.301.481.95Interviewed,easytolocateinterviewed,hardtolocate.91.68.12.05Temporaryrefusal.interviewed,easytolocate1.801.541.80.87 :Divorcedmotherswhohavephysicalcustodyandtheircorrespondingdivorcingspousesareincludedinthissample.Thereferenceperiodforallvariablesis1986.TheCRDanalysisincludes1375cases(caseswithfinaldispositionof"other"areexcluded[N=11]);thePSanalysisincludes964divorcedresidentmothersand786divorcednonresidentfathers.CellentriesarestatisticsbasedontheBonferronimethodofmultiplecomparisons(/2*g;n),where1-isthefamilyconfidencecoefficient,gisthenumberofgroupcomparisons,andnisthenumberofcases).Forthejointestimationofninecomparisons(CRD),the90thpercentileofthedistribution(two-sided)with1375degreesoffreedomis(1-.10/2*9,1375)=2.54,the95thpercentileis(1-.05/2*9,1375)=2.77,andthe99thpercentileis(1-.01/2*9,1375)=3.26.Forthejointestimationofthreecomparisons(PS)amongmothers,the90thpercentileofthedistribution(two-sided)with964degreesoffreedomis(1-.10/2*3,964)=2.13,the95thpercentileis(1-.05/2*3,964)=2.39,andthe99thpercentileis(1-.01/2*3,964)=2.93.Forthejointestimationofthreecomparisons(PS)amongfathers,the90thpercentileofthedistribution(two-sided)with964degreesoffreedomis=2.13,the95thpercentileis(1-.05/2*3,786)=2.39,andthe99thpercentileis thegroupsoftemporaryorfinalrefusersdiffersignificantlyfromeachotherforeithermothersorfathers.Similarly,forbothmothersandfathers,thetwogroupsofunlocatedparentsaresimilartoeachotherandtothehard-to-locaterespondents.Theselattertwonulleffectsarecompatiblewiththemodel.Butthedistinctionsthemodelpredictsdonotappear.Althoughmothersandfatherswhoareeasytolocatedifferfromthosewhoarenotlocated,atleastinsupportpaid,theydonotdifferfromthosewhoaredifficulttolocate.Similarly,althoughmotherswhoultimatelyrefusedifferfromthosewhoareeasytolocateforsupportowedintheresultsinTable4,thecomparisonfailswhenlogsareanalyzed,andthelattergroupdoesnotdifferfromthetemporaryrefusers;neitherdifferenceissignificantformothersforsupportpaidorforeithervariableforfathers.Moreover,theresultsinTable4indicatethatnoneofthesedifferencesaresignificantinthePS;theoverreportingbyalmostallgroupsofrespondentsappearstoreduceeventhedifferencesbetweenthemthatappearintheCRD.Theseresultsmakeitseemplausiblethat,atleastinthiscase,therearedifferenttypesofnonparticipants,whoprobablydifferfromparticipants,sothatsomethingliketheclassesmodelmightbeapplicablehere.AlthoughthepatternsofmeansformothersandfathersinTable2andTable3appearratherdifferent,thepatternofsignificantresultsisverysimilarforbothgroups.Theresultsarelesspromising,however,aboutthegroundsforusinggroupsofparticipantstoestimatevaluesfornonparticipants.AlthoughtheresultsinTable4donotjustifytreatingthosewhoweredifficulttolocateorwhoweretemporaryrefusersasdifferentfromthosewhowereeasytolocateandinterview,furtherexaminationoftheclassesmodelrequiresthatweproceedasthoughatleastsomeofthesedifferencesweresignificant. CORRECTINGSAMPLEESTIMATESOFMEANSFORNONPARTICIPATIONBIASTousegroupsofparticipantstoestimatethecharacteristicsofnonparticipants,Stinchcombeetal.(1981)assumedthatpeoplewhowerenevercontactedwould,ifcontacted,participateandrefuseintheproportionsobservedamongthosewhowerecontacted.Thus,peoplewhoarenotcontactedcanbedividedintotwogroups,potentialrespondentsandpotentialrefusers.Stinchcombeetal.thenassignedsurveyestimatesbasedonrespondentstothepotentialrespondentgroup,andassignedsurveyestimatesbasedontemporaryrefuserstothegroupofpotentialrefusersandfinalrefusers.Thisprocedureallowsadhocestimatesofthepopulationmeanandnonparticipationbiastobeestimated.Althoughthereareseveralwaysthatthisgeneralapproachcouldbeextendedtothecurrentstudydesign,ourfirstestimatesareessentiallysimilartothoseofStinchcombeetal.,exceptthatwepreservethedistinctionbetweensoftandhardtemporaryrefusals.Wethenofferaslightmodification.ThecalculationsformothersandfathersarepresentedinAppendixC.Table5presentstheestimatesandrelatedmeansforcomparisonsusingtheCRDandthePS.UsingtheCRDhastwoadvantages.First,itprovidesanestimateofthepopulationmeantouseinestimating"true"nonparticipationbiasandhenceevaluatingwhetherthemethod'sestimateofnonparticipationbiasisaccurate.Second,itpermitsustoevaluatewhetherthisadhocmethodofadjustingfornonparticipationisaccuratewhenthereisnoresponseerror.ComparingtheCRDmeanfortheachievedsample(thoseactuallyinterviewed)(row1)withtheCRDmeanforthetotalsample(row2),wecanestimatetheactualbiasinthemeanoftheachievedsample(row1-row2).Forbothmothersandfathers,themeanfortheachievedsampleisgreaterthanthatforthetotalsample,becauseasubstantialproportionofthesamplewhowerenotlocatedhave,onaverage,lowervaluesthanthesampleasawhole.However,usingtheclasses-basedmethodwiththeCRDdata(row3),onewouldconcludethattheachievedsampleunderestimatesthepopulationmean(row1-row3).Similarly,forthePS,themeanfortheachievedsampleisconsistentlylargerthantheactualmean TABLE5EstimatesofTotalSampleMeansandNonresponseBiasUsingtheCourtRecordDataandParentSurvey Mothers Fathers MeanDollarsMeanDollarsMeanDollarsMeanDollarsofChildofChildofChildofChildEstimatesSupportOwedSupportPaidSupportOwedSupportPaid A.CourtRecordData Meanforachievedsample(1)3393251436722790Meanfortotalsample(2)3348238733452381Estimatedmeanfortotalsample(3)3562267836882875Biasinachievedsample(1-2)45127327409Estimatedbiasinachievedsample(1-3)-169-164-16-85B.ParentSurvey Meanforachievedsample(4)3714289740003622Estimatedmeanfortotalsample(5)3873303041593854Biasinachievedsample(4-2)3665106551241Estimatedbiasinachievedsample(4-5)-159-133-159-232 :Divorcedmotherswhohavephysicalcustodyandtheircorrespondingdivorcingspousesareincludedinthissample.Thereferenceperiodforallvariablesis1986.TheCRDanalysisisbasedon1386cases;thePSanalysisisbasedon964divorcedresidentmothersand786divorcednonresidentfathers.Theanalysisexcludescaseswithfinaldispositionof"other." fromtheCRD(row4-row2).Themethod,however,leadstotheconclusionthatthemeanforthetotalsampleisunderestimatedbytheachievedsample(row4-row5).Furthermore,themethodsuggeststhattheabsolutesizeofthebiasissmallerforfathersthanformothers(usingtheCRD)orroughlysimilarforthetwogroups(usingthePS),wheninfactthebiasissubstantiallygreaterforThemeansinTable3suggestthattheestimatesmightbeimproved--atleastformothers--byassumingthatallpotentialrespondents(.17ofthetotalsampleofmothersand.22ofthetotalsampleoffathers)wouldbelikethosewhowerehardtolocate,andthisisanadjustmentthataninvestigatormightreasonablymakeaprioriforthisstudy.(SeeAppendixCforthecalculations.)TheresultsinTable6showthatthemodifiedmethodimprovesmostoftheestimatesofnonparticipationbias,buteventhedirectionoftheestimatedbiasisnowcorrectforonevariableusingtheCRD.Althoughitmightbepossibletofurtheradjustthemethodtoobtainbetterestimatesofnonparticipationbias,suchadjustmentswouldrequireusingknowledgeoftheCRD,knowledgethatwouldnotbeavailableinmostrealisticsituations.ApplyingStinchcombeetal.'smethodleadsustoconcludethatthemeanforthetotalsampleisunderestimatedbytheachievedsample,whenthemeanfortheachievedsampleisactuallygreaterthanthatforthetotalsample.InthecaseexaminedbyStinchcombeetal.,thosewhoweredifficulttocontactappearedsimilartothosewhowereeasytocontact,andtheyconcludedthatnonparticipationbiaswasmostlikelytoresultfromrefusals.Thus,theirmethodimputesthemeanforallthosewhowereinterviewedtoallpotentialrespondents.Acomparableconclusion--thatthemeanforthosewhowereinterviewedcanbeimputedtothosewhowerenotlocated--doesnotappearjustifiableinourcase.Thelargemajorityofournonparticipantswerenotlocated(seeTable1).The TABLE6ImprovedEstimatesofTotalSampleMeansandNonresponseBiasUsingtheCourtRecordDataandParentSurvey Mothers Fathers MeanDollarsMeanDollarsMeanDollarsMeanDollarsofChildofChildofChildofChildSupportOwedSupportPaidSupportOwedSupportPaid A.CourtRecordData Meanforachievedsample(1)3393251436722790Meanfortotalsample(2)3348238733452381Estimatedmeanfortotalsample(3)3439254637612777Biasinachievedsample(1-2)45127327409Estimatedbiasinachievedsample(1-3)-46-32-8913B.ParentSurvey Meanforachievedsample(4)3714289740003622Estimatedmeanfortotalsample(5)3753295341373845Biasinachievedsample(4-2)3665106551241Estimatedbiasinachievedsample(4-5)-39-56-137-223 :Divorcedmotherswhohavephysicalcustodyandtheircorrespondingdivorcingspousesareincludedinthissample.Thereferenceperiodforallvariablesis1986.TheCRDanalysisisbasedon1386cases;thePSanalysisincludes964divorcedresidentmothersand786divorcednonresidentfathers.Theanalysisexcludescaseswithfinaldispositionof"other." meanamountsofchildsupportowedandpaidinthesamplewerelowestforunlocatedparents(seeTable2),inpartbecausetheproportionofparentsowingorpayingnochildsupportishighestamongunlocatedparents.Thus,whentheCRDmeanforparticipantsisassignedtounlocatedparents,theresultingestimatedmeanforthetotalsampleisnotonlygreaterthanthemeanfortheachievedsample,butalsomuchgreaterthanthe"true"mean.WhenthePSisused,theestimateofnonparticipationbiasisevenworsebecause,onaverage,participantsoverestimatechildsupportowedandpaid.Toattempttoimproveestimatesofnonparticipationbiasforthesedata,weimputedthemeanamountsofchildsupportowedandpaidforhard-to-locaterespondentstoallpotentialrespondents.AsTable6showed,thisadjustmentdecreasestheabsolutesizeoftheestimatedbiasformothersalthoughtheestimateddirectionisstillwrong.However,thisadjustmentprovidesnoimprovementforfathers.Thedifferencebetweenmothersandfathersoccursbecausehard-to-locatemothersarebetter"proxies"forunlocatedmothersthanhard-to-locatefathersareforunlocatedfathers.Themeanamountsofchildsupportowedandpaidforunlocatedmothersareonlyslightlylowerthanthoseforhard-to-locateparticipatingmothers(forowed,$2577vs.$2598;forpaid,$1572vs.$1649),andtheproportionsofunlocatedmotherswithnochildsupportowedorpaidareclosetothoseforhard-to-locateparticipatingmothers(forowed,17percentvs.12.8percent;forpaid,49.7percentvs.43.6percent).However,themeanamountsofchildsupportowedandpaidforunlocatedfathersaremuchlowerthanthoseforhard-to-locateparticipatingfathers(forowed,$2667vs.$4043;forpaid,$1540vs.$2342),andtheproportionsofunlocatedfatherswithnochildsupportowedorpaidaremuchhigherthanthoseforhard-to-locateparticipatingfathers(forowed,17.7percentvs.5.7percent;forpaid,48.5percentvs.28.6percent).Stinchcombeetal.(1981,p.373,375)notethatattheleasttheirmethodmightmakeinvestigatorscautiousaboutignoringnonparticipationbias.Inthepresentcase,however,theestimates ofbiasmightmakeinvestigatorsoverlysanguine--whenthemethodisappliedtodatawithoutresponseerrorforfathers,itsuggeststhattheabsolutevalueofnonparticipationbiasis0.4percentto4percentofthemean,althoughtheactualbiasis10percentto17percentofthemean.Furthermore,themethodleadstoincorrectconclusionsaboutthedirectionofbiasandtherelativesizeofbiasformothersandfathers.Thisfailureoccurseventhoughthereappeartobesomesignificantdifferencesbetweenparticipantsandnonparticipantsand,atleastformothers,somedifferencessuggestedbythemodelappearinthesample(eventhoughtheyarenotalwayssignificant).Themotivationtosearchforaneasy,adhocmethodtoadjustsamplemeansfornonparticipationiseasytoappreciate.Theseanalysessuggest,however,thatneitherthecontinuum-of-resistancemodelnortheclassesmodelprovidesadependableanswertothequestionofhownonparticipationmightbiasestimatesofmeansfromagivensample.Giventhecomplexprocessesdeterminingsurveyparticipation,thismayhardlyseemsurprising.Furthermore,thepresentationclarifiesthatthesemethodsreallyareadhoc:theyrequirealargenumberofsmalldecisions--suchaswhattouseasanindicatorofthelevelofeffort,howtodistinguisheasy-to-locatefromhard-to-locaterespondents,orwhatclassestouse--thatmustbemadewithoutagreatdealofguidancefromeithertheoryordatabutwhichmaygreatlyaffecttheresultingestimatesofnonparticipationbias.Thestraightforwardprojectionsofthecharacteristicsofnonparticipantssuggestedbythecontinuumofresistancecannotbeassumedtoberoutinelyjustified,andeventhemorecomplexmethodthatrecognizesdifferenttypesofnonparticipantswasunsuccessful.Inaddition,theestimatesofnonparticipationbiasareaffectedbyresponseerror,anditisunclearhowonemightadapteithermethodtotakeresponseerrorsintoaccount.Theconsequencesofresponseerrorforsuchmethodshavenotbeenconsideredsystematicallyinpastinvestigations.Ouranalysesalsosuggestthatresponseerrorsmayhaveseriousimplicationsformorerigorousmethodsofcorrectingfor nonparticipation,suchas"hotdeck"procedures,thatimputeinformationtakenfromparticipantstononparticipants(Ford1983).Itispossiblethatthesemethodsarenotusefulherebecauseofthesomewhatspecializeddesignandnatureofthissampleorbecauseofthelowresponserate,butthesemethodshavebeenproposedandexaminedwithsimilarlyspecializedsamplesand,despitetheresponserate,ournonparticipantsappeartobesomewhatdistinctfromparticipants.Althoughasimpleadhocmethodforadjustingfornonparticipationwouldbewelcome,thereislittleevidencethateitherofthesemethodscurrentlyprovidesone. APPENDIXAQuestionWordingTheParentSurveyusedthefollowingquestionstodeterminewhetherchildsupportwasowedin1986andtheamountofanysupportowed.Versionsshownarethoseusedforcustodialparents.Theversionsfornoncustodialparentsareexactlyparallel.J.30Didyouand(CHILD)'s(father/mother)havealegalagreementthatwasineffectduring1986whichspecifiedANYof(his/her)rightsorobligationsasaparent,suchascustody,rightstosee(CHILD),ortheobligationtohelpsupport(CHILD)?By"legalagreement"Imeanonethathasbeenapprovedbyajudgeoranotherofficerofthecourt.J.31ASKIFJ.30=NO.Didyouand(CHILD)'s(father/mother)haveaninformalagreementofthiskindthatwasineffectduring1986?J.59ASKIFJ.30ORJ.31=YES.Didtheagreementyouand(CHILD)'s(father/mother)hadin1986specifythat(he/she)provideANYkindoffinancialsupportorhealthinsurancecoverageforJ.62ASKIFJ.59=YES.Theregularpaymentsthatparentsmakeforthesupportoftheirchild(ren)arereferredtoaschildsupportpayments.Thenextquestionsareaboutchildsupportpaymentsforallof1986.Accordingtotheagreementyouhadin1986,was(CHILD)'s(father/mother)tomakeREGULARPAYMENTS,suchaseveryweekoreverymonthfor(CHILD)'ssupport?J.65ASKIFJ.62=YES.Now,pleasetellmejustabouttheagreementregardingchildsupport,whetherornotyoufolloweditexactly.Accordingtotheagreementthatyouhadduring1986,howmuchwas(CHILD)'s(father/mother)SUPPOSEDtopayregularlyfor(CHILD)'sand(his/hersiblings')support?Donotincludeotherfinancialhelpthatthechild(ren)'s(father/mother)mayhaveagreedtoprovideforyouorthechild(ren)inadditiontoregularsupportpayments.J.69Howoftenweretheregularchildsupportpaymentsyouweresupposedtoreceivemissedcompletelyin1986?In1986wereallpaymentsmissedcompletely,somepaymentsmissedcompletely,orwereallpaymentsmade?(appendixcontinues) APPENDIXAJ.71ASKIFJ.69=SOMEMISSED.Whatwasthedollaramountofthechildsupportpaymentsthatyoureceivedin1986?J.78ASKIFSOMEORALLPAYMENTSMADE.Did(CHILD)'s(father/mother)makeanypartofthesepaymentsDIRECTLYTOYOUduring1986?J.79In1986,whatwasthedollaramountofthepaymentsthatyoureceiveddirectlyfromthe(CHILD)'s(father/mother)? APPENDIXBNumberofCasesforEachFigure,byNumberofCallsandNumberofTelephoneNumbers Figure1:Mothers Figure2:Fathers ofPhoneFigure3:Mothers Figure4:Fathers No.ofCallsOwedPaidOwedPaidNumbersOwedPaidOwedPaid 1182184107108158955041339521781801081092218201223209312612887883535477764858676784232018185697062625+1816242365052373774142313283737303093434333410-114346434412-132727313114-162222393917-191516242420+35356061Nonparticipants304309446453 APPENDIXCEstimatesofMeansCorrectedforNonparticipationBiasInapplyingthemethodsuggestedbyStinchcombeetal.(1981),weretainedmorecategoriesanddetailthantheyused,toimprovethemethod'schanceofsuccessandtotakethestructureofthisstudydesignintoaccount.Steps1±8shownbelowgivethecalculationsfortheestimatesinTable5.TheseestimatesarecomparabletothoseprovidedbyStinchcombeetal.,exceptthatwedistinguishsoftandhardtemporaryrefusals.Step9describestheadjustmentusedtoobtaintheestimatesinTable6.Ourdescriptionofthestepsbelowreferstothecalculationsformothers;thenumberscorrespondtothecalculationsshownbelow.(1)Wecalculatetheprobabilityofarefusalgiventhattherespondentislocated.Weaddthenumberofsofttemporaryrefusals,hardtemporaryrefusals,andfinalrefusalstoformthenumerator;thenumberofinterviewedrespondents,temporaryrefusals,andfinalrefusalsaresummedtoformthedenominator.Theresult,.18formothers,forexample,istheestimatedproportionofallmotherswhowouldrefusetheinterviewatleastonce.(2)Themodelassumesthatrespondentswhoarenotcontactedwould,ifcontacted,respondandrefuseinthesameproportionsobservedamongthosewhoarecontacted.Weestimatetheproportionofthesampleofmotherswhoareunlocatedbutwhoarealsopotentialrefusalsas.18multipliedbytheproportionofthesamplewhowereunlocated.(3)Thenwesubtracttheproportionofmotherswhoarepotentialrefusalsfromtheproportionofunlocatedmothers(i.e.,.21-.04=.17)toobtaintheproportionofthesamplewhoareunlocatedbutpotentialrespondents.(4and5)Thenumberoftemporaryrefuserswhoaresoftandhardrefusalsaresimilar.Wedividethepotentialrefusalgroupintotwogroups,2percentofthewholesamplewhoareunlocated butwho,oncelocated,wouldrespondassofttemporaryrefusalsdo,and2percent(i.e.,.04*.50=.02)who,oncelocated,wouldrespondashardtemporaryrefusalsdo.Theproportionofmotherswhoareeasytolocateandareinterviewedis.96(i.e.,848/848+39)ofthemotherswhoneverrefuse;theproportionofthesamplewhoarehardtolocateandareinterviewedis.01(i.e.,39/848+39)ofthemotherswhoneverrefuse.Therefore,wecandividethepotentialrespondents(17percentofthewholesampleofmothers)intotwocategories:(6)16percentofthesamplewould,iflocated,respondlikethosewhowereeasytolocateand(7)1percentofthesamplewhowouldrespondlikethosewhowerehardtolocate.(8)Theseproportionsareusedtoestimatethemeanthatwouldhavebeenobservediftherehadbeennononparticipation:formothers,16percentofthesamplehavetheestimateobtainedfromtheeasytolocaterespondents(groupA);1percenthavetheestimateobtainedfromthehardtolocaterespondents(groupB);2percenthavetheestimateobtainedfromthesofttemporaryrefusals(groupC);2percenthavetheestimateobtainedfromthehardtemporaryrefusals(groupD).(9)Adjustedestimatesassumethatallpotentialrespondentsarelikethosewhoweredifficulttolocate.(appendixcontinues) APPENDIXC(continued) Mothers EasytoLocate HardtoLocate CSoftTemporaryRefusal DHardTemporaryRefusal FinalRefusal Unlocated N=8483935371222941375 .03 .03 .03 .09 .21 (1)P(refuse/locate)=(35+37+122)/(848+39+35+37+122)=.18(2).21*.18=.04proportionoftotalsamplewhoareunlocatedandpotentialrefusals(3).21-.04=.17proportionoftotalsamplewhoareunlocatedandpotentialrespondents(4).04*(35/35+37)=.02proportionoftotalsamplewhoareunlocatedandpotentialrefuserssimilartosofttemporaryrefusals(5).04*(37/35+37)=.02proportionoftotalsamplewhoareunlocatedandpotentialrefuserssimilartohardtemporaryrefusals(6).17*(848/848+39)=.16proportionoftotalsamplewhoareunlocatedandpotentialrespondentssimilartoeasy-to-locaterespondents(7).17*(39/848+39)=.01proportionoftotalsamplewhoareunlocatedandpotentialrespondentssimilartohard-to-locaterespondents(8)Estimateofmeancorrectedfornonparticipationbias=(.62+.16)*A+(.03+.01)*B+(.03+.02)*C+(.03+.09+.02)*D(9)Improvedestimateofmeancorrectedfornonparticipationbias=.62*A+(.03+.17)*B+(.03+.02)*C+(.03+.09+.02)*D(appendixcontinues) APPENDIXC Fathers EasytoLocate HardtoLocate CSoftTemporaryRefusal DHardTemporaryRefusal FinalRefusal Unlocated N=6513546451794071363 .03 .03 .03 .13 .30 (1)P(refuse/locate)=(46+45+179)/(651+35+46+45+179)=.28(2).30*.28=.08proportionoftotalsamplewhoarepotentialrefusalsamongthoseunlocated(3).30-.08=.22proportionoftotalsamplewhoarepotentialrespondentsamongthose(4).08*(46/46+45)=.04proportionoftotalsamplewhoareunlocatedandpotentialrefuserssimilartosofttemporaryrefusals(5).08*(45/46+45)=.04proportionoftotalsamplewhoareunlocatedandpotentialrefuserssimilartohardtemporaryrefusals(6).22*(651/651+35)=.21proportionoftotalsamplewhoareunlocatedandpotentialrespondentssimilartoeasy-to-locaterespondents(7).22*(35/651+35)=.01proportionoftotalsamplewhoareunlocatedandpotentialrespondentssimilartohard-to-locaterespondents(8)Estimatedmeancorrectedfornonparticipationbias=(.48+.21)*A+(.03+.01)*B+(.03+.04)*C+(.03+.13+.04)*D(9)Improvedestimatedmeancorrectedfornonparticipationbias=.48*A+(.03+.22)*B+(.03+.04)*C+(.03+.13+.04)*D Anotherapplicationofmodelsoftherelationshipbetweenparticipantsandnonparticipantsistoidentifythepointatwhichfurtherfieldeffortsbecomeunnecessarybecausesampleestimatesnolongerchangewhenadditionalsamplemembersareinterviewed(foradiscussion,seee.g.,Groves1989).Theseapplicationsareaimedprimarilyatevaluatingsurveycostsandfalloutsidethescopeofthispaper.Inthisstudy,residentparentsarethosewithphysicalcustody.Physicalcustodyisdefinedaslivingwiththechildrenforatleastsixmonthsin1986.Themotheristheresidentparentinthelargemajorityofdivorcecases.Becausethedistributionsoftheamountofchildsupportowedandpaidarepositivelyskewed,wealsoconductanalysesandsignificancetestsusingthenaturallogarithmsofthesevariables.Onewasaddedbeforetakingthenaturallogarithmsbecauseofthenumberofzeros.ResultsfortheproportionofmothersowedorpaidanysupportcanbefoundinLin(1990).TheimpactofresponseerrorisconsideredmorefullybelowinthediscussionoftheclassesAllnonparticipantsincludefinalrefusersandunlocatedmothers.Untilarespondentislocated,wecannotdistinguishafailuretocontactfromafailuretolocate.Weconsiderallcallsmadebeforetheparent'slocationwasconfirmedaslocatingcalls.Onlythreeparentswhoselocationwasconfirmedwerenotcontactedattheendofthefieldperiod.Wealsoexaminedthisrelationshipusingtheindicatorsdescribedintheprevioussection.Forbothmothersandfathers,theprobabilityoflocatingtheparentgenerallyincreasesasthetotalnumberofcallsandthenumberofcallstothelasttelephonenumberincrease(datanotshown).Althoughtheserelationshipsappeartobefairlysmooth,levelingoffafterapproximatelyeightcalls,wehave alreadyseen(Figure1andFigure2)thatthereisnoclearrelationshipbetweeneitherindicatorandthemeanamountsofchildsupportowedorpaid.Formothers,etasquaredforchildsupportowedandpaidisverysmallbothinthePS(.02forowedandpaid)andtheCRD(.01forowed;.02forpaid).Thus,thelinearrelationshipbetweenthenumberoftelephonenumbersandthepaymentvariablesisevensmaller.TherelationshipisalsoweakforfathersinthePSandtheCRD(etais.02forbothowedandpaid).Althoughpreviousapplicationsofthismodelhavenotexaminedwhethersuchgroupsdiffersignificantlyfromeachother,wetestedtheresultsofthisdecisionusingtheBonferronimethodofmultiplecomparisons.Classifyingthoseforwhommorethanthreenumberswerecalledashard-to-locateandanalyzingthePS--towhichaninvestigatorapplyingthemodelwouldberestricted--wefoundthatthemeanchildsupportowedorpaidtohard-to-locatemothersorfathersdoesnotdiffersignificantlyfromthatowedorpaidtoeasy-to-locatemothersorfathers�(p.20).TheresultsarethesameifoneexaminestheCRD.SeeTable4below.Althoughwerecognizethattheremaybeconsiderableinterviewervariabilityintheuseofthetworefusalcategories,wepreservethembecausedifferencesamongtypesofrefusalsarepotentiallyimportant.Respondentswhoeverrefusedareclassifiedinthe"temporaryrefusal"categories,regardlessofwhethertheywereeasyordifficulttolocate.Itisnotclearwhichclassificationshouldtakepriority,andthisrulemayincreasetheheterogeneityoftherefusalcategories,buttherefusersaresimilartotheeasy-to-locateinthenumberofdifferenttelephonenumbersrequiredforlocation,providingsomesupportforthisdecision.Itdoesnotseemthatthedifferentpatternsformothersandfathersariseonlybecauseofafewoutliers.Ifoutliersareomittedsomeofthepatternsinthistabledobecomemoresimilarformothersandfathers,butotherpatternsbecomemoredissimilar. ThetestsperformedusinglogsoftheamountsowedandpaidgivethesameresultsasthosepresentedinTable4withthetwoexceptionsnotedinthetextandwiththefollowingadditionalexceptions:Asthemodelsuggests,finalrefusersandunlocatedfathersdiffersignificantlyforsupportowedintheCRD(p)contrarytothemodel'sassumptions,theunlocatedandhard-to-locatefathersdiffersignificantlyforsupportowed(p)andpaid(.05)intheCRD;andthehardtemporaryandsofttemporaryrefusalsdiffersignificantlyforfathersforsupportpaidinthePSThedifferencebetweenthemeanforthetotalsampleandthemeanforthe"population"ofCRDcasesfromwhichthesamplewasdrawnistriviallysmall(i.e.,approximatelyfourdollarsforsupportowedandfortydollarsforsupportpaid;seeSchaefferetal.[1991]).WeignorethissamplingvariabilityinthediscussionbecauseitwouldcomplicatethepresentationwithoutaffectingtheAccordingtotheCRD,thepercentageswithnochildsupportowedforfinalrefusals,hardtemporaryrefusals,softtemporaryrefusals,easy-to-locateparticipants,hard-to-locateparticipants,andunlocatednonparticipantsare8.2,10.8,11.4,7.7,12.8,and17.0formothers.Thecomparablepercentagesforfathersare7.3,8.9,8.7,6.9,5.7,and17.7.Thecomparablepercentagesformotherswithnochildsupportpaidare33.6,27.0,34.3,32.3,43.6,and49.7.Forfathers,theproportionswithnochildsupportpaidare36.9,31.1,43.5,29.5,28.6,and48.5. Bartfeld,Judi.1991."ParentSurvey2:FieldReport."Unpublishedmanuscript.ChildSupportDemonstrationProject,InstituteforResearchonPoverty,UniversityofWisconsin-Madison.Duncan,Greg,andNancyA.Mathiowetz.1985.AValidationStudyofEconomicSurveyData.AnnArbor,Michigan:InstituteforSurveyResearch.Dunkelberg,WilliamC.,andGeorgeS.Day.1973."NonresponseBiasandCallbacksinSampleJournalofMarketingResearch10:160±168.Ellis,RobertA.,CalvinM.Endo,andJ.MichaelArmer.1970."TheUseofPotentialNonrespondentsforStudyingNonresponseBias."PacificSociologicalReview13:103±109.Filion,F.L.1976."ExploringandCorrectingforNonresponseBiasUsingFollow-upsofPacificSociologicalReview19:401±408.Fitzgerald,Robert,andLindaFuller.1982."IHearYouKnockingbutYouCan'tComeIn:TheEffectsofReluctantRespondentsandRefusersonSampleSurveyEstimates."MethodsandResearch11:3±32.Ford,BarryL.1983."AnOverviewofHot-DeckProcedures."InIncompleteDatainSampleSurveys,Volume2,TheoryandBibliographies,ed.WilliamG.Madow,IngramOlkin,andDonaldB.Rubin,pp.185±207.NewYork:Academic.Groves,RobertM.1983."Discussion."InIncompleteDatainSampleSurveys,Volume3,ProceedingsofTheSymposium,ed.WilliamG.MadowandIngramOlkin,pp.61±63.NewYork:Academic._______________.1989.SurveyErrorsandSurveyCosts.NewYork:JohnWiley&Sons.Hawkins,DarnellF.1975."EstimationofNonresponseBias."SociologicalMethodsandResearch Lin,I-Fen.1990."UsingRespondentsasProxiestoEstimatetheCharacteristicsofNonrespondents."UnpublishedMaster'sthesis,DepartmentofSociology,UniversityofWisconsin-Madison.Madow,WilliamG.,IngramOlkin,andDonaldB.Rubin,eds.1983.IncompleteDatainSampleSurveys,Volume2,TheoryandBibliographies.NewYork:Academic.Neter,John,WilliamWasserman,andMichaelH.Kutner.1989.AppliedLinearRegressionModelsBoston:Irwin.O'Neil,MichaelJ.1979."EstimatingtheNonresponseBiasDuetoRefusalsinTelephoneSurveys."PublicOpinionQuarterly43:218±232.Platek,R.,M.P.Singh,andV.Tremblay.1978."AdjustmentforNonresponseinSurveys."InSurveySamplingandMeasurement,ed.N.KrishnanNamboodiri,pp.157±174.NewYork:Schaeffer,NoraCate.Inpress."ErrorsofExperience:ResponseErrorsinReportsaboutChildSupportandTheirImplicationsforQuestionnaireDesign."InAutobiographicalMemoryandtheValidityofRetrospectiveReports,ed.NorbertSchwarzandSeymourSudman.NewYork:Schaeffer,NoraCate,JudithA.Seltzer,andMariekaKlawitter.1991."EstimatingNonresponseandResponseBias:ResidentandNonresidentParents'ReportsaboutChildSupport."SociologicalMethodsandResearch20(1):30±59.Smith,TomW.1984."EstimatingNonresponseBiaswithTemporaryRefusers."27:473±489.Stinchcombe,ArthurL.,CalvinJones,andPaulSheatsley.1981."NonresponseBiasforAttitudePublicOpinionQuarterly45:359±379. Thomsen,Ib,andErlingSiring.1983."OntheCausesandEffectsofNonresponse:NorwegianExperiences."InIncompleteDatainSampleSurveys,Volume3,ProceedingsofTheed.WilliamG.MadowandIngramOlkin,pp.25±59.NewYork:Academic.Traugott,MichaelW.1987."TheImportanceofPersistenceinRespondentSelectionforPreelectionPublicOpinionQuarterly51:48±57.