Definitional test of complex systems - PDF document

Chapter2DeÞnitionalTestofComplexSystemsCreatedforthecurrentstudy,theDenitionalTestofComplexSystemsisourattemptatanexhaustivetoolfordeterminingtheextenttowhichacomplexsystem’sdenitiontsatopic.TheDTCSisnot,however,astandardizedinstrument.Assuch,wehavenotnormedorvalidatedit.Instead,itisaconceptualtoolmeanttomovescholarstowardempirically-driven,syntheticdenitionsofcomplexsystems.Todoso,theDTCSwalksscholarsthroughanine-question,four-stepprocessofreview,method,analysis,andresults—seeTableFollowingGatrell,theDTCSdoesnotseektodetermineifaparticularcasetsadenition;instead,itseekstodetermineifadenitiontsaparticularcase.AsGatrellexplains[30],thechallengeinthecurrentliteratureisnotwhetherplacesarecomplexsystems;asitwouldbehardtoprovethemotherwise.Instead,thequestionis:howdowedenethecomplexityofaplace?And,doessuchadenitionyieldnewinsights?Giventhisfocus,Question9oftheDTCSfunctionsasitsnegativetest,focusingonthreerelatedissues:thedegreetowhichadenition(a)isbeingforcedorincorrectlyused;(b)isnotarealempiricalimprovementoverconventionaltheoryormethod;or(c)leadstoincorrectresultsortoideasalreadyknownbyanothername.ScholarscanmodifyorfurthervalidatetheDTCStoexamineitsfurtherutility.LetusbrieyreviewthestepsoftheDTCS:Step1:ToanswertheDTCS’sinitialvequestions,researchersmustcombthroughtheirtopic’sliteraturetodetermineifandhowithasbeentheorizedasacomplexsystem.Ifsuchaliteraturedoesexist,thegoalistoorganizethechosendenitionofacomplexsystemintoitssetofkeycharacteristics:self-organizing,pathdependent,nonlinear,agent-based,etc.Forexample,ifourreviewofthecommunityhealthscienceliterature,weidentiedninecharacteristics.Ifnosuchliteratureex-ists,oriftheresearcherschoosetoexamineadifferentdenition,theymustexplainhowandwhytheychosetheirparticulardenitionanditssetofcharacteristics,in-cludingaddressingepistemologicalissuesrelatedtotranslatingortransportingthedenitionfromoneeldtoanother.Step2:Next,toanswertheDTCS’ssixthquestion,researchersmustdecidehowtheywilldeneandmeasureadenitionanditskeycharacteristics.Forexample,doestheliteratureconceptualizenonlinearityinmetaphoricalorliteralterms?And,ifmeasuredliterally,howwillnonlinearitybeoperationalized?Oncethesedecisions©SpringerInternationalPublishingSwitzerland2015B.Castellanietal.,PlaceandHealthasComplexSystems,SpringerBriefsinPublicHealth,DOI10.1007/978-3-319-09734-3_2 2DenitionalTestofComplexSystemsTable2.1Denitionaltestofcomplexsystems(DTCS) Step1:Literaturereviewandformulationofthedenition QuestionSet1:Whatdenitionofacomplexsocialsystemwillbeused?1.Whatisthedenition?a.Forexample,isthedenitiondictionaryinformorencyclopedic?b.Whatareitskeycharacteristics?2.Wheredoesthedenitioncomefrom?a.Forexample,isthedenitioncurrentlyusedintheeld,orisitanewdenition?3.Whatarethedenition’sepistemologicalassumptions?a.Forexample,isitpostmodern,criticalrealist,naïverealist,constructionist,etc?4.Whatisthetheoreticalbasisforthedenition?a.Forexample,isthedenitionmeanttobemetaphorical,literalorprescriptiveorsome5.Doesthedenitionoranyofitskeycharacteristicsseemtobeempiricallyortheoreticallyproblematic?a.Forexample,arethereexamplesintheliteraturewhereusageofthedenitionledto(i)poorlydesignedstudies,(ii)faultyempiricalresults,or(iii)awedoruncleartheoreticalconclusions? Step2:Methods QuestionSet2:Howwillthedenitionbeoperationalizedandtested?6.Howwillthecurrenttestbeconducted?a.Forexample,whatmeasureswillbeused?b.Whatcasestudywillbeused?c.Whatanalytictechniqueswillbeusedforthetest? Step3:Runtest Step4:Determineresults QuestionSet3:Whatconclusionsaboutthevalidityandvalueofthedenitionwere7.Didthetestsuggestthatthedenitionisempiricallyvalid?8.Didthetestsuggestthatthedenitionistheoreticallyvaluable?9.IntermsoftheDTCS’snegativehypothesis:a.Didthedenitionoranyofitskeycharacteristicsleadthetesttofaultyempiricalb.Didthedenitionoranyofitskeycharacteristicsleadthetesttoawedtheoreticalc.DoesthedenitionobeyOccam’srazor;orisitalotofworkforlittleempiricalortheoreticalyield? aremade,researchersmustdecidewhichmethodstouse.Aswehavealreadyhigh-lighted,choosingamethodologicalframeworkanditsassociatessetofmethodsisnoeasytask.So,socialscientistsarefacedwithamajorchallenge:theDTCSrequiresthemtotestthevalidityoftheirdenitionsofacomplexsystem,butsuchtestingnecessitatethemtoemployanewmethodology,whichmanyarenotequippedtouse.Itisbecauseofthischallengethat,forthecurrentproject,weemployedthe DenitionalTestofComplexSystemsSACSToolkit,whichwediscussnext.First,however,weneedtoaddressthenaltwostepsoftheDTCS.Step3:Oncequestions1through6havebeenanswered,thenextstepistoactuallyconductthetest.Thegoalhereistoevaluatetheempiricalvalidityofeachofadenition’scharacteristics,alongwiththedenitionasawhole.Inotherwords,alongwithdeterminingthevalidityofeachcharacteristic,itmustbedeterminedifthecharacteristicsttogether.Havingmadethatpoint,werecognizethatnotallcomplexitytheories(particularlymetaphoricalones)seektoprovidecomprehensivedenitions;optinginsteadtooutlinetheconditionsandchallenges,forexample,thateducationaladministratorsfacewhencomingtotermswiththecomplexityoftheirorganizations[51].Nonetheless,regardlessofthedenitionused,itscriterianeedtobemet.Step4:Finally,withtheanalysiscomplete,researchersneedtomaketheirnalassessment.Todoso,thefollowquestionneedstoasked:Intermsofthenegativetestfoundinquestion9andthenullhypothesisoftheDTCS,towhatextent,andinwhatwaysis(orisnot)thechosendenition,alongwithitslistofcharacteris-tics,empiricallyvalidandtheoreticallyvaluable?Withtheanswertothisquestiondetermined,thetestiscomplete.