visualrepresentationofadesignconceptisasharedviewforallinvolvedinthedesignprocessInadditiontheofconstructingandre ID: 146132
Download Pdf The PPT/PDF document "Astudyofprototypes,designactivity,anddes..." is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Astudyofprototypes,designactivity,anddesignoutcomeMariaC.Yang,DanielJ.EpsteinDepartmentofIndustrialandSystemsEngineering,3715McClintockAvenue,GER201,UniversityofSouthernCalifornia,LosAngeles,CA90089,USAThebuildingofprototypesisanimportantfacetoftheproductdesignprocess.Thispaperexaminesfactorsinprototyping,includingpartcountandtimespentonvariousdesignactivities,andtheircorrelationswithdesignoutcome.Theresearchquestionsasked:Dosimplerprototypesmeanamoresuccessfuldesign?Doestheamountoftimespentonaproject,bothoverallandondierentactivitiesoveraprojectcycle,relatetodesignsuccess?Anddoesitmatterwhenthistimeisspent?Oneofthemainndingsofthisstudyisthatprototypeswithfewerpartscorrelatewithbetterdesignoutcome,asdoprototypesthathavefewerpartsaddedtothemoverthecourseofdevelopment.Thispaperalsondsthatcommittingmoretimetoaprojectisnotnecessarilyassociatedwithasuccessfuldesignoutcome.2005ElsevierLtd.Allrightsreserved.Keywords:designprocess,engineeringdesign,prototyping,designhebuildingofprototypesisanimportantfacetoftheproductdesignanddevelopmentprocess.Simulatingadesignthroughprototypingcanreducedesignriskwithoutcommittingtothetimeandcostoffullproduction(HoudeandHill,1997).Bybuildingprototypesofdesignconcepts,questionsaboutadesignorspecicaspectsofadesigncanbeansweredconcretely.Willamaterialbesucientlysti?Willadesigncongurationperformasexpected?Furthermore,prototypescanbeaneectivewaytocomparedesignalternativesandaidinconceptselection.Wardetal.(1995)describethepracticeofbuildinglargenumbersofprototypestoexploredesignalternativesbeforeselectinganaldesign.Thisiscontrarytocommondesignwisdomthatcallsfordeepexplorationintheconceptualstage,beforefabrication.PrototypesarealsoameanstocommunicateanideatoothersKolodnerandWills,1996;SchrageandPeters,1999).Atangible,Correspondingauthor:Dr.MariaC.Yangwww.elsevier.com/locate/destud0142-694X$-seefrontmatterDesignStudies(2005)649doi:10.1016/j.destud.2005.04.0052005ElsevierLtdAllrightsreservedPrintedinGreatBritain visualrepresentationofadesignconceptisasharedviewforallinvolvedinthedesignprocess.Inaddition,theofconstructingandreninga3-Dphysicalprototypecanbringupdesignissuesinwaysthatalternativerepresentationsoftencannot.Whiletheprocessofde-velopingofa2-Ddrawingorevenacomputergeneratedsolidmodelcangenerateagreatdealofinformationaboutadesign,thatinformationwilllikelybedierentthanthatgainedbyphysicallycuttingmetal,forThispaperexaminesfactorsinprototyping,includingpartcountandtimespentondesignandfabricationactivity,andtheircorrelationswithdesignoutcome.Thisworkwasdoneinthecontextofamechanicalengineeringcourseinwhichteamsdesignedandfabricatedelectro-mechanicaldevices,andisbasedonpreliminaryresearchpresentedearlierbytheauthor(Yang,2004Questionsaboutprototypesthispaperexplores:Indesign,thereisanadage:Keepitsimple.Dosimplerdesignprototypesmeanamoresuccessfuldesignoutcome?Doestheamountoftimespentonaproject,bothoverall,andondesignandprototypingactivitiesoveraprojectcycle,relatetodesignIsthetimingofactivitiesinthedesigncycleassociatedwithcertaindesignoutcomes?RelatedworkAprototypeisanearlyembodimentofadesignconcept.Prototypescanrangefromsimple2-Dsketchesthatrepresentdesignthinking(etal.,1990;Goel,1995;SuwaandTversky,1997)tofoamcoremock-upstosophisticated3-Drapidprototypingdesignsthatarenearlyindistinguishablefromamanufactureditem.Bydenition,prototypesarenotproductionstagedesign..1WaysofbuildingprototypesPrototypesinmechanicalengineeringareoftenthoughtofintermsofproductiontechnologies.Manytraditionalmethodsofmanufac-ture,suchasmachining,casting,andmolding,areemployedinprototyping(Kieferetal.,2004).Rapidprototypingemployslayeredmanufacturingtoquicklybuildrealisticphysicalprototypes.Virtualprototypingrepresentsadesignconceptthroughdetailedcomputer650DesignStudiesVol26No.6November2005 .2PrototypingasadesignlanguageThereisasubstantialbodyofworkthatdescribeshowsketchingisadesignlanguage,awaytoexpressdesignthinking(Ullmanetal.,1990;SuwaandTversky,1997;Cross,1999).Thebuildingof3-Dprototypesmayalsobethoughtofasatypeofdesignlanguageinthesensethatitrepresentsandembodiesdesignthought.However,thecreationofprototypesrequiresasetofdesignskillsandtimecommitmentthatarealevelremovedfromhandsketching.Littleoverheadisrequiredfortheproverbialcocktailnapkindrawingofanidea,butmoreresourcesareneededtoconstructsomethingin3D,evenoutofsimplematerials..3PurposesofprototypesPrototypescanbethoughtofintermsoftheir,orthecategoriesofquestionstheyansweraboutadesign.Ullman(2003)describesfourclassesofprototypesbasedontheirfunctionandstageinproductTobetterunderstandwhatapproachtotakeindesigningaproduct,proof-ofconceptprototypeisusedintheinitialstagesofdesign.Later,aprototypeclariesadesignsphysicalembodimentandproductionfeasibility.prototypeshowsthattheproductionmethodsandmaterialscansuccessfullyresultinthedesiredproduct.Finally,aprototypedemonstratesthatthecompletemanufacturingprocessiseective.HoudeandHill(1997)proposeatrianglemodelofearlystageproof-of-conceptprototypesasshowninFigure1Eachcornerofthetrianglerepresentsadierentpurposeforaprototype:Tounderstandadesigns,theabilityofaproposeddesigntooperateinadesiredway.Inthemechanicalworld,aworkingprototypedemonstratestheessentialfunctionalityofadesign. Look and feel(Form)Implementation(Function) Figure1Whatprototypesprototype(HoudeandHill,Studyofprototypes,designactivity,anddesignoutcome651 prototyperepresentstheformandappearanceofadesign.Anexampleisanon-functionalindustrialdesignmodelmadeoffoamorrenderedina3-Dmodel(Ferguson,1992prototypesgiveasenseoftheusabilityofadesign.Storyboardsareoftenusedtoillustratehowandunderwhatcontextsaproductmightbeemployedbyanenduser.Aprototypeoftenservesmultiplepurposesatonce,movingalongpointsinthetriangle.Forexample,aprototypemaybeaproof-of-conceptforfunctionalperformance,andatthesametimegiveasenseofrolethroughtheprototypesdimensionsandweight.Iftheprototypeisbuilttoconsiderfunction,look-and-feel,androletogether,thenitisconsideredan.4StagesofprototypingPrototypesmayalsobecharacterizedbytheirstageindevelopment.Insoftwareengineering,Sommerville(1995)classiesprototypesasthrowaway,evolutionary,orincremental.Throwawayprototypesarerough,earlystageprototypesthathelpclarifyrequirements.Anevolutionaryprototypegoesthroughiterativestagesofbuildingandevaluation.IncrementalprototypesaremodicationstoexistingPetroski(1996)detailstheincrementalpathofseveralwellknownphysicalproducts.Buddeetal.(1992)describeaclassicationsystemsimilartoSommerville,includingevolutionary,experimental,andexploratoryprototypes..5FidelityofprototypesAprototypemaybeconsideredbyitslevelofrealism,or.Thebuildingofprototypesisoftenatrade-obetweendelityandthetime,eort,andcostrequiredtoproducethatprototype.Ideally,designersshouldchoosethecheapestprototypethatisstilleective,meaningaprototypethatcanbebuiltquicklyandinexpensivelybutstillprovidetheinformationthatthedesignerislookingfor(Dijketal.,1998).Forexample,adesignerneednotmachineacompleteworkingprototypeoutofaluminumwhenitissucienttoquicklyconstructasinglemechanismoutoffoamcoretounderstandacongurationissue.Supposeadesignteamwantstoexplorevariousdesignalternativesforasoftwareinterface.Interfacedesignersspeakofmockinguppaperprototypesthataresimplehand-sketchesofsoftwareinterfacesusedtoelicitvaluableuserfeedback(Wagner,1990).Ratherthaninvestinginwritingafullyfunctionalsoftwareprogram,eortisspentonlyontheinterfacedesignissuesathand.Likewise,productdesignersuseminimallyfunctional3Dprototypesforusabilitytesting.652DesignStudiesVol26No.6November2005 Eachoftheclassicationsabovecanbeusedtodescribeprototypes.Theintentofaprototype(function,look-and-feel,orrole)canberepresentedbyanystageofdevelopmentorlevelofdelity.Thatis,alook-and-feelmodelmaybeathrowawayprototype,oritcanbeahighlyfaithfulrepresentationofadesign..6DesigncorrelationsThisstudyisfurtherconcernedwithcorrelatingfactorsinprototypingwithdesignoutcome.Thereisagrowingbodyofresearchoncorrelatingfactorsinthedesignprocesswithdesignoutcome.MabogunjeandLeifer(1996)lookedattherelationshipbetweennoun-phrasesindesigndocumentationanddesignoutcome.Dongetal.(2004)considertheroleofcoherencyinteamdocumentationindesignoutcome.PreviousworkSongandAgogino(2004)Yang(2003)examinedsketchinganddesignoutcome.OneofthendingsbyYangwasthatdimensioneddrawingsappearedtobemoresignicantthandrawingsthatdonotincludedimensions.Itwasproposedthatdimensioneddrawingswereimportantbecausetheyaremoreconcretethandrawingswithoutdimensions,andastepclosertophysicalprototyping.Itisfromthisobservationaboutthevalueofprototypingthatthisstudygrows..1TestbedThisstudyexaminestheworkproducedinanadvancedmechanicalengineeringdesigncourseforjuniorsandseniorsattheCaliforniaInstituteofTechnologyintheFallof2002(Class1)andagainwithadierentsetofstudentsintheFallof2003(Class2).Eachcourseincludes23studentsdividedinto12teamsoftwo(oneteamconsistedofastudentandateachingassistant).Intheseteams,eachstudentisrequiredtodesign,buildandtesthisorherownstandalonedevicetocompeteinanaldesigncontest.Althoughteammatesareencouragedtodiscussideastogetherandwillcompeteintandem,eachdeviceisgradedindependentlysothattheymaybeassessedfairly.Eachclassisassigneditsownuniquedesignchallenge.InClass1,eachteamsdevicesstartoneithersideofalargecirculartable,thenmustremoveanaluminumagfromtheirside,transportittotheiropponentsside,andplanttheagthere.InClass2,eachteamstartsinsidealargealuminumcageplacedonthecenterlineofatable.Eachteammustmovethecagetotheiropponentssideofthecenterlinebytheendoftheallottedtime.Theclasseswereidenticalintermsofresourcesavailabletotheteamsandcoursematerials.However,theabilitiesofaclassmaychangefromStudyofprototypes,designactivity,anddesignoutcome653 yeartoyear.Onecomparisonpointistheaveragegradeofeachclass.TheaveragenumberofnalpointsinClass1was88.9outofapossible100pointswithastandarddeviationof8.4.InClass2,theaveragenumberofpointswas83.0withastandarddeviationof5.8.TheseaveragessuggestthatClass1mayconsistofslightlybetterperformersthanClass2.Eachparticipantbuildshisorherdeviceusingonlyelementsfromakitcontainingawidevarietyofpartsandmaterials,suchassteel,aluminum,andplasticsheet.Asidefrommotors,bearings,andgears,eachparticipantfabricatesvirtuallyeverycomponentofhisorherowndeviceinthemachineshop,includingtransmissionandwheels.AtypicaldeviceisshowninFigure2StudentsinbothClass1andClass2hadtakenarequiredintroductorycourseinengineeringdesignthatpresentedbasicdesignmethodologiesandmachineshopskillssuchasmillingandturning.InasurveyofthepreviousdesignexperienceoftheparticipantsinClass1(Yang,2003),itwasfoundthatpreviousself-reportedengineeringfabricationskillcorrelatedstatisticallysignicantlypositivelywithnalgrade.Itmakessensethatengineeringfabricationskilliscriticalinthetypeofhands-ondesignprojectsthatarethefocusofthisstudy,anditispossiblethatprototypingabilityoutweighsotherskillsorprocessesthataretaughttostudentsasfactorsindesignoutcome.However,engineeringintuitionasself-reportedonavepointscalecorrelatedstatisticallysignicantlynegativelywithcontestresult.Itissomewhatsurprisingthatstudentswhothoughttheypossessedbetterengineeringintuitiondidmorepoorlythanstudentswhodidnot.Thesendingsmaybeaneectofhavingnoviceengineersassesstheirownabilities. Figure2Exampleelectro-mechanicaldesignprototype654DesignStudiesVol26No.6November2005 Atthebeginningoftheproject,eachteamcreatesthreescenariosofpotentialdesignalternativesalongwithanon-functionalformmockupmadeofsoftmaterials.Theteamthenselectsonedirectiontopursueinthefourremainingmilestones.Ineachofthesemilestones,teamspresentprototypesorsub-assembliesandsketchestoshowtheirprogress.AscheduleofthesemilestonesislistedinTable1.Thenalweek,week10,ismarkedbythepubliccontestinwhichtheteamsfaceeachotherinadoubleeliminationcompetition.Theworkinthispaperisbasedonassessmentsofthenumberofpartsateachmilestoneaswellastimespentontheactivitiesindesign.Eachparticipantmaintainedalogoftimespenteachweekondesign,fabrication,debugging,andclassattendance.Studentswereinformedthatthehourstrackedwereforrecordkeepingpurposesonly,andwouldnotbetakenintoaccountforgrading..2DesignoutcomeThesuccessofadesignmaybeconsideredinmanyways.Deviceperformanceisanobviousmeasureofdesignoutcome,butinthiscourse,thefunctionalperformancemetricsofdevicesarenotpre-specied.Thisisinpartbecausetherearemultiplepossibledesignsolutionstotheproblem,andthenalcontestconditionscanchangedramaticallydependingontheopponentateamfaces.Forthepurposesofthisstudy,nalgradeandcontestperformancewereusedasmetricsfordesignoutcome.Thenalgradeinthecourseisacumulativemeasurethatevaluatesboththedesignprocessusedtoachievethedesignandthedesignitself.Themeasureofdesignprocessincludesassignmentsthatdevelopprojectobjectives,requirementsandfunc-tions,threeinitialdesignalternatives,anon-functional3-Dmockupoftheselectedapproach,andassessmentofeachstudentslogbook.Thedevicedesignitselfisjudgedatfourmilestones,andcriteriaconsideredincludetheoverallconcept,thedetailsofthedesign,thefabricationofthedevice,andtheplanninginvolvedinthedevice. Table1Scheduleofprojectmilestones(Class1andClass2)MilestoneWeekDescription14Fabricateakeycomponentorsub-assemblyofthedesign26Presentprototypewithadditionalcomponentsorsub-assembliesconstructedsincelastmilestone39Demonstratetheassembled,operationaldevice410DeviceisfullyfunctionalandcontestreadyStudyofprototypes,designactivity,anddesignoutcome655 Thenalcontestranksateamsperformanceincompetition.Notethatcontestperformanceandgradeareindependentofeachother.Astudentcanearnagoodgradebasedonhisorherperformanceinthecourseassignments,butdopoorlyinthecontest,andviceversa.Themainconstraintsthatstudentsfacearetimenecessaryfordesign,fabrication,anddebugging,andbythematerialsavailableforprototypingintheirkit.TheSpearmanRankingCorrelation(Siegel,1956)fornonparametricpopulationswasemployedtotestforcorrelationsbetweendesigndataanddesignoutcome.TheSpearmancorrelationcoecientcomputedinEq. isthenumberofindividualsandaretheordinalranksofthevariablesbeingcorrelated,inthiscasedesigndataanddesignoutcome.cantakeonavaluebetween1and1.If0,thereisanegativecorrelationbetweenthetwodatasets.If01,thereisapositivecorrelation.Forexample,inapopulationofcollegestudentswhohavetakenintroductoryandadvancedversionsofacourse,itmightbeexpectedthatthestudentrankingintheintroductorycoursewillcorrelatesomewhatwiththerankingintheadvanced.Ifthetopstudentbyrankintheintroductorycourse()isthefourthrankedstudentintheadvancedcourse(andthatthedierence3.Intheseclasses,23,andifthecorrelationcoecientexceeds0.415,itisconsideredforasignicancelevel,oraprobabilityoferror,of(twotailed).Resultsanddiscussion.1PrototypesThefollowingresultsforprototypesarefromClass2only.NodataonpartcountwaskeptforClass1becausethestudyofprototypesinClass2infactgrewoutofpreliminaryresearchonsketchinginClass1.Theseprototypescouldallbedescribedasproof-of-concepts,usingcategorizations.Prototypeswerefabricatedusingmaterialremovalmethods(millingandturning)onmetalsandplasticsandpolymercasting.ApplyingtheclassicationschemeforprototypesofHoudeandHill(1997),theintentoftheprototypeswasfoundtobe.Prototypespresentedateachmilestonewere656DesignStudiesVol26No.6November2005 constructedtoseeiftheywouldoperateasintended.Appearanceandusabilityweresecondaryconsiderations.PrototypescouldbefurthercategorizedasSommerville,1995)becausegenerallyeachwasaniterationofthepreviousone.Componentsintheprototypesweregenerallyfabricatedofnalkitmaterialsratherthantemporarysoftmaterials.Becausetheseprototypeswouldbelikelyusedinthenaldevice,thelevelofdelityoftheprototypesisconsideredtobehigh.Asexpectedwithrealisticprototypes,theresourcesrequiredtoproducethesepartswerealsohigh.Animportantelementofprototypingisthelearningthatthedesigneracquiresduringfabrication.Inlecture,studentswereencouragedtoposequestionsabouttheirdesignsandthenanswerthesequestionsthroughbuilding.Thisiscloselyrelatedtothetraditionaldesignbuildtestcycleofdesign.Intheirlogbooks,studentsaskedquestionsthatrangedfromthegeneral(Willmyideawork?)tothespecic(Will1/8plasticmakeastrongenoughchassis?).Itwasobservedthatstudentssoondiscoveredthatansweringtheirquestionsthroughbuildingprototypeswouldleadtootherquestionsthatcouldnothavebeenconsideredbefore(1/8plasticisstrongenoughforachassis,butthepartImadeshowsthatcracksformwhereholesaredrilledclosetoeachother.).Ineect,prototypingOnemeasurerelevanttoprototypingispartcount.Thisisthenumberofcomponentsineachprototypepresentedateachmilestone,notincludingmanufacturedcomponentssuchasmotorsandscrews.Figure3liststheaveragenumberofindividualpartsfoundineachdeviceateachmilestone. Milestone 1Milestone 2Milestone 3Milestone 4 Figure3Averagenumberofpartsperdevice(Class2only)Studyofprototypes,designactivity,anddesignoutcome657 Noticethatthenumberofpartsisalwaysincreasing.Iftherewasalevelingout,ordrop,inthenumberofparts,itcouldbeinferredthatdesigniteration(swappingoutpartsforothers)orrenement(reducingthenumberofparts)wasoccurring.Thissteadyriseinpartcountsuggeststhatdesignandprototypingeortisspentonincrementallybuildingupaprototype.Thisisconsistentwithphotographicrecordsthatsuggestthatnewpartsarebeingaddedincrementally.Table2showsthecorrelationofnumberofpartswithdesignoutcomeateachofthemilestones.Statisticallysignicantnegativecorrelations(valuesinitalic)areseenwiththelastmilestoneandnalgrade,andwiththethirdandfourthmilestonesandcontestresults.Thisnegativecorrelationmeansthatoverallnumberofpartsinthenaldevicecorrelateswithgradeandcontestranking.Inbothcontestandgrade,thecorrelationtendstoincreasewithtime.Table3showsthechangeinthenumberofpartsfrommilestonetomilestonecorrelatedwithdesignoutcomemeasures.Thiscorrelationisstatisticallysignicantbetweenthelastmilestoneattheendoftheprojectforbothgradeandcontestranking.Broadly,thissuggeststhatthenumberofparts,thebetterthecorrelationwithimproveddesignoutcome.CoupledwiththendingsfromTable2thissuggeststhataprototypethatnotonlystartsoutwithfewercomponents,buthasfewerpartsaddedtoitisassociatedwithabetterdesignoutcome..1.1LevelsofintegrationCommondesignwisdomholdsthatintegrationofdesignprototypesearlyonwillleadtoabetterprototypeintheend.Byassemblingcomponentsandsub-assembliesaheadoftime,theliteralnuts-and-bolts Table2Correlationbetweennumberofpartsanddesignoutcome(Class2only)withgradewithcontest20.150.415isconsideredstatisticallysignicantfor0.05(twotailed).Inthesecases,datafromoneparticipantwasunavailable,making22insteadof23.Forsuchcases,thethresholdvalueincreasesto0.425for0.05(twotailed).658DesignStudiesVol26No.6November2005 issuesassociatedwithprototypeintegrationareworkedoutovertimeratherthanallatonceattheendofaproject.Onemeasureofhowquicklyintegrationtakesplaceistheratioofassembledpartsateachmilestonetothenumberofassembledpartsinthenalprototype(Milestone4).Partsthathavebeenfabricatedbutnotyetassembledarenotincludedinthiscount.Table4showsthatthisratiogenerallyincreaseswitheachsuccessivemilestone.Table5showsthecorrelationoftheseratioswithgradeandcontestThereisastatisticallysignicantcorrelationbetweentheratioofassembledpartsinMilestone2andMilestone4withgrade.Milestone1(week4)istherstreviewinwhichstudentsmustpresentapieceofhardwarefabricatedfromkitmaterials.ByMilestone2(week6),theprototypesareexpectedtohavesomedegreeoffunctionality,anditwouldmakesensethatbythispointstudentsshouldhavemadesignicantprogresstowardstheirnaldesign.However,therearenostatisticallysignicantcorrelationswiththeratioofMilestones3to4,whichmaybeinpartbecausebythatstageoftheproject,morestudentshaveintegratedtheircomponents,andtheratioislessofadierentiatorbetweenteams. Table3Correlationbetweenchangeinnumberofpartsanddesignoutcome(Class2only)ChangeinnumberofpartswithgradewithcontestMilestone2Milestone3Milestone4Milestone40.415isconsideredstatisticallysignicantfor0.05(twotailed).Inthesecases,datafromoneparticipantwasunavailable,making22insteadof23.Forsuchcases,thethresholdvalueincreasesto0.425for0.05(twotailed). Table4Ratioofassembledpartsateachmilestonetothefinalnumberofassembledparts(Class2only)AverageratioMilestone1/Milestone40.11Milestone2/Milestone40.28Milestone3/Milestone40.77Milestone4/Milestone41.00Studyofprototypes,designactivity,anddesignoutcome659 .2TimeWenowconsidertheroleoftimespentondesignanddevelopmentactivities.TheseresultsareforbothClass1andClass2.Table6theaveragenumberofhoursspentinfourproject-relatedactivitiesoveronesemester,includingdesign,fabrication,debugging,andclasstime,alongwiththepercentageoftotaltimespentoneachactivity.isconcernedwithconceptualdesignincludingcongurationlayoutandcomponentdesign.relatestoeortspentconstructingandassemblingthedevice.Thisisusuallycloselylinkedtotheamountoftimespentbythestudentdesignerinthemachineshop.focusesontestingthedevicetoassessperformance.istimespentintheclassroom.ThedierenceintotaltimebetweenClass1andClass2isabout9%,mostofwhichcanbeattributedtomorefabricationtimeforClass2.Figures4and5showtheaveragenumberofhoursspentperweekeachactivityforbothclasses.Theoverallpatternisfortheamountoftimeontheprojecttoincreasesteadily.AsubtledierenceisseeninClass1wheretheincreaseintimespentovertherst8weeksofclassis Table5Correlationbetweenratioofassembledpartsateachmilestonetothefinalnumberofassembledparts(Class2only)withgradewithcontestMilestone1/Milestone4Milestone2/Milestone4Milestone3/Milestone40.380.190.415isconsideredstatisticallysignicantfor0.05(twotailed).Inthesecases,datafromoneparticipantwasunavailable,making22insteadof23.Forsuchcases,thethresholdvalueincreasesto0.425for0.05(twotailed). Table6AveragetimespentonactivitiesandasapercentageoftotaltimeActivityClass1Class2Averagehours%TotalAveragehours%TotalDesign46244722Fabrication1005211354Debugging31163215Class168199Total193100211100660DesignStudiesVol26No.6November2005 moregradualthaninClass2,meaningthatmoretimespentearlyon(asapercentage)thaninClass1.TimespentdesigningdropsoinMilestone1(week4),presumablyasstudentsturntheirfocusonfabrication.Thisisoneconnectionbetweendrawingandprototyping.Designtimeincludesdrawing,sketching,andplanning.Afterideasaresolidiedinstudentssketchbooksintwodimensions,thestudentsmovetothemachineshop(fabricationtime)toimplementtheideasin3D.ByMilestone2(week6),thereisasteadyincreaseinprototypingeorts.ByMilestones3and4(weeks9and10),however,proportionatelymoretimeisspentdebuggingtheprototype.Notethatweeks9and10areshortweeksduetoaholidayandthecontestitself,andareaccompaniedbyadropintheamountoftimespent.Table7listshowtimespentonindividualdesign-relatedactivitiesoverthelifeoftheprojectcorrelatewithdesignoutcome.ForClass2,Table7showstwostatisticallysignicantnegativecorrelationsbetweentotaltimespentoverthetermonfabricationandcontestperformance,andtotaltimespentonthecourseoverthetermandcontest.Thatis,timeonfabrication,aswellaslesstimeoverallcorrelateswithbetterperformanceinthenalcontest.Inthepreviousyears 12345678910Average hours per participant debug fabrication design class Figure4Averagenumberofhoursspentperparticipantperweekbyactivity(Class1) 12345678910 debug Figure5Averagenumberofhoursspentperparticipantperweekbyactivity(Class2)Studyofprototypes,designactivity,anddesignoutcome661 course,nocorrelationisfoundwithanyoftheactivitiesandeithergradeorcontestperformance.ThismaybeinpartbecauselesstimewasspentoverallinClass1thaninClass2,andthetimewasspentearlieron.Itisalsousefultounderstandhowtimespentweeklyonvariousactivitiescorrelateswithoutcome,asshowninTable8(Class1)andTable9(Class2)..InbothClass1(inweek2)andClass2(inweek3),thereisastatisticallysignicant,positivecorrelationwithtimespentondesignandgrade.Thisisconsistentwiththendingsimpliedby.Thatis,timespentsketchingdimensioneddrawingsearlyoncancorrelatewithbetterdesignoutcome.Class1furthershowsastatisticallysignicantnegativecorrelationinweek7withcontest,meaningthattimespentondesignlaterintheprojectdoesnotcorrelatewellwithcontestperformance. Table7Correlationbetweentimespentonactivitiesanddesignoutcome(Classes1and2)Class1Class2withgradewithcontestwithgradewithcontestDesign0.340.240.22Fabrication0.080.01Debugging0.270.240.38Totaltime0.14Inthesecases,datafromoneparticipantwasunavailable,making22insteadof23.Forsuchcases,thethresholdvalueincreasesto0.425for0.05(twotailed). Table8Design,fabrication,debugging,andtotaltimespentperweekcorrelatedwithgradeandcontest,byweek(Class1only)DesigntimeFabricationtimeDebuggingtimeTotaltimeGradeContestGradeContestGradeContestGradeContestWeek10.220.320.390.110.190.290.410.32Week20.110.250.31Week30.080.270.030.320.110.29Week40.110.500.450.260.260.380.15Week50.230.420.000.210.300.280.200.01Week60.260.230.050.140.360.170.20Week70.350.110.080.300.140.05Week80.310.080.370.360.14Week90.030.10Week100.240.000.110.070.120.160.425for0.05,(two-tailedtest).662DesignStudiesVol26No.6November2005 .InClass1,therearepositive,statisticallysignicantcorrelationswithfabricationtimeanddesignoutcomeinweeks3and4thatsuggestthattimespentonbuildingearlyoncorrelateswellwithdesignoutcome.Furthermore,thelevelofcorrelationgenerallydeclinesovertime.Incontrast,Class2hasnegative,statisticallysignicantcorrelationswithfabricationtimeanddesignoutcomeinweeks3,4,6,9,and10,andthelevelofcorrelationdoesnothaveacleartrendovertime.Interestingly,Class2spentmoretimeonfabricationoverallthanClass1.PossiblereasonsthatthepatternofcorrelationdieredforthetwoclassescouldbethatClass1workedmoreecientlythanClass2,orthattheypossessedmorefabricationskills.Clearly,thisisanavenueforfurtherstudy..NocorrelationswerefoundwithtimespentdebuggingandoutcomeforClass1.Class2showsanegative,statisticallysignicantcorrelationinweeks6and7withcontestperformance,meaningthemoretimespentondebuggingaprojectcorrelateswithpoorercontestranking.Onepossiblereasonforthisisthat,asapercentageofoveralltime,thoseinClass2spentmoretimelaterondebugging,ratherthanearlierasClass1had.Notethatinweeks1noneoftheparticipantsspentanytimedebuggingsonocorrelationcoecientcouldbecomputed..Apositive,statisticallysignicantcorrelationisfoundbetweenweek3anddesigngradeforClass1.Class2showsaseriesofnegative,statisticallysignicantcorrelationswithcontestinthesecondhalfoftheprojectcycle.Aswiththeresultsforfabricationtime(above),theClass1and2resultsmayactuallybethoughtofasconsistentbecausetheysuggestthatoveralltimespentearlyoncorrelateswell,buttimespentlaterondoesnotcorrelatewell. Table9Design,fabrication,debugging,andtotaltimespentperweekcorrelatedwithgradeandcontest,byweek(Class2only)DesigntimeFabricationtimeDebuggingtimeTotaltimeGradeContestGradeContestGradeContestGradeContestWeek1Week2Week3Week40.240.190.090.010.21Week50.200.210.220.340.08Week60.33Week7Week80.270.400.26Week90.090.320.23Week100.000.130.030.425for0.05,(two-tailedtest).Studyofprototypes,designactivity,anddesignoutcome663 Nowconsidertheamountoftimespenteachweekonallactivitiesasapercentageofthetotalspentbyeachparticipant.Thisnormalizesforthefactthatsomestudentsmayworkatadierentpacethanothers,and,moreimportantly,showstheeectoftimespentatdierentpointsovertheprojectcycle.ThisrunningpercentageoftimespentbyweekisshowninTable10Table10,Class1showsapositive,signicantcorrelationbetweenthepercentageoftimespentontheprojectinweeks7and8andwithdesigngrade.ForClass2,thereisapositive,signicantcorrelationbetweenthepercentageoftimespentonaprojectandcontestresultsfromweek3through5.Thehigherthepercentageoftotaltimespentintheseweeks,thebetterthecontestresults.Becausethisisarunningtotal,thisimpliesthatadesignerinClass2whospendsacriticalamountoftimebythethirdweekandcontinuestodosofortheremainderoftheprojectwillcorrelatewithabettercontestresult.ForClass1,thistimeframeisshifteddowntoweek7.ThisshiftmaybepartiallyattributedtotheamountoftimespentearlieroninClass1.Wenowconsidertheanswerstothequestionsposedintheintroductioninlightofthendingsofthisstudy..1Dosimplerprototypesmeanamoresuccessfuldesign?Thendingsinthisstudysuggestthatoverallnumberofpartsinadevicecorrelatewithgradeandcontestrankingforsome Table10Correlationbetweenrunningpercentageoftimespentbyeachparticipantwithfinalgradeandcontest(Class1andClass2)%ofoverallClass1Class2withgradewithcontestwithgradewithcontestWeek10.190.300.010.28Week20.250.270.060.35Week30.110.370.30Week40.180.360.36Week50.210.280.36Week60.320.260.230.40Week70.200.070.38Week80.320.090.36Week90.190.100.150.410.425for0.05,(two-tailedtest).Inthesecases,datafromoneparticipantwasunavailable,making22insteadof23.Forsuchcases,thethresholdvalueincreasesto0.425for0.05(twotailed).664DesignStudiesVol26No.6November2005 milestones.Simplicityisacommongoalindesign,andonemeasureofthisispartcountateachmilestone.Intuitively,havingfewerpartsmeanslesstodesign,fabricate,assemble,debug,andmaintain.Considertherelationshipofsimplicitytodesignquality.Often,verysimpleproductsarethoughtofasmoreelegantandbetterthoughtoutthanmorecomplicatedones.Infactintheseprojects,itwasobservedthatthebasicdesignswereoftenverysimilar.Inbothclasses,manyteamssettledonatleastone3-or4-wheeledcarthatincludedacustomattachmentfortheparticulardesignchallenge,suchasawinchorapairofgrippers.Thisclassofsolutionswasjudgedbytheteachingstatohavelowriskoffailurewithareasonableprobabilityofsuccessinthecontest,andwasconsideredrelativelystraightforwardtodesignandfabricate.Ingeneral,informalobservationoftheteamsshowedthatsimplerdevicesgenerallydidbetterthanothersinboththecontestandingrading.Notsurprisingly,oneofthekeyguidelinesinDesignforAssembly(DFA)methodology(BoothroydandDewhurst,1989;Ullman,2003)istheminimizationofpartcount.InDFA,itisoftenthecasethatpartcountisreducedbyattributingadditionalfunctionalitytoexistingparts.Inthisstudy,thecomplexityofpartsthemselveswasnotassessed,soitisnotknownwhetherapartwasintendedtoservemultiplefunctions,oronlyone.Thisstudyalsoshowsapositiveassociationbetweendesignsthatlimitthenumberofpartsaddedovertimeanddesignoutcome.Thesteadyincreaseinpartsovermilestonessuggeststhatpartfabricationwaslikelyfornewcomponents.However,thiswaslikelyasubstantialamountof,andofexistingpartsinadditiontothecreationofnewcomponents,althoughjusthowmuchofthisactivityoccurredwasnottracked..2Doestheamountoftimespentonaproject,bothoverallandondierentactivitiesoveraprojectcycle,relatetodesignsuccess?Timeisacritical,limitedresourceonanydesignproject.Intuitively,itwouldmakesensethatspendingmoretimeonaprojectwouldleadtoabetterdesignresult.Thisbeliefwasbylookingatthetimespentdebugginganditscorrelationwithnalgrade.Moretimespentdebuggingadesignsuggeststhataprototypewillfunctionmorereliablyorconsistently.Studyofprototypes,designactivity,anddesignoutcome665 InClass2,itwasfoundthatspendingtimeonfabricatingaprototypewascorrelatedwithbettercontestresultsasdiscussedabove.Thisisnotablebecausethepercentageoftimespentbytheteamsonfabricationwasgreaterthantheotherthreeactivitiescombined.Thetimespentinbuildingadesign,thetherankinginthenaldesigncontest.Whilethismaybecounterintuitive,itcouldbeseenasconsistentwiththendingthatsimplerdesignscomprisedoffewerpartswereassociatedwithmoresuccessfuldesignoutcomes.Havingfewerpartsinadesignimpliesthatlesstimewillbespentonallphasesofthedesigncycle,includingfabrication.Therewas,howeveraverydierentcorrelationfortotalfabricationtimeinClass1.InClass1,thereisapositivecorrelationearlyonwithdesignoutcome,whilearoundthesametimeperiod,thecorrelationisnegativeinClass2.Class2spentmoretimetotal,andslightlymoretimeasapercentage,onfabricationthanClass1did.ThisadditionaltimeonfabricationinClass2couldimplyanumberofthings:thatClass2sprojectsweremoreambitiousthanClass1s,thatClass2sstudentswerelessskilledatfabricationandrequiredmoretimetocompletetheirprojects,orthatClass2sstudentssimplychosetospendmoretimeinthemachineshop.WealsoknowthatClass2soverallgradewasslightlylowerthanClass1s.Inthisstudy,theoverallquantityoftimespentonallactivitiesintheprojectcorrelatedinastatisticallysignicant,negativewaywithcontestperformanceinClass2.Again,thisissomewhatunexpected,butitmaybeappropriatetoexplainthisresultbyreferringbacktotheearlierndingsaboutsimplicity.Simplerprototypesarelinkedwithbetterdesignoutcomes,anditmightbereasonabletoassumethatsuchdesignswouldrequirelesstimetodesign,build,anddebug.Thisndingalsosuggeststhatmerelyputtinginthetimeisnotsucientfordesignsuccess,butthatitisrelatedtohavingtheforesighttocomeupwithamanageabledesignscope.Timeisnotaproxyforquality.OnecanspendendlesshoursinthemachineshopandstillnotproduceagoodForClass1,thecorrelationforoveralltimewasalsonegative,butnotinastatisticallysignicantway.ThisdiscrepancymaybepartiallyexplainedbythefactthatClass1spentlesstimeoverallthanClass2,whichcouldbeinterpretedtomeanthattheyworkedsmarter.Itwasfoundthattheproportionofoveralltimespentontheproject,week-by-weekonallactivitieshadasignicantcorrelationwithcontest666DesignStudiesVol26No.6November2005 results.Thisresultimpliesthatdesignerswhomeetathresholdleveloftimecommitment(asapercentageoftheiroveralltime)andmaintainthatcommitmentaresomehowlinkedtodoingbetter.Aparticipantwhoslacksoforthersthalfoftheprojectisunlikelytocatchuplateron.Thisisconsistentwiththedescriptionofadesignactivityknownaspatching(Ullman,2003)duringwhichadesignisalteredwithoutaectingitslevelofdetail.Thisisdierentthanrening,inwhichmoredetailisaddedtoadesign.Ingeneral,thegoalofpatchingistomakeanexistingdesignworkbyrearrangingexistingpartsorotherwisemodifyingthem.Inasituationwheretimeisoftheessence,suchasoccurslateinthedesigncycle,patchingmaytheonlywaytomakeadesignfunctional,ratherthandesigningandfabricatingnewThisstudyshowscorrelationsthatsuggestthatitismoreusefultospendtimeconsistently,andtoputfortheortsonawellscopeddesign..3Isthetimingofactivitiesinthedesigncycleassociatedwithcertaindesignoutcomes?Finally,intermsoftiming,Tables8and9suggesttwotrendsthat,takentogether,maypointtothesameconclusion.Table8impliesthatworkperformedearlieron(byWeek4)tendedtohavepositive,statisticallysignicantcorrelationswithcertaindesignoutcomes.Table9asimilarresultfordesigntimeforClass2.Inaddition,itshowsstatisticallysignicant,negativecorrelationswithoutcomeinthelaterstagesofdesign(Week3forfabricationtime,andWeek6fordebuggingandtotaltime),suggestingthattimespentlateronisnotassociatedwithItisinterestingtonotethatintheearlyweeksoffabrication,Class1exhibitsapositivecorrelation,whileClass2exhibitsanegativecorrelation.Thesendingsareatodds,butmaybeexplainedinpartbythewayeachclassapportioneditsfabricationtime.Class1spentmoretimeproportionallyupfrontthanClass2,suggestingthatstudentsinClass2mayhavespentthelatterpartofthetermplayingcatchup.Thisworkhasimplicationsforthedevelopmentofsoftwaretoolstosupportprototyping.First,theresultsfromthisstudyfurthervalidatethenotionthatsimplerdevicesareassociatedwithbetterdesignoutcome.Italsoshowsthatspendingproportionatelymoretimeintheearlystagesofdesignonprototypingcorrelateswithbetterdesign.Takentogether,theseresultsimplythatearlystageprototypesareanimportantareatofocuson.SuchprototypesarelikelythrowawayStudyofprototypes,designactivity,anddesignoutcome667 prototypesintendedtofacilitatethinkingaboutadesign.CurrentsolidmodelingandCADtoolsarehighlysophisticated,andareexcellentforrepresentingdesignsattheirlaterstages,buttheyrequirealevelofeort,certaintyandrenementinadesignthatcanmakethemapoorchoiceforanearlystageprototype.Thisstudypointstoaneedforresearchinanddevelopmentofsoftwaretoolstobettersupportearlystageconceptgenerationandprototyping.Theauthorgratefullyacknowledgesthesupportandguidanceoftheinstructorsofthecourse,Prof.ErikAntonsson,Prof.JoelBurdick,andDr.CurtisCollinsattheCaliforniaInstituteofTechnology,andthecommendabledesigneortsofthestudentsthatarethebasisofthisresearch.Theauthoralsoacknowledgesthegeneroussponsorsofthecourse:AppliedMaterials,Amerigon,Dr.DavidandMrs.BarbaraGroce,Honeywell,Idealab!,MabuchiMotor,NorthropGrumman,TheSanDiegoFoundation,andToro.Boothroyd,GandDewhurst,PProductdesignforassemblyBoothroydDewhurstInc.,Wakeeld,RIBudde,R,Kautz,KandKuhlenkamp,KPrototyping:anapproachtoevolutionarysystemdevelopmentSpringer,BerlinCross,N(1999)NaturalintelligenceindesignDesignStudiesVol20No1pp25Dijk,L,Vergeest,JSMandHorvath,I(1998)TestingshapemanipulationtoolsusingabstractprototypesDesignStudiesVol19No2pp187Dong,A,Hill,AWandAgogino,AM(2004)Adocumentanalysismethodforcharacterizingteam-baseddesignoutcomesJournalofMechanicalVol126No3pp378Ferguson,ESEngineeringandthemindseyeTheMITPress,Cambridge,MAGoel,VSketchesofthoughtMITPress,Cambridge,MAHoude,SandHill,C(1997)Whatdoprototypesprototype?inMHelanderTLandauerandPPrabhuHandbookofhumancomputerinteractionElsevierScience,AmsterdamKiefer,S,Silverberg,LandGonzalez,M(2004)Acasestudyofprototypingmethodsanddesignformanufacture:electrostaticwindowblindsofEngineeringDesignVol15No1pp91Kolodner,JLandWills,LM(1996)PowersofobservationincreativeDesignStudiesVol17No4pp385Mabogunje,AandLeifer,L(1996)210-NP:measuringthemechanicalengineeringdesignprocess,inTwenty-sixthAnnualFrontiersinEducationConferenceonTechnology-basedRe-engineeringEngineeringEducation,SaltLakeCity,UTPetroski,HInventionbydesign:howengineersgetfromthoughttoHarvardUniversityPress,Cambridge,MA668DesignStudiesVol26No.6November2005 Schrage,MandPeters,TSeriousplay:howtheworldsbestcompaniessimulatetoinnovateHarvardBusinessSchoolPress,Boston,MASiegel,SNonparametricstatisticsforthebehavioralsciencesMcGraw-Hill,NewYork,NYSommerville,ISoftwareengineeringAddison-Wesley,Wokingham,Song,SandAgogino,AS(2004)Insightsondesignerssketchingactivitiesinproductdesignteams,in2004ASMEDesignEngineeringTechnical,SaltLakeCity,UtahSuwa,MandTversky,B(1997)Whatdoarchitectsandstudentsperceiveintheirdesignsketches?AprotocolanalysisDesignStudiesVol18No4ppUllman,DGThemechanicaldesignprocessNewYork,NYUllman,DG,Wood,SandCraig,D(1990)TheimportanceofdrawinginthemechanicaldesignprocessComputers&GraphicsVol14No2ppWagner,A(1990)Prototyping:adayinthelifeofaninterfacedesigner,inBLaurelTheartofhumancomputerinterfacedesign,Addison-Wesley,NewYorkWard,A,Liker,JK,Sobek,DandCristiano,J(1995)ThesecondToyotaparadox:howdelayingdecisionscanmakebettercarsfasterManagementReviewVol36No3pp43Yang,MC(2003)Conceptgenerationandsketching:correlationswithdesignoutcome,in2003ASMEDesignEngineeringTechnicalConferencesChicago,ILYang,MC(2004)Anexaminationofprototypinganddesignoutcome,in2004ASMEDesignEngineeringTechnicalConferences,SaltLakeCity,UtahUSAStudyofprototypes,designactivity,anddesignoutcome669