Assessingd.learning:CapturingtheJourneyofBecomingaDesignThinkerShelley - PDF document

Assessingd.learning:CapturingtheJourneyofBecomingaDesignThinkerShelleyGoldman,MaureenP.Carroll,ZandileKabayadondo,LeticiaBritosCavagnaro,AdamW.Royalty,BernardRoth,SweeHongKwek,andJainKimAbstractTheresearchexploredtherelationshipoflearningdesignthinkingandassessingthatprogress.Itaddressedthequestions:Howcanweunderstandwhatislearnedindesignthinkingclasses,andhowassessmentsmightcontributetothatprocessinauthenticways?Thestudyfollowedareciprocalresearchanddesignmethodologywherebasicresearchandthedesignofassessmentsolutionswereongo-ing,reciprocal,andrelatedtoeachotherinorganicways.Welearnedthatthelearningofdesignthinkingdispositionsandmindsetsisanemergentjourney—withvariouslevelsofsophistication,transformation,application,andintegration.Weintroducetheconceptoftorepresentthedevelopingandnascentepistemologicalviewpointsandinstinctsthatarestrengthenedwhilebecomingadesignthinker.WeS.Goldman(StanfordUniversitySchoolofEducation-Professor(Teaching)ofEducationand,bycourtesy,ofMechanicalEngineeringM.P.CarrollStanfordUniversitySchoolofEducation-ResearchStaffZ.KabayadondoStanfordUniversitySchoolofEducation-PhDCandidateL.B.CavagnaroStanfordUniversityTechnologyVenturesProgram-AssociateDirectorA.W.RoyaltyStanfordUniversityHassoPlattnerInstituteofDesign-AcademicStaffB.RothStanfordUniversitySchoolofEngineering-ProfessorandHassoPlattnerInstituteofDesign-AcademicDirectorS.H.KwekStanfordUniversitySchoolofEducation-Master’sDegreeGraduateJ.KimStanfordUniversitySchoolofEducation-Master’sDegreeGraduate H.Plattneretal.(eds.),DesignThinkingResearch,UnderstandingInnovation,DOI10.1007/978-3-642-31991-4_2,Springer-VerlagBerlinHeidelberg2012 reviewdesignsfortoolsthatwerebasedontheconceptofmindshiftsthatincludereectiveandperformanceassessmentsandanassessmentdashboard.1IntroductionandBackgroundAsstudentsbecomedesignthinkers,theyemergewithsignicantchangesintheirapproachestoproblemsolvingandtonewchallenges.Theystarttodevelopasenseofresiliencythatenablesthemtothink“outsidethebox.”ThisresearchexploredtheexperiencesofstudentswhiletheywereenrolledinauniversitycourseattheHassoPlattnerInstituteofDesignatStanfordUniversityandtookupquestionsabouttherelationshipbetweenlearningdesignthinkingandassessingprogress.Topursuethemetaphor,assessmentisoftendescribedasa“blackbox”activity,withstudentsbeingtestedandassessedoutsideoftheactuallearningprocess.BlackandWiliam()describehowpoliciesintheU.S.andinmanyothercountriesseemtotreattheclassroomasablackbox,wherecertainfromtheoutsidearefedintothebox,andfromwhichstudentsemergewithcertaincompetencies.Ourgoalwastounderstand,document,anddesigntoolsthatcouldhelpassesswhatstudentswerelearningindesignthinkingcourses.Wesoughttobringassessment“intothebox”oftheuniversitydesignthinkingcoursesasstudentswerelearninghowtothink“outsidethebox”.Wesoughttounderstandthelearningexperienceofthestudentsinwaysthatwouldprovideareectionofprogressmadetowardsdesignthinkingbasedontheirauthentic,course-basedexperiences.Wehopedtheworkwouldhaveimplicationsforbothstudentsandtheircourseinstructors.Studentscouldbetterreectonandhaveinformationabouttheirevolvingdesignlearning.Instructorswouldbeabletoreectontheprogressoftheindividualandtheirdesignteam,andusetheinformationtoreorganizeteachingandcurriculumtobettermeettheneedsofstudents.Theresearchaddressedthefollowingkeyquestions:Howcanweunderstandwhatislearnedindesignthinkingclasses,andhowmightassessmentscontributetothatprocessinauthenticandhelpfulways?Thestudyfollowedareciprocalresearchanddesign(RR&D)processandmethodology(Alexanderetal.).IntheRR&Dmodel,basicresearchandthedevelopmentofsolutionsareon-going,andarereciprocalandrelatedtoeachotherinorganicways.Inthisproject,interimndingsofstudiesofstudentsindesignthinkingcoursesandactivitieswereutilizedasneedsassessmentsforemergentassessmentdesigns.Thiswasdoneinsuchawaythataftereachroundofdatagatheringandanalysis,thepreliminaryresultscontributedtothecreationofperfor-manceandreection-basedassessments.DesignthinkingaspectsoftheRR&Dmodelinuencedtheresearch,whichincludedinterviews,observations,analysis/synthesis,andideationtoiteratesuccessiveprototypesofassessmenttoolsforbothstudentsandtheirinstructors.Onesuchimplementemergingfromthisprocesswastheconceptofadashboardwithtoolsfortrackinghowstudentsarelearningtheartandpracticeofdesignthinking.14S.Goldmanetal. Thispaperchroniclestheresearchanddesignprocessanditsresults.TheresearchbeganbybuildingonandadaptingndingsfromapriorstudywecompletedonassessmentthatoccurredinK-12educationalsettingsandresultedinthedevelopmentofaDesignThinkingAssessmentRubric.Thatresearchresultedinaprototypeofarubricfordocumentingandassessingtheskillsandprocessesthatstudentscomeincontactwithwhilelearninginthecontextofdesignthinkingclassrooms.Therubricallowsthedocumentationofskillsandprocesseslearnedsuchasinterviewing,brainstormingandprototyping.Thatrubricprovidedastartingpointfortheresearchonassessmentreportedherein,whichfocusedongraduatestudentdesignthinkingclassroomsatStanfordUniversity’sHassoPlattnerInstituteofDesign.2FromMindsetstoMindshiftsThroughthisstudywehavebeenabletorethink,redevelop,andredeneideasabouttheassessmentchallengesrelatedtodesignthinking.Wewereabletoredenetheparametersandcomplexityofdesignthinkingassessmentsandrenearubricanditsusesforstudentsandtheirinstructors.Ultimately,thestudyhelpedusgainfurtherunderstandingofthedifferentskills,stages,andmindsetsofdesignthinking,theirevolutionamongstudents,andtheirrelationshipstoeachother.Thislastndingwasmostsignicant:wecoulddevelopassessmentstocapturethecomplexityandnestedrelationshipsofdesignthinkingskills,processesandmindsets.Onenuanceofthiscomplexitywastoconsiderandcapture,insteadofstaticmindsets,mindshiftsthatresultfromtheprocessoflearningdesignthinkingcameintobeing.Sinceweconsiderthelearningofdesignthinkingdispositionsasanemergentjourney—withvariouslevelsofsophistication,transformation,appli-cation,andintegration—weusethetermmindshiftstorepresenttheactiveshiftsthatstudentsaremaking.Mindshiftsarethedevelopingandoftennascentepiste-mologicalviewpointsandinstinctsthatarestrengthened(orinstilled)andmadeobservablethroughchangeinalearner’sorientationsandactionsasadesignthinker.Mindshiftaresignicantinthedesignthinkinglearningarc.Formanystudentstheyarethere-synthesisandreorientationoftheirworldviews,routes,andpropensitiesinproblemsolving.Theyrequirenewcategoriesandsensibilities.Evenasstudentsaremakingchangestotheircategoriesorworldviews,theyholdontoandintegrateoldandnewwaysofthinkingandacting.Thereisalonghistoryofideasabouthowconceptualshiftscanbeunderstood.Wallace()examinedhowchangesinculturalthinkingandworldviewswerepossible.Sometimestheshiftswereviolentandintense,andatothertimestheyweremoregradual,withabackandforthbetweenoldandnewviewsoftheworldandhowtoactinit.Mindshiftsarenotalwaysattheculturallevel,andcanoccuratmorelocallevels.CarolDweck’swork()onxedandgrowthmindsetsshowshowgrowthmindsetscanbelearned—hardworkandapropensitytowardsproblemsolvingandactioncanleadtogreatlearningandtoproductivity.Studiesinphysicseducationalsoshowhowstudents’naveideasaboutthephysicalworldcouldbeAssessingd.learning:CapturingtheJourneyofBecomingaDesignThinker15 reorganizedthroughinstruction(Smithetal.).Wethinkofmindshiftsinasimilarvein.Designthinkingstudentsaccomplishnewmindshiftsastheylearnandbecomeexperienced.Designthinkingmindshiftsdifferfromdesignskillsandprocesses.Theyoftenseemsynthetictootherskillsandprocesses,andarethesuperstructureofalearneddesignthinker.Theyaredifculttoobserve,butitispossibletouncoverandidentifythem.Theseideasaboutmindshiftsledtoaframeworkforreectiveassessmentactivities,aperformance-basedrubricforskillsandprocesses,andanassessmentdashboardformanaginginformationaboutwhatislearned.Assuch,theconceptionofmindshiftschallengedourassumptionsabouthowtoassessdesignthinking.Weinitiallyfocusedprimarilyonprocessandskilldevel-opmentandonhowtoassessthese,butlaterprioritizedtheidenticationofmindshifts,whichwedenedasepistemologicalviewpointsthatareevidentbyachangeinone’sorientationasadesignthinker(Fig.).Thisunderstandingledustoadeeperinvestigationofthetransitionalnatureofwhatitmeanstoadopttheorientationofadevelopingdesignthinker.Oftheconceptualshiftsweidentied,wefocusedonfourkeymindshifts:human-centered;experimental;collaborative;andmetacognitive.2.1Human-CenteredThehuman-centeredmindshiftischaracterizedbyacentralfocusonempathyforothers.Wefocusedourworkspecicallyonassessmentsthathelpmakevisibleanddocumentthedevelopmentofstudents’human-centeredmindsets.Whenstudentsaredevelopingahuman-centeredmindset,theybegintomovebeyondegocentricviewsoftheworldandnolongerdesignbasedontheirownneeds,desires, Fig.116S.Goldmanetal. experiencesorpreferences.Becominghuman-centeredisauidanddynamicprocesswherestudentsactivelyseeksolutionstoproblemsthatmeettheneedsofotherswhomightbenetfromtheirinnovationordesign.Eachengagementtakesstudentstowardbeingmorehuman-centeredandcanhelpthemsee,consider,interactwith,andhaveempathyforothers.Ahuman-centeredmindsetisanintegral,necessaryanddistinguishingelementofdesignthinking.Whileweseethatstudentscanaccomplishhuman-centeredmindshifts,wesuggestthatthetasksdevelopedandresearchedcanbeperformasneartransfertasksforexaminingstudents’developmentofa“humancentered”mindset.2.2ExperimentalTheexperimentalmindshiftischaracterizedbyarealizationthateverythingmaybeconsideredaprototype.Havinganexperimentalstancechangesone’sapproachtoproblemsolvingbyallowingonetodo,make,andvisualizeasintegralpartsofthinkingandoftheevolvingideas.2.3CollaborativeThecollaborativemindshiftischaracterizedbyabeliefthatcollaborationisakeycomponentofproblemsolving,andthatradicalcollaborationsundergirdtransfor-mativeinnovation.2.4MetacognitiveThemetacognitivemindshiftischaracterizedbyanawarenessthatitisessentialtobeawareofwhereoneisinthedesignthinkingprocessinordertoagilelyrespondtochangingparametersofaproblem.3NeedsItiscriticalthatstudentsmasterboththeknowledgeofcoresubjectsandthecriticalskillsnecessaryforreadinessintheinnovationeconomyofthetwenty-rstcentury.Twenty-rstcenturyskillsincludecriticalthinkingandproblemsolving,communi-cation,collaboration,andcreativityandinnovation(Partnershipfor21stCentury).Contentlearning,lifeskills,innovation,anduencywithtechnologiessuchasmediaandwebfunctionalitiesarealsoincluded.Theseskillsareconsideredessentialfordevelopingaparticipatory,innovative,andtechnologicallysophisti-catedculture;andcontributetothepersonal,educational,professional,andcivicAssessingd.learning:CapturingtheJourneyofBecomingaDesignThinker17 livesoflearners(Jenkins;Jenkinsetal.).Designthinking,withitsfocusonproblemsolvingandcreativecondence,isnestedwithinthisviewofculturalshift.Suchnewlearninggoalscreateacriticalneedtounderstandhowdesignthinkingisintegratedintoclassroomlearningenvironments.Aneedfornewmetricsandassessmentmethodologiesaccompaniesthesenewwaysoflearning,aseducatorsbecomeincreasinglyawarethatmeasuresofstudentprogressneedtoalignwiththenewskillsrequiredforschool,entrytothework-force,andlife(COSEPUP).TheUSNationalScienceFoundationTaskForceonCyberlearning()describedthisneedasfollows:Despitetherevolutionswroughtbytechnologyinmedicine,engineering,communications,andmanyotherelds,theclassrooms,textbooks,andlecturesoftodayarelittledifferentthanthoseofourparents.Yettoday’sstudentsusecomputers,mobiletelephones,andotherportabletechnicaldevicesregularlyforalmosteveryformofcommunicationexceptlearning.Thetimeisnow—ifnotlongoverdue—forradicalrethinkingoflearningandofthemetricsforsuccess.PresidentObamahasalsorecognizedthecriticalneedtomeasuretwenty-rstcenturyskills,callingonthenation’sgovernorsto:...developstandardsandassessmentsthatdon’tsimplymeasurewhetherstudentscanllinabubbleonatest,butwhethertheypossess21stcenturyskillslikeproblem-solvingandcriticalthinkingandentrepreneurshipandcreativity(ObamaMostoftheassessmentdonetodayisafterthefactanddesignedtoindicateonlywhetherstudentshavelearned.Notenoughisdonetocollectandaggregatedataaboutstudentlearningthatmakestheinformationvaluabletoandaccessiblebystakeholderswhomightsupportcontinuousimprovementandinnovation.Veryfewassessmentsareauthentictoclassroom-supportedlearningactivitiesortotheappli-cationofwhatislearnedoutofschoolsituations(Baker2010).Generally,assessmentsdonotgivestudentsaccessto,oractiveparticipationin,theirownlearningprogressions.Silva(2008)describesthecreationofeffectivemeasuresoftwenty-rstcenturyskills,andtheneedforbetterandmorecomprehensiveassessmentsthatmeasureboththeabilitytothinkcreativelyandtoevaluateinforma-tion,andstudent’smasteryofcorecontentorbasicskillsandknowledge.Thesenotionsofcomplex,holisticandperformance-basedmeasuresrelatingtoinnovationlearningwerecentraltothisproject’sresearchfocusandthedevelopmentofassess-menttoolsthatcouldbegintocapturethecomplexityofdesignlearninginprogress.4TheoreticalPerspectivesThevekeyphasesofthedesignthinkingprocessareempathize,dene,ideate,prototypeandtest(HassoPlattnerInstituteofDesign).Together,theyprovideeachlearnerwitharelevant,sociallysituated,complexproblem-solvingenviron-mentinwhichtogeneratesolutions.Whilelearningtobecomedesignthinkers,studentsfocusondeningtheparametersofaproblem,identifyingneeds,dealingwithvaryinglevelsofambiguity,activelysolvingproblemsandmakingconnections18S.Goldmanetal. betweentheirlivesinsideandoutsideofschool.Asresearchersofdesignthinkingwhoarealsoconcernedwithlearning,wewereguidedbyatheoreticalstancethatisbasedinexperientialandsociocognitiveviewsoflearning.AsVygotsky()described,opportunitiestointeractwithotherswhohavevaryingdegreesofexpertiseinasocialenvironmentbecomecrucialtolearning.Thehuman-centeredfocusofdesignthinkingandthedeepandradicalcollaborationsthatdenetheprocessprovideadeeplysocialprocessforlearning.Designthinkingisanapproachtowardlearningthatencompassesactiveproblemsolvingbyengagingwith(Dewey),andchanging,theworld.Languageiscentraltothisview,aswecommunicateandengageindialoguewithothers(Bakhtin).Learning,asFreireandMacedo)describeit,demandsacriticalwayofcomprehendingandofrealizingthereadingofthewordandthatoftheworld,thereadingoftextandofcontext.DesignthinkingsupportsthisFreireannotionofimpactfulchangewhichfocusesonlearningasaprocesswhereknowledgeispresentedtous,thenshapedthroughunderstanding,discussionandreectionand,ultimately,actiononbehalfofothers.Furthermore,designthinkingisalsoinformedbyPapert’s()viewofconstructionism,whichdescribeshowideasgetformedandtransformedwhenlearnersareinvolvedinmakingtangibleobjectsthatareexpressedthroughdifferentmedia,actualizedinparticularcontexts(KafaiandResnick;Todd).Thistransformation,whichHaskell()describesastheacquisitionofaknow-howskillandnewtechnologies,inspiresthesubjectwithanirresistibleurgetochangethelivesofstrangerswhothosetechnologiesandskillscanimpact.Designthinkingprovidesarobustscaffoldfordivergentproblemsolving,asitengendersasenseofcreativecondencethatisbothresilientandhighlyoptimistic.Theneedforthiskindofapproachistimely.AccordingtotheCarnegieFoundationCommissiononMathematicsandScience(),theUnitedStatesneedsaneducatedyoungcitizenrywiththecapacitytocontributetoandgainfromthecountry’sfutureproductivity,understandpolicychoices,andparticipateinbuildingasustainablefuture.Likewise,researchhasindicatedthatmanyrst-yearcollegestudentsneedfurtherdevelopmentincriticalthinkingandproblemsolving(Lundelletal.).Theneedforknowledgeandskillsfromscience,technology,engineer-ing,andmathematicsareconsideredcrucialtovirtuallyeveryendeavorofindivid-ualandcommunitylife.And,inatimeofeconomicuncertainty,designthinkinghasthepotentialtoaddressthedemandsofdeveloping,recruitingandretainingtopstudentsandagentsofinnovation.Designthinking,asamodeofinquirythatputs“doing”and“innovating”atthecenterofproblem-solving,promisestoaddressfutureneedsoftheglobe.Ithasthepotentialtoengagestudentsinwaysthatareinclusiveoftheirdiversity,makesschoollearningrelevantandreal,pressinglocalandglobalissueswhichcanenhanceone’smotivationtolearn.Itcreatesa“thirdspace”(Gutierrez),aninterdisciplinaryspacewherestudentsrespondtodesignchallengeswithaclearlyarticulatedsenseoftheircondenceandagencyand,morespecically,oftheiridentitiesaschangeagents.Attheuniversitylevel,thecommitmentsofbothStanfordUniversityandtheUniversityofPotsdamtodesignthinkingeducationandresearchareleadingtheAssessingd.learning:CapturingtheJourneyofBecomingaDesignThinker19 waytobetterunderstandingsofwhatistakingplaceindesignthinkingeducationandcreatingtoolsthathelpcaptureandimprovetheprocessattheuniversitylevel(Meineletal.2011;thisvolume).Designthinkingeducationatanylevelhasthepotentialtoimpactlearningskillssuchasworkingingroups,followingaprocess,deningproblems,andcreatingsolutions(Barron2006).SomeresearchintheK-12arenashowsthatdesign-basedlearningcanbeproductiveevenbeforetheuniversityyears(Carrolletal.2010;Hmeloetal.2000;Goldman2002).Earlyworkinthisdomainhasindicatedthepotentialfordesigntocontributetoyoungpeople’smetacognitive(Kolodneretal.2000and2003)andsociallearning(CognitionandTechnologyGroupatVanderbilt1997)aswellasinspecicsubjectareas(Goldmanetal.1998MiddletonandCorbett1998).VandeZande(2007)characterizesdesignthinkingasameansofcreativeproblem-solvingthatrelatesthoughtandactiondirectlyanddynamically.Themethodologiesofdesignthinkingandrapidprototypingplayimportantrolesindevelopingtransformativeadvancesinlearningandteaching(Cobbetal.2003;Design-BasedResearchCollective2003).Mimickingthisorienta-tiontowardscreativeproblem-solving,radicalcollaborations,andlearningthroughdesigninoursubjectmatter,adesignthinkingresearchmethodology,theteamwasabletoorientitselftothecomplexproblemofassessingdesignthinking.Twotypesofstudiesdenedtheresearchagendaandprocess.Therstsetofstudiestookanexploratoryapproach,usingobservationsandinterviewmethodstogenerateadetailedunderstandingofstudents’designlearningjustaftertheywereensconcedincourses.Thesecondseriesofstudieswereaimedattestingtheassessmenttoolprototypesthatweredesignedinconjunctionwithourexploratorystudyresults.Thetwotypesofstudieswerereciprocal,takingourteambackandforthbetweenthemtorenendingsinaniterativemodeofunderstandinglearningandassessment.Here,itwasourintentionandpracticeforndingsbasedonexploratorystudiestocontributetoaprototypeassessmenttool.Whenanalyzingresultsoftestingonthetools,wethenpouredthoseresultsbackintomorerenedornuancedunderstandingsoftheoriginalbasicresearchndings.5ResearchMethodsandAnalysisQualitativemethodologies(BogdanandBiklen)wereemployedindatacollectionandanalysis.Theexploratorystudiestookastudent-centered,emicapproachandwereconductedwithstudentswhowereinStanforddesignthinkingcourses.WeconductedobservationsandinterviewedstudentswhowereenrolledintwoofStanford’sHassoPlattnerInstituteofDesign(d.school)classes:ThinkingBootcamp;andDesignGarage.TheobservationstookplaceduringtheFallandWinteracademicquarters.Observations,whichfocusedonthefullclassandonsmallerprojectteams,wererecordedineldnoterecords.Attheendofthecourse,observationswerefollowedwithopen-endedinterviewswithstudentparticipants.Interviewquestionscoveredtopicsrelatingtotheoverallexperienceintheclass.Someaskedthemtoreecton,anddescribe,whenandwhattheythoughttheywerelearning.Samplequestionsfromtheinterviewincluded:20S.Goldmanetal. Whathasbeenyourbiggeststruggle?Whathasbeenyourbiggestsurprise?Pleasedrawavisualrepresentationofdesignthinkingasyouunderstandit.Tellusaboutanexperienceofbootcampthatmadeanimpressiononyou.Whatdoyouthinkwillstaywithyouaftertheclassiscompleted?Withtheobservationsandinterviewscompleted,analyseswereconductedtoidentifythekindsoflearningexperiencesstudentswerehavingandtodeterminehowtheymetthegoalssetforthembytheirinstructors.Analysistechniqueswerequalitativeandincludedopencodingoftheeldnotesandinterviews,searchesforimportantlanguage,events,and“learnings”thatstudentsdescribed,andplaceswherestudentsexpresseduncertaintyaboutwhatorwhethertheywerelearning.Wesearchedfortimes,placesandeventswherefeedbackcameintoplayforthestudent:howtheyunderstoodthefeedbacktheyweregetting;whentheyreportedtheywantedfeedbackthattheydidnotreceive;andwhattheythoughtthatmeantforthemasbuddingdesignthinkers.Thisprotocolhelpedusidentifyopportunitiesforfeedbackorreectiononperformance;expressionsoflearning;anddocumen-tationoflearning.Forexample,inoneclassweobservedstudentsaskingforconrmationthattheywere“gettingthisideationprocesscorrect”duringclass,andbeingtold,“Don’tworryaboutwhetherornotyouaregettingitatthispointintime.Justgiveitatry!”Studentswerenotalwaysreassuredatthosetimes,butforgedonward.Forafewstudents,wecouldactuallytracehowthoseclassroommomentsplayedoutintheirownthinkingorinthefeedbacktheyreceivedonclassprojects.Thisisanexampleofaseriesofassessmentopportunitiesthatwecouldidentifyandtrytounderstand.Withthepreliminaryexploratorystudyndingsathand,webegantodesignasetofassessmenttools.Weconductedformativeuser-testswiththesetoolstoimprovethemiterativelyandtofurtherreneinsightsfromourexploratoryndings.Throughuser-testingfeedbackontheassessmenttoolsandexploratoryobservations,wewereablesolveourongoingdilemmaabouthowdifcultitseemedtoassessdesignthinkingmindsets.Thestudiesindicatedthatmindsetdevelopmentwasdifculttoobserveanddocumentduringclassroomactivitieseventhoughtheyseemedpresentwhenyoutalkedtostudentsabouttheirexperiencesanddesignprojects.Thecyclingback-and-forthamongthestudieshelpedusseethattheassessmentrubricwasnotanappropriatevehicleforassessingthestudents’evolvingmindshifts.Werealizedmindshiftscouldbestbeassessedandbenchmarkedthroughspecialtyperformanceassessmenttasksandprompts.Inordertodevelopperformanceassessmenttasksthatcouldmeasurethedevelopmentandpresenceofmindsets,weobservedandinterviewedschoolchil-dren,universitystudents,andinstructorsandexperienceddesignthinkers.Wefocusedourstudiesononemindset—beinghumancentered.Wedevelopedandresearchedtwotasks—onetaskforstudentgroupsandonetaskforindividuals—whichwouldshowhowstudentsapproachedandworkedthroughchallenges.Wecouldthusexplorehowstudentswereexhibitingahuman-centeredmindsetanddocumentthelevelofsophisticationatwhichthisoccurred.WeadministeredtheAssessingd.learning:CapturingtheJourneyofBecomingaDesignThinker21 taskstomiddleschool,undergraduate,graduatestudents,anddesignthinkinginstructors.Thisenabledustoexamineandhopefullydistinguishamongperformancesofnovice,intermediaryandsophisticateddesignthinkers.Aswehoped,ananalysisofstarkdifferencesbetweenthenovicesandexpertsrevealedacomplexpatternofdevelopment,andindicatedthatmindsetsmightbemoreinuxthan“set”.Awaytoconceiveofanddescribesomeofthecommonevolvingbehavioralmovesfornovice,intermediateandsolidiedmindsetswasthereforethestartingpointfortheconceptofevolvingmindshiftsandanewroundofdevelopmentandvalidationofthereectiveassessmentrubricandtheperformance6AssessmentToolPrototypesIntroductionstotheassessmenttoolsthatwereprototypedandstudiedfollow.Eachintroductionincludesabasicdescriptionofthetool.WeconductedobservationsandinterviewedstudentswhowereenrolledatStanford’sHassoPlattnerInstituteofDesign(d.school)classes:DesignThinkingBootcampDesignGarage,andalsostudentsfromStanford’sSchoolofEducationLearning,Design&Technologyprogram.Withinsightsfromtheseresearchexperiencesinplace,webegantoiterateasetoftoolsthatwouldmeasurebothmindshift(theactiveprocessofdevelopingamindset)andprocess.Wefocusedonthecreationofperformance-basedtoolsandreectivetoolsthatassessedhowonewouldapproachbothproblemdenitionandproblemsolving.Thesewouldcapturetheessentialshiftsthatoccurwhenonedevelopsasadesignthinker.Ourimplementsinclude(1)theReßectiveAssessmentRubric;(2)theteam-basedWindaloobahExperimenttask;(3)theDesigningTwenty-ÞrstCenturyLearningSpacesTask;and(4),theAssessmentDashboard6.1TheReßectiveAssessmentRubricWithaninitialprototypeofadesignthinkingassessmentrubricdevelopedinapriorresearchproject,weusedtheiterativedesignthinkingapproachtocontinueevolvingtherubric.Ouractivitiescenteredon:interviews,observations,analysis/synthesis,ideation,prototypingandtesting,asfocalpointsofthedesignprocess.Thelearningfromeachtypeofprototypeinformedsuccessiveversionsoftheassessmentrubric.Figureillustratesasamplepagefromanadvancedprototypeofthereectiveassessmentrubric.Observationsofexpertandnovicedesignersledtoamatrixofskills,processesandmindsetsrelatedtodesignthinking.Whenweconsideredhowtheywerenoticedandcouldbedocumented,werealizedthatdifferentkindsofassessmentswereneededforassessingskills,processes,andmindsets.Somewerenested;others22S.Goldmanetal. wereumbrellaskills.Somewerediscrete;otherswerebundled.Somecouldbe documentedandmuchrepetitionandpracticeputthemintoplace;otherscouldbe accomplishedonceandthenceforthseemtotakeroot.Somedesignthinkingskills overlappedwithskillsfromotherdomains,forexample,interviewing,observation, andpersistenceskills.Otherskills,suchasthoserelatedtohuman-centeredness, ideation,andprototypingweremoreexclusivetodesignthinking.Wereconsidered therubric’scategoriesandchosespecictypesofactionstobeincludedthatbest exemplieddesignthinking.Wealsoseparatedmindshiftsfromrubricassessment, andturnedattentionto mindshifts . Todate,therubricconsistsofmindshifts,skills,andthreelevelsofexpertise.It alsoindicateswhereaspecicskillisexhibitedintheprocessandwhereskillstend tooverlapandinteractwithmindshifts.Theidentiedskillsincludethefollowing: Interviewing Prototyping Synthesis Persistence Resilience Adaptability Risk-taking Fig.2 Samplepagefromthereectiveassessmentrubric Assessingd.learning:CapturingtheJourneyofBecomingaDesignThinker23 BrainstormingBiasTowardsActionStorytellingProcessVocabularyCollaborationAspecicskillmaypresentitselfinmorethanonephaseoftheprocess.Forexample,onemightinterviewduringboththeempathyandtestingphaseofthedesignthinkingprocess,whichwill,consequently,reectdifferentcomponentsoftheskill.Studentsofdesignthinkinglearnatdifferentrates,andthethreelevelsareapproximaterepresentationsofbehaviorsandactionsfornovice,intermediateandskilleddesignthinkers.Whenanalyzingdata,wealsofoundthatstudentsexhibitedarangeofskillsthatwereindicativeoftheirabilitiesasdevelopingdesignthinkers.Levelsofexpertisevariedforeachphaseoftheprocess.Whileworkingontherubricweconsideredwhowouldbestbeinapositiontoassessdevelopingskillsandmasteryofprocesses.Whilemanyitemscalledforassessmentbyaninstructor,wealsoidentiedmanyitemsontherubricthatcouldbedocumentedbystudents.Forexample,studentscouldtracktheirworkandprogressrelatingtotopicssuchaspersistence,self-regulation,anddevelopmentofmetaskills.Therubricisavailableelectronicallyandwecontinuetovalidateitbasedonfeedbackfromstudentsandinstructors.6.2TheWindaloobahExperimentTaskOncewediscoveredthattherubricwasaninappropriateinstrumentforassessingmindshifts,wetransferredoureffortstothedesignofperformance-basedassess-menttasks.Wecreatedanassessmenttask,TheWindaloobahExperiment,togaugestudents’understandingofthedesignthinkingmindsets.Thetaskwasdesignedtogivestudents,workinginteams,opportunitiesandresourcesforexhibitingempathyandhuman-centeredness.Itwasadministeredtotwoclassesofmiddleschoolstudentsandtwogroupsofmaster’slevelgraduatestudents.Inthemiddleschool,oneclasshadrecentlycompletedanintegratedmathematicsanddesignthinkingchallenge.Thesecondclasshadnotyetbeenintroducedtodesignthink-ing.Thegraduatestudentswerediverseinexperience,rangingfromnonetoastudentwhowasinthed.schoolclassDesignThinkingBootcampTheWindaloobahExperimentwasstructuredasanhourlongdesignchallenge.ThestudentswereaskedtoworkinteamstodesignaWindaloobah,however,thedenitionofaWindaloobah(amadeupword)wasdeliberatelyambiguous.Studentswatchedabriefintroductoryvideo.Figurecontainsthevideoscriptfromthevideo.Afterwatchingthevideo,thestudentsweregiveninformationpacketswithprolesofpeoplewhowouldbepartofthecommunityembarkingonajourneyontheWindaloobah.Theprolesconsistedofaphotographofeachcommunity24S.Goldmanetal. Welcome to the Experiment. It is the year 2125. A group of brave andadventurous explorers have joined the experiment. For the next 15months they have no permanent home. Your job is to design a WindaloobahWhat is a A Windaloobahmust have room for 5 people.A must travel through water, skies, snow, it must be able tomove through cities.It must be ready for unexpected adventures.Windaloobahis a home away from home. Fig.3TheWindaloobahExperimentvideoscript Fig.4communitymemberproleAssessingd.learning:CapturingtheJourneyofBecomingaDesignThinker25 member,andtextdescribinghisorherinterests,needsandchallenges(SeeexampleproleinFig.).TheWindaloobahcommunityincludedagrandmother,a3-year-oldboy,ashy13-year-oldgirl,a23-yearoldmusician,anda19-year-oldvideographer.Thestudentswerealsogivenanenvelopewithatravelitinerarydescribingtheconditionsthetravelerswouldencounter,andasetofpictures(akite,asignwith5mphonit,andasquirrel.)Thestudentshadtoascertainwhattheymightdowithambiguousinformation.Eachgroupwasalsogivenacardboardboxlledwithanassortmentofprototypingmaterials.TheWindaloobahExperimenttaskenabledustoobserveanddocumenthuman-centerednessasgroupsofstudentsengagedadesignchallenge.Withthetask,wecoulddiscriminatehuman-centeredness,separatingouthowhuman-centereddesignthinkersapproachedthetaskdifferently.Studentswhohadevenanintro-ductorylevelofdesignthinkingshowedmorehuman-centeredbehaviorsandactionsthanthosewhowerecompletelyinexperienced6.3TheDesigningTwenty-FirstCenturyLearningSpacesTaskTheresearchteamwasalsointerestedindesigningareection-basedtaskforindividualsinordertogainfurtherinsightintohuman-centeredmindshifts.Studentswereinvitedtoawebsite,DesigningTwenty-ÞrstCenturyLearningSpaces,wheretheywereaskedtoprovideinput(Fig.).Theywereaskedtowatchavideothatconsistedoftwocomponentsofexploringtheissue.Therstpartofthevideofocusedonaclassroomlledwithanarrayoftechnologymaterials.Therewerenostudentsintheroom.Thesecondpartofthevideofeaturedastoryaboutateacher,Nate,inanurbanclassroomwhowasconcernedaboutmeetingtheneedsofhisstudents.Attheendofthevideo,theresearchsubjectsweredirectedtoasurveyandaskedtosharetheirthoughts.Thesurveyconsistedoftwoquestions:Inyouropinion,whatisthemostimportantaspecttoconsiderinthedesignoftheclassroomofthefuture?WhatwouldyoudoifyouhadtodesignaclassroomforNate?Thesurveyalsoincludedaquestionregardingtheresponder’sexperienceswithandexposuretothedesignthinkingprocess.Preliminarydataanalysisindicatedthatstudentswithnodesignthinkingexperienceshowedabaselineoflittletonohuman-centeredness.IftheydidorienttotheNate’sstatedneeds,itwaslimitedtoalevelof“noticing”whichdidnotextendtodesignsolutions.Atpresent,thetaskneedsguidelinesforscoringresponses,andinfutureworkwewouldliketovalidatethetaskwithalargenumberofparticipants.26S.Goldmanetal. 6.4TheAssessmentDashboardFromthetimewerstprototypedtheassessmentrubric,wewereawarethatinstructorsandstudentswouldwantawaytokeeptrackofdesignlearningovertime.Designthinkingisacomplexofinterwovenandinterdependentskills,processesandmindshifts.Evenstudentsinthesamecourseanddesignchallengearelikelytodevelopdifferentskills,capabilitiesandmindsets.Thus,ineachstudywecompleted,welookedforwaystothinkaboutandorganizetheassessmentprocessandthesnapshotonlearningthatitgenerates.Weinitiallyhadtheideaofalongitudinalportfoliothatwouldfollowstudentsandtheirlearningovertime.Atthetimeallofoureffortwasfocusedontheassessmentrubric.Oncewemovedtoanotionofvariedpaletteofassessmenttypes,wehadtorethinkhowtoorganizeassessmentinformationsoitwasusefulandtransfer-abletostakeholders.Wetrieddifferentwaystoorganizemindshiftswithskillsandpractices.Wedecidedtotreatmindshiftsasreectivetaskswherestudent-simulatedapproachestodesignthinkingactionwouldhelprevealthemorehiddenpropensitiesofmindshifts.Inordertoallowstudentsandteacherstoeffectivelyvisualizetheserelationships,makeexplicitconnectionswiththeirprojects,anddocumentastudent’sprogressindevelopingasadesignthinker,ourworkturnedtodesigninganonlinedashboard.Thedashboardwasconceivedtoconsidermanyoftheneeds Fig.5Designingtwenty-rstcenturylearningspacesAssessingd.learning:CapturingtheJourneyofBecomingaDesignThinker27 wewerediscovering.Figuresillustratetheframeworkforadashboardthatmightmeettheneedsofindividualstudents,studentsworkinginteams,andinstructorswhomustweighinonlearningforbothinstances.Thisframeworkwouldallowstakeholderstodocumentthedifferentaspectsofdesignlearningandwouldserveasanassessmentenvironmentthattraveledwiththestudentovertime.Thedashboardwouldarchivemediarelatedtodesignchallengesandbenchmarksindesignlearning,helpingtracethechanginglevelsofsophisticationasadesignthinkergrewovertime.Todate,wehavedevelopedarstprototypeofthedashboardandwillbemovingtowardsuserfeedbackandfurtheriterations.Thedashboardisdesignedtoenableinstructorsandlearnersto:VisualizetheskillsthatarerelevanttostagesofthedesignthinkingprocessVisualizetheskillsandprocessstagesthatcontributetoeachmindshift(Fig.Buildaportfolio(Fig.),selectingthosestagesandskillsfocusedoninagivenproject.Developingmindshiftsareincluded.Assessandviewtheperformanceofindividualsandteamsfordifferentprojects,progressovertime,andforeachofthemindshifts(Fig.Thedashboardwillalsoallowtheresearchteamtoprototypeandrenetheassessmentoftheskillsthatareincludedintherubricbytrackingitsusebyteachers.Atthemomentthedashboardisintheuserfeedbackphase. Fig.6Aviewofthedashboardfromthemindsetview28S.Goldmanetal. 7SummaryandDiscussionThroughaseriesofreciprocalresearchanddesignstudiesweexploredcriticalelementsoflearningthedesignthinkingprocessandpossiblewaystodocumentthem.Ourfocusonthedevelopmentofperformance-basedassessmenttasks,onreection-basedassessmenttasks,andonthefurtherdevelopmentandrenementofanassessmentrubricmadeforrichlearning.Theresultsoftheprojecthelpedenrichourassessmenttoolsandourconceptualizationofdesignlearning.Firstly,weunderstoodthecomplexrelationshipanddifferencesbetweendesignthinkingskillsandprocesses,andthedevelopmentofdesignthinkingmindsets.Thesedifferenceshadimplicationsfortheassessmenttools.Secondly,weconceivedmindshifts,whichexpressthedevelopmentaljourneytowardsmindsetsthataresoimportantwhenapersonislearningtobeadesignthinker.Mindshiftsareepistemologicalviewpointsinux.Theyarepartoftheprocessofbecomingdesignthinkers.Wewereabletoidentifyfourcentralmindshifts—becominghuman-centered,experimental,collaborative,andmetacognitive—eachofwhicharerepresentativeofdifferentaspectsofone’sdevelopmentasadesignthinker.Ourperspectivechangetoshiftsgaveusnewinsightsintothemostsigni-cantwaysthatone’sbehaviorandorientationchangeasonegrowsasadesignthinker.Amindshift,whennallyset,isanunderlyingstructureforthinkingandactingasadesignthinker—itisthedeepstructureandworld-viewthatislearnedinconjunctionwith,andsupports,thedevelopmentofdesignskills.Forexample,one Student 2Student 3Student 4 Fig.7VisualizingprogressoverdesignprojectsAssessingd.learning:CapturingtheJourneyofBecomingaDesignThinker29 mightbeagoodprototypedeveloper,butsomeonewithanexperimentalmindshiftunderstandsthevalueofprototypingandseemstohaveinstinctsforbringingprototypingintotheprocessearlyandoften.Onemightbeaskilledinterviewer,butbeinganempatheticinterviewer—themanifestationofthehuman-centeredmindshift—isessentialtobeingadesignthinker.Priortounderstandingthemindsetsasshiftsinperspectivesanddevelopmentofpredilectionsinstincts,wefocusedontheskillsandprocessesofdesignthinkingthatwereobservableanddocumentedbasedontheperformanceoftasksandactivities.However,afterthischangeinorientation,wefocusedourworkspeci-callyonassessmentsthathelpmakevisibleanddocumentthedevelopmentofstudents’human-centeredmindshift.Whenstudentsaredevelopingahuman-centeredmindset,theymovetowardssolutionsthatresonatewiththeneedsandlivesofothers,markingasharpcontrasttoegocentricviewsoftheworldthatarecharacteristicofafailuretoadopthuman-centeredness.Mindsetdevelopmentisauidanddynamicprocess,yetforsomestudents,developingamindsetmightseemlikea“switchhasbeenippedfromofftoon”.Forothers,mindshiftscomeinmorecomplexandincrementalways.Eachstepastudenttakestowardbeingmorehuman-centeredcanhelphimorhersee,consider,interactwith,andhaveempathyforothers.Ourresearchteamandotherdesignthinkingeducatorstookforgrantedthatsomeorallstudentswouldgainmindsets,butduringthisprojectwewereactuallyabletowitnessstudentsreachingbenchmarkpointsintheevolutionofthesemindsets.Suchstudentswereintheprocessofmindshifts.Wesawthatstudentsmighthavenoinclinationtowardhuman-centeredness,theymighttendtoactinhuman-centeredwaysintermittentlyoratconvenientstagesofthedesignprocess;ortheymightconsistentlyorientthemselvesinhuman-centeredwaysnomatterwheretheywereinthedesignprocess.Ourreectivetasksaidedusinseeingdifferencesinthosewhowere“mind-shifting”andthosewhowerenot.Likewise,ourresearchwithstudentswhohadlittleornoexperiencewithdesignthinkingcoursesshowedastarkcontrastwiththosewhoweretakingcoursesandthosewhowerealreadyconsideredmoreprofessionaldesignthinkers.Wewereabletoexpandandrenedesignthinkingskillsthatwereincludedintherubricandidentifyactionsandbehaviorsthatmightbepartofthedocumenta-tionprocess.ThroughtheRR&Dprocess,weidentiedtopicsforfurtherresearchanddesignofassessmenttoolsforthedesignthinkingclassroom.Thefollowingquestionswillbethefocusoffuturework:1.Howeffectivelydotheperformance-basedassessmentrubricandreection-basedtasksdocumenttheskillsandmindshiftsinherentinthedesignthinkingprocess?Howusefularetheyastoolsforbothstudentsandinstructors?2.Arethetoolsusefulinavarietyofeducationalsettingsasindicatorsofdesignlearning?3.Sincethetoolsweredevelopedtoassess,document,andhelpindividualsandtheirinstructorsindesignthinkingeducation,whatneedstobeaccountedforindevelopingassessmentsofgroupswhoarelearningdesignthinking?Evenwithademonstratedstrongneedforsuchtools,aretheseassessmentspracticalandusablegiventhestructureofcourses?30S.Goldmanetal. Wemadeprogresstowardourgoalofprovidingadetailedreectiveframeworkforstudentsandteacherstodocument,reecton,andconrmtheirengagementsandprogressinrelationtoadiverserangeofdesignthinkingactivities.Theframeworkisaccompaniedbyasetoftoolsfortrackingandcertifyingstudents’progressinrelationtodesignthinkingskills,processesandmindsets.Inaddition,thetoolscanprovideasharedresourcefordesignthinkingteamsandinstructors.Itisessentialtodevelopacomprehensiveunderstandingofdesignthinkingfromtheoreticalandappliedvantagepoints.Itisourhopethatourresearchwillcontrib-utetothisunderstanding.Wewouldliketothankthestudentsandinstructorswhoparticipatedinourstudies.Theyhavemadeitpossibleforustobetterunderstandwhatitmeanstobecomeadesignthinkerandhowassessmenttoolscouldaddvaluetodesignlearning.AgrantfromtheHassoPlattnerDesignThinkingResearchProgrammadethisworkpossible.FindingsandopinionspresentedarethoseoftheauthorsanddonotrepresenttheHPDTRP.AlexanderA,BlairKP,GoldmanS,JimenezO,NakaueM,PeaR,RussellA(2010)GoMath!Howresearchanchorsnewmobilelearningenvironments.In:Proceedingsofthesixthinter-nationalIEEEconferenceonwireless,mobile,andubiquitoustechnologiesineducation(WMUTE).Kaohsiung,pp57–64BakerE(2010)Whatprobablyworksinalternativeassessment.CRESSTReport772.NationalCenterforResearchonEvaluation,Standards,andStudentTesting,LosAngelesBakhtinMM[1930s](1981)Thedialogicimagination:fouressays.In:HolquistM(ed)(Trans:EmersonC,HolquistM).UniversityofTexasPress,Austin/LondonBarronB(2006)Interestandself-sustainedlearningascatalystsofdevelopment:learningecologyperspective.HumDev49(4):193–224BlackP,WiliamD(1998)Insidetheblackbox:raisingstandardsthroughclassroomassessment.PhiDeltaKappan,80(2):139–148,October1998BogdanR,BiklenS(1992)Qualitativeresearchforeducation:anintroductiontotheoryandmethods.Allyn&Bacon,BostonCarnegieFoundationCommissiononMathematicsandScience(2009)Excellenceandequityinmathematicsandsciencetotransformeducation.Accessed29Jan2011CarrollM,GoldmanS,BritosL,KohJ,RoyaltyA,HornsteinM(2010)Destination,imagination,andthereswithin:designthinkinginamiddleschoolclassroom.IntJArtDesignEduc29CobbP,DiSessaA,LehrerR,ScaubleL(2003)Designexperimentsineducationalresearch.EducRes21(1):9–13CognitionandTechnologyGroupatVanderbilt(1997)TheJasperProject:LessonsinCurriculum,Instruction,Assessment,andProfessionalDevelopment.LawrenceErlbaumAssociates,Inc.,Mahwah,NJCommitteeonScience,EngineeringandPublicPolicy(COSEPUP)(2006)Beyondbiasandbarriers:fulllingthepotentialofwomeninacademicscienceandengineering.NationalScienceFoundation,ArlingtonNationalScienceFoundationTaskForceonCyberlearning(2008)Fosteringlearninginthenetworkedworld:thecyberlearningopportunityandchallenge.http://www.nsf.gov/pubs/2008/Assessingd.learning:CapturingtheJourneyofBecomingaDesignThinker31 nsf08204/nsf08204.pdfhttp://www.nsf.gov/pubs/2008/nsf08204/nsf08204.pdf.Accessed20Dec2010Design-BasedResearchCollective(2003)Design-basedresearch:anemergingparadigmforeducationalinquiry.EducRes32(1):5–8DeweyJ(1916)Democracyandeducation.Macmillan,NewYorkDweckC(2007)Hownottotalktoyourkids:theinversepowerofpraise.NewYorkMagazine..Accessed8Nov2011FreireP,MacedoD(1987)Literacy:readingthewordandtheworld.BerginandGarvey,SouthGoldmanS(2002)Instructionaldesign:learningthroughdesign.In:GuthrieJ(ed)Encyclopediaofeducation,2ndedn.MacmillanReferenceUSA,NewYork,pp1163–1169GoldmanS,KnudsenJ,LatvalaM(1998)Engagingmiddleschoolersinandthroughreal-worldmathematics.In:LeutzingerL(ed)Mathematicsinthemiddle.NationalCouncilofTeachersofMathematics,Reston,pp129–140rrezKD(2008)Developingasociocriticalliteracyinthethirdspace.ReadResQuart43HaskellTL(1985)Capitalismandtheoriginsofthehumanitariansensibility,parts1and2.AmHistRev.90(3,4):339–361,547–566HassoPlattnerInstituteofDesignatStanford(2007)Designthinkingprocess.StanfordUniversity,PaloAltoHmeloC,HoltonD,KolodnerJ(2000)Designingtolearnaboutcomplexsystems.JLearnSci9JenkinsH(2006)Convergenceculture:whereoldandnewmediacollide.NewYorkUniversityPress,NewYorkJenkinsH,ClintonK,PurushotmaR,RobinsonAJ,WeigelM(2006)Confrontingthechallengesofparticipatoryculture:mediaeducationforthe21stcentury.TheMacArthurFoundation,KafaiY,ResnickM(eds)(2000)Constructionisminpractice:designing,thinking,andlearninginadigitalworld.LawrenceErlbaumAssociates,MahwahKolodnerJ,GrayJT,FasseBB(2000)Promotingtransferthroughcase-basedreasoning:ritualsandpracticesinlearningbyDesignTMclassrooms.CognSciQ1:183–232KolodnerJL,CampPJ,CrismondD,FasseB,GrayJ,HolbrookJ,PutambeckarS,&RyanM(2003)Problem-basedlearningmeetscase-basedreasoninginthemiddle-schoolscienceclassroom:PuttingLearningByDesignIntoPractice.TheJournaloftheLearningSciences,12(4):495–547LundellDB,HigbeeJL,HippS,CopelandRE(2004)Buildingbridgesforaccessandsuccessfromhighschooltocollege:proceedingsofthemetropolitanhighereducationconsortium’sdevel-opmentaleducationinitiative.CenterforResearchonDevelopmentalEducationandUrbanLiteracy,UniversityofMinnesota,MinneapolisMeinelC,LeiferL,PlattnerH(eds)(2011)Designthinking:understand–improve–apply.Springer,HeidelbergMiddletonJA,CorbettR(1998)Sixth-gradestudents’conceptionsofstabilityinengineeringcontexts.In:LehrerR,ChazanD(eds)Designinglearningenvironmentsfordevelopingunderstandingofgeometryandspace.LawrenceErlbaumAssociates,Mahwah,pp249–266ObamaB(2009)AddresstotheHispanicChamberofCommerce.U.S.DepartmentofEducation,Assessment:measurewhatmatters.http://www.ed.gov/technology/netp-2010/assessment-.Accessed20Dec2010PapertS(1980)Mindstorms.Children,computersandpowerfulideas.BasicBooks,NewYorkPartnershipfor21stCenturySkills(2008)21stCenturySkills,Education&Competitiveness:AResourceandPolicyGuide.Tucson:Partnershipfor21stCenturySkillsSilvaE(2008)Measuringskillsforthe21stcentury.EducationSector,pp1–10..Accessed28Nov201032S.Goldmanetal. SmithJP,diSessaAA,RoschelleJ(1993)Misconceptionsreconceived:aconstructivistanalysisofknowledgeintransition.JLearnSci3(2):115–163(1993–1994)ToddR(1999)Designandtechnologyyieldsanewparadigmforelementaryschooling.JTechnolStud25(2):26–33VandeZandeR(2007)Designeducationascommunityoutreachandinterdisciplinarystudy.JLearnThroughArts3(1):1–22VygotskyLS(1934/1976)Thoughtandlanguage.MITPress,Cambridge,MAWallaceA(1956)Revitalizationmovements:sometheoreticalconsiderationsfortheircompara-tivestudy.AmAnthropol58:264–281Assessingd.learning:CapturingtheJourneyofBecomingaDesignThinker33