Judith Olson University of California Irvine Today Im going to cover Three recent events that inspired this call to arms Your role in this discussion What it means to have impact ID: 639923
Download Presentation The PPT/PDF document "Broader Impacts Research you can use" is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1
Broader Impacts
Research you can use
Judith Olson
University of California IrvineSlide2
Today I’m going to cover….
Three recent events that inspired this
“call to arms”
Your role in this discussion
What it means to have impactKinds of impactRecap of what it means to have impactScope, Cost, Timeline…Your pledge about making an impactSlide3
Collaboration Success Wizard
Theory based on
The literature on teams
Own own observations
and interviews of over 50
Science
Collaborations
Corporate virtual teamsTo verify theoryNeed dataOnline survey with advice to motivate participationThey get the help and we get the dataSlide4
Collaboration Success Wizard
Web accessible assessment tool
Assesses
Strengths
Challenges
How to overcome the challengesSlide5
We are having an impact
NSF
Had us give a talk to Federal funders in general
“I have needed this for the last 10 years! Thank you.”
Teams who were assessed welcomed advice“It drew out patterns in the way our members work that we were not conscious of, confirmed some of our impressions, and allowed us to hear frankly from our members.…useful as an independent evaluation tool not tied to a funding agency or other review panel”.Slide6
Clinical and Translational Science Awards
(CTSA)
Object of study
…”to speed the translation
of laboratory discoveries into treatment for patients.”“from bench to bedside.”
National Institute of Health:
60 CTSA awards in 30 states plus DCSlide7
Clinical and Translational Science Awards
National Institutes of Health
Since 2006
$733 MSlide8
NSF “Dear Colleague”
“While most researchers know what is meant by Intellectual Merit
, experience shows that many researchers have a less
than clear understanding
of the meaning of Broader Impacts.”Slide9
Impact
Many of us came to this field to change the digital world
Technology had gone awry
Many early people attracted
to HCI were “Children of the 60s”Slide10
Then…
Our careers were caught up in the reward structures
Industry
Create new products
Disincentive to makefindings available to othersAcademiaPublish new findingsStay on topic, build a reputationSlide11
Worry…..
Where have all the impacts gone
Long time passingSlide12
How we will proceed….
I will describe what I think it means to have impact
I will list a number of ways we do and can have an impact
You pledge…
The card on your seatWhat other ways can you have impactHow are you going to have an impactCollected by SVs at the door as you leaveSlide13
What it means to have impact
What “counts”
Theory gets used
Downloads/views
ProfitsDegrees/EducationTechnologiesLives changed…..Who is impacted?StudentsDevelopers
Consultants
Specific populations
The general publicSlide14
What it means to have impact
Scopes differ
You affect some people directly
Interventions, teaching
You enable others to be better at making better productsToolkitsYou set policyAffect a large number of peopleSlide15
What it means to have impact
Time scales differ
Now
e.g. Action
research1-3 yearse.g., Publications20-30 yearse.g., Theory Assessment Tools
40-50 years
e.g.,
Cyberinfrastructure development?e.g., Policy (like SOPA/PIPA)Slide16
What it means to have impact
Access?
Free
Toolkits…
Wizard….FeesCommercial Assessment ToolsProductsEducational degree….Slide17
Kinds of Impacts
Theories
Assessment tools
Technological innovations
Guidelines, templates, patterns, toolkits and standardsPoliciesNew media disseminationAction researchTeaching and teaching materials…
What else?
Which kinds of impacts will YOU make?Slide18
Theories
“There is nothing so practical as a good theory”
Kurt
Lewin
“He who loves practice without theory is like the sailor who boards ship without a rudder and compass andnever knows where he may cast”
Leonardo Da VinciSlide19
Theories
Who
Other researchers
Consultants
Tool developersHowRead and build on/test theoryScopeSmall at firstTime scale1-3 or more yearsAccessFreeSlide20
Kinds of Impacts
Theories
Assessment tools
Technological innovations
Guidelines, templates, patterns, toolkits and standardsPoliciesNew media disseminationAction researchTeaching and teaching materials
What else?
Which kinds of impacts will YOU make?Slide21
Theories delivered as
Assessment Tools
Collaboration Success
Wizard
GlobesmartMyers-Briggs Personality AssessmentCogTool…Slide22
Assessment Tools: GlobeSmart
B
ased on academic theories of cultural differences
David Matsumoto
Handbook of Culture and PsychologyLike the Wizard, they collect data to adjust their assessmentsRecent upgrade used data from 400,000 users from over 60 countriesSlide23
Assessment Tools: GlobesmartSlide24
Assessment Tools: GlobeSmartSlide25
Assessment Tools: GlobeSmartSlide26
Assessment Tools: GlobeSmartSlide27
Assessment Tools: Myers Briggs
Based on the work of Carl Jung
Developed further by Myers
and Briggs
Like the GlobeSmart you can see differences in values and habits with people you interact withDimensions of discussionSome professional helpSlide28
Assessment Tools: CogTool
Based on work of Bonnie John
Based
on
Card, Moran, & NewellGOMS and the Model Human ProcessorA general purpose UI prototyping toolIt automatically evaluates your design with a predictive human performance modelA “cognitive crash dummy”Slide29
Assessment Tools: CogToolSlide30
Assessment Tools: CogTool
“You can compare expert use task time without recruiting participants…An excellent choice for completely new systems that don’t already have experts.”Slide31
Assessment Tools
Who
General public
How
Take the assessmentScopeCould be hugeTime scaleImmediateAccessSome are free; some cost moneySlide32
Kinds of Impacts
Theories
Assessment tools
Technological
innovationsGuidelines, templates, patterns, toolkits and standardsPoliciesNew media disseminationAction researchTeaching and teaching materials
What else?
Which kinds of impacts will YOU make?Slide33
Technical Innovations: Alice
3-D programming environment
For telling a story
Playing an interactive gameTeaching tool for introductory programmingFormally shown to improve learning and performance
Randy
PauschSlide34
Technical Innovations: Alice
Caitlin Kelleher,
2006Slide35
Technical Innovations: Alice
U
sing storytelling to make computer programming attractive to middle school girls
Storytelling Alice users
spent 42% more time programming were more than three times as likely to sneak in extra time to continue working on their programs
Caitlin Kelleher Slide36
Technical Innovations: Alice
10% of the nation’s colleges now use Alice
An accompanying textbook, lessons, test banks
88% of “at risk” students who had Alice in a pre-CS1 course were retained through CS2
3.03 GPASlide37
Which then inspired….
iMuse
A requirements engineering environment where both developers and stakeholders could understand the flow
Kristina
WinbladhSlide38
Technical Innovations: More...
Hypertext Transfer Protocol HTTP
HTTP/1.1 spec
Fielding,
Gettys, Mogul, Frystyk and Berners-LeeWebDAV extension“Architecture of the Web”Fielding and TaylorSlide39
Technical Innovations: More….
Aspect Oriented Programming
Difference lies in the power, safety
and
usability of the constructs providedOriginal article downloaded6,681 times16,600 articles in Google Scholar with “Aspect Oriented Programming”
Crista
LopesSlide40
Technical innovations
Who
Students
The
general publicOther developersHowUse the technology that makes things possibleScopeHugeTime scale5-10 yearsAccess
Often free (though products cost money)Slide41
Kinds of Impacts
Theories
Assessment tools
Technological
innovationsGuidelines, templates, patterns, toolkits and standardsPoliciesNew media disseminationAction researchTeaching and teaching materials
What else?
Which kinds of impacts will YOU make?Slide42
Guidelines, templates, and patterns
All provide conventions
So there is little new to learn
Where things go, what they look like
Sometimes task flow guideSlide43
Guidelines, templates, and patterns
What are they based on? Are they consistent?
(Human Interface Guidelines)Slide44
Guidelines, templates, and patterns
Principles, patterns and
practices for improving
use experience
Early instance:Christian Crumlish
&
Erin Malone
Christopher AlexanderSlide45
Guidelines, templates, and patterns
Their effectiveness depends on
The research they are based on
The context in which they arose
Their fit to the context they are being applied toSlide46
Toolkits
UI Development environments
With extra features
Highly interactive
GraphicalDirect manipulationAutomatic undoSupport for animationGesture recognitionAmulet - C++Garnet – Common Lisp, X11, and Mac
Brad MyersSlide47
Standards
Keeping these up to date….Slide48
Who
DevelopersEnd usersHow
Find and use relevant templates….
Scope
Speeds development, makes software consistent Time scaleImmediateAccessFreeGuidelines, templates, patterns, toolkits and standardsSlide49
Kinds of Impacts
Theories
Assessment tools
Technological
innovationsGuidelines, templates, patterns, toolkits and standardsPoliciesNew media disseminationAction researchTeaching and teaching materials
What else?
Which kinds of impacts will YOU make?Slide50
Policies
Stop Online Piracy Act (SOPA)/Protect Intellectual Property Act (PIPA)
Network neutrality
Participatory design in Scandinavia
Open access vs. commercial production of educational materialsData sharing policies……Slide51
Who
EveryoneHow
Dictates
what’s possible
ScopeHugeTime scale?AccessWho gets to be in the conversation?
PoliciesSlide52
Kinds of Impacts
Theories
Assessment tools
Technological
innovationsGuidelines, templates, patterns, toolkits and standardsPoliciesNew media disseminationAction researchTeaching and teaching materials
What else?
Which kinds of impacts will YOU make?Slide53
New Media Dissemination
Total views = 14,660,471Slide54
New Media Dissemination
All videos viewed 6,813,795 times
Hans
Rosling
A
Swedish medical doctor, academic, statistician and public speaker. He is Professor of International Health at
Karolinska Institute
[2] and co-founder and chairman of the Gapminder Foundation, which developed the Trendalyzer software system.Slide55
Who
The publicStudents
How
YouTube
, TedTalks….ScopeHugeTime scaleImmediateAccessFree
New Media
D
isseminationSlide56
Kinds of Impacts
Theories
Assessment tools
Technological
innovationsGuidelines, templates, patterns, toolkits and standardsPoliciesNew media disseminationAction researchTeaching and teaching materials
What else?
Which kinds of impacts will YOU make?Slide57
Action Research
Helping teachers of autistic children assess behavioral incidents
Helping caretakers and clinicians of
preterm infants monitor their
movement and other key factors
Gilllian
HayesSlide58
Who
Target population starting with a small groupHow
New technologies to help critical situations
Scope
Small at first, larger as results are generalizedTime scaleImmediateAccessFreeAction researchSlide59
Kinds of Impacts
Theories
Assessment tools
Technological
innovationsGuidelines, templates, patterns, toolkits and standardsPoliciesNew media disseminationAction researchTeaching and teaching materials
What else?
Which kinds of impacts will YOU make?Slide60
Teaching and Teaching Materials
Undergraduate teaching
6,970 students in a career
Ph.D. students ~40
A multiplier because they go on to teachTeaching materialsBooks for classesCases, exercisesSlide61
Teaching and Teaching Materials
Online resources
that educate
UsabilityFirst.com
Hcibib.org
Useit.comSlide62
Teaching and Teaching Materials
Teaching or action kits
National Center for Women
in Information Technology
NCWITSlide63
Teaching and Teaching Materials
NCWIT Slide64
Teaching and Teaching MaterialsSlide65
Teaching and Teaching MaterialsSlide66
Teaching and Teaching Materials
David Evans &
Sebastian
ThrunSlide67
Who
Students How
Exposed to lectures, exercises, assessments
Scope
Digital media is the multiplierTime scale1-2 yearsAccessSometimes free; sometimes requires tuitionTeaching and Teaching MaterialsSlide68
Kinds of Impacts
Theories
Assessment tools
Technological
innovationsGuidelines, templates, patterns, toolkits and standardsPoliciesNew media disseminationAction researchTeaching and teaching materials…
What else?
Which kinds of impacts will YOU make?Slide69
Recap on what counts as impacts
Who is impacted
How
Scope
Time ScaleAccessDecisions you have to make…Slide70
What it means to have impact
What “counts”
Theory gets used
Downloads/views
ProfitsDegrees/EducationTechnologiesLives changed…..Who is impacted?StudentsDevelopers
Consultants
Specific populations
The general publicSlide71
What it means to have impact
Scopes differ
You affect some people directly
Interventions, teaching
You enable others to be better at making better productsToolkitsYou set policyAffect a large number of peopleSlide72
What it means to have impact
Time scales differ
Now
e.g. Action
research1-3 yearse.g., Publications20-30 yearse.g., Theory Assessment Tools
40-50 years
e.g.,
Cyberinfrastructure development?e.g., Policy (like SOPA/PIPA)Slide73
What it means to have impact
Is it free?
Yes
Khan Academy
Open Knowledge
Standards, toolkits, patterns
Wizard
No
Udacity
Meyers Briggs
GlobeSmart
Textbooks
Degree programs
ProductsSlide74
A continuing dialog
How to translate our research to have broader impacts?
How to guarantee quality?
E.g. evidence based medicine
How to make it accessible?How to evaluate impact?Slide75
Indirect but important impact
How careers are advanced now
Product innovation
Publications
Future+ ImpactIt takes the evaluators to change the systemPromotion policySlide76
What impact will you make?
Theories
Assessment Tools
Popular technologies that become standards
Guidelines, templates, patterns, toolkits and standards
Policies
New media dissemination
Action Research
Teaching and teaching materials
…
Student Volunteers will collect on the way outSlide77
Thank you
In the interest of
potential
impactA video of this will appear on the ACM website and the ACM-W websitejsolson@uci.edu