Aerospace Vehicle Design Alexandra E Coso Dissertation Defense Presentation Friday March 28 th 2014 Advisor Dr Amy Pritchett Committee Dr Brian German Dr Wendy Newstetter ID: 545249
Download Presentation The PPT/PDF document "Preparing Students to Incorporate Stakeh..." is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1
Preparing Students to Incorporate Stakeholder Requirements in Aerospace Vehicle Design
Alexandra E. Coso
Dissertation
Defense Presentation
Friday,
March 28
th
, 2014
Advisor:
Dr. Amy Pritchett
Committee:
Dr. Brian
German,
Dr
. Wendy
Newstetter,
Dr. Jerry Seitzman,
Dr. Robin Adams, and Dr. Alan Jacobsen
Slide2
2
Image references at end of presentationSlide3
Stakeholders in Aircraft Design
Coso - Ph.D. Thesis Defense - March 28th, 2014
3
Stakeholders:
All those affected by the system. They may be users or non-users. They need not be clients or decision-makers. Stakeholders may be major or minor, and the ways in which they interact with a large-scale system are myriad.
- Gibson, Scherer, Gibson 2007, p.315
Non-Users
Design Firms
Airlines
Flight Attendants
& Passengers
Manufacturers & Maintainers
Ground Personnel
PilotsSlide4
Stakeholders in Complex System Design1Maguire, 2001; 2
Green & Jordan, 1999;
3Hupe, 2011;
4Chua & Feigh, 2011;
5Pew, 2008; 6Landsburg et al., 2008
4
Stakeholders are critical to the successful design of products in many fields1,2
International Civil Aviation Organization
(ICAO) has
called for
more integrated consideration of stakeholders
, through social sustainability efforts
3
Human-related design criteria are part of
certification requirements
established by regulators
Additional
efforts are needed to examine how to better incorporate stakeholder requirements consistently within the
design process4-6
AE industry members have
explored a variety of tools and resources for better incorporating stakeholdersSlide5Engineering Design Education
Accreditation
Board for Engineering and Technology (ABET) defines
program
outcomes
that include need to graduate engineers who understand impact of design solutions
on environment, economy, and society1 1ABET, 2011; 2ASEE,
2013; 3Hinkle, 2013; 4Dym et al., 2001; 5Ro et al., 2012; 6Jordan & Lande 2012; 7Zoltowski 2010;
8
McMasters
,
2003
5
Movement
within some fields of engineering to design learning environments which
place a higher value on the needs
and
limitations of stakeholders6-7
Recently, ASEE and NSF
report based on a survey of industry representatives, highlights a need to improve how we prepare students
to synthesize engineering, business, and societal perspectives in the design
process2
Studies of design and engineering students’ pathways support the need to determine how to better prepare students to incorporate stakeholder considerations
into design3-5 Slide6Engineering Design Education
Accreditation
Board for Engineering and Technology (ABET) defines
program
outcomes
that include need to graduate engineers who understand impact of design solutions
on environment, economy, and society1 1ABET, 2011; 2ASEE,
2013; 3Hinkle, 2013; 4Dym et al., 2001; 5Ro et al., 2012; 6Jordan & Lande 2012; 7Zoltowski 2010;
8
McMasters
,
2003
6
There
is a
movement
within some fields of engineering to design learning environments which
place a higher value on the needs and limitations of stakeholders.6-7
Recently, ASEE and NSF
report based on a survey of industry representatives,
highlights a need to improve how we prepare students to synthesize engineering, business and societal perspectives in the design
process2
Studies of design and explorations of engineering students’ pathways in industry support the need to determine how to better prepare students to incorporate stakeholder considerations into design.3-5
Research is limited in terms of
how to improve students’ understanding of the broader context of design and ability to balance the performance and stakeholder-related considerations within a vehicle design8
Research Goals and ContributionsExamine how stakeholder considerations are currently integrated into aerospace vehicle systems designExamine how stakeholder considerations are currently integrated into aerospace engineering design education
Design empirically-informed and theoretically-grounded educational interventions and assessment tools that support students’ integration of stakeholder considerations Isolate the characteristics of the interventions and learning environment that support students’ integration of stakeholder considerations Slide7
RQ#4 -
What
educational interventions
can enhance students’ understanding of and ability to integrate stakeholder considerations into the design of an aerospace vehicle?
Phase I: Examination of Current Practice in Industry and Education
Phase II: Design of Educational Interventions and Assessment Tool
RQ#1 – What conditions enhance or inhibit the integration of stakeholder considerations into the
practice of aerospace vehicle systems design
?
RQ#3 – To what extent do
aerospace engineering students
understand and take into consideration the effects of design decisions on stakeholders?
Coso - Ph.D. Thesis Defense - March 28th, 2014
7
RQ#2 – To what extent and how does the
aerospace engineering design curricula
take into account stakeholder considerations?
Research OverviewSlide8
Research & Presentation Overview8
Phase I:
Examination of Current Practice in Industry and Education
Phase II:
Design of Educational Interventions and Assessment Tool
Practice of Aerospace
E
ngineering
D
esign
Contributions and Future Work
Aerospace Engineering Design
C
urricula
Aerospace Engineering Students’ Perceptions
Educational Intervention SpecificationsSlide9
Examination of Practice of AE Vehicle DesignRQ#1 – What conditions enhance or inhibit the integration of stakeholder considerations into the
practice of aerospace vehicle systems design
?
9
Human factors & related research
Addressed need
and provided recommendations for how to integrate stakeholder considerations into complex system design
Aircraft design research
Examined stakeholder considerations within
multidisciplinary design
,
aerodynamic
analyses
, and studies of the
operational environment
However, this
literature has not
empirically studied
how stakeholder considerations are integrated within
aerospace vehicle design
industry’s
multidisciplinary design
teamsSlide10
Units of
Analysis
Functional
Groups
Embedded Multiple Case Study
Coso - Ph.D. Thesis Defense - March 28th, 2014
10Disciplinary Backgrounds
Individuals
Research
Setting
Aerospace Vehicle Design Firm
Time Frame:
3 Months
Data
Sources
First Round Interviews
Semi-Structured Protocol
Second Round Interviews
Remaining Questions
Topics included:
Overview of the
design process
at the firmThe challenges
affecting the integration of stakeholder considerationsPreviously used interventions to improve integrationSlide11
Results: Group/Team-Level11
Contributing Factors
Group Structure
That supports addressing stakeholder considerations
Common Goal
That addresses stakeholder considerations
Human Factors Specialists
Design Engineers
Pilot Compartment
Group
In-House Pilots
Coso - Ph.D. Thesis Defense - March 28th, 2014
Design Engineers
HF Specialists
Other Functional GroupsSlide12
Results: Individual-Level & Interaction-Level12
Contributing Factors
Differing P
erspectives
About collaborations
with individuals from other disciplinesShared Language &
Boundary Objects To support the exchange of ideas
Image References included at end of presentationSlide13
Educational Intervention SpecificationsFrom Industry Case Study
Coso - Ph.D. Thesis Defense - March 28th, 2014
13
Group/Team-Level
ID1. Stakeholder-centric learning environment
Interaction-LevelID2. Language and vocabulary for discussing stakeholder considerationsID3. Tools and resources to support integration of stakeholder considerations
Individual-LevelID4. Cross-disciplinary problem-solving experiencesID5. Value of designs created from multiple perspectivesSlide14
Research & Presentation Overview14
Phase I:
Examination of Current Practice in Industry and Education
Phase II:
Design of Educational Interventions and Assessment Tool
Practice of Aerospace Engineering Design
Contributions and Future Work
Aerospace Engineering Design Curricula
Aerospace Engineering Students’ Perceptions
Educational Intervention SpecificationsSlide15
Examination of AE Design EducationRQ#2 – To what extent and how does the
aerospace engineering design curricula
take into account stakeholder considerations?
15
A review of
publically-available syllabi from
top aerospace engineering programs and relevant published work
Review of the design processes presented in
classical aerospace design textbooks
(including Roskam
; Raymer; Nicolai & Carichner)
commonly used within aircraft design capstone course
Coso - Ph.D. Thesis Defense - March 28th, 2014Slide16Components of AE Design Curricula
1Peet & Mulder, 2004
16
Elective course offerings
& systems
engineering coursesIsolated course may be
insufficient in training students to integrate these topics into in their capstone courses and in the future1
Isolated Courses
Design Textbooks
Design Projects
Quantitative approximations
(i.e. surrogates)
are used to model some stakeholder characteristics
Mention of
customer needs
and
certification at beginning of text
,
but not emphasized within the design processes themselves
[with exception of Nicolai and Carichner, 2010]
Distinctive from product design
due to life span, purpose, and size of traditional aerospace vehicles
Impacts extent to which students
experience entire design cycleContact with clients or stakeholders is limitedSlide17
Educational Intervention SpecificationsFrom Industry Case Study
ED1. Awareness of contextual and stakeholder considerations
ED2. Importance of considering broad context and wide variety of stakeholders
ED3. Methods for incorporating novel or qualitative stakeholder considerations
ED4. Interactions with clients or other stakeholders
Coso - Ph.D. Thesis Defense - March 28th, 2014
17From Examination of Design Curricula and Students’ Prior Knowledge
Group/Team-Level
ID1. Stakeholder-centric learning environment
Interaction-Level
ID2. Language and vocabulary for discussing stakeholder considerations
ID3. Tools and resources to support integration of stakeholder considerations
Individual-Level
ID4. Cross-disciplinary problem-solving experiences
ID5. Value of designs created from multiple perspectivesSlide18
Research & Presentation Overview18
Phase I:
Examination of Current Practice in Industry and Education
Phase II:
Design of Educational Interventions and Assessment Tool
Practice of Aerospace Engineering Design
Contributions and Future Work
Aerospace Engineering Design Curricula
Aerospace Engineering Students’ Perceptions
Educational Intervention SpecificationsSlide19
Engineering Students’ Prior KnowledgeRQ#3 – To what extent do
aerospace engineering students
understand and take into consideration the effects of design decisions on stakeholders?
1
Carberry et al., 2010; 2
Adams & Fralick, 2010; 3Ro et al., 201219
Design Understanding and Self-Efficacy
Self-Efficacy in Design Scale
1
and
Design Ranking Test
2
Understanding of Contextual Considerations
Contextual Competence Scale
3
and
Open-Ended Follow-Up Question
Perceptions of the Role of Stakeholders in Design
S
cenario-Based Design TaskSlide20
Senior Aerospace Engineering Students
Coso - Ph.D. Thesis Defense - March 28th, 2014
20
Sample:
80 senior AE students enrolled in an aircraft design capstone course
8
women
(~10
%)
5
international students
(~
6%)
30 students (~38%) completed a
previous design
course
52
students
(~65%) described having
industry experience50 students (~63%) took a problem-based learning course
within the departmentSlide21
Design Understanding
Coso - Ph.D. Thesis Defense - March 28th, 2014
21
Important Design Activities
Problem-Scoping
Communication
Not Considered
“Most Important”
Iteration
Making Trade-offsSlide22
Design Understanding
Coso - Ph.D. Thesis Defense - March 28th, 2014
22
Important Design Activities
Problem-Scoping
Communication
Not Considered
“Most Important”
Iteration
Making Trade-offs
Consistent
with aircraft design students from other aerospace engineering
departmentsSlide23
Understanding of Contextual & Stakeholder Considerations
Coso - Ph.D. Thesis Defense - March 28th, 2014
23
Consistent
with aircraft design students from other aerospace engineering
departments
Important Design Activities
Problem-Scoping
Communication
Not Considered
“Most Important”
Iteration
Making Trade-offs
Students consider…
Technical and logistical considerations
Design’s effects on customer
Economic and mission constraints
Students do not see
impact of…
Broader context
(i.e. political, historical considerations)
Stakeholders other than customer
(i.e. pilots, maintainers, passengers)Slide24
Educational Intervention SpecificationsFrom Industry Case Study
ED1. Awareness of contextual and stakeholder considerations
ED2. Importance of considering broad context and wide variety of stakeholders
ED3. Methods for incorporating novel or qualitative stakeholder considerations
ED4. Interactions with clients or other stakeholders
ED5. Importance of iteration and trade-offsCoso - Ph.D. Thesis Defense - March 28th, 2014
24From Examination of Design Curricula and Students’ Prior Knowledge
Group/Team-Level
ID1. Stakeholder-centric learning environment
Interaction-Level
ID2. Language and vocabulary for discussing stakeholder considerations
ID3. Tools and resources to support integration of stakeholder considerations
Individual-Level
ID4. Cross-disciplinary problem-solving experiences
ID5. Value of designs created from multiple perspectivesSlide25
Phase I Contributions 25
Phase I:
Examination of Current Practice in Industry and Education
Practice of Aerospace Engineering Design
Aerospace Engineering Design Curricula
Aerospace Engineering Students’ Perceptions
Educational Intervention Specifications
Empirical
evidence of
conditions which enhance or inhibit the integration
of human factors and stakeholder considerations into the aircraft design process.
Evaluation
method
to explore
engineering students’ perceptions of how stakeholder considerations are integrated
into a complex system designSlide26
Research & Presentation Overview26
Phase I:
Examination of Current Practice in Industry and Education
Phase II:
Design of Educational Interventions and Assessment Tool
Contributions and Future Work
Educational Intervention Specifications
Requirements Lab
Stakeholders in Design LabsSlide27Requirements Lab
Coso - Ph.D. Thesis Defense - March 28th, 2014
27
GOAL:
Introduce
students to requirements, their importance, and how and why stakeholder-requirements should be incorporated into the requirements definition phase of the design
process
Design ProcessSlide28Requirements Lab
Coso - Ph.D. Thesis Defense - March 28th, 2014
28
Student Characteristics
(from In-Class Evaluation)
Wide spectrum of design knowledge and industry experience
Nature of the Subject
Design should be flexible for use within other aerospace courses and programs
Context of Teaching/Learning Situation
(Study-Specific)
Two
50-minute lectures per week (80 students),
One
three-hour lab session (40 students per lab session)
Design ProcessSlide29Requirements
Lab Coso - Ph.D. Thesis Defense - March 28th, 2014
29
LO#3
-
Students will be able to
identify different stakeholders affected by a design problem Based on ID3 & ED3
LO#4 - Students will be able to discuss
how requirements affect design
and the design process
Based on
ED1
LO#6
-
Students will be able to breakdown a problem and
write requirements for performance and stakeholder considerations
Based on
ID1
Design ProcessSlide30Requirements
Lab Coso - Ph.D. Thesis Defense - March 28th, 2014
30
Opportunities for Direct & Immediate Feedback
Class-wide discussions about requirements and stakeholders
“Forward-looking” Assessment/Check for Understanding
Design case provides an open-ended, real-life context for students to practice developing requirements and selecting verification and validation procedures
Further Opportunities to Assess Students’ Understanding
Anonymous Student Feedback Forms
Design ProcessSlide31Requirements
Lab 1Kolb & Kolb, 2009
31
Adaptation of Kolb’s Learning Cycle
1
Design ProcessSlide32Requirements
Lab 1Kolb & Kolb, 2009
32
Adaptation of Kolb’s Learning Cycle
1
Designed such that students will
create knowledge about
stakeholders
and requirements
through activities that require them to
experience
and
reflect
on the relationships between stakeholders, requirements, and design
Design ProcessSlide33
Research & Presentation Overview33
Phase I:
Examination of Current Practice in Industry and Education
Phase II:
Design of Educational Interventions and Assessment Tool
Contributions and Future Work
Educational Intervention Specifications
Requirements Lab
Stakeholders in Design LabsSlide34
Stakeholders in Design Labs Coso - Ph.D. Thesis Defense - March 28th, 2014
34
GOALS:
Have
students consider how stakeholder requirements and concerns can be integrated throughout the aircraft design process
Have students define how stakeholder requirements will be incorporated into the design process they will follow during their team design project
Design ProcessSlide35
Stakeholders in Design Labs Coso - Ph.D. Thesis Defense - March 28th, 2014
35
Student Characteristics
(from In-Class Evaluation)
Wide spectrum of design knowledge and industry experience
Nature of the Subject
Design should flexible for use within other aerospace courses and programs
Context of Teaching/Learning Situation
(Study-Specific)
Two
50-minute lectures per week (80 students),
One
three-hour lab session (40 students per lab session
)
Implemented several weeks into the semester
Design ProcessSlide36
Coso - Ph.D. Thesis Defense - March 28th, 201436
LO#1
-
Students will be able to
identify relevant stakeholders
of a fixed wing design and explain how their concerns affect (or affected) the design solution Based on specifications ID1, ED1,
& ED2LO#2
-
Students will be able to
assess methods
for integrating stakeholder considerations
into the design process
Based
on
ID1, ID3, & ED3
Stakeholders in Design Labs
Design ProcessSlide37
Stakeholders in Design Labs Coso - Ph.D. Thesis Defense - March 28th, 2014
37
Opportunities for Direct & Immediate Feedback
Student-generated design processes
Case study evaluations
Class-wide discussions
Additional Checks for UnderstandingCase study evaluationsStudent reflectionsAnonymous student feedback forms
Design ProcessSlide38
Stakeholders in Design Labs 1Kim,
2001;
2Bucciarelli, 1988
38
Social Constructivist Theory
1Knowledge is constructed through social interaction
Based on ID4 and ID5
Design as Reflective Practice
2
Design requires negotiation among disciplines and perspectives
Based on
ED5
Design ProcessSlide39
Stakeholders in Design Labs Coso - Ph.D. Thesis Defense - March 28th, 2014
39
Lab #1
Framing the Lessons
Design Process Negotiation #1
Pre-Lab Reflection
Individual Design Process Drawing
Design ProcessSlide40
Stakeholders in Design Labs Coso - Ph.D. Thesis Defense - March 28th, 2014
40
1. Understand the Problem
3. Seek Information
Planning
Evaluating
Generate Alternatives
Making Trade-offs
Make Initial Decisions
Testing
Building
2. Visualize & Brainstorm
Identify Constraints
Communicate Throughout!
Iterating
Design ProcessSlide41
Stakeholders in Design Labs Coso - Ph.D. Thesis Defense - March 28th, 2014
41
Lab #1
Framing the Lessons
Design Process Negotiation #1
Introduction to Stakeholder Mapping
Pre-Lab Reflection
Individual Design Process Drawing
Tool for isolating stakeholder considerations that can affect a design
Design ProcessSlide42
Stakeholders in Design Labs Coso - Ph.D. Thesis Defense - March 28th, 2014
42
Government Regulators
(e.g. FAA, EASA)
Passengers
Commercial Aircraft
Pilots
Passengers
Maintainers/ Ground Personnel
Flight Attendants
Non-Users (e.g. people who live near an airport)
Design Firm/ Manufacturers
Suppliers
Wider seat widths
Improved Passenger Experience
Larger fuselage provides greater headroom when walking up and down the aisle
Airlines
Stakeholder Mapping Technique
(adapted from a technique used in Civil and Environmental Engineering)
Design ProcessSlide43
Stakeholders in Design Labs Coso - Ph.D. Thesis Defense - March 28th, 2014
43
Lab #1
Framing the Lessons
Design Process Negotiation #1
Introduction to Stakeholder Mapping
Introduction to Case Study Assessment
Case Study Assessment
Wrap-Up
Pre-Lab Reflection
Individual Design Process Drawing
Tool for isolating stakeholder considerations that can affect a design
Real-world examples of aircraft design process and impact of stakeholders on design
Design ProcessSlide44
Stakeholders in Design Labs Coso - Ph.D. Thesis Defense - March 28th, 2014
44
Lab #2
Conclusion
Design Process Negotiation #2
Human-Centered Design Methods
Case Study Presentations
Framing the Lessons
Student groups deconstruct an aircraft case study
Outcomes include a
stakeholder mapping and a design
process
Introduction to 5 HCD methods
Discussion of advantages and disadvantages of methods and applicability to aircraft design
Modification of initial design processes based on lessons learned in labs
Indications of where stakeholders would be accounted for in the process
Final reflections
Anonymous Student Feedback
Design ProcessSlide45
Contribution of Intervention Design45
Phase I:
Examination of Current Practice in Industry and Education
Phase II:
Design of Educational Interventions and Assessment
Tool
Requirements Lab
Stakeholders in Design Labs
Empirically-informed
and
theoretically-grounded instructional designs
that can
be easily integrated into an aircraft design capstone course or adapted for complex systems design
course Slide46
Research & Presentation Overview46
Phase I:
Examination of Current Practice in Industry and Education
Phase II:
Design of Educational Interventions and Assessment Tool
Contributions and Future Work
Requirements Lab
Stakeholders in Design Labs
Implementation and Evaluation of Educational InterventionsSlide47
Evaluation of Educational Interventions: Research Questions
RQ#4 - What
educational interventions
can enhance students’ understanding of and ability to integrate stakeholder considerations into the design of an aerospace vehicle?
47
RQ4.1 - What characteristics of the educational interventions support students’ abilities to integrate stakeholder considerations into the design of an aerospace vehicle?
RQ4.2 - To what extent can the interventions help students integrate stakeholder considerations into the design of an aerospace vehicle?
RQ4.3 -
What
characteristics of the engineering design learning experience and learning environment
support or hinder students’ integration of stakeholder considerations into the design of an aerospace vehicle?
Coso - Ph.D. Thesis Defense - March 28th, 2014Slide48
Research & Presentation Overview48
Phase I:
Examination of Current Practice in Industry and Education
Phase II:
Design of Educational Interventions and Assessment Tool
Contributions and Future Work
Requirements Lab
Stakeholders in Design Labs
Evaluation of Educational Interventions
Evaluation of the Learning Environment
Stakeholders in Design RubricSlide49
1Adapted from Moon et al 2011 & Kaufman et al 1995
49
Evaluation of Educational Interventions:
Evaluation Framework
1
RQ4.1 - What
characteristics of the educational interventions
support students’ abilities to integrate stakeholder considerations into the design of an aerospace vehicle?
RQ4.2 -
To what extent
can the interventions help students integrate stakeholder considerations into the design of an aerospace vehicle?Slide50
Results Summary: Data from Interventions
Coso - Ph.D. Thesis Defense - March 28th, 2014
50
Requirements Lab
[data from design
c
ase & feedback forms]
Students
formulated well-constructed high-level requirements
and many students were able to identify a diverse group of
stakeholders
Students
considered both performance and stakeholder considerations
and most groups included at least three different verification and validation
procedures
S
takeholder-centric
focus of the lab was not clear to students
Stakeholders in Design Labs [data from
all activities & feedback forms]Stakeholder-centric
focus supported students’ integration of stakeholder considerations in a variety of different ways
in final design processes
Case studies illustrated challenges of cross-disciplinary designMisconceptions about HCD methods may have also developed over the course of these lab sessionsSlide51
Results Summary: Post-Intervention Evaluation
Coso - Ph.D. Thesis Defense - March 28th, 2014
51
Design Understanding at Post-Evaluation
Iteration
:
23.8%
of the students in the
pre-evaluation
selected it, as compared with
69.6%
of students in the
post-evaluation
Making Trade-offs:
33.8
% of the students in the
pre-evaluation selected it, as compared with
48.2% of students in the post-evaluation
Stakeholder Considerations
Students reported
significantly more stakeholder-related considerations in the post-evaluations as compared with the pre-evaluationsSlide52
Research & Presentation Overview52
Phase I:
Examination of Current Practice in Industry and Education
Phase II:
Design of Educational Interventions and Assessment Tool
Contributions and Future Work
Requirements Lab
Stakeholders in Design Labs
Evaluation of Educational Interventions
Evaluation of the Learning Environment
Stakeholders in Design RubricSlide53
Stakeholders in Design RubricREFERENCES: Zoltowkski, 2010; Crismond and Adams, 2012
53
Stakeholder Integration by Design Stage
Design Understanding
Stakeholder Integration
Examines students’
integration of stakeholder considerations into 3 stages of design
Specifically, students’
intent,
application
of
methods,
and
overall success
Scores students based on their
application of the design
process
0pts
: Lack of knowledge about the process
4pts: abstract the process and be a reflective designer
Scores students based on the extent to which they integrated stakeholder
considerations0pts: Lack of appreciation
4pts: Leveraging multiple perspectives for a more innovative solutionSlide54
Examination of Students’ Team Design ProjectsCoso - Ph.D. Thesis Defense - March 28th, 2014
54
Problem Definition Stage
Technology Integration Stage
Overall Design
Most teams
did not state an intention
to incorporate stakeholder considerations
[
as noted by 67.6% of judging panels’ scores
]
Or
apply a design process
that supported integration of stakeholder considerations
[
as noted by
58.8% of judging panels’ scores
]Most teams
did not apply a design process that supported the integration of stakeholder considerations
[as noted by 60.6% of judging panels’ scores]
68.6% of judges’ scores suggested that students were somewhat successful at integrating stakeholder considerations into their final design
solutionsSlide55
Examination of Students’ Team Design Projects
Example of Variations in Scores Across Teams
Team E received 0’s from all 3 judges
Team C received two 3’s and one 2 from their judges
Coso - Ph.D. Thesis Defense - March 28th, 2014
55
Median: 2Applies design process as presented in course and demonstrates how design solution improved through an iterative design process
Median: 2
Identifies stakeholder considerations
at isolated points
in the design process, but overall these
considerations are not the basis for design decisions
and are not addressed consistently through design
Design Understanding
[0 to 4pt Scale]
Stakeholder Integration
[0 to 4pt Scale
]Slide56
Research & Presentation Overview56
Phase I:
Examination of Current Practice in Industry and Education
Phase II:
Design of Educational Interventions and Assessment Tool
Contributions and Future Work
Requirements Lab
Stakeholders in Design Labs
Evaluation of Educational Interventions
Evaluation of the Learning Environment
Stakeholders in Design RubricSlide57
1Adapted from Moon et al 2011 & Kaufman et al 1995
57
Evaluation of Educational Interventions:
Evaluation Framework
1
RQ4.3 -
What
characteristics
of the engineering design learning experience and learning environment
support or hinder students’ integration of stakeholder considerations into the design of an aerospace vehicle?Slide58
Examination of the Learning EnvironmentCoso - Ph.D. Thesis Defense - March 28th, 2014
58
TASK
[Ames, 1992]
Team Design Project
Requirements
Characteristics of Educational
I
nterventions
Reliance on Design
Process
Structure
Available
Tools and ResourcesSlide59
Examination of the Learning Environment
TASK
[Ames, 1992]
EVALUATION
PRACTICES
Team Design Project Requirements
Design of an Assessment Rubric
Characteristics of Educational
I
nterventions
Reliance on Design
Process
Structure
Available
Tools and Resources
Use of Project Requirements as Evaluation Strategy
Evaluation through Interactions with External Audience
Coso - Ph.D. Thesis Defense - March 28th, 2014
59Slide60
Examination of the Learning Environment
TASK
[Ames, 1992]
EVALUATION
PRACTICES
DISTRIBUTION OF AUTHORITY
Team Design Project Requirements
Design of an Assessment Rubric
Characteristics of Educational
I
nterventions
Reliance on Design
Process
Structure
Available
Tools and Resources
Use of Project Requirements as Evaluation Strategy
Evaluation through Interactions with External Audience
Coso - Ph.D. Thesis Defense - March 28th, 2014
60
Transition from Instructor-centered, Individual-based Learning Environment
to
Student-centered, Team-based Learning EnvironmentSlide61
Additional Phase II Contributions61
Phase I:
Examination of Current Practice in Industry and Education
Phase II:
Design of Educational Interventions and Assessment Tool
Requirements Lab
Stakeholders in Design Labs
Evaluation of Educational Interventions
Evaluation of the Learning Environment
Stakeholders in Design Rubric
Empirical evidence of conditions within the learning environment
that impact
students’ integration of stakeholder considerations into a complex systems design process.
Assessment tool
to
support
students’ understanding
of how to design with stakeholders in mind, and
to assess
students’ abilities
to consider the impact of complex system design decisions on stakeholders
.Slide62
Research & Presentation Overview62
Phase I:
Examination of Current Practice in Industry and Education
Phase II:
Design of Educational Interventions and Assessment
Tool
Contributions and Future WorkSlide63
Research Contributions & Future Work63
Industry Case Study
–
empirical evidence of conditions
that enhance or inhibit the integration of human factors & stakeholder considerations into the aircraft design process
In-Class Evaluation & Stakeholders in Design Rubric – evaluation methods to (1) explore engineering students’ perceptions of how stakeholder considerations are integrated into the vehicle design process
,
(2)
support
students’ understanding
of how to design with stakeholders in mind, and
(3)
assess
students’ abilities
to consider the impact of complex system design decisions on stakeholders
Requirements Lab & Stakeholders in Design Labs - empirically-informed and theoretically-grounded instructional designs
that can be easily integrated into an aircraft design capstone course or adapted for complex systems design course.
Examination of Learning Environment
- empirical evidence of conditions within the learning environment that impact students’ integration of stakeholder considerations into a complex systems design
processCoso - Ph.D. Thesis Defense - March 28th, 2014Slide64
Research Contributions & Future Work64
Discipline-based education research
that
informs teaching and learning within aerospace engineering
and supports continued research into this
interdisciplinary fieldStarting point for future research into pedagogical techniques which support students’ integration of stakeholder considerations into the design of complex systems
Coso - Ph.D. Thesis Defense - March 28th, 2014Slide65
Questions?
Acknowledgments
NSF Graduate
Research Fellowship under Grant No. DGE-0644493.
NSF under
Grant #EEC-1059472 to the American Society for Engineering Education.
Thank you to all of the interview participants, the instructors, and the students for their assistance with this
study.
Thank you to
my amazing peer debriefers, Subject
M
atter
E
xperts, and the former and current members
of the Cognitive Engineering Center at Georgia
Tech!Slide66Related Publications
Journal Papers(Submitted) Coso, A. and Pritchett, A. (2013) “The Role of Design Teams in the Integration of Stakeholder Considerations.”
Journal of Aircraft
Conference Proceedings
(Accepted) Coso, A. and Pritchett, A. (2014) “The Development of a Rubric to Evaluate and Promote Students’ Integration of Stakeholder Considerations into the Engineering Design Process.”
American Society for Engineering Education: Proceedings of the 2014 ASEE Annual Conference and Exposition.
Indianapolis, INCoso, A. and Pritchett, A. (2014) “Incorporating Stakeholder Considerations in the Aircraft Design Process: A Focus on Aircraft Design Education.” Proceedings of AIAA SciTech. National Harbor, MD. Coso, A. and Pritchett, A. (2013) “The Integration of Stakeholder Requirements within Aerospace Engineering Design Education.” American Society for Engineering Education: Proceedings of the 2013 ASEE Annual Conference and Exposition.
Atlanta, GA. Coso - Ph.D. Thesis Defense - March 28th, 2014
66Slide67Image References
http://claesjohnson.blogspot.com/2012_07_01_archive.html
http://www.amazon.com/Airplane-Design-Part-Preliminary-Airplanes/dp/188488542X
http://www.amazon.com/Mission-Analysis-Edition-TECHNOLOGY-LIBRARY/dp/0792359011
https://www.aiaa.org/PubDetail.aspx?id=1796
http://www.aviationweek.com/Blogs.aspx?plckBlogId=Blog:27ec4a53-dcc8-42d0-bd3a-01329aef79a7&plckPostId=Blog:27ec4a53-dcc8-42d0-bd3a-01329aef79a7Post:6e334c9f-0810-4874-a0be-d7e023b58135
http://orbiterchspacenews.blogspot.com/2010/11/nasa-selects-companies-for-advanced.htmlhttp://www.disinfo.com/tag/spacex/http://www.open.ac.uk/about/campus/jennie-lee-research-labs/files/jennieleeresearchlabs/imagecache/node_standard/HCI%202%20Lab-1.jpg
http://www.manvitas.com/images/Corporate.jpghttp://www.inc.com/uploaded_files/image/speed-up-your-prototyping-process-pop_9632.jpghttp://www.kelvinyoung.com/v6/work/storyboard_01.jpghttp://www.nngroup.com/reports/paper-prototyping-training-video
/
For literature references, view dissertation text
Coso - Ph.D. Thesis Defense - March 28th, 2014
67