Engineering Elegant Systems Engineering at the System Level Consortium Team UAH George Washington University Iowa State Texas AampM University of Colorado at Colorado Springs UCCS Missouri University of SampT ID: 783883
Download The PPT/PDF document "11 July 2017 Michael D. Watson, Ph.D." is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1
11 July 2017Michael D. Watson, Ph.D.
Engineering Elegant Systems: Engineering at theSystem Level
Consortium Team
UAH
George Washington University
Iowa State
Texas
A&M
University of Colorado at Colorado Springs (UCCS)
Missouri University of S&T
University of Michigan
Doty Consulting Services
AFRL
Wright
Patterson
Slide2OutlineUnderstanding Systems EngineeringPostulatesHypothesisPrinciplesSystems Engineering DomainSystem IntegrationSystem State VariablesGoal Function TreeState Analysis Model
System Value ModelSystem Integrating PhysicsSystem AutonomyMultidisciplinary Design Optimization (MDO)Engineering StatisticsMethods of System IntegrationDiscipline Integration
Sociological Concepts in Systems Engineering
Information Flow
System DynamicsCognitive SciencePolicy and Law in ApplicationSupporting ProcessesSummary
2
Slide3Understanding Systems Engineering
Slide4MotivationSystem Engineering of Complex Systems is not well understoodSystem Engineering of Complex Systems is ChallengingSystem Engineering can produce elegant solutions in some instancesSystem Engineering can produce embarrassing failures in some instancesWithin NASA, System Engineering does is frequently unable to maintain complex system designs within budget, schedule, and performance constraints
“How do we Fix System Engineering?”Michael D. Griffin, 61st International Astronautical Congress, Prague, Czech Republic, September 27-October 1, 2010Successful practice in System Engineering is frequently based on the ability of the lead system engineer, rather than on the approach of system engineering in general
The rules and properties that govern complex systems are not well defined in order to define system elegance
4 characteristics of system elegance proposed as:
System EffectivenessSystem EfficiencySystem RobustnessMinimizing Unintended Consequences
4
Slide5ConsortiumResearch Process Multi-disciplinary research group that spans systems engineering areas Selected researchers who are product rather than process focusedList of Consortium MembersMichael D. Griffin, Ph.D.Air Force Research Laboratory – Wright Patterson, Multidisciplinary Science and Technology Center: Jose A. Camberos
, Ph.D., Kirk L. Yerkes, Ph.D.Doty Consulting Services: John Doty, Ph.D.George Washington University: Zoe Szajnfarber, Ph.D. Iowa State University: Christina L. Bloebaum, Ph.D., Michael C. Dorneich, Ph.D.Missouri University of Science & Technology: David Riggins, Ph.D.
NASA Langley Research Center: Peter A. Parker, Ph.D.
Texas A&M University: Richard Malak, Ph.D.
Tri-Vector Corporation: Joey Shelton, Ph.D., Robert S. Ryan, Kenny MitchellThe University of Alabama in Huntsville: Phillip A. Farrington, Ph.D., Dawn R. Utley, Ph.D., Laird Burns, Ph.D., Paul Collopy, Ph.D., Bryan Mesmer, Ph.D., P. J. Benfield, Ph.D., Wes Colley, Ph.D., George Nelson, Ph.D.The University of Colorado – Colorado Springs: Stephen B. Johnson, Ph.D.The University of Michigan: Panos Y. Papalambros, Ph.D.
The University of Texas, Arlington: Paul
Componation
, Ph.D.
The University of Bergen: Erika Palmer
Previous Consortium Members
Massachusetts Institute of Technology: Maria C. Yang, Ph.D.
Stevens Institute of Technology – Dinesh
VermaSpaceworks – John Olds (Cost Modeling Statistics)Alabama A&M – Emeka Dunu (Supply Chain Management)George Mason – John Gero (Agent Based Modeling)Oregon State – Irem Tumer (Electrical Power Grid Robustness)Arkansas – David Jensen (Failure Categorization)
~50 graduate students and 15 undergraduate students supported to date
5
Slide6Understanding Systems EngineeringDefinition – System Engineering is the engineering discipline which integrates the system functions, system environment, and the engineering disciplines necessary to produce and/or operate an elegant system.Elegant System - A system that is robust in application, fully meeting specified and adumbrated intent, is well structured, and is graceful in operation.
6
Primary Focus
System Design and Integration
Identify system couplings and interactions
Identify system uncertainties and sensitivities
Identify emergent properties
Manage the effectiveness of the system
Engineering Discipline Integration
Manage flow of information for system development and/or operations
Maintain system activities within budget and schedule
Supporting Activities
Process application and execution
Slide7Systems Engineering PostulatesPostulate 1: Systems engineering is product specific.Postulate 2: The Systems Engineering domain consists of subsystems, their interactions among themselves, and their interactions with the system environmentPostulate 3: The function of Systems Engineering is to integrate engineering disciplines in an elegant mannerPostulate 4: Systems engineering influences and is influenced by organizational structure and culture
Postulate 5: Systems engineering influences and is influenced by budget, schedule, policy, and lawPostulate 6: Systems engineering spans the entire system life-cyclePostulate 7: Understanding of the system evolves as the system development or operation progresses
7
Slide8System Engineering HypothesesHypothesis 1: If a solution exists for a specific context, then there exists at least one ideal Systems Engineering solution for that specific contextHypothesis 2: System complexity is greater than or equal to the ideal system complexity necessary to fulfill all system outputsHypothesis 3: Key Stakeholders preferences can be accurately represented mathematically
8
Slide9Systems Engineering PrinciplesPrinciple 1: Systems engineering integrates the system and the disciplines considering the budget and schedule constraintsPrinciple 2: Complex Systems build Complex SystemsPrinciple 3: The focus of systems engineering during the development phase is a progressively deeper understanding of the interactions, sensitivities, and behaviors of the systemSub-Principle 3(a): Requirements are specific, agreed to preferences by the developing organizationSub-Principle 3(b): Requirements and design are progressively defined as the development progresses
Sub-Principle 3(c): Hierarchical structures are not sufficient to fully model system interactions and couplingsSub-Principle 3(d): A Product Breakdown Structure (PBS) provides a structure to integrate cost and schedule with system functionsPrinciple 4: Systems engineering spans the entire system life-cycleSub-Principle 4(a): Systems engineering obtains an understanding of the systemSub-Principle 4(b): Systems engineering models the system
Sub-Principle 4(c): Systems engineering designs and analyzes the system
Sub-Principle 4(d): Systems engineering tests the system
Sub-Principle 4(e): Systems engineering has an essential role in the assembly and manufacturing of the systemSub-Principle 4(f): Systems engineering has an essential role during operations and decommissioning
9
Slide10Systems Engineering PrinciplesPrinciple 5: Systems engineering is based on a middle range set of theoriesSub-Principle 5(a): Systems engineering has a mathematical basisSystems Theory BasisDecision & Value Theory Basis (Decision Theory and Value Modeling Theory)Model BasisState Basis (System State Variables)Goal Basis (Value Modeling Theory)Control Basis (Control Theory)
Knowledge Basis (Information Theory)Predictive Basis (Statistics and Probability)Sub-Principle 5(b): Systems engineering has a physical/logical basis specific to the systemSub-Principle 5(c): Systems engineering has a sociological basis specific to the organizationPrinciple 6: Systems engineering maps and manages the discipline interactions within the organization
Principle 7: Decision quality depends on the system knowledge represented in the decision-making process
Principle 8: Both Policy and Law must be properly understood to not overly constrain or under constrain the system implementation
Principle 9: Systems engineering decisions are made under uncertainty accounting for risk
10
Slide11Systems Engineering PrinciplesPrinciple 10: Verification is a demonstrated understanding of all the system functions and interactions in the operational environmentIdeally requirements are level and balanced in their representation of system functions and interactionsIn practice requirements are not balanced in their representation of system functions and interactionsPrinciple 11: Validation is a demonstrated understanding of the system’s value to the system stakeholdersPrinciple
12: Systems engineering solutions are constrained based on the decision timeframe for the system need11
Slide12Methods of System IntegrationGoal: Techniques to Enable Integrated System Design and Assessments by the Systems Engineer
Slide13System State VariablesGoal: Utilize system state variables to understand the interactions of the system in relation to system goals and system execution
Slide14System State ModelsSystem Stage Models represent the system as a whole in terms of the hardware and software states that the system transitions through during operationGoal Function Tree (GFT) Model“Middle Out” model of the system based on the system State VariablesShows relationship between system state functions (hardware and software) and system goalsDoes not contain system physical or logical relationships and is not executableSystem State Machine Model
Models the integrated State Transitions of the system as a whole (i.e., hardware states and software states)Confirms system functions as expectedChecks for system hazardous, system anomalies, inconsistent state progression, missing states, improper state paths (e.g., short circuits in hardware and/or software design)Confirms that the system states progress as stated in the system design
Executable model of system
14
Slide15Booster – CS Ascent GFT
System Works
Slide16System State Machine ModelThe state analysis model is split into two main components:Manager software modelSystem PlantModeled using MATLAB StateflowAllows the software model to look like the SysML Activity DiagramsAllows the
SystembPlant to be modeled as State MachinesAllows those two models to interact with each other within the MATLAB environmentFacilitates the ability to generate custom analysis toolsReads in command sequence to execute model
16
Slide17State Analysis Model for SLS M&FM
Commands
From Launch
Countdown Doc
Control
(
SysML
to Stateflow)
Plant
(State Machines)
Commands
Sensor
Values
Faults
Physics Values
14% of R12 modeled
Over 7,200 Transitions in the Vehicle and Software
Over 3,500 States in the Vehicle
Slide18System ValueGoal: Utilize system state variables to understand the interactions of the system in relation to system goals and system execution
Slide19System Value ModelA System Value Model is a mathematical representation of Stakeholders Preferences (Expectations) for the systemThe basic structure is straight forwardThe sociology/psychology of representing the Preferences can be a challengeThe System Value Model is the Basis of System Validation!!!The Requirements and Design Models form the basis of System VerificationThe System Value Model forms the basis of System Validation
Constructing an SLS Value Model to compare to System Validation resultsCan expand to Integrated Stack with input from MPCV and GSDOSystem Value model also provides basis for a measure of System RobustnessHow many mission types are supported by the system?
19
Slide20Integration with the Capability ModelCongressional Model
SLS AttributesMission 1
Mission 2
Mission 3
Capability Value
Development Cost
Slide21𝑌𝑇 = 𝐴𝑅∗[𝛿∗𝐾
𝑇(𝜎−1)/𝜎+(1−𝛿)∗𝐿
𝑇
(𝜎−1)/𝜎
]
𝜎/(𝜎−1)
+Other Preferences
𝑌
𝑇
= Total output (Best Value from substituting
𝐾
𝑇
and 𝐿𝑇
)𝜎 = Elasticity of Substitution𝛿 = Distribution Parameter
Value between 0 and 1
𝐴
𝑅
= Productivity Factor
𝐾
𝑇
= Value of capital/services from Program 1 (SLS)
Societal Benefits, Resource Value, and New Information from program 1
𝐿
𝑇
= Value of capital/services from Program 2 (Other)
Societal Benefits, Resource Value, and New Information from program 2
Production Function
Slide22Fundamental ObjectivesValue model must quantitatively answer the question: “How does the SLS contribute to NASA’s top-level objectives?”Concrete contributions of the SLS are shown below as “sub-goals”. Note that SLS contributes to the four positive objectives indirectly (for the most part) by enabling missions – the missions themselves are the actual engines of value generation.Prior work on launch vehicle value models is in the context of specific missions – these value models, while useful, are really value models for the missions, not the launch vehicle itself.
A value model for the SLS should value the mission-enabling capability of the vehicle in the generic sense.
Slide -
22
Enable Missions
Increase Military Capability
Support Economic Growth
Further Scientific
Knowledge
Create Jobs & Support Industry
Minimize Negative Impacts
Minimize Environmental Damage
Minimize Loss of Life
Generate National Prestige
Impress other Nations
Impress Public
Sub-Goals
Top-Level Objs.
Impress other Nations
Impress the Public
Create Jobs & Support Industry
Minimize Environmental Damage
Minimize Loss of Life
Enable Missions
Slide23Constructing the Value ModelSlide - 23A generic value model maps a design, described by an envelope of top-level attributes, to a scalar value.This model can (and should) be capable of accepting uncertain attributes and propagating them through to output an uncertain value.This uncertain value can then be adjusted for risk attitude if desired, or (if decision-maker is risk-neutral) the expected value can be used as the final “score” for the design.
A proposed valuation paradigm for the mission enabling capability of SLS should have two distinct components:Capability model (from system behavioral models – “what can it do?”)Value model (from preference elicitation – “how does this benefit us?”)
Capability Model
(physics, software, etc.)
Top-level Attributes
Value Model
(preferences)
Design variables, environmental variables, operational variables
(includes uncertainties)
System Value
Lower
-level Attributes
Slide24Capability Model
(physics, software, etc.)
Top-level Attributes
Value Model
(preferences)
Design variables, environmental variables, operational variables
(includes uncertainties)
System Value
Lower
-level Attributes
Constructing the Value Model
Modifications to basic value model framework for SLS:
Because the SLS does not create value directly (instead enabling missions that create value), the basic value model framework must be modified.
Instead of mapping capability directly to value, instead capture all relevant capabilities of the vehicle and interface them with a portfolio of individual mission value models.
Slide -
24
Slide25The Capability EnvelopeThe first step in constructing the value model is to determine the appropriate abstractions for capturing the capabilities of any given launch vehicle.What are the “top-level attributes” for a launch vehicle?A top-level attribute is defined as any system attribute that directly impacts value delivery – or to put it another way, any system attribute that a potential “customer” of SLS might care about.It is important to identify which system attributes are top-level attributes.Top-level attributes (such as cost, payload mass to orbit, and weather envelope) are derived from other attributes (such as materials used, engine specific impulse, and controllability) via various behavioral models.Any given system design is represented by an N-dimensional envelope in top-level attribute space.
This envelope is here called the “capability envelope”.It is then mapped to value by interfacing with mission value models.Slide -
25
Slide26The Capability EnvelopeThe capability envelope is the fundamental abstraction that allows for valuation of any given launch vehicle design.Why an envelope specifically?Systems are commonly represented with vectors in attribute space.“Envelope” implies instead a bounded region of attribute space.For many top-level attributes, a vector is sufficient to represent capability.However, certain relationships should be expressed as continuous relationships or envelopes in order to accurately model value.Definition of “capability envelope”:
Slide -
26
The
Capability Envelope
is the combination of all top-level
attribute
values
and all
relationships
between top-level attributes that fully define any
individual
design concept.
Slide27The Capability EnvelopeOne of the most important parts of the capability envelope is the answer to the question “how much can it carry, and how far?”This is expressed as an envelope bounded by the curve representing the relationship between max payload mass and total energy imparted.Can be expressed as “C3 vs payload mass” or “
DV vs payload mass”.Converting between the “C3 vs payload mass” and “DV vs payload mass” curves is trivial as long as the required DV to some reference orbit can be assumed or estimated from aerodynamic and trajectory models.
Slide -
27
OR
Slide28The Capability EnvelopeOther top-level attributes included in the capability envelope:Cost (including production, integration, launch, etc.)Reliability (probability of mission success for any given launch)Payload services provided (power, thermal conditioning, etc.)Load factors imparted to payload (max Q and during dynamic events)Shock loads imparted to payload (max Q and during dynamic events)Controllability envelope (wind speeds vs fairing dims vs CoG offset)Allowable payload volumes (related to fairing dimensions)
Time between launches (including assembly time, pad time, etc.)Injection accuracy (altitude and inclination)Ability to operate in loss of communication (autonomy)Important to capture relationships that may restrict variables, even if the variables may affect different aspects of value.Example: controllability envelope relates wind speeds (affects value by determining how often vehicle can launch) to fairing dimensions (affects value by determining which payloads can be carried).
Slide -
28
Slide29The Capability EnvelopeNote that the capability envelope is quite specifically the inherent capabilities of any given single design (see slide 15). The capability envelope is not a design trade-space envelope.The former is an abstraction of a single design in the space of all possible designs, while the latter is a representation of all designs that are possible.Example: the relationship “max wind speed vs fairing diameter” is part of the capability envelope (assuming multiple fairings exist for a single design), while the relationship “cost vs reliability” is not (unless you’ve designed an SLS that literally allows for trading those off on any given launch).The capability envelope is not a mission trade-space envelope.
The former is the inherent capabilities of the vehicle, while the latter is related to specific mission planning.Example: the relationship “max payload mass vs max transfer energy” is part of the capability envelope, while the relationship “flight duration vs transfer energy” (as shown on a porkchop plot) is not.
Slide -
29
Slide30Mapping Capability to ValueThis is only the basic framework for the valuation model.Implementation may vary, but these core elements should remain.There is not necessarily a “correct” specific implementation of this framework – the implementation is limited in detail only by the designer’s knowledge of their preferences and/or ability to codify them.The “mission portfolio” valuation paradigm is beneficial because it allows for the consideration of the entire gamut of launch vehicle applications throughout its lifetime.The following slide presents a general illustration of how the mission portfolio maps the capability envelope to value.
Mission A
20,000 m/s
dV
required
Value = $50000 * m
Demand = 25% of total
Mission B
15,000 m/s
dV
required
Value = $30000 * m
Demand = 60% of total
Mission C
32,000 m/s
dV
required
Value = $80000 * m
Demand = 15% of total
Slide31Mapping System Capability to Value
“Will it work?”
(Reliability)
“What can it carry?”
Load Factors
Shock Loads
Payload Volume
Payload Services
Injection Accuracy
“How expensive is it?”
Production cost
Launch cost
e
tc.
Missions Attempted
Missions Succeeded
Total Value Delivered by Launch Vehicle
&
31
Slide32Mapping Capability to ValuePossible variations on this valuation framework:Mission Classifications:While location is used in the example as the sole classifier for missions, it is far from being the only potentially useful classifier. Missions could also be broken up by purpose (science/military/commercial), manned/unmanned, etc. – it is up to the designer to decide what is most useful.The important thing is that each unique mission archetype has a characteristic C3/D
V requirement, valuation function, and demand.Value Curve:Instead of having a required C3/DV for each mission and value being a function of mass, value for each mission could simply be a function of C3/D
V and mass.
This is a more complicated formulation that would be much more cumbersome to construct, but it would allow for more realistic representations of certain situations (such as scientific mission to a specific location that can accomplish more with a smaller payload and greater
DV than with a larger payload and smaller DV).
Again, it is up to the designer to decide which formulation will best serve the project – in essence trading model fidelity for model effort.
Slide -
32
Slide33Mapping Capability to ValueSlide - 33Possible variations on this valuation framework:Mission Demand:Various restrictions could potentially be placed on any given mission’s demand to more accurately reflect real-world limitations – “no more than X launches of Y mission type every Z years”, etc.
These would allow for more realistic modeling of the change in value delivery granted by having a vehicle that can launch more frequently.Example: It could be the case that no matter how often the launch vehicle can launch, you do not want to (or cannot) launch more than 2 Jupiter missions every 10 years, but would instead do many more LEO and Luna missions).General:Various restrictions could potentially be placed on
individual missions
– “this mission requires at least X kg of payload to be attempted”, that sort of
thing.Any small tweaks which increase the fidelity of the model are welcome, however designers must remain conscious of the increased effort that will result.It is recommended to start with a low-fidelity model (similar to that which is shown in the example formulation) and refine as needed instead of attempting to construct a highly detailed model from the start.
Slide34System Physics and System Integrating PhysicsGoal: Utilize the key system physics to produce an elegant system design
Slide35System Integrating PhysicsConsortium is researching the significance of identifying and using the System Integrating Physics for Systems EngineeringFirst Postulate: Systems Engineering is Product Specific.States that the Systems are different, and therefore, the Integrating Physics for the various Systems is differentLaunch VehiclesThermodynamic System
SpacecraftRoboticIntegrated through the bus which is a thermodynamic systemEach Instrument may have a different integrating physics but integrates with the bus thermodynamicallyCrew ModulesIntegrated by the habitable volume (i.e., ECLSS)A thermodynamic system
Entry, Descent, and Landing (EDL)
Integrated by thermodynamics as spacecraft energy is reduced in EDL
Other Thermodynamic SystemsFluid SystemsElectrical SystemsPower PlantsAutomobiles
Aircraft
Ships
Not all systems are integrated by their Thermodynamics
Optical Systems
Logical Systems
Data Systems
Communication Systems
Biological SystemsSystem Integrating Physics provides the engineering basis for the System Model
Slide36Launch Vehicle System Exergy Efficiency36
S-1C Center Engine Cut-Off
S-1C Stage Separation
S-II Center Engine Cut-Off
S-II Stage Separation
S-II Engine Mixture Ratio Shift
S-IVB Burn
1
Cut-Off
LEO Insertion
S-IVB Burn 2 Cut-Off
S-!VB Separation
Max Q
S-IVB Burn 2 Engine
Mixture Ratio Shift
.
Slide37Spacecraft Integration Model
37
Slide38Crew Module Exergy Balance: ISS ECLSS
38
Calculate Efficiency
Slide39System AutonomyGoal: Establish system interfaces to provide autonomy algorithms with system information necessary and sufficient to manage system
Slide40System Engineering of Autonomous Systems40Elegant System Engineering requiresUnderstanding the Mission ContextSystem ApplicationsSystem Environments (operational, test, abort, etc.)Understanding the Physics of the System
System Interactions with themselves and with their environments are governed by their physicsInformation Theory provides linkages between physical state representations and actual physical statesManaging the organizational influences on system design and the system context influences on the organizationUnderstanding Policy and Law Constraints
National Space Policy
International Space Treaties and agreements
Space Debris, Contamination, Property
Slide41Autonomy in Context: What and Why?41Spacecraft and Surface System Autonomy is the enabling capability for Human Exploration beyond Lunar Sortie MissionsAutonomy is necessary for complex system operations
Timely response to unplanned or unscheduled eventsPropulsion, Structure, Thermal Conditioning, ECLSS, Electrical Power, Avionics, RCS, Communication are all understood sufficiently to allow engineered solutions to be reliably producedChallenges do exist in terms of Space Environmental Effects, efficiency, compact sizeRadiation Hardened computer processors neededPhysics and demonstrated solutions are available from which to engineer a vehicle
Operations are sufficiently understood for terrestrial based execution, not on-board execution
Manual operations provide a rich knowledge base of planning and execution processes
Manual operations have a generic template (derived from Apollo/Saturn) applied uniquely to each spacecraftTerrestrial based manual operations will not support operations beyond 5 light minutes from Earth
Autonomous Operations are essential to Human Exploration of the Solar System
Slide42Operations Concept Drivers42Small Crew Size (4-6)1 crew member per shift available for vehicle operationsLimited systems expertsComplex Systems
Nuclear Power and Propulsion SystemsLife Support and Environmental ProtectionUSN Attack Submarines are similar complexity systems but have 134 crew members~525 high level functions to manage an interplanetary crewed spacecraft.Abort ScenariosUnambiguous determination
Extremely low latency
Fully autonomous/automated (crew incapacitated conditions)
Vehicle reconfiguration necessaryLong Communication Latency/Blockages15 minutes one way, 30 minutes round trip to MarsGround based intelligence not responsive to maintain crew safety
1 hour blockage by Moon each Lunar orbit
Harsh Environment
Solar flare radiation
Meteorites
Slide43SUN
5 min.
15 min.
2.5 min.
Mercury
Venus
1.28 sec.
Earth
Mars
AU
.387
(Not to Scale)
.723
1.0
1.5236
25 min.
Moon
1 min.
Slide44Spacecraft Systems Overview44Beyond Earth Orbit (BEO) crew transport vehicle are comprised of several unique and intricately integrated subsystemsPropulsionStructureElectrical Power
AvionicsThermal ManagementFlight control systemCommunication and TrackingVehicle Management (Guidance, Navigation and Control (GN&C) and Mission and Fault Management (M&FM))
Environmental Control and Life Support Systems (ECLSS)
Each of these subsystems are driven by unique physics and information theory relationships
Control Theory governs the control of each subsystem both independently and at the vehicle level
Slide45State Variable MethodologyGoal/Function TreeState Variable to define System PerformanceState variables are defined as inputs and outputs to functions: y=f(x)x = inputs to the functions ff transforms the inputs into the outputs yGoals = Requirements => define intended range of the output state variables yFailure = state (value) of output state variable y is out of intended range
State variables enforce strong connection of the functional decomposition to the system’s physical laws and causationThe state variables are the connection between function and design—exist in both function and design representationsAllows system to be analyzed in each mission phase and goals which can have different ranges and values for each state variableAllowed leak rates vary inversely with time from Earth Return date
45
Function f
≤
≤
G
Slide46Mars Mission simplified GFT ExampleMars Landing
Deliver to Destination
Keep crew alive
Control
Trajectory
Control
Attitude
keep capsule intact
limit
accelerations
on crew
provide life support
control structural temp
control structural loads
provide desired
Pos/
Vel
Measure actual Pos/Vel
Control Error
Between
Desired & Measured Pos/
Vel
X,
V
X, V
Crew A
P, Cabin T, O2
Capsule A,
Capsule
Struc
T
Capsule
Struc
T
Des X
Des V
Meas
X
Meas
V
Capsule A
A,
Cabin
T,
O2,
Θ
dot
Θ
Control Crew
Roll Rate
Maintain
Struc
Integrity
Control
Structure
Defects
Control
LV
Struc
Loads
Control
LV
Struc
Temps
LV
Struc
A
LV
Struc
T
Struc
Crack
Size, Alloy
Purity
Control
Attitude
Error
Θ
Err
Complete Mars Science
Achieve Mars Orbit
Land on Earth
y
= f (x)
x
f
y
Functioning Crew Performs Mars Mission & Returns to Earth
X, V,
Θ
dot, A
,
Struc
T, Cabin T, O2
Reach Mars Vicinity
Achieve Earth Orbit
A, LV
Struc
T
X, V,
Θ
dot, A
,
Struc
T, Cabin T, O2
X, V,
Θ
dot, A
,
Struc
T, Cabin T, O2
LV “Ready”,
Struc
T, Cabin T, O2
Θ
dot
X
Err, V Err
Crewed BEO Mission Goal Types
Transportation
Crew health and safety
Scientific and
Technical
Slide47Transportation Goals47Position, Velocity, AccelerationEarth Departure, Mars DeparturePropulsion SystemFlight Control SystemInterplanetary Coast
Propulsion SystemFlight Control SystemPlanetary Orbital InsertionPropulsiveAero BrakingSurface DescentPropulsiveAero SurfacesPlanetary Mobility
Drive force
Control
System
Slide48Crew Health and Safety Goals48Provides link between human health and System PerformanceBiologicalPsychologicalBiological State Variables are linked directly with System State Variables
BiologicalHeart rateRespiration rateFood intakeWater intakeSolid and Liquid waste production rate Spacecraft SystemsBreathable air (oxygen
concentration, carbon dioxide concentration, atmospheric
pressure)
Oxygen can be stored as LOX and converted to gas as neededDrinkable water (mass)Consumable food (mass)Solid and Liquid waste processing/disposal (mass)Vehicle acceleration rates (
linear and rotational
accelerations)
Crew Cabin/Suit temperature (temperature and humidity)
Activity (work and exercise)
and
sleep times (hours or minutes / crew day)
Communication System (family communications (email, video, audio), entertainment, etc.)
Ranges vary with mission phases
Slide49Science and Technology Goals49Information ReturnCommunication systemsTransmission ratesradiated powersignal strength
beam widthSample ReturnContainment System (mass, pressure, leakage rate)Samples (mass)
Slide50Autonomy Stack50Autonomy must operate consistent with the physical control laws of the vehicle systemsMultiple subsystems exist within the vehicleManagement algorithms must match subsystem physical control lawsVehicle level integration is a unique set of relationships dependent on the subsystem types chosen
Type of PropulsionType of Flight Control System(s)Type of ECLSSType of Electrical Power GenerationEtc.
Slide51Autonomy Stack51Vehicle Autonomy has 5 distinct functionsControl Monitoring (sensing)Performance DeterminationDiagnostics
PrognosticsSubsystems Autonomy has the same 5 distinct functionsControlMonitoring (sensing)Performance DeterminationDiagnosticsPrognostics
ISHM
FDIR
Control System
ISHM
FDIR
Control System
Slide52Subsystem Management Functions for System Control
Performance
Diagnostics
Prognostics
Monitoring
Control
Subsystem
Slide53System
System Control
ISHM
Vehicle Control
Vehicle ISHM
Mission Execution
Mission Objectives & Constraint Data
Mission Planning
Autonomy System Stack
Detection
Diagnostics
Prognostics
Actuation
Control Logic
Slide54Candidate Autonomous Algorithms for Spacecraft Systems54Several classes of Autonomous AlgorithmsExpert SystemsNeural NetworksBayesian B
elief NetworksModel Based ReasoningFuzzy LogicDemonstrated in marine, space, industrial, and aviation applicationsVerification and Validation (V&V) approaches will need to be defined for these algorithms, both individually and as an integrated set
Formal V&V Methods
(e.g., model checkers
) need to be properly appliedNon-deterministic V&V methods need definition
Slide55Candidate Autonomous Algorithms for Spacecraft Systems55Expert SystemsExpert rules establish decision structureKnowledge base contains rules and relationshipsServes well as a central authority where rules/relationships are clearly established
Can be processing intensive with high data storage requirements depending on rules and rule relationship complexitiesWell suited for:Mission Planning, Crew and Mission Constraint ManagementSubsystems with clear cut physical equations and well understood interrelationships
Slide56Candidate Autonomous Algorithms for Spacecraft Systems56Neural NetworksGradient Descent MethodsDeterministic due to the underlying mathematicsIdeal for nonlinear and interpolative applications/situation
Static NetworksLearning during training operations onlyQuality of application based on quality of training casesDynamic NetworksLearning during real time operationValidation and predictabilityImplementation
Hardware (fast)
Software
Complexity can be difficult to verify and may require specialized chips (e.g., ASIC)Ideal for Control of highly nonlinear subsystemsPropulsion, Flight Control System transientsInterpolationGood where there is limited knowledge of complex physical interactions
Real time adaptation in the event of spacecraft subsystem reconfiguration (failure response)
Slide57Candidate Autonomous Algorithms for Spacecraft Systems57Bayesian Belief NetworksApplies Bayes Rule to Determine System StatePrior StatesCurrent Belief probability
Best employed as an information source for other subsystem or vehicle autonomous algorithmsHelps clarify/validate uncertaintyAids inference and reasoning (e.g., augments Expert Systems) Well Suited for:
Performance Determination
Vehicle
Subsystem
Slide58Candidate Autonomous Algorithms for Spacecraft Systems58Model Based ReasoningModels based on extensive domain knowledge
Can leverage design modelsUncertainty based on fidelity of model implementedSoftware architecture must addressEfficient Programming LanguageOperating
S
ystem
capable of dealing with Conflict resolutionEfficient processingEmbedded systems for mission critical applications (i.e., software health management)Well Suited for:
Vehicle and Subsystem Diagnostics
GN&C (
Kalman
Filter)
Slide59Candidate Autonomous Algorithms for Spacecraft Systems59Fuzzy LogicClassical Mathematical Set TheoryRequires deep knowledge of subsystem physical rules and interactions to properly trainProvides support to Reasoning Systems (e.g., Model Based Reasoning)
Well Suited for:Flight Control Systems
Slide60Autonomous Algorithm Integration603 LevelsMission Execution and PlanningVehicle ManagementSubsystem Integration BasedPhysics form basis of subsystem interactions
Form basis of normal or failed statesSubsystem LevelPhysics based
Slide61Autonomous Algorithm Integration61Subsystem Level AutonomyKeys: Understanding the physics of the system Selecting an autonomous algorithm that can
effectively manage the system physics(take the necessary actions based on all interactions) and responsively manage the system physics (take the necessary action in a timely manner) System physics are driven by the internal system processes, interactions with other systems, and interactions with the environment, all of which must be managed by the
algorithm
System-level algorithm matching involves knowledge of the system transfer functions which include external system and environment
interactionsControl Theory is important in implementation. The physics will define the poles and zeros of the control system and the relative proximity of the system response to these locations.
System Transfer
F
unctions
must be defined and matched with the characteristics of the autonomous
algorithms
Slide62Autonomous Algorithm Integration62Vehicle Level AutonomyKeys: Integration of the systems autonomous algorithms into a cohesive and response management system
Algorithms taking proper responses to planned and unplanned conditionsManaging the subsystem physics effects on the vehicle are essentialManage interactions between systemsVehicle must manage cooperative vs. competitive subsystem responses such that
subsystems
do not counter each other’s actions leaving the vehicle in a failed
state
Slide63Autonomous Algorithm Integration63Mission Execution and PlanningKeys: Mission ExecutionManages the total execution of the all mission aspects from a vehicle stand pointProper knowledge of the current vehicle states
Progress toward specific mission objectivesMitigates subsystem interaction effects through adjustment to system control parameters in response to specific physical events. Mission Planning Based on
Proper
knowledge of the current vehicle
statesProgress toward specific mission objectivesConducts Re-planning (with crew approval) to ensure future vehicle states will stay within mission objectives and constraintsThree Levels
Strategic: Earth-based
controls will also be involved
Tactical: Crew input and approval
Emergency: Automated to prevent loss of mission, crew, or compromise of crew safety
Slide64Summary64Human exploration outside of the Earth planetary system (beyond Earth orbit) requires autonomous operation of the vehicleCommunication LatenciesCrew size Limits
Vehicle ComplexityA fully autonomous vehicle of this complexity will require multiple autonomous algorithms working cooperatively within a set of mission objectives and system constraintsThe understanding of the physics of the systems, system interactions, and environmental interactions is essential to the system engineering of this complex system
The
Goal-Function Tree methodology provides a system engineering approach to define the vehicle state variables and their interactions.
Algorithms at the vehicle level will need to handle future projected states to enable safe mission execution and planning. Verification and validation approaches will need to be defined for these algorithms, both individually and as an integrated
set
V&V will also need to borrow from Formal Methods (e.g., model checkers)
Applications
looking at autonomous system cooperation will be essential to the development of human rated spacecraft operated away from the Earth planetary
system
Slide65System Design and OptimizationGoal: Apply system design and optimization tools to understand and engineer system interactions
Slide66Multidisciplinary Design OptimizationMartins, J. R R. A., Lambe
, A. B., “Multidisciplinary Design Optimization: A Survey of Architectures”, AIAA Journal, Vol. 51,No. 9, September 2013, pp 2049 – 2075
Slide67Engineering StatisticsGoal: Utilize statistical methods to understand system uncertainties and sensitivitiesSystems Engineering makes use of Frequentist Approaches, Bayesian Approaches, Information Theoretic Approaches as appropriate
Slide68Maximum Likelihood Estimate (MLE)Maximum Likelihood Estimate (MLE) forms a basis to select a model which best fits the available data parametersThis is not a statistical analysis of the dataThis is a statistical estimate of the best model to fit the given data
Where the model is the specific equation of the physical phenomena,
data (x) is the dataset the model is being evaluated for fit,
and
q
is the set of data parameters in the model
The likelihood of the individual data entries fitting the associated parameters is the product of the likelihood functions
and is often evaluated as the log likelihood:
Slide69Maximum Likelihood Estimate (MLE)The MLE is found by maximizing or
First and Second Derivative
Numerical
How do we apply this to the determination of measurements prior collecting the data?
Kullback-Liebler Information
, or
K-L Information is an MLE that measures the distance between f(x) and the approximating model g(
x|
q
), i.e., the likelihood that g(
x,
q
) accurately approximates f(x)
69
Slide70Maximum Likelihood Estimate (MLE)In K-L Information f(x) can be considered to represent the physical reality of the system
Essentially, the expectation of the physical truth is constant across all model comparisons and can be removed from the evaluation yielding
Which is equivalent to the MLE as
Where K is the number of parameters in the model plus 1 for the variance parameter,
s
2
Note, that the measurement data (x) is viewed as an approximation of the actual physical truth
Allows for measurement and communication errors in the data
70
Slide71InformationInformation is viewed as the number of meaningful parametersParameters with sufficient measurements to be reasonable estimatesThere is an optimum fit for a given set of models used to evaluate the same phenomenaToo few parameters limits the informationToo many parameters (with respect to the data measurements made) leads to large uncertainties in the parameter value and therefore limited information is contained by the parameterFisher Information MatrixDefines information as the matrix of partial second derivatives
Information is the amount of parameters with non zero values (so provides an indication of structure)This value converges to a maximum as the number of parameters goes to infinityDoes not contain an optimum, always increases with added parameters
71
Slide72Akaike Information CriteriaAkaike Information Criteria (AIC) uses this relationship to define a measure of model fit as AIC = -2
Where -2 allows a fit for some statistical
c
2
problems
and
is the value of
q
producing the MLE
When the number of parameters is small, a corrected AIC addresses the bias generated by small data sets and larger numbers of parameters
AIC
c = -2
The best fit is determined simply as the model with the lowest
AIC
c
value (magnitude is not telling) by computing the
AIC
c
differences,
D
i
D
i
=
AIC
c,i
–
AIC
c,min
D
min
= 0
72
Slide73Akaike Information CriteriaThe AICc differences, Di , can be used to calculate weight functions to evaluate how substantial the minimum (or best fit model) compares to the other candidate models in the set
,
and
1
Thus,
D
i
and
w
i
provide measurement information indicating how well a model fits with physical truth
This allows identification of the most parsimonious model (model which best fits the data with the fewest parameters)
Avoids over dispersion of data in model causing difficult to identify estimation errors
73
Slide74Akaike Information CriteriaThe AICc weights, wi, can also be used to evaluated the relevance of particular parameters within the evaluated model setThe w
i can be summed for each model in the evaluation set which contains a particular parameters.
The sum of these
w
j
for each parameter indicates the importance of the parameter in the model (equation)
74
Slide75Optimal Sensor InformationConfigurationApplying Akaike Information Criteria (AIC) corrected (AICc) to assess sensor coverage for a systemTwo Views of Information ContentAIC Information
Information is viewed as the number of meaningful parametersParameters with sufficient measurements to be reasonable estimatesFisher Information MatrixDefines information as the matrix of partial second derivativesInformation is the amount of parameters with non zero values (so provides an indication of structure)
This value converges to a maximum as the number of parameters goes to infinity
Does not contain an optimum, always increases with added parameters
AIC/AICc has an adjustment factor to penalize sensor arrangements where: number of sensors < 3x(number of measurements)Provides an optimization tool for use with System Models
75
Slide76Flat Plate FEA Analysis and Akaike Information Criterion (AIC)Slide - 76Results: DOFs kept, Method 1Overlaid on mode shapes
Exergy-Based Information for Systems Analysis, Doty
From Paper:
From my FEA analysis and overlay of Method 1 DOFs:
Red circle: Method 1 DOFs kept
Black circle: DOFS removed due to clamped BC
Slide77Flat Plate FEA Analysis and Akaike Information Criterion (AIC)Slide - 77Numerical results (decomposed)
Exergy-Based Information for Systems Analysis, Doty
corrected Akaike Information Criterion (AICc)
allocation of information
Source
Model 1
Model 2
Model 3
Model 4
Model 5
Model 6
n dofs
32
30
76
36
28
116
K
33
31
77
37
29
117
SS
error
1428.37
1461.59
1428.91
857.73
1762.84
1092.14
Bias
606.56
616.70
606.72
381.65
699.34
488.20
2K
66
62
154
74
58
234
AIC
672.56
678.70
760.72
455.65
757.34
722.20
2nd order correction
5.51
4.85
33.09
6.98
4.23
85.49
AICc
678.07
683.55
793.81
462.63
761.57
807.68
Bias
Bias Correction
Small Sample Correction
Bias
Bias Correction
Small Sample Correction
Slide78Flat Plate FEA Analysis and Akaike Information Criterion (AIC)
Slide - 78
Results
Sources of AICc:
Bias:
mean square error (MSE)
Total Corrections: “penalties” for over-fitting
Bias
Bias Correction
Small Sample Correction
Total Corrections
MWEI ‘Best’ is worst bias and
Slide79Flat Plate FEA Analysis and Akaike Information Criterion (AIC)Slide - 79
Slide80Optimal Sensor InformationConfigurationApplying Akaike Information Criteria (AIC) corrected (AICc) to assess sensor coverage for a systemTwo Views of Information ContentAIC Information
Information is viewed as the number of meaningful parametersParameters with sufficient measurements to be reasonable estimatesFisher Information MatrixDefines information as the matrix of partial second derivativesInformation is the amount of parameters with non zero values (so provides an indication of structure)
This value converges to a maximum as the number of parameters goes to infinity
Does not contain an optimum, always increases with added parameters
AIC/AICc has an adjustment factor to penalize sensor arrangements where: number of sensors < 3x(number of measurements)Provides an optimization tool for use with System Models
80
Slide81Flat Plate FEA Analysis and Akaike Information Criterion (AIC)
Slide - 81
Results
Sources of AICc:
Bias:
mean square error (MSE)
Total Corrections: “penalties” for over-fitting
Bias
Bias Correction
Small Sample Correction
Total Corrections
MWEI ‘Best’ is worst bias and
Slide82Verification ProcessMethod 1: ‘Intelligent’ GuessSlide - 82Final Solution:
Exergy-Based Information for Systems Analysis, Doty
Overlaid on Peaks
Projected to XY plane
Note:
2 initial guesses ‘removed’ (red)
NEW points added (blue)
MOST
initial guesses ‘survive’ (
green)
Slide83Sensor LocationSensor Placement is determined by locations of highest residual errorIndicates lowest level of information about the systemSystem model allows determination of highest residual error locationMust properly model physics of the system to be measured and associated interactionsPlacing the first sensor here changes the information available and biases all other locationsProvides keystone for locating sensors appropriatelyProvides an objective method to determine proper sensor measurement locations
83
Slide84Methods of System IntegrationGoal: System Design and Analysis
Slide85System Models Contain an Understanding of the System
Goal FunctionTree (GFT)
Goals
Value Model
System State Transition
Model
System Functions &
State Variables
System Integrated
Physics Model
(System Exergy)
Discipline Physics
Models
System Functions &
State Variables
Engineering
Statistics
State
Variables
Multidisciplinary Design
Optimization (MDO)
MagicDraw
Enterprise (
SysML
)
Matlab
Matlab
StateFlow
Microsoft
Excell
Allow systems engineers to:
Define system functions based on the system state variables
Understand stakeholders expectations on system value (i.e., capabilities)
Integrate discipline engineering models into a system level physics based model (e.g., system exergy)
Design and Analyze system responses and behaviors at the System level
Slide86System Design and Integration
Slide87Methods of Engineering Discipline IntegrationGoal: Understand How Organizational Structures influence Design and Operations Success of Complex Systems
Slide88Sociological Concepts in Systems EngineeringSpecification of Ignorance is important in the advancement of the understanding of the systemConsistent use of Terminology is important for Communication within the OrganizationOpportunity StructuresProvide opportunity to mature ideasTask teams, working groups, communities of practice, etc.Socially Expected Durations will exist about the project
Both Manifest and Latent Social Functions exist in the organizationSocial Role SetsIndividuals have a set of roles for their positionCultural Subsets will formi.e., disciplines can be a subset within the organization
Insider and Outsider attitudes can form
Be Aware of the Self-Fulfilling Prophecy, Social Polarization
Reconsiderations Process (i.e., Reclama Process)Provides ability to manage social ambivalenceMust be able to recognize social beliefs that may be contributing to the disagreementHelps to avoid putting people in to social dysfunction or complete social anomie
Conformity
Innovation
Ritualism
Retreatism
Rebellion
88
Slide89Information FlowInformation Flow through a program/project/activity is defined by Information TheoryOrganizational communication pathsBoard StructureDecision Making follows the First PostulateDecision Process is specific to the decision being madeTracked 3 SLS CRs, with 3 separate task team processes, all had equally rated effectiveness
89
Margin is maintained by the Organization, not in the margin management tables
Biased Information Sharing
Margin Management is focused on Managing the Disciplines (informed by the System Integrating Physics)
SLS Organizational Structure was defined by the LSE as a recommendation to the Chief Engineer and the Program Manager
Slide90Decision Structure Information FlowInformation Theory ModelInformation Theory can be used to understand decision making structures and information flow
Practitioner’s Guidance
Understand and define the scope of each needed decision body
Ensure that each decision body has all affected or contributing disciplines represented, including understanding of the types and magnitudes of uncertainties affecting decisions within that decision body’s
scope, but no more
Minimize
the number of decision bodies based on scope. The efficiency of the structure decreases with distributed and overlapping scopes.
90
Slide91SLS Organizational Structure ModelingInterviewed 12 Marshall engineers/designers (w/J. Shelton)Understand strategies used to integrate subsystems with each otherCommon strategy across subsystems – marginsKeep some percentage of a parameter in “back pocket” as hedge for future negotiationsBiased Information Sharing(Here, “margins” different from “safety margin”)How does maintaining a margin affect optimality of the final design?
Model as simple 2 Player System with 3 design parameters15 problem test suite
91
Slide92Simulation Results No margin :
Slide -
92
Static margin, m= 1.3
Descending margin
, 𝑚=1.3−.1∗𝑖 until 𝑚=1
No margin condition reaches optimality quickest
Descending margin still reaches optimal, but requires more iterations
Margins
are an
issue
Interviews
highlight real-world
consequences
Simulations
quantify extent of the
problem
Still possible to achieve optimal design with descending margin, but takes additional time to achieve
Slide93Discipline Integration Models
Goal Function Tree (GFT)
Organizational
Structure &
Mapping
System Functions
MagicDraw
Enterprise (
SysML
)
Matlab
Matlab
StateFlow
JAVA
Anylogic
Extend
Allow systems engineers to:
Understand information flow through the development and/or operations organization
Integrate discipline information into a system level design
Analyze information flow, gaps, and blind spots at the System level
Agent Based Model (ABM)
System Dynamics Model
Goals
Value Model
Value
Attributes
Discrete Event Simulation
Organizational
Values
Slide94What is System Dynamics?
A methodology that has been used extensively in business and engineering environments to
develop better understanding of the interactions that define the operation and behavior of complex systems
Based on the concept that
complex systems can be sub-divided into simpler, more manageable components
The
high-level operation of an overall system can be defined by examining and defining the behavior and interactions of the individual components.
“While it is often difficult for scientists to intuitively understand the overall operations of complex systems there is often a great deal of in-depth knowledge on the operations of the structural components that constitute the system.”
Slide95Tools and Methodologies
Tools and techniques have been developed using the System Dynamics methodology that make it possible to efficiently decompose complex systems and to quickly set-up and test models of system operation.
Tools promote understanding through visual diagramming and modeling.
Slide96Roles of System Dynamics
Creates a
Structured View
of the Problem Space.
Allows
Visualization and Testing
of Assumptions and Postulated Relationships.
Dynamically Traces
Potential Results of Modifications.
Communicates
Operational Understanding to Others.
Slide97System Dynamics is frequently used to support long-term strategic decision-making
Modeling of strategic decision space often requires at least some tactical level modeling
Art is in decomposing model to a level that captures driving relationships but is not too complex to accurately or efficiently model
Stochasticity is often an important driver in system behavior
Typical strategic models are run probabilistically, with major drivers represented as prob. distributions
Strategic Analysis
Slide98Methodology has been applied across many disciplines which involve complex systems with limited high-level operational understanding:
- Business and monetary systems
- Power generation and distribution
- Information technology networks
- Aircraft /spacecraft design and operation
- Nuclear reactor operation
- Military
operations
- NASA STS/ISS Transportation/Operations Analysis
Develop
a methodology to evaluate strategic level options for the re-supply and operation of the ISS-STS
system
Evaluate various potential transportation
systems
Analyze
impact (financial and performance) on Exploration System
Applications
Slide99STS-ISS Transportation / Operation Analysis
Slide100ISS-STS Transportation / Operations Analysis
Slide101Cognitive ScienceResearch Goal: Identify some of the key cognitive and organizational challenges in engineering complex systems and the implications to Systems Engineering University of Michigan, Design ScienceTopic: Cognitive Science Perspective of Systems ThinkingMapping Engineering Terminology to Cognitive Science Terminology to provide a scientific basis for the engineering cognitive concepts
Investigating Mediated Learning as a method to teach system thinking101
Cognitive Competencies from Frank,
2012
Related Concepts from Cognitive
Psychology
Understand the whole system and see the big picture
Sensemaking
; information integration; mental model formation; generalization
Understand interconnections
Induction; classification; similarity; information integration
Understand system synergy
Deductive inference
Understand the system from multiple perspectives
Perspective taking (direct mapping)
Think creatively
Creativity (direct mapping)
Understand systems without getting stuck on details
Abstraction; subsumption
Understand the implications of proposed change
Hypothetical thinking
Understand a new system/concept immediately upon presentation
Categorization; conceptual learning; inductive learning/inference
Understand analogies and parallelism between systems
Analogical thinking (direct mapping)
Understand limits to growth
Information integration
Ask good (the right) questions
Critical thinking
(Are) innovators, originators, promoters, initiators, curious
Inquisitive thinking
Are able to define boundaries
Functional decomposition
Are able to take into consideration non-engineering factors
Conceptual combination
Are able to “see” the future
Prospection
Are able to optimize
Logical decision-making
Slide102Space Policy andSystems EngineeringImpact of Government Oversight Time Allocation StudyMotivation: Industry and government leaders agree that government oversight leads to cost growth, but there is less agreement on how much and through what mechanisms.Research Plan:Build an empirical basis for measuring the extent and nature of the impact of oversightNon-invasive “Time Allocation Study:” Statistically valid aggregated observations of how engineers actually spend their time throughout a product’s life cycle.
Part One: Collect time-recall diaries to develop a composite list of activities performedPart Two: Survey Population over several months at random times per day to accurately observe amount of time spent on activitiesData collection is complete and analysis is in processMost non-value added oversight is internal company drivenGovernment generated insight/oversight is a small % of work done (< 10%)
Corporate Communication and Administrative work drive non-value added work from viewpoint of practicing systems engineers within the company
102
Slide103Percentage of total time spent on each oversight category103Brainard
, S. M., Zsajnfarber, Z., “Understanding the burden of government oversight on engineering work: adding empirical data to the debate”, submitted to Space Policy
Slide104System Engineering Supporting ActivitiesProcess Application and Execution for the Specific System
Slide105ProcessesWell defined in NASA/SP-2007-6105 Rev1, NASA Systems Engineering HandbookSEMP is essential to capture appropriate application of processes to the specific systemProcess application is specific to the system being developedTailoring is not a special exception, it is the norm
Slide106System Engineering Standards in Practice
106
Slide107UAH SE Consortium - Comparing the Relationship between Systems Engineering Process and Project Success in Commercial and Government Research and Development Efforts, 2012 – 2014.
Processes with
>
3
Correlations ≥ .4
Processes with < 3
Correlations ≥ .4
Original Study
Correlations
107
Agriculture
AerospaceDefense and securityTransportation
Communications
Electronics
Energy
Infrastructure
Slide108UAH SE Consortium - Comparing the Relationship between Systems Engineering Process and Project Success in Commercial and Government Research and Development Efforts, 2012 – 2014.
Processes with
>
3
Correlations ≥ .4
Processes with < 3
Correlations ≥ .4
Original Study
Correlations
108
Slide109Engineering the SystemUnderstand the System ApplicationDefine System Architecture and Mission RequirementsDesign the SystemModel the System
Understand the System Physics, Environments, and InteractionsUnderstand System Sensitivities and Uncertainties
Integrate Discipline Designs
Maintain Technical Solution within Cost and Schedule
Test the SystemDevelopment
Verification
Validation
Support System Fabrication
Support System Operations
System Maintenance Responses
System Obsolescence Upgrades
System Capability Upgrades for New Applications
System Decommissioning Planning and Execution
Manage Flow of Information Through OrganizationConfiguration ManagementTechnical Data ManagementRecommend Board Structure based on Information Flow
Advocate Opportunity Structures
Support
reclama
path
Manage System Engineering Processes as Defined in SEMP
Understand
Policy and Law Impacts on Technical Solution
Mission Context
Technical
Integration
(Physics Basis
Focus)
Organizational
Structure &
Information Flow
Policy
& Law
109
Slide110SummaryDiscussed approach to Engineering an Elegant SystemDiscussed Systems Engineering Framework and PrinciplesSystem IntegrationEngineering Discipline IntegrationDiscussed several methods and tools for conducting integrated system design and analysisSystem IntegrationSystem Integrating Physics
System Design and OptimizationEngineering StatisticsState Variable AnalysisSystem ValueDiscipline IntegrationSociological Principles and Cognitive ScienceDecision Making
Policy and Law Application
Processes Application
Systems Engineering Approach defined in two documents“Engineering Elegant Systems: Theory of Systems Engineering”“Engineering Elegant Systems: The Practice of Systems Engineering”
110
Slide111Backup111
Slide112System Engineering ProcessesStakeholder ExpectationsTechnical Requirements DefinitionLogical DecompositionDesign Solution DefinitionProduct Implementation
Product IntegrationProduct VerificationProduct ValidationProduct Transition
Product Operation and Sustainment
Technical Planning
Technical Risk Management
Technical
Assessment
Decision
Analysis
Configuration
Management
Technical Data Management
Requirements ManagementInterface Management
Mission Context
Technical Integration
(Physics Basis Focus)
Organizational
Structure &
Information Flow
Policy
& Law
Focus on the intent of the processes not the processes themselves
112