A System Analysis Study - PowerPoint Presentation

Slide1

A System Analysis Study Comparing Reverse Engineered Combinatorial Testing to Expert Judgment

Atlee M. Cunningham, Jr., Jon Hagar, Ryan

J. Holman

Lockheed Martin

Jon.d.hagar@lmco.comSlide2

AgendaIntroduce the trade study space: the F16 and Combinatorial Test (CT) problemDefine the F16 Failure

Present the steps of the CT study

Cover the results

ConclusionsSlide3

Introduction: F16 Ventral Fin Study and Applying Combinatorial Testing

Evaluate the use of Combinatorial Testing (CT) to a real “problem”

Used a historic F-16 problem and data

See if CT could be used in place of or to support an expert

Save time and/or people

Mixed

example, between what was done and what could have been

done

Problem space

Interacting factors (good for CT)

Outside of the software testing

System-Hardware failure resolution and design evaluation

Demonstrate CT is viableSlide4

F16 Failure Case Study During production and maintenance of the F-16 fighter aircraft a structural problem immerged

Buffeting of the F-16 ventral fins has provided a classic example of structural fatigue of such aerodynamic surfaces by an upstream source of severe turbulent wakes

These fins are very thin surfaces, about 5 ft. chord and 2 ft. span, composed of three wedge like surfaces that taper down to edge thicknesses of 0.05 inches, all of which makes the fins susceptible to turbulence buffeting

Examples of possibly interacting turbulence sources include: various centerline stores, side slips and inlet lip spillage during rapid decelerations

The historic work done by Atlee M. Cunningham, Jr. and Ryan J. Holman

The Combinatorial analysis and case study was primarily done Jon Hagar with support from Atlee Cunningham as the “expert”Slide5

Original F-16 Problem Details

Added 2 avionics LANTIRN pods on the F-16 just aft of the inlet on the lower fuselage directly upstream of the ventral fins

Avionics pods in general are often not very aerodynamic in shape and hence can produce very turbulent wakes

The damage to the right hand ventral fin on first flight with LANTIRNS

Originally, the primary source of the fin’s fatigue and loss was high speed throttle chops that produced severe turbulence from inlet lip spillage during rapid decelerations where the throttle was suddenly pushed to idle position

A comparison of the ventral fin response to LANTIRN and throttle chop turbulence was done

The response levels are about the same; however, constant buffeting by the LANTIRNS produced much higher fatigue damage per flight hour as compared to that due to the transient throttle

chop

As a result, several major structural re-designs of the fins and other associated structures followed over the following years that incrementally improved the fatigue life of these componentsSlide6

F16 with LANTIRN Pod and Ventral FinSlide7

Failure: Damaged Ventral Fin

But Why (what parameters and interactions)? Slide8

Original Analysis HistoryAfter a number of years more the problem continued.As a result, a detailed analysis of the flight data was performed by Atlee Cunningham , yielding

Showed that the most severe buffeting of the

ventrals

consistently occurred with only LANTIRN pods on the aircraft and with high speed throttle chops at Mach 0.95 on the clean aircraft

Anomalous trends were also seen in throttle chop data with LANTIRNs where response levels were 3-to-4 times as high as level flight with LANTIRN

Recognizing that the very thin

ventrals

(leading edge thickness is 0.05 in.) would probably be subject to leading edge separation at small angles of side slip

Flow change resulted in a large increase in the slope of side force with side slip angle, which would have a significant impact on dynamic loads due to large amplitude turbulenceSlide9

Do More Testing and Analysis (by experts)A low speed small scale wind tunnel test was conducted to

Explore various aspects of ventral aerodynamics and effects of modifications

Data were obtained with 1/5 scale models of the fin mounted on the wind tunnel wall and rotated for incidence effects

Testing was to determine “sensitivities” (variables and values) but had to be designed by expert

Flight Tests were conducted

Three of these four fins, plus several early block ventral fins, were tested on an early Block 15 F-16. The fins consisted of:

(1) the baseline fin, “BSLN,” the standard Block 40 ventral fin;

(2) the “MMC” fin, the Block 40 fin with 40% stiffer skins of MMC aluminum material

(3) the “MMCNC” fin, the MMC fin with an added rounded “nose cap” glove with a NACA 0012 airfoil section of 5 inch chord

(4) the “NACA” fin, the Block 40 modified to have a full span, full chord airfoil section that eliminated the sharp leading edge and sharp tip section of the fin

An expert had to define test program (combinations) for these too = hundreds of hoursSlide10

Defining conditions for CTWhat was the situation(s) that brought the failure on? Factors considered include:

Aircraft (AC)

Maneuver

Speed (Mach)

Altitude

Aircraft add on structures (tanks, pods, etc.)

Which design solutions (4 fin designs) might solve the problem with different aircraft configurations:

Block 15

Block 40Slide11

Idealized Analysis Steps using CT (Hypothetical Reconstruction)NIST ACTS Combinatorial Tool was used to “reverse” engineer a test program

Other tools were considered

Open source nature was deciding factor

This can be viewed as a “reverse” or “Re” engineering case study

We were trying to see if the tool would replicate the historic test

program without an system expert

Test planning

using a CT tool

(not the expert

)

A series of idealized steps were done using the toolSlide12

CT Step 1Historic “first” test program - clean baseline configuration, which in the example are F16s in block 15 and 40 in “clean” configuration, and apply “testing” to points associated

Input to

tool (equivalence classes):

Tool

produced: 90 test cases (similar to actual effort

) with 2 way

Aircraft

15 , 40

Altitude (s)

5k, 10k, 15k, 20k, 30k, 40k, 50k

Maneuvers

hi-speed throttle , slow

accel

/dwell , L/R 5deg side slip , L/R 360 roll , R/L 360 roll, R/L 5deg side slip, Med

accel

/dwell, R-L-R-L banking, Hi-speed to Low, 360 nose roll

Mach(100

th

)

40, 50, 60, 70, 80, 90, 100, 110, 120

Parameters: Variable: Slide13

Step 2: Refinement of the test programStep 2 a more refined set of analyses would have been done based on information from:Step 1Historic databases

More detailed wind tunnel analyses

Supplemental water tunnel analysis

Flight

data and

constraints

were av

ailable

This effort confirmed design work

Produced 30 test

cases from 2 way coverage

Parameters:

Variable:

Aircraft

15 , 40

Alt

5, 10, 15

Mach(100

th

)

60, 80, 85, 90, 95

LANTIRN

on, offSlide14

Step 3: Final Design Flight Test Program

Parameters: Variable:

AC-BLK&Ventral-Fin-Config

Blk15-Blk15 ventral,

Blk15-Blk40 ventral,

Blk15-MMC ventral,

Blk15- MMC ventral + cap,

Blk40-Blk40 ventral, Blk40-MMC ventral, Blk40-MMC Ventral+ nonosecap,

Blk40-NACA

LANTIRN

[on, off]

Alt

[5, 10, 15]

Mach (100ths)

[60, 70, 80, 85, 90, 95]

Maneuver Block

[basic, basic +]

Tests for a flight test program

Number of test (cases)

generated with 2 way coverage:

72Slide15

ConclusionsAn example where combinatorial test could have aided Provided

another test design method for teams to use

Reduce the "shotgun” approach and expert judgment needed for situations dealing with “many” parameters

Showed CT can support a system failure (fault isolation)

analysis

Historic data useful in a CT

proof on concept

(case study example)

Lockheed Martin will continue advocating CT as a technique

Looking for pilots and more data points

Would be interesting to compare to DOE

approaches

How to get Engineers to start using

Other items noticed

Tool interchange (operability), particularly into a test automation framework

Constraints were “tricky”

Interface to/from Model based testing would be usefulSlide16

SummaryDemonstrated Combinatorial Test tool could have supported the F16 problem (or other hardware, software, system test/analysis)Expert felt results would have been similar

Approach could support other programs

Open source tool

worked

Commercial tools worked too

Supports move from “theory” to real use

Supported an “non” software area

System Design/Failure Evaluation