Gaussian process emulation - PowerPoint Presentation

Slide1

Gaussian process emulation of multiple outputs

Tony O’Hagan, MUCM, SheffieldSlide2

Outline

Gaussian process emulatorsSimulators and emulatorsGP modelling

Multiple outputs

Covariance functions

Independent emulators

Transformations to

independence

Convolution

Outputs as extra dimension(s)

The multi-output (separable) emulator

The dynamic emulator

Which works best?

An exampleSlide3

Simulators and emulators

A simulator is a model of a real processTypically implemented as a computer codeThink of it as a function taking inputs x and giving outputs y

y = f(x)

An emulator is a statistical representation of the function

Expressing knowledge/beliefs about what the output will be at any given input(s)

Built using prior information and a training set of model runs

The GP emulator expresses f as a GP

Conditional on

hyperparametersSlide4

GP modelling

Mean functionRegression form h(x)T

β

Used to model broad shape of response

Analogous to universal kriging

Covariance function

Stationary

Often use the Gaussian form

σ

2

exp{-(x-x′)

T

D

-2

(x-x′)}

D is diagonal with correlation lengths on diagonal

Hyperparameters

β

,

σ

2

and D

Uninformative priorsSlide5

The emulator

Then the emulator is the posterior distribution of fAfter integrating out

β

and

σ

2

,

we have a t process conditional on D

Mean function made up of fitted regression

h

T

β

*

plus smooth interpolator of residuals

Covariance function conditioned on training data

Reproduces training data exactly

Important to validate

Using a validation sample of additional runs

Check that emulator predicts these runs to within stated accuracy

No more and no less

Bastos

and O’Hagan paper on MUCM websiteSlide6

Multiple outputs

Now y is a vector, f is a vector functionTraining sampleSingle training sample for all outputs

Probably design for one output works for many

Mean function

Modelling essentially as before, h

i

(x)

T

β

i

for output

i

Probably more important now

Covariance function

Much more complex because of correlations between outputs

Ignoring these can lead to poor emulation of derived outputsSlide7

Covariance function

Let fi(x) be

i-th

output

Covariance function

c((

i,x

),

(

j,x

′)

) =

cov

[

f

i

(x),

f

j

(x

′

)]

Must be positive definite

Space of possible functions does not seem to be well explored

Two special cases

Independence: c((

i,x

), (

j,x

′)

) = 0 if

i

≠

j

No correlation between outputs

Separability: c((

i,x

), (

j,x

′

)

) =

σ

ij

c

x

(x, x

′

)

Covariance matrix

Σ

between outputs, correlation

c

x

between inputs

Same correlation function

c

x

for all outputsSlide8

Independence

Strong assumption, but ...If posterior variances are all small, correlations may not matterHow to achieve this?

Good mean functions and/or

Large training sample

May not be possible in practice, but ...

Consider transformation to achieve independence

Only linear transformations considered as far as I’m aware

z(x) = A y(x)

y(x) = B z(x)

c((

i,x

), (

j,x

′)

) is linear mixture of functions for each zSlide9

Transformations to independence

Principal componentsFit and subtract mean functions (using same h) for each yConstruct sample covariance matrix of residuals

Find principal components A (or other

diagonalising

transform)

Transform and fit separate emulators to each z

Dimension reduction

Don’t emulate all z

Treat

unemulated

components as noise

Linear model of

coregionalisation

(LMC)

Fit B (which need not be square) and

hyperparameters

of each z simultaneouslySlide10

Convolution

Instead of transforming outputs for each x separately, consider y(x) = ∫

k(

x,x

*) z(x*)

dx

*

Kernel k

Homogeneous case k(x-x*)

General case can model non-stationary y

But much more complexSlide11

Outputs as extra dimension(s)

Outputs often correspond to points in some spaceTime series outputs

Outputs on a spatial or

spatio

-temporal grid

Add coordinates of the output space as inputs

If output

i

has coordinates t then write

f

i

(x) = f*(

x,t

)

Emulate f* as single output simulator

In principle, places no restriction on covariance function

In practice, for single emulator we use restrictive covariance functions

Almost always assume separability -> separable y

Standard functions like Gaussian correlation may not be sensible in t spaceSlide12

The multi-output emulator

Assume separabilityAllow general Σ

Use same regression basis h(x) for all outputs

Computationally simple

Joint distribution of points on multivariate GP have matrix normal form

Can integrate out

β

and

Σ

analyticallySlide13

The dynamic emulator

Many simulators produce time series output by iteratingOutput y

t

is function of state vector

s

t

at time t

Exogenous forcing inputs

u

t

, fixed inputs (parameters) p

Single time-step simulator f*

s

t+1

= f*(

s

t

, u

t+1

, p)

Emulate f*

Correlation structure in time faithfully modelled

Need to emulate accurately

Not much happening in single time step but need to capture fine detail

Iteration of emulator not straightforward!

State vector may be very high-dimensionalSlide14

Which to use?

Big open question!This workshop will hopefully give us lots of food for thoughtMUCM toolkit v3 scheduled to cover these issues

All methods impose restrictions on covariance function

In practice if not in theory

Which restrictions can we get away with in practice?

Dimension reduction is often important

Outputs on grids can be very high dimensional

Principal components-type transformations

Outputs as extra input(s)

Dynamic emulation

Dynamics often driven by forcingSlide15

Example

Conti and O’Hagan paperOn my website: http://tonyohagan.co.uk/pub.html

Time series output from Sheffield Global Dynamic Vegetation Model (SDGVM)

Dynamic model on monthly

timestep

Large state vector, forced by rainfall, temperature, sunlight

10 inputs

All others, including forcing, fixed

120 outputs

Monthly values of NBP for ten yearsSlide16

Multi-output emulator on left, outputs as input on right

For fixed forcing, both seem to capture dynamics well

Outputs as input performs less well, due to more restrictive/unrealistic time series structureSlide17

Conclusions

Draw your own!