Computer Audio/ fmod CGDD 4603 - PowerPoint Presentation

Slide1

Computer Audio/fmod

CGDD 4603Slide2

Why fmod?

Sound you might have in a game:

Background music

Sound effects

Dialog

How do you currently use these in your games?

fmod

is

very

popular

Powerful – both 2D and 3D sound, DSP/effects…

Cross platform (PS3, Wii, PC…)

Has a designer, sandbox and APISlide3

Partial List of Games that use fmod

Allods Online

Audition Online

Battlestations

: Pacific

BioShockBioShock2Brutal LegendClive Barker's JerichoCortex CommandCrysisDarkfallDJ HeroBatman: Arkham AsylumDe BlobDragon Age: OriginsDogfighterFar CryForza Motorsport 2Forza Motorsport 3Guild Wars

Guitar Hero IIIGuitar Hero: AerosmithGuitar Hero: World TourHeavenly SwordHeroes of NewerthHellgate: LondonJurassic Park: Operation GenesisJust Cause 2League of LegendsLego UniverseLittleBigPlanet/LBP2Metroid Prime 3Natural Selection 2Need for Speed: ShiftNicktoons Unite!Nicktoons: Across the Second Dimension

Plants

vs

Zombies

Pure

Rise of Flight: The First Great Air War

Second Life

Shatter

Shattered Horizon

[4]

Silent Hill: Shattered Memories

StarCraft II: Wings of Liberty

Stargate

Worlds

Stranglehold

TimeShift

TNA

iMPACT

!

Tom Clancy's Ghost Recon

Tom Clancy's Rainbow Six Vegas 2

Tomb Raider: Underworld

Torchlight

Tropico

3

vSide

World of Tanks

World of

WarCraft

You Don't Know Jack

ZumaSlide4

Things You Should Know…

ADC/DAC – Audio to Digital Conversion

Sampling rate

# of times per second the computer “listens”

44.1 KHz is CD

22 KHz is goodBits per sampleComputers have to approximate4 bits = 16 levels of approximation8 bits = 256 levels of approximationSlide5

Original SoundSlide6

Low Sampling Rate

TIMESlide7

Low Sampling Rate

What the computer hears

TIMESlide8

High Sampling Rate

TIMESlide9

High Sampling Rate

TIMESlide10

2 bits per sample

4 Approximations

TIMESlide11

StairStep Effect

TIMESlide12

3 bits per sample

8 Approximations

TIMESlide13

Less StairStep

TIMESlide14

Capturing Sounds

Usually done with:

a microphone (such as voice)

Line in

CD

Hollywood Edge®Computer has sound cardInput types (RCA, MIDI, mini, ¼”, XLR)Card has quality (plays 16-bit sound)Need some kind of softwareSoundForge/AudacityWindows SoundRecorder (gag)

                                                                         Slide15

3D Sound

Sounds have position and velocity

There is a listener component

Relationship between the two

Attenuation (with distance)

Occlusion (low-pass filter)Doppler (relative velocities)Lots of “psycho-acoustic” optionsSlide16

Back to fmod

There are three primary components:

The Sound Designer

The Sandbox

The API

We’ll use all threeStart with Sound DesignerSlide17

Setting up a Project

Take the advice of the video tutorials

Have a separate folder

Copy sounds into a “sounds” directory

Keeps the safe

Can have sub-directoriesCreate an “output” directorySlide18

Designer Interface

Can delete this if you want, but need to have at

l

east one groupSlide19

Events

Used to define sound properties for an event

Can be comprised

of

One sound

Several sounds randomly chosenMonophonic vs PolyphonicCan randomize pitch and attenuationEvents can beSimpleMulti-trackSlide20

A Simple EventSlide21

Granular Sounds

These sounds aren’t

looping

Allows for a sounds to

occur between time

rangesAllows for polyphonySlide22

Event Options

Can vary attenuation

(

dropoff

with distance)

Can vary pitchCreates a sound considerably smaller than a “soundtrack” Plays forever and is always random!Slide23

Sound Definitions

Required for multi-track eventsSlide24

Multi-track Events

Comprised of sound

defs

in layersSlide25

Multi-track Events

Careful when adding sounds

Choose sound

def

Ambient noise is looping

Other sounds are granular, so choose “Oneshot”Have parametersNot based on time!Slide26

Multi-track Events

Can cross-fade and set fade out timeSlide27

EffectsSlide28

Parameter Properties

Can define range and velocity (to simulate time)Slide29

Engine Designer

fmod

can specifically work with engine sounds

Need for Speed 2

Based on the “load” of the engine

Right-click on a sound instance->propertiesAuto-pitchWindow->fmod Engine DesignerSlide30

The Build Process

Know which platform you’re targeting

Changes are applied only to that platform

Project->Clean, Project->BuildSlide31

Interactive Music

Comprised of several

short

segments of music

Intro music

Darkening or discoveryFighting/intense fightingReleaseBefore you begin, you must knowTempo of music (beats per minute – bpm)Beats per measure (time signature)Slide32

CuesSlide33

Themes and AuditioningSlide34

Transitioning via ParametersSlide35

Other Things

Supports deployment of multiple languages

Can deploy different builds based on

Platform

LanguageSlide36

A Look at the API

Basic API

Designer API

Built on basic API

Can read .

fev filesWritten for C/C++#include <fmod.h> // C#include <fmod.hpp> // C++Read the documentation fmodex.chmSlide37

API Parts

A

System

is the

fmod

engineA Sound is the raw resource 2D uses FMOD_2D 3D uses FMOD_3Dresult = system->createSound(“music.wav", FMOD_2D, 0, &sound);Slide38

API Parts

A

Channel

is an

instance

of a SoundEach time you play a sound, you get a new channelChannels can start out pausedYou can set the Volume (0.0f – 1.0f)Pan (-1.0f – 1.0f) Frequency (in Hz as a float)Always use FMOD_CHANNEL_FREE to pick for youSlide39

#include

<

iostream

>

#include <fmod.hpp>#include

<fmod_errors.h>using namespace std;

using

namespace

FMOD;

void

main

()

{

FMOD_RESULT

result;

System

*

system;

// Create the fmod system. We only need one of these

result =

System_Create(&system);

// Specify a max number of simultaneous channels result

= system->init

(100, FMOD_INIT_NORMAL,

0); Sound

* sound;

// Decompress the entire mp3 into 16-bit PCM in memory

result = system->

createSound("winning.mp3",

FMOD_DEFAULT, 0,

&sound);

if (result

!= FMOD_OK) {

cout

<< "Couldn't open it! "

<<

FMOD_ErrorString

(result)

<<

endl

;

}

Channel

*

channel

;

// Used for setting volume, pan, pausing...

// You have the option of passing that channel as the last parameter

result

=

system->

playSound

(FMOD_CHANNEL_FREE,

sound,

false

,

&channel);

channel->

setFrequency

(44100.0f);

if

(result

!=

FMOD_OK)

{

cout

<<

"

Couldn't play it!

"

<<

FMOD_ErrorString

(result)

<<

endl

;

}

while

(

true

)

{

cout << "1

"; system->update(); // Now required in fmod

}}Slide40

Virtual Voices

Supports “virtual voices” when hardware is limited

Set from

System

::

init()System::getCPUUsage()Try to keep < 1000May need to set the priority of a channel if it’s importantchannel->isVirtual()Slide41

3D Sound in fmod

Sound attenuation

Logarithmic

Set the

mindistance

of a sound channel to start attenuationBee = 0.1fJet = 50.0fLeave max distance alone (default is 10,000)Set 3D settings with System::set3DSettings()DopplerDistance factor (in cm, m, feet)Rolloff scale (attenuation models)Slide42

3D Sound

f

mod

uses a left-handed coordinate system

result=system-

>init(100, (FMOD_MODE)(FMOD_INIT_3D_RIGHTHANDED|FMOD_3D), 0);With velocity, you must pass it as meters per secondvelx = (posx-lastposx) * 1000 /

timedelta;vel = 0.1 * 1000 / 16.67 = 6 meters per secondvel = 0.2 * 1000 / 33.33 = 6 meters per secondSlide43

void

main () {

FMOD_RESULT result;

FMOD_VECTOR

soundPos

, soundVel; FMOD_VECTOR listenerPos, listenerVel, listenerForward, listenerUp; System* system; result = System_Create(&system); result = system->init(100, (FMOD_MODE)(FMOD_3D), 0); int numHC = 0; result = system->getHardwareChannels(&numHC);

cout << "Hardware channels: " << numHC << endl; Sound* sound; result = system->createSound(“train.mp3", FMOD_3D, 0, &sound); if (result != FMOD_OK) { cout << "Couldn't open it! " << FMOD_ErrorString(result) << endl; } Channel* channel; result = system->playSound

(FMOD_CHANNEL_FREE, sound, false, &channel);

channel->

setFrequency

(44100.0f);

if (result != FMOD_OK) {

cout

<< "Couldn't play it! " <<

FMOD_ErrorString

(result) <<

endl

;

}

float

dsp, stream, geometry, update, total; soundPos.y = soundVel.x = soundVel.y

= 0.0f; soundPos.z = -100.0f; soundPos.x = 5.0f; soundVel.z = 6.0f;

channel->set3DMinMaxDistance(10, 10000); listenerPos.x = listenerPos.y = listenerPos.z = 0.0f; listenerVel.x = listenerVel.y

= listenerVel.z = 0.0f; listenerForward.x = listenerForward.y = listenerUp.x = listenerUp.z = 0.0f;

listenerForward.z = listenerUp.y = 1.0f; system->set3DListenerAttributes(0, &listenerPos, &listenerVel, &listenerForward, &listenerUp

); while (true) { system->update(); system->getCPUUsage(&dsp, &stream, &geometry, &update, &total);

cout << total << endl; soundPos.z+=0.01f; channel->set3DAttributes(&soundPos, &

soundVel); }}Slide44

Interfacing with Sound Designer

You should have received

.

fev

file (Designer file – event data)

.fsb file (raw audio data).txt file (describes the events and parameters)May want to start by creating a helper functionvoid checkErrors(FMOD_RESULT result) { if (result != FMOD_OK) { cout

<< "fmod error: " << FMOD_ErrorString(result) << endl; exit(1); }}Slide45

Imagine That This is our ProjectSlide46

#include <

iostream

>

#include <fmod.hpp>

#include <fmod_event.hpp>

#include <fmod_errors.h>using namespace std;using namespace FMOD;void main () { EventSystem* eventSystem = NULL; EventGroup* eventGroup = NULL; Event* myEvent = NULL; EventParameter* eventPar;

FMOD_RESULT result = EventSystem_Create(&eventSystem); checkErrors(result); result = eventSystem->init(64, FMOD_INIT_NORMAL, 0, FMOD_EVENT_INIT_NORMAL); eventSystem->setMediaPath("..//"); result = eventSystem->load("fmodTest.fev", 0, 0);

result

=

eventSystem

->

getGroup

("

fmodTest

/beeps", false, &

eventGroup

);

result

=

eventGroup

->

getEvent

("PossessedComputer", FMOD_EVENT_DEFAULT, &myEvent); result = myEvent->getParameter("proximityToComputer

", &eventPar); myEvent->start();

float dir = 0.0001f; float currentParVal = -1.0f; eventPar->setValue

(0.1f); while (true){ cout << currentParVal << endl;

eventPar->getValue(&currentParVal); currentParVal+=dir

; eventPar->setValue(currentParVal); if ((currentParVal >= 1.0)||(

currentParVal <= 0.0)) { dir = -dir; } }

}Slide47

Interactive MusicSlide48

void main () {

EventSystem

*

eventSystem

= NULL; MusicSystem* musicSystem = NULL; MusicPrompt* introPrompt, *fightPrompt, *fight2Prompt, *releasePrompt

; FMOD_MUSIC_ITERATOR cueIter; FMOD_MUSIC_ITERATOR paramIter; FMOD_MUSIC_ID intensityParID = -1.0f;

FMOD_RESULT result =

EventSystem_Create

(&

eventSystem

);

result =

eventSystem

->

init

(64, FMOD_INIT_NORMAL, 0,

FMOD_EVENT_INIT_NORMAL

);

result =

eventSystem

->load("

acid_test.fev

", 0, 0);

result =

eventSystem->getMusicSystem(&musicSystem

); musicSystem->

setVolume(1.0f); result = musicSystem->

getCues(&cueIter);

cout << "Iter ID: " <<

cueIter.value->id << endl;

result = musicSystem->prepareCue(

cueIter.value->id, &fightPrompt);

result = musicSystem->

getNextCue(&cueIter);

cout << "Iter

ID: " << cueIter.value->id << endl;

result = musicSystem->prepareCue

(cueIter.value->id, &fight2Prompt);

result = musicSystem->getNextCue

(&cueIter); result =

musicSystem->prepareCue(cueIter.value

->id, &introPrompt); result

=

musicSystem

->

getNextCue

(&

cueIter

);

result

=

musicSystem

->

prepareCue

(

cueIter.value

->id, &

releasePrompt

);

musicSystem

-

>

getParameters

(&

paramIter

);

intensityParID

=

paramIter.value

->id;

cout

<< "

paramIter.value

->

name is "

<<

paramIter.value

->name <<

endl

;

musicSystem

->

setParameterValue

(

intensityParID

, 7.5f);

result =

introPrompt

->begin();

while (true){

eventSystem

->update();

}

}Slide49

Other Things You Should Look Into…

The 3D Reverb API

Asynchronously loading data (by default)

Memory

management for non-PC platforms

FMOD::Memory_Initialize() // fix size of fmodUsing compressed samplesBuilt-in DSPSlide50

Configuration Notes

You’ll be working with

Header files

C/C++ -> General->Additional Include Directories

Lib files

Linker->General->Additional Library DirectoriesLinker->General->Additional DependenciesDLLs (which you should put into Windows/System32)There’s also a FMOD.NET site…Slide51

Configuration Notes

You have the option of 64-bit and 32-bit libraries

You might have to create a new platform