The Timbre ModelUniversitetsparken 1, 2100 Copenhagen - PDF document

The Timbre ModelUniversitetsparken 1, 2100 Copenhagen ¯, Denmark timbre. In extension to the timbre model incorporating expressions isis divided into four parts: an overview of the on theperception of timbre, an overview of the signal processing aspects dealing with sinusoidalcaused by gestures. Expressionsor seems to be a multi-is timbre consists of the spectral envelope, anwhich can be attack,additive a goodThis paper presents an modeling the additive an easilytimbre model models each partial in a fewparameters. Furthermore, the rate andextend of the vibrato and tremolo are modeled.The timbre model has a fixed the the model have an intuitiveFurthermore, by interpolating obtained from different sounds,to resynthesize theall of the themodel is detailed. Next a novel set ofexpression additions to the timbre model ispresented, other applications to the model The timbre model is derived from conclusionsthe auditory perception methodologies have been used in thisresults are given inthe which thetimbre model is based. For a larger overview ofof to auditory perception [102] and psycho-attributes presented here can be found in [48].The mpeg defined the popularmp3 compression standard, are currently definingin finding common terms formusic noise and harmonic sounds, 2.1 Timbre Definitionsounds with the same pitch,thedifficulty of timbre identity research is often increasedmany timbre morea high pitched piano sound are closer to theto alow-pitched piano humanperception or cognition generally identifies theinstrument correctly. Unfortunately, not muchTimbre is best defined in the human community[98] had subjects rate speech, musicalpair dull-sharp, the second was determined by the tone/noisethe timbre of different musicalsubjects to judge thethe A[33] found the timbreand synchronicity was that the attack contained all theonly or thestudies Krimphoff et refined theanalysis, and found the timbrebrightness, attack the& ratingsthe centroid of the spectral envelope, dimension of the timbre. Intime) to correlate with the second dimension of thethe third dimension of the timbre.McAdams far do notattribute, but it was later discarded inno noisy sounds were included in the testthe mostcommon analysis methods doesnÕt permit thethen cannot be interesting way of examining the qualities ofto is theauditory stream segregation [12].theto thesame source) sounds with components falling insounds are several groups, assuccessive tones in a rapid percept may be ambiguous.Experimenters have taken auditorytimbre perception. Using a three tone by means of adjusting the middle tone they tried toinvestigate how the isthe where for the bitimbral case there were changes in thetime and the number of harmonics in thethan for theof partials than for the envelope changes.the modification. One recent such study isMcAdams et al. [64], in which the additivethe modifications (311.1 Hz), 2 secs and equal subjectiveis strong. instrument isparticipants The results from the simplification are that to the constituent (Mostsalient feature dominate in combined simplifications).flux (amplitude good discrimination), andamplitude micro-variations poordiscriminations).In conclusion, several have beenused to determine the dimensions of timbre.envelope,known analysis/synthesisanalysis/synthesis systemsthe loris program [25] and theadditive program [81].choice of underlying model is the additive(sinusoidal) model, for its well-understood parameterswavelet analysis/synthesis [55], the atomicnon-homogenous.Other non-homogenous additions include thetransients [97].3.2 Additive ModelThe additive analysis consists in associating atime-varying amplitudes (ak(t)) and (fkNwith a high s(t)=akk=1Nå(t)× fk( )d ) =0tò .(1)In order to have a good resynthesis Several methods exist for determining the time-the partials.for the time-varying analysis of theanalysis [66], and the linear and Qian [19] has presented an interestingthe residual, improved and dubbed adaptivemany objective (outside thecommunity to evaluate analysis/synthesisevaluated the spectral/time with and[84] evaluated the PCA-based data reduction with Recently, however, the additive analysis/synthesisseveral well-controlled listeningtest experiments. The hidden the linear systems and longermusical sequences. In addition [3] found thatmean error in the amplitude and TheThe timbre model is inspired by the sounds using the analysis by synthesismethod [77] and by literature studies into these model has beenbrightness and the aassociated with pitch andinharmoncity. It also consists of an amplitudestart, attack,startthe amplitude and (shimmer and jitter). The shimmer and jitter deviation and bandwidth. The amplitudeenvelope is associated with important timbreattributes, such as attack the irregularity isalso slowthe Group Additive [22],a classification of attack base of the timbre model is the additivespectral envelope, amplitude envelope andby the mean the jitterof a clean envelope irregularity (shimmer)the spectral envelope value, andof a static value andcorrelation, standarddeviation and bandwidth. Spectral Envelope amp amp Clean Envelope Shimmer filter Static Frequency Jitter filter freq Clean Envelope Shimmer filter Static Frequency Jitter filter freq Envelope Parameters Frequency Parameters Irregularity Parameters Common shimmer Individual shimmer + Individual shimmer Common jitter Individual jitter + + + + + + Individual jitter + Std & BW Corr, Mean freqs (asr), t, a, cf. Max amps. Figure 1. Timbre model diagram. The model consists of a number of sinusoidals, with amplitude as a sum of the 4.1 The Spectral Envelopespectral envelope is very important for thethe sound; indeed, the alone is often enough to is especially true for thesound with realism.[88]. Back in 1966 [94] synthesized windinstruments with a combination of spectral andwith different source theadditive model.function [52]. There exists an easycalculation and brightness with theseformulas [52].‰kweightedof ö f k .Most sustained instruments are supposed to be f k=kö f 0 visualized divided by the harmonicpiano partial 4.3 Amplitude EnvelopesThe envelope of each partial is modeled in fiveallthe amplitude (the spectral envelope), and theor logarithmic form. The k(t) the very smoothed time- (t)=ak(tg (tg (t)=12 e-t22 2 . (2)finding the maximum and the timederivative of the smoothed envelope, maxmin Lt, (t),Lt, (t)= t env (t) (3) Lt, is close to zero (about one tenth of thethe the clean have theenvelopes. The same holds true for theirregularities are divided intomodeled for the attack, sustain and It is supposed to have a Gaussianamplitude of the noise is thenof the noise is assumed low-pass andinclude themusic analysis, [24] introduced the bandwidththemusic sounds include the analysis of aperiodicityclean amplitudes and the mean shimmerk=ak(t)- k(t)ö a k (4) jitterk=fk- ö f kö f k (5)The jitter and shimmer is then assumed to be(and laterthe timbreThis periodicity is modeled by There are individual vibrato and tremolovibrato and tremolo found bytheeach partial) or amplitude(subtracted by the clean amplitude curve). Thisuses a for the vibratoestimation. is and it has hightimbre model now beenpresented, and an overview of the methods has been given. A morethe timbre attributes. Many of them are bestplotted logarithmically, and all the attributes with fixed axes, to facilitate comparisonsbetween sounds.There are 12 different timbre attributes, some ofto the spectral envelope, theand fundamental, and envelope andenvelope percents, the envelope rightshimmer and jitter correlation and thetremolo/vibrato strength.5 theTimbre Modelincluding the analysis of the basis for the inclusion of a number ofcontext, and adapted for real-time synthesis [60],where possible.variants, andtremolo, and other expressions, such aslegato/staccato.order to[31] gives a lot ofinformation, which can also be gathered from researchmusical associated with the expression[100], [101] [43]. The vibratoproblem [18] is an interesting or stretch(glissando) a time-scaled continuous expression5.1 Variants common in all the expression styles of thethe expression is theintroduced bythe performer. The expression can be seen as thenotthebyof variants, which is assumed to be theequivalent of the sounds coming from The variants are introduced an idealcurve is then assumed to befor of the deviation issound a clearly altered timbre.Some of the timbre attributes have ideal ak=a0BB-1 è ç ö ø ÷ -k (6)where akka0B is the estimated fk=kf01+ k2 (7)where bhowever, are fitted with asimple exponential curve, ck=v0×ev1k (8)where v0 is the fundamental value and v1 is themodel bothalmost linear curves with small v1, but alsoexponential behaviors. The the curves are found byusing the Marquardt algorithm [71], except for the B=(kakk=1Nåakk=1Nå The deviations from the ideal timbre attributes dk=ck- c k where ck c k the ideal parameters and dkdkof theparameters, there can be by bad between attributes, or in the original timbre attribute different. One way of remove some of the discrepancies. the variants influence is scaled, k=ö c k+v× ö d k+(1-v)×dk (11)vtimbre attribute deviations ( ö d k ) for additionally becreating new, or altered, timbre modelrules that encompasses all possible musicalinstead, a simple interpolationof at least the important values.rather has thedouble handling the timbre attribute correctly, and assuring continuousvariations in the resulting the pitch iseffects, caused by for instanceAn have shown that thispresent in many musical instruments,clarinet) to 10 (flute), and 15 dB/octavedB/octave) and included into the expressive model.velocity, is another importantbrighter. [10] showed that the change of brightness in the pianowhen changing the velocity of the hammer isthe Hz/dB domain. val=(vM-vm×e- ×v) ,(12)where valvMvm vvM, vm and can be determined from theenough velocity executions a useful expressionparameter, since it defines how bright the soundbecomes, when increasing the velocity to awith thethat the change of velocity (of the pianoduration is of course also a very importantexpression parameter. Since the timbre modelnecessary. This strategy could be found by k(t)=a0+(aT-a0)ct/T-1c-1 ,(13) where c is the curve form coefficient, T is thesegment duration, and a0aTonly part of the sustain/decay segment is used, ‰(t)=‰0dB+bt ,(14)where ‰0bknownsplit-bset to 100%, then nodecay in durational contrasts. This is aninteresting not beenThe vibrato and tremolo are important expressionspecific values defined by thetremolo is not desirable in Somebut inherent in the particular, the case for the amplitudethe rate (speed), the strength and theidentical toadding vibrato [68]. Inin bethe it containing periodicity is assumed to be part of theidentity of the not controlled by theis found, it isother expression in classicalmusic include mainly styles (legato/staccato, foreasily integrated into the timbre available to perform muchtimbre attribute changes have been found whenanalyzing executions of different styles [52].a matter of duration, which is easilytime-varying amplitude,difficult to attribute sets withexpression is the various mappingstrategies [51], [101].is an important topic today, forabout the importance of the timbrereal-time learning classification scheme wasand sound manipulation is anotherhas aeasy to morph between the sounds by simplyThe interpolatedtimbre model also be used to manipulate the additive A similarin [95]. [83] interpolates the additiveand [88] uses the to modify morphingsignal high fidelity.by consists of a five-segment amplitudeenvelope model with individual use of the model in a musical context. Theand an expression part. Thedeviationfrom the identity part of the a newinstance at each new execution. interesting novelty is added to each new soundif with the same pitch,loudness and duration. Allen, term spectral analysis,synthesis and modification by discrete fouriertransform. IEEE Trans. on Acoustics, Speech3, American Standard Association. Acoustical sounds based on psycho acousticWorkshop on current [4] Time Time ScaleRepresentations by the Reassignment Method,IEEE Transactions on Signal Processing, vol. . Barrire, J-P (editor). [8] Beauchamp, Unix for synthesis of musical sounds, 94th AES3479, 1993.[9] Benade, A. H. Fundamentals of musicalacoustics. Dover publications inc. New York.1990.[10] Ystad. ,of the 2000.[11] Jensen. AdditiveAnalysis/Synthesis Using Analytically , Proceedings of the DAFX,Trondheim, Norway, 1999.[12] Scene Analysis, The time Music and Audio Applications, Doctoraldissertation, Computer Science, University of [15] algorithms. Eng. Soc. tones, Soc. Depalle HŽlie. Extraction of spectralpeak parameters using a short time Fourier.to Audio and Acoustics, Mohonk, times. Computer a combined quadratic polynomial-phase,Eng. Soc. Vol. 45, No. 7/8, [20] of frequency on higher orderand classification. Proc of the Int. aperiodicity. Proc of the Int. Eaglestone, Analytic . Proc. of the ICMC,1990.[23] TransientPreservation Under anAdditive Sound Model, Proceedings of theInternational Computer Music Haken. Project: net/projects/loris � , Springer-Verlag. 1990.[28] . Soc. Am. Vol. 41, No. 4,1967.[29] Friberg, A. Generative rules for musicperformance: A formal description of a rulesystem MacMillan. Realtime Gabrielson, A. The performance of music, in Thepp.501-602, AP press, San Diego, USA, 2nd Gordon, The perceptual attack time of[33] of synthesized musical instrument tones, J.Acoust. Soc. 62, No. 2, August1977.[35] modification on musical timbres. J. [37] et modŽlisation delinŽaires. UniversitŽ dÕAix-Marseille II, 1994.[38] issues intimbre research, in (editor), pp. 253-306, AP press, San Harris, the use of harmonic analysis discrete Fouriertransform[40] Helmholtz, H. On the Sensations of Tone, parameterization in the spectral modelingsynthesis framework, Proc. DAFx, Segmentation for MusicContent Description: a ofInstrument ,Proceedings of International Design in Music Working Group, http://www.notam.uio.no/icma/interactive Iverson, Krumhansl. Isolating the Jaffe, D. of thealgorithm.[46] JŠrvŠinen, VŠlimŠki, synthesis. ICMC Proceedings, [48] Perception,Papers from the 1st Jensen, Binary TreeMusical Instruments, Jensen, K. Pitch Independent Prototyping ofMusical Sounds, Proceedings of the IEEE [51] ,of the NAM. 1996.[52] , Kleczowski, Additive Synthesis. Krimphoff, acoustiques et quantificationJournal de Physique IV, Kronland-Martinet, R. and processing of music sounds. Computer Music Journal,12(4), 1988.[56] Lindeberg, T. CVAP Report, Karaoke Applications, Proceedings ofInternational Computer Music Conference 2000.[58] , Proceedings of the 2nd COST G-6[59] Marchand, Spectral AnalysisSignal In Proc. DAFX98 DigitalAudio Effects Workshop, pages 114--118,Barcelona, November 1998.[60] Marentakis, Jensen. Timbre Engine:[61] coding of speech at , Speech Mart’nez M. of the (version telecomitalialab.com/standards/mpeg-7/mpeg- 7.htm � E. David.Soc. Am. Vol. 33, No. 2, [64] of musical instruments soundsresynthesized parameters Perceptual scaling ofsynthesized musical timbres: Commondimensions, specificities, and latent subjectclasses. Psychological Research, 58, pp. 177- McAulay, based on a sinusoidalrepresentation, IEEE Trans. on Acoustics, McIntyre, Schumacher, J.Woodhouse. Aperiodicity in bowed-stringmotion H. Wakefield. A modelof the Proc M¿ller, A. Akustisk . MasterThesis, Computer Science Department,University of Copenhagen, 1996.[70] audiospectra by means of discrete summationformulas[71] Implementation and theory. in[72] Pielemeier, W. H. Wakefield, of musicalsignals. Proc. of the IEEE, Vol. 84, No. 9.September 1996.[73] Rayleigh, theory of sound.reprinted in 1945 by Dover [74] small impairments inaudio systems, including multichannel soundsystems[75] modification of the speech aperiodiccomponent noises synthesis using random formantwaveforms. Acousticthe royal Academy of analysis by synthesis:Representation, variants formusical composition. in Representations ofmusical de ,of the 1999.[79] Rocchesso, . and Audio Processing[80] G. and synthesis methods based onspectral envelopes and . European Speech The additive analysis-synthesispackage, � et indexation des Instrumental Gestural MappingStrategies as Expressivity Determinants inComputer Music Kansei- TheGenova - , http://sparky.parmly.luc.edu/sandell/sharc/timbr edef.html L. Martens. Perceptualevaluation of of musical timbres. Soc. objets musicaux, of aperiodicity in nearly periodic signals. J. Schwarz, Representation for SoundAnalysis-Synthesis, Proc. 351-354, Sekey, Hanson. Improved 1-bark based on adeterministic plus stochastic decomposition,[91] timbre the perceptualJournal Slaney, 1996.Automatic audio morphing, Proc. IEEE Int. of Synthesis of wind- Tellman Haken, Timbrenumbers offeatures9,September 1995.[96] Truax, inner real-timegranulation technique Synthesis: A flexibleanalysis/synthesis tool for transient signals, von Bismarck, G. Timbre of steady sounds. quality consistent analysis-synthesis in (eds). Trends inGestural Music. Ircam - Centre Wanderley, M. Interaction musician-instrument:synthesesonorel'universitŽ Paris Warren, Perception: A newanalysis and synthesis. Cambridge university Weinreich, G. Coupled piano strings, J.[104] and models, 2nd Germany, 1999.