/
Song-level Multi-pitch Tracking by Heavily Constrained Clustering Song-level Multi-pitch Tracking by Heavily Constrained Clustering

Song-level Multi-pitch Tracking by Heavily Constrained Clustering - PowerPoint Presentation

mrsimon
mrsimon . @mrsimon
Follow
343 views
Uploaded On 2020-06-23

Song-level Multi-pitch Tracking by Heavily Constrained Clustering - PPT Presentation

Zhiyao Duan Jinyu Han and Bryan Pardo EECS Dept Northwestern Univ Interactive Audio Lab httpmusiccsnorthwesternedu For presentation in ICASSP 2010 Dallas Texas USA Multipitch Estimation amp Tracking Task ID: 784390

pitch northwestern audio music northwestern pitch music audio http lab interactive university clustering frequency time constraints objective constrained estimates

Share:

Link:

Embed:

Download Presentation from below link

Download The PPT/PDF document "Song-level Multi-pitch Tracking by Heavi..." is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.


Presentation Transcript

Slide1

Song-level Multi-pitch Tracking by Heavily Constrained Clustering

Zhiyao Duan, Jinyu Han and Bryan PardoEECS Dept., Northwestern Univ.Interactive Audio Lab, http://music.cs.northwestern.eduFor presentation in ICASSP 2010, Dallas, Texas, USA.

Slide2

Multi-pitch Estimation & Tracking Task

Given polyphonic music played by several monophonic harmonic instruments (Num known) Estimate a pitch trajectory for each instrumentNorthwestern University, Interactive Audio Lab. http://music.cs.northwestern.edu2

Slide3

Potential ApplicationsAutomatic music transcription

Harmonic source separationOther applicationsMelody-based music searchChord recognitionSource localizationMusic education……Northwestern University, Interactive Audio Lab. http://music.cs.northwestern.edu3

Slide4

The 2-stage Standard Approach

Stage 1: Multi-pitch Estimation (MPE): estimate pitches in each single time frameZ. Duan, B. Pardo and C. Zhang. , “Multiple Fundamental Frequency Estimation by Modeling Spectral Peaks and Non-peak Regions”, IEEE Trans. Audio Speech Language Process., in press.Stage 2: Multi-pitch Tracking (MPT): connect pitch estimates across frames into pitch trajectories4

Time

Frequency

Slide5

State of the Art of MPTWhat existing MPT methods do

Form short pitch trajectories within a note, (note-level) according to local time-frequency proximity of pitch estimatesOur contributionForm long pitch trajectories through multiple notes (song-level) using a new constrained clustering algorithmNorthwestern University, Interactive Audio Lab. http://music.cs.northwestern.edu5

Slide6

Try Clustering by TimbreEach trajectory is a cluster of pitch estimates

One cluster per instrumentClustering principle: maintain timbre consistency in each clusterNorthwestern University, Interactive Audio Lab. http://music.cs.northwestern.edu?

Slide7

Timbre Feature of Pitch EstimatesHarmonic structure:

relative amplitudes of first 50 harmonics

Time

Frequency

Slide8

Minimize This Objective FunctionNorthwestern University, Interactive Audio Lab. http://music.cs.northwestern.edu

A partitioninto K clustersThe 50-d harmonicstructure of i-thpitch estimateNumber ofClustersCenter of k-th clusterFor all pitch estimates in k-th cluster

Slide9

Objective Function Is Not EnoughNorthwestern University, Interactive Audio Lab. http://music.cs.northwestern.edu

Slide10

Add Pitch-locality ConstraintsMust-link

: pitch estimates close in both time and frequency should be in the same clusterCannot-link: simultaneous pitches should not be in the same cluster (only for monophonic instruments)10

Time

Frequency

Slide11

Properties of Our ProblemObjective: timbre consistency

Constraints: pitch localityPrevious constrained clustering algorithms do not apply due to the following properties:Inconsistent constraints: pitch estimates sometimes erroneous may make constraints unsatisfiable Heavily constrained: nearly every pitch estimate is involved in at least one constraintNorthwestern University, Interactive Audio Lab. http://music.cs.northwestern.edu

Slide12

The Proposed Clustering Algorithm

: clustering in n-th iteration; : {all constraints satisfied by } ; 1. Start from an initial clustering , which satisfies , a subset of all constraints; n=1; 2. Find a new clustering that decreases the objective and also satisfies ; 3. = {all constraints satisfied by } ; 4. Repeat 2-4 until the objective (nearly) cannot be decreased; Northwestern University, Interactive Audio Lab. http://music.cs.northwestern.edu

Slide13

Initial ClusteringTrivial one

: a random partition : constraints satisfied by , may be emptyA more informative one for MPT : label pitches according to pitch order in each frame: highest, second-highest, third.., fourth… : will contain all cannot-links

Time

Frequency

Time

Frequency

Northwestern University, Interactive Audio Lab. http://music.cs.northwestern.edu

Slide14

1. Satisfy current constraints2. Decrease the objective function

: satisfied cannot-link : unsatisfied cannot-link : satisfied must-link : unsatisfied cannot-linkSwap set: A connected subgraph between two clusters. Traverse all swap sets until finding a new clustering that decreases the objective function4237

8

3

1

5

6

Find A New Clustering

Northwestern University, Interactive Audio Lab. http://music.cs.northwestern.edu

4

2

3

7

8

3

1

5

6

4

2

3

7

8

3

1

5

6

Slide15

Algorithm ReviewNorthwestern University, Interactive Audio Lab. http://music.cs.northwestern.edu

: partition of points into clusters : feasible solution space under constraints

Slide16

Experiments

Data set10 J.S. Bach chorales (quartets, played by violin, clarinet, saxophone and bassoon)Each instrument is recorded individually, then mixedGround-truth pitch trajectoriesUse YIN on monophonic tracks before mixingInput pitch estimatesOur previous work in [1]Input accuracy: 70.0+-3.1% [1] Zhiyao Duan, Bryan Pardo and Changshui Zhang, “Multiple Fundamental Frequency Estimation by Modeling Spectral Peaks and Non-peak Regions”, IEEE Trans. Audio Speech Language Process., in press.16Northwestern University, Interactive Audio Lab. http://music.cs.northwestern.edu

Slide17

Overall Multi-pitch Tracking Results

Northwestern University, Interactive Audio Lab. http://music.cs.northwestern.eduMean % of correct pitch estimates

Slide18

Among Correctly Estimated Pitches

Northwestern University, Interactive Audio Lab. http://music.cs.northwestern.edu

Slide19

An Example

Northwestern University, Interactive Audio Lab. http://music.cs.northwestern.edu

Slide20

An Example

Northwestern University, Interactive Audio Lab. http://music.cs.northwestern.edu

Slide21

ConclusionFormulate the song-level Multi-pitch Tracking problem as a constrained clustering problem

Objective: timbre consistencyConstraints: pitch localityExisting constrained clustering algorithms do not apply due to problem propertiesPropose a new constrained clustering algorithmExperimental results are promisingNorthwestern University, Interactive Audio Lab. http://music.cs.northwestern.edu

Slide22

Thanks you!22

Northwestern University, Interactive Audio Lab. http://music.cs.northwestern.edu