By Rong Yan Alexander G and Rong Jin Mwangi S Kariuki 200811629 Quiz Whats Negative PseudoRelevance feedback in multimedia retrieval Introduction As a result of high demand of content based access to video information ID: 375804
Download Presentation The PPT/PDF document "Pseudo-Relevance Feedback For Multimedia..." is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1
Pseudo-Relevance Feedback For Multimedia Retrieval
By Rong Yan, Alexander G. and Rong Jin
Mwangi
S.
Kariuki
2008-11629Slide2
QuizWhat’s Negative Pseudo-Relevance feedback in multimedia retrieval?Slide3
IntroductionAs a result of high demand of content based access to video information.
Content based implies that searching can be done not only through manually indexed terms-directly evaluate if the video content(image and the audio) is similar to the query.Need to allow users to query and retrieve based on the audio information and the imagery of the video, Content-based video retrieval (CBVR) or Content based multimedia retrievalUsing pattern recognition technique.NPRF retrieves images/items which are not similar to the query or relevant information.Slide4
CBVR rely on pre-defined generic similarity which determines the distance between the two images. Limitations include:-visual features representation limited to capturing fairly low-level physical features(color, texture or shape).
Different query scenarios require different similarity metrics to model the distribution of examples. E.g. Sky and water(sea)Slide5
Standard relevance system iteratively asks the user more training examples as relevant or non-relevant for the learning algorithms.After a interactive relevance feedback, the system must then re-build a new classifier.
The top-ranked example from generic similarity metric doesn’t always make the correct result due to poor performance of the current visual information retrieval in applications.e.g.-cars shape Slide6
The Information Digital Video Library System.
Focuses on information extraction from video.Involves the integration of speech, image and natural language.After retrieving the metadata, the system enables full content search and retrieval of the spoken language and visual documents.Slide7
Informedia interface provides multiple levels of abstractions including:-
Visual Icons with relevance measureShort titles or headlinesTopic identification of storiesFilmstrip(storyboard) viewsTranscriptDynamic mapsActive video skimsFace detections and recognitionImage retrieval Slide8
Relevance and Pseudo-Relevance Feedback in Information Retrieval
Main retrieval techniquePseudo-Relevance Feedback is an automatic retrieval approach without any user intervention.Starting with a small no. of positive examples and no negative examples, then extract the strong negative to train the classifier.Transductive learning and co-training are two of paradigms to utilize the information of unlabeled data.Co-training is used to the multimedia retrieval since redundant information is available from different modalities.Slide9
Pseudo-Relevance Feedback
Define the query-text description plus audio, image or video.Video retrieval algorithm retrieves a set of relevant video shots from given data collections.Taking target(T)and query(Q) the retrieval algorithm should provide permutation of the video shots t(i) in T which is sorted by their similarity to the user queries q(i) in Q.The difference between two video segments is measured through a similarity metric between their feature vectors.Then the video collected are separated into two parts of each query positive examples (T+) and Negative examples T(-).Precision and recall are performance measures for retrieval systems, But we use mean average precision since we want the rank.Precision after every retrieved relevant shot is computed and these precisions are averaged. The average precision of this average precision gives the mean average precision.The main idea in PRF approach is to automatically feed back the data which are identified based on generic similarity metricSlide10
Analysis
We can define the positive distance d+ as the distance between the positive data T+ and the queries.The negative distance d- is defined also.The distance d+ and d- will converge towards a gaussian distribution when the no. of examples goes to infinity.Therefore the probability density function(pdf) p(x) for both distance are in form of,Slide11
Which sometimes is also called the error function
er f(x). Slide12
Statistical Model for average precision
Let p(t) be the probability density of T for the data distribution, p(+)t (positive) and P(-)t(Negative) distributions.Slide13
Probabilistic Output and combination
Fusion, combining the base metric and PRF metric.Reduce the prediction variance and offer more stable results.Linearly normalize the scores to a certain interval e.g.[-1,1] As a result all scores(-,+) are gaussian distributed, then we can obtain the probability by applying bayes rule. Parametric sigmoid model to fit the posterior directly Slide14
Base Similarity Metric
Algorithm used to generate the base retrieval scores.Expressed as follows;Can handle multiple examples in arbitrary metric spaces.Retrieval algorithm is assigned a score for each video frame, while the basic unit is a video shot(Multiple frames)-choose the maximal retrieval score of a frame within a video shot’s retrieval score.Slide15Slide16
Sampling strategy
No. of feedback training examples will be sampled as the input to a learning algorithm(+ e.g.)Subset of the e.g. that are dissimilar to the queries will be considered as (- e.g.).Slide17
Classification Algorithm
SVMs are known to yield good generalization performance compared to other classification algorithms. The decision function is of the form;Slide18
Query Slide19
ResultsSlide20
ConclusionImproved information retrieval, negative pseudo-relevance feedback.
Using learning algorithm for classification-very successful.Multimedia query e.g. provide the (+) training e.g. for machine learning theory(-) training e.g. are obtained from the initial simple Euclidian similarity metric.SVM classifier that learns to weight the discriminating features-improves retrieval performance.NPRF shows the ability to separate Gaussian distribution of the (-) and (+) image reducing the variances.Slide21
Answer 3rd slide.Questions?
Thanks