Back Ground Prepared By Anand Bhosale Supervised Unsupervised Labeled Data Unlabeled Data X1 X2 Class 10 100 Square 2 4 Root X1 X2 10 100 2 4 Distance Distance Distances ID: 1026685
Download Presentation The PPT/PDF document "Classification Using K-Nearest Neighbor" is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
1. Classification Using K-Nearest NeighborBack GroundPrepared By Anand Bhosale
2. Supervised Unsupervised Labeled Data Unlabeled DataX1X2Class10100Square24RootX1X21010024
3. Distance
4. Distance
5. DistancesDistance are used to measure similarityThere are many ways to measure the distance s between two instances
6. DistancesManhattan Distance|X1-X2| + |Y1-Y2|Euclidean Distance2
7. Properties of DistanceDist (x,y) >= 0Dist (x,y) = Dist (y,x) are Symmetric Detours can not Shorten Distance Dist(x,z) <= Dist(x,y) + Dist (y,z) XyzXyz
8. DistanceHamming Distance
9. Distance Measure – What does it mean “Similar"? Minkowski DistanceNorm:Chebyshew DistanceMahalanobis distance: d(x , y) = |x – y|TSxy1|x – y| Distances Measure
10. Nearest Neighbor and Exemplar
11. ExemplarArithmetic MeanGeometric MeanMedoid Centroid
12. Arithmetic Mean
13. Geometric MeanA term between two terms of a geometric sequence is the geometric mean of the two terms.Example: In the geometric sequence 4, 20, 100, ....(with a factor of 5), 20 is the geometric mean of 4 and 100.
14. Given: a set P of n points in RdGoal: a data structure, which given a query point q, finds the nearest neighbor p of q in PNearest Neighbor Searchqp
15. K-NN(K-l)-NN: Reduce complexity by having a threshold on the majority. We could restrict the associations through (K-l)-NN.
16. K-NN(K-l)-NN: Reduce complexity by having a threshold on the majority. We could restrict the associations through (K-l)-NN.K=5
17. K-NN Select 5 Nearest Neighbors as Value of K=5 by Taking their Euclidean Disances
18. K-NNDecide if majority of Instances over a given value of K Here, K=5.
19. ExamplePointsX1 (Acid Durability )X2(strength)Y=ClassificationP177BADP274BADP334GOODP414GOOD
20. KNN ExamplePointsX1(Acid Durability)X2(Strength)Y(Classification)P177BADP274BADP334GOODP414GOODP537?
21. Scatter Plot
22. Euclidean Distance From Each PointKNNEuclidean Distance of P5(3,7) from P1P2P3P4(7,7)(7,4)(3,4)(1,4)Sqrt((7-3) 2 + (7-7)2 ) =Sqrt((7-3) 2 + (4-7)2 ) =Sqrt((3-3) 2 + (4-7)2 ) =Sqrt((1-3) 2 + (4-7)2 ) =KNNEuclidean Distance of P5(3,7) from P1P2P3P4(7,7)(7,4)(3,4)(1,4)
23. 3 Nearest NeighBourEuclidean Distance of P5(3,7) from P1P2P3P4(7,7)(7,4)(3,4)(1,4)Sqrt((7-3) 2 + (7-7)2 ) =Sqrt((7-3) 2 + (4-7)2 ) =Sqrt((3-3) 2 + (4-7)2 ) =Sqrt((1-3) 2 + (4-7)2 ) =Class BADBADGOODGOODEuclidean Distance of P5(3,7) from P1P2P3P4(7,7)(7,4)(3,4)(1,4)Class BADBADGOODGOOD
24. KNN ClassificationPointsX1(Durability)X2(Strength)Y(Classification)P177BADP274BADP334GOODP414GOODP537GOOD
25. Variation In KNN
26. Different Values of K
27. ReferencesMachine Learning : The Art and Science of Algorithms that Make Sense of Data By Peter FlachA presentation on KNN Algorithm : West Virginia University , Published on May 22, 2015
28. Thanks