Computer Vision Industrial IoT - PowerPoint Presentation

1. Computer VisionIndustrial IoTSoftware and Services GroupIoT Developer Relations, Intel

2. 2

3. 3

4. What is the Intel® CV SDK?4The Intel® Computer Vision SDK is a new software development package for development and optimization of computer vision and image processing pipelines for Intel System-on-Chips (SoCs).Intel-optimized implementation of the Khronos OpenVX 1.1 APIPre-built and fully-validated community OpenCV 3.3 binariesVision Algorithm Designer (VAD)Deep Learning Model Optimizer toolDeep Learning Inference Engine

5. 5

6. What is OpenVX*?6OpenVX* is about standardized, portable, power-efficient vision processing.Faster development of computer vision applicationsBetter optimization via directed graph scheduling of image kernels/algorithms. Each graph node can run on a separate piece of hardware.OpenVX chooses which hardware to run each step of your computer vision graph (e.g. CPU, GPU, VPU). Smaller number of algorithms than OpenCV. However, Khronos maintains a conformance test suite that can validate a vendors OpenVX implementation. Camera InputColor ConversionExtract ChannelImage PyramidOptical FlowHarris CornersImage OutputFN2FN1OpenVX NodesOpenVX Graph (Video Pipeline)

7. Algorithms in OpenVX* 1.17Absolute DifferenceAccumulateAccumulate SquaredAccumulate WeightedArithmetic AdditionArithmetic SubtractionBitwise AndBitwise ORBox FilterCanny Edge DetectorChannel CombineChannel ExtractColor CovertConvert bit depthCustom ConvolutionDilate ImageEqualize HistogramErode Image Fast CornersGaussian FilterHarris CornersHistogramImage PyramidIntegral ImageMagnitudeMean and Standard DeviationMedia FilterMin Location Max LocationOptical Flow PyramidPhasePixel-wise MulitplicationRemapScale Image Sobel 3x3TableLookupThresholdingWarp AffineWarp Perspective

8. OpenVx* Vs. OpenCV*8OpenCV*OpenVX*ImplementationCommunity driven open source libraryOpen standard API designed to be implemented by hardware vendorsConformanceExtensive OpenCV Test Suite but no formal Adopters programImplementations must pass defined conformance test suite to use trademarkConsistencyAvailable functions can vary depending on implementation / platformAll core functions must be available in all conformant implementationsScopeVery wide 1000s of imaging and vision functions Multiple camera APIs/interfacesTight focus on core hardware accelerated functions for mobile vision – but extensible Uses external/native camera APIEfficiencyMemory-based architecture Each operation reads and writes to memoryGraph-based execution Optimizable computation and data transferTypical Use CaseRapid experimentation and prototyping - especially on desktopProduction development & deployment on mobile and embedded devicesEmbeddedDeploymentRe-usable codeCallable library

9. OpenCV*Open Source library for computer visionWritten in C++Bindings for most popular languagesBlock-like programming structureImage data is sequentially passed through functionsFundamental concepts allow for powerful image processing toolscodingbox.com# python code snippets go hereimport cv2import numpy as np

10. Computer Vision Algorithms

11. uint8arraysCadin Batrackimg = cv2.imread( ‘image.jpg’) # open imagecv2.imshow( ‘Title’, img ) # show imagecv2.imwrite( ‘image2.jpg’, img ) # write image fileImage Binary Representation

12. ColorspacesintensityBGRHSVGrayscaleBGRHSVGrayscaleimg2 = cv2.cvtColor( img, cv2.COLOR_BGR2HSV )

13. MaskinginRangethresholdBinaryoperationsImageMaskCombineImage +Maskret, thresh = cv2.threshold( single_channel, min_value , set_value_to, cv2.THRESH_BINARY )mask = cv2.inRange( hsv_img, lower_color , higher_color )cv2.bitwise_and( img, img, mask = mask )

14. FilteringNoise Removal (LPF)Sharpness increase (HPF)Operate on Kernel

15. Filtering: SmoothingUses 2D ConvolutionBlurring removes noiseKernel size determines blur amountTypes:Mean - averageMedian – less prone to outliersGaussian – applies Gaussian curveBilateral – blurs & preserves edges# 5x5 mean filterkernel = np.ones( ( 5, 5 ) , np.float32 ) / 25res = cv2.filter2D( img, -1, kernel )

16. Filtering: GradientsHigh Pass FiltersUsed to find edges in an imageSpecify Kernel sizeTypes: Sobel – directional derivativesScharr – better for smaller kernelLaplacian – relation on Sobel derivativesOpenCV Documentationlaplace = cv2.Laplacian( img, cv2.CV_64F )sobelX = cv2.Sobel( img, cv2.CV_64F, 0, 1, ksize = 5 )

17. Gradients for Canny Edge DetectionOriginalSmoothedCanny# 4x4 mean filterkernel = np.ones( ( 4, 4 ) , np.float32 ) / 16wreck= cv2.filter2D( img, -1, kernel )# apply canny edge detectionedges = cv2.Canny( wreck, 0, 200 )

18. Filtering: Morphological TransformationsOriginalNoisy ImageErosionDilationCombinationkernel = np.ones( ( 5, 5 ) , np.float32 ) / 25erosion = cv2.erode( img, kernel, iterations = 1 )dilation = cv2.dilate( img, kernel, iterations = 1 )opening = cv2.morphologyEx( img, cv2.MORPH_OPEN, kernel )closing = cv2.morphologyEx( img, cv2.MORPH_CLOSE, kernel )

19. ContoursJoin all continuous points along boundaryDetect from logical maskContour hierarchies allow selection of a contour based on its relation to othersimg2, contours, hierarchy = cv2.findContours( mask, cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE )cv2.drawContours( img, contours, contour_number, ( 0, 255, 0 ), thickness )

20. Contour FunctionsContours can be mathematically analyzedOpenCV functions give:Moments & CentroidsArea & PerimeterConvex hullsBounding RectanglesFit curves & shapesOutermost pointsShape Matching

21. Histogram:  juzaphotohist = cv2.calcHist( [img], [channel], mask, [number_bins], [min_val, max_val] )

22. Histogram Equalization

23. James Fishbaugheq = cv2.equalizeHist( img )

24. 2D Histogram:  HueSaturationfrequencyhist = cv2.calcHist( [img], [0, 1], mask, [180, 256], [0, 180, 0, 256] )

25. Histogram BackprojectionReturns similarity of pixels in image region to the histogramUseful for object detectionExample of a mountainFind similar mountains

26. Select regioncreate into maskCalculate region histogramBack projectDistancemapping

27. Haar Cascades & Classifiers

28. cascade = cv2.CascadeClassifier( ‘haarcascade_frontalface_default.xml’ )gray_img = cv2.cvtColor( img, cv2.COLOR_BGR2GRAY )faces = cascade.detectMultiScale( gray_img, scaleFactor = 1.25, minNeighbors = 5, minSize = ( 30, 30 ),)for (x, y, w, h) in faces: cv2.rectangle( img, x, y, (x+w, y+h), (0, 255, 0), 2)

29. Computer Vision Graph Examples

30. Crosshair follows object of distinct colorCaptureVideo fromWebcamMask colors in rangeSet MousecallbacksColorRangeTrackbarsConvert frame to HSVMorph. close mask to fill holesFind contoursFind largest contourFind centroidDraw crosshair on centroid

31. Video capture from online webcamModularity: using an online video streamLoad video URLStart streamParse streamDecode to frameDo whatever processing is necessary

32. Presence detection from an online cameraCapture onlinevideoTake image of empty roomClick to capture empty roomGenerate frame bufferTake distance between current and empty pixelsTake threshold of distancesDilate map to close holesIf buffer is full, draw contoursDisplay videoFind contoursFind contour of largest areaUpdate buffer

33. Storing video upon face detectionCaptureVideo fromWebcamCreate cascade classifier from Haar CascadeGenerate bufferConvert color to grayscaleCreate face detection cascadeDraw rectangle around facesWait until buffer endsVideo saved to fileGenerate video writer

34. 34