# computer vision

## Chapter 5 Image Gradient Processing

### 1. What is Image Gradient

- The image gradient calculates the speed at which the image changes. For the edge part of the image, the gray value changes greatly, and the gradient value is also large; on the contrary, for the relatively smooth part of the image, the gray value changes less, and the corresponding gradient value is also small. In general, the image gradient calculates the edge information of the image.

### 2. Template operation

- The template (filter) is a small image W of size n*n (n is generally an odd number, called the template size), and the value w at each position is called the weight. During the calculation, the center of the template is aligned with the pixel P, and the pixel value of the neighborhood N in the same range as the template in the original image is selected as the input. Template operations are divided into template convolution and template sorting.
- The calculation of the template convolution is to multiply the aligned corresponding position pixels, and then accumulate them as the output value of the pixel P position. Note that the pixel gray value of the original image is s, and the calculated value is d, then the output value of point P

d = ∑ w i s i ∑ w i d = \frac{\sum w_is_i}{\sum w_i} d=∑wi∑wisi

- In the calculation of template sorting, the pixel values of the neighborhood N are sorted, and the gray value in a specific order is selected as the output value of the pixel P position, such as the maximum value, the minimum value, and the median.

### 3. Mean filter

- The mean filter refers to a filter whose template weights are all 1. It takes the average value of the neighborhood of the pixel as the output result. The mean filter can smooth the image and remove noise, but the image will become more blurred as the size of the template increases. Often used as fuzzification.

### Fourth, Gaussian filtering

- In order to reduce the size of the template and increase the blurring of the image, a Gaussian filter can be used. The template of the Gaussian filter determines the template coefficients according to the Gaussian distribution, and the weight close to the center is larger than that of the edge. A Gaussian filter of 5 looks like this:

### 5. Median filter

- The median filter belongs to the filter of the template sorting operation. The median filter replaces the original pixel value with the sorted median value of the pixels in the neighborhood. It retains the sharpness of the original image while implementing the noise reduction operation without modifying the grayscale value of the original image.
- The use of median filtering is very common. It has a good effect on the suppression of salt and pepper noise, and can effectively protect the edges from blurring while suppressing random noise. However, median filtering is a nonlinear change, which may destroy the linear relationship in the image. It is not suitable for images with more details such as points and lines and high-precision image processing tasks.

### 6. Edge detection

- The edge of the details in the image can be obtained by gradient calculation. In order to reduce the influence of noise while sharpening the edge, different edge detection operators are developed by improving the gradient method:
- First-order gradient: Prewitt gradient operator, Sobel gradient operator
- Second-Order Gradients: Laplacian Gradient Operator

- Edge detection effect

### 7. Sharpening

- Image sharpening and image smoothing are the opposite operations. Sharpening is to reduce the blur in the image by enhancing the high-frequency components, enhance the edges and contours of the image details, and enhance the grayscale contrast, which is convenient for the identification and processing of the target in the later stage. Sharpening increases the noise of the image while enhancing the edges of the image.
- The image sharpening operation can be realized by superimposing the obtained edge on the original image according to a certain coefficient ratio. For example, a template for sharpening using the Laplacian gradient operator, where A is a coefficient greater than or equal to 1:

### 8. Code

# image blurring import cv2 import numpy as np im = cv2.imread("../data/lena.jpg", 0) cv2.imshow("im", im) # median filter im_median_blur = cv2.medianBlur(im, 5) cv2.imshow("im_median_blur", im_median_blur) # mean filter im_mean_blur = cv2.blur(im, (3, 3)) cv2.imshow("im_mean_blur", im_mean_blur) # Gaussian filter im_gaussian_blur = cv2.GaussianBlur(im, (5, 5), 3) cv2.imshow("im_gaussian_blur", im_gaussian_blur) # Custom Gaussian kernel to perform filtering calculation gaussian_blur = np.array([ [1, 4, 7, 4, 1], [4, 16, 26, 16, 4], [7, 26, 41, 26, 7], [4, 16, 26, 16, 4], [1, 4, 7, 4, 1] ], np.float32) / 273 # Use filter2D to perform filtering calculation im_gaussian_blur2 = cv2.filter2D(im, # original image -1, # target image depth, -1 means the same gaussian_blur as the original image) # filter cv2.waitKey() # wait for the user to press a key cv2 .destroyAllWindows() # Destroy all created windows

# image sharpening import cv2 import numpy as np im = cv2.imread("../data/lena.jpg", 0) cv2.imshow("im", im) # sharpening operator 1 sharpen_1 = np.array([[-1, -1, -1], [-1, 9, -1], [-1, -1, -1]]) im_sharpen1 = cv2.filter2D(im, -1, sharpen_1) cv2.imshow("im_sharpen1", im_sharpen1) # sharpening operator 2 sharpen_2 = np.array([[0, -1, 0], [-1, 8, -1], [0, 1, 0]]) / 4.0 im_sharpen2 = cv2.filter2D(im, -1, sharpen_2) cv2.imshow("im_sharpen2", im_sharpen2) cv2.waitKey() # Wait for the user to press a key cv2.destroyAllWindows() # Destroy all created windows

## Chapter 6 Image Outlines

### 1. What is an image outline

- Although edge detection can detect edges, the edges are discontinuous, and the detected edges are not a whole. The image contour is a whole formed by connecting the edges, which is used for subsequent calculations.
- The image contour is a very important feature information in the image. Through the operation of the image contour, we can obtain the size, position, direction and other information of the target image.
- Image contour operations include: finding contours, drawing contours, and contour fitting.

### 2. Find and draw contours

- A contour corresponds to a series of points, these points represent a curve in the image in some way, and drawing these points into lines of different styles is the contour search and drawing

### Three, contour fitting

- When calculating the contour, the actual contour may not be needed, but only an approximate polygon close to the contour is required. Drawing this approximate polygon is called contour fitting.

### 4. Rectangular bounding box

### 5. The smallest enclosing circle

### 6. Best Fit Ellipse

### 7. Approximate polygons

### 8. Code

#### 1. Find and draw contours

# find and draw contours import cv2 import numpy as np im = cv2.imread("../data/3.png") cv2.imshow("im", im) # Grayscale im_gray = cv2.cvtColor(im, cv2.COLOR_BGR2GRAY) # Binarization ret, im_binary = cv2.threshold(im_gray, 127, 255, cv2.THRESH_BINARY) # cv2.imshow("im_binary", im_binary) # Find contour cnts, hie = cv2.findContours(im_binary, # Original image (binarized) cv2.RETR_EXTERNAL, # Only detect outer contour cv2.CHAIN_APPROX_NONE) # Store all contour points # print(type(cnts) ) # tuple # for cnt in cnts: # # print(type(cnt)) # <class 'numpy.ndarray'> # print(cnt.shape) # """ # (257, 1, 2) # (371, 1, 2) # (336, 1, 2) # (228, 1, 2) # """ # draw contours im_cnt = cv2.drawContours(im, # original image cnts, # contour data, return value of findContours -1, # draw all contours (0, 0, 255), # contour color: red 2) # contour thickness cv2 .imshow( "im_cnt" , im_cnt) cv2.waitKey() # Wait for the user to press a key cv2.destroyAllWindows() # Destroy all created windows

#### 2. Draw a rectangular bounding box

# Draw the outline bounding rectangle import cv2 import numpy as np im = cv2.imread("../data/cloud.png", 0) cv2.imshow("im", im) # binarization ret, im_binary = cv2.threshold(im, 127, 255, cv2.THRESH_BINARY) cnts, hie = cv2.findContours(im_binary, cv2.RETR_LIST, # Do not establish hierarchical relationship cv2.CHAIN_APPROX_NONE) # Store all coordinate points of contour # print(cnts[0].shape) # (498, 1, 2) # Generate bounding rectangle parameters according to the outline x, y, w, h = cv2.boundingRect(cnts[0]) # draw rectangle brcnt = np.array([[[x, y]], [[x+w, y]],[[x+w,y+h]], [[x, y+h]]]) cv2.drawContours(im, [brcnt], -1, (255, 255, 255), 2) cv2.imshow("result", im) cv2.waitKey() # Wait for the user to press a key cv2.destroyAllWindows() # Destroy all created windows

#### 3. Draw a circular enclosing circle

# Draw the smallest circular enclosing circle of the outline import cv2 import numpy as np im = cv2.imread("../data/cloud.png", 0) cv2.imshow("im", im) # binarization ret, im_binary = cv2.threshold(im, 127, 255, cv2.THRESH_BINARY) cnts, hie = cv2.findContours(im_binary, cv2.RETR_LIST, cv2.CHAIN_APPROX_NONE) (x, y), radius = cv2.minEnclosingCircle(cnts[0]) # Minimum enclosing circle parameters to generate contour print((x, y), radius) # (204.5, 91.5) 86.53621673583984 center = (int(x) , int(y)) # Convert the coordinates of the center of the circle to an integer radius = int(radius) # Convert the radius to an integer cv2.circle(im, center, radius, (255, 255, 255), 2) # draw a circle cv2.imshow( "result" , im) cv2.waitKey() # Wait for the user to press a key cv2.destroyAllWindows() # Destroy all created windows

#### 4. Draw the best fit ellipse

# draw the best fit ellipse import cv2 import numpy as np im = cv2.imread("../data/cloud.png") cv2.imshow("im", im) im_gray = cv2.cvtColor(im, cv2.COLOR_BGR2GRAY) ret, im_binary = cv2.threshold(im_gray, 127, 255, cv2.THRESH_BINARY) # extract contours cnts, hie = cv2.findContours(im_binary, cv2.RETR_LIST, cv2.CHAIN_APPROX_NONE) ellipse = cv2.fitEllipse(cnts[0]) # Generate the best fit ellipse data print( "ellipse" , ellipse) # ellipse ((204.24952697753906, 91.23741149902344), (69.94644165039062, 162.95944213867188), 84.70285034179688) cv2.ellipse(im, ellipse, (0, 0, 255), 2) # draw an ellipse cv2.imshow( "result" , im) cv2.waitKey() # Wait for the user to press a key cv2.destroyAllWindows() # Destroy all created windows

#### 5. Approximate polygons

# Use polygons to fit contours import cv2 import numpy as np im = cv2.imread("../data/cloud.png") cv2.imshow("im", im) # Convert grayscale image im_gray = cv2.cvtColor(im, cv2.COLOR_BGR2GRAY) # Binarization ret, im_binary = cv2.threshold(im_gray, 127, 255, cv2.THRESH_BINARY) # find contours cnts, hie = cv2.findContours(im_binary, cv2.RETR_LIST, cv2.CHAIN_APPROX_NONE) # accuracy one adp = im.copy() epsilon = 0.005 * cv2.arcLength(cnts[0], True) # Accuracy, calculated according to perimeter approx = cv2.approxPolyDP(cnts[0], epsilon, True) # Construct polygon, return polygon data adp = cv2.drawContours( adp, [approx], 0, (0, 0, 255), 2) # draw cv2.imshow( "result_0.005" , adp) # precision two adp2 = im.copy() epsilon = 0.01 * cv2.arcLength(cnts[0], True) # Accuracy, calculated according to perimeter approx = cv2.approxPolyDP(cnts[0], epsilon, True) # Construct polygon, return polygon data adp = cv2.drawContours( adp2, [approx], 0, (0, 0, 255), 2) # draw cv2.imshow( "result_0.01" , adp2) cv2.waitKey() # Wait for the user to press a key cv2.destroyAllWindows() # Destroy all created windows