# computer vision

## Chapter 5 Image Gradient Processing

### 1. What is Image Gradient

• The image gradient calculates the speed at which the image changes. For the edge part of the image, the gray value changes greatly, and the gradient value is also large; on the contrary, for the relatively smooth part of the image, the gray value changes less, and the corresponding gradient value is also small. In general, the image gradient calculates the edge information of the image.

### 2. Template operation

• The template (filter) is a small image W of size n*n (n is generally an odd number, called the template size), and the value w at each position is called the weight. During the calculation, the center of the template is aligned with the pixel P, and the pixel value of the neighborhood N in the same range as the template in the original image is selected as the input. Template operations are divided into template convolution and template sorting.
• The calculation of the template convolution is to multiply the aligned corresponding position pixels, and then accumulate them as the output value of the pixel P position. Note that the pixel gray value of the original image is s, and the calculated value is d, then the output value of point P

d = ∑ w i s i ∑ w i d = \frac{\sum w_is_i}{\sum w_i} d=∑wi​∑wi​si​​

• In the calculation of template sorting, the pixel values ​​of the neighborhood N are sorted, and the gray value in a specific order is selected as the output value of the pixel P position, such as the maximum value, the minimum value, and the median.

### 3. Mean filter

• The mean filter refers to a filter whose template weights are all 1. It takes the average value of the neighborhood of the pixel as the output result. The mean filter can smooth the image and remove noise, but the image will become more blurred as the size of the template increases. Often used as fuzzification.

### Fourth, Gaussian filtering

• In order to reduce the size of the template and increase the blurring of the image, a Gaussian filter can be used. The template of the Gaussian filter determines the template coefficients according to the Gaussian distribution, and the weight close to the center is larger than that of the edge. A Gaussian filter of 5 looks like this:

### 5. Median filter

• The median filter belongs to the filter of the template sorting operation. The median filter replaces the original pixel value with the sorted median value of the pixels in the neighborhood. It retains the sharpness of the original image while implementing the noise reduction operation without modifying the grayscale value of the original image.
• The use of median filtering is very common. It has a good effect on the suppression of salt and pepper noise, and can effectively protect the edges from blurring while suppressing random noise. However, median filtering is a nonlinear change, which may destroy the linear relationship in the image. It is not suitable for images with more details such as points and lines and high-precision image processing tasks.

### 6. Edge detection

• The edge of the details in the image can be obtained by gradient calculation. In order to reduce the influence of noise while sharpening the edge, different edge detection operators are developed by improving the gradient method:
• Edge detection effect

### 7. Sharpening

• Image sharpening and image smoothing are the opposite operations. Sharpening is to reduce the blur in the image by enhancing the high-frequency components, enhance the edges and contours of the image details, and enhance the grayscale contrast, which is convenient for the identification and processing of the target in the later stage. Sharpening increases the noise of the image while enhancing the edges of the image.
• The image sharpening operation can be realized by superimposing the obtained edge on the original image according to a certain coefficient ratio. For example, a template for sharpening using the Laplacian gradient operator, where A is a coefficient greater than or equal to 1:

### 8. Code

# image blurring
import cv2
import numpy as np

cv2.imshow("im", im)

# median filter
im_median_blur = cv2.medianBlur(im, 5)
cv2.imshow("im_median_blur", im_median_blur)

# mean filter
im_mean_blur = cv2.blur(im, (3, 3))
cv2.imshow("im_mean_blur", im_mean_blur)

# Gaussian filter
im_gaussian_blur = cv2.GaussianBlur(im, (5, 5), 3)
cv2.imshow("im_gaussian_blur", im_gaussian_blur)

# Custom Gaussian kernel to perform filtering calculation
gaussian_blur = np.array([
[1, 4, 7, 4, 1],
[4, 16, 26, 16, 4],
[7, 26, 41, 26, 7],
[4, 16, 26, 16, 4],
[1, 4, 7, 4, 1]
], np.float32) / 273

# Use filter2D to perform filtering calculation
im_gaussian_blur2 = cv2.filter2D(im,   # original image
-1,   # target image depth, -1 means the same
gaussian_blur as the original image)   # filter
cv2.waitKey()   # wait for the user to press a key
cv2 .destroyAllWindows()   # Destroy all created windows

# image sharpening
import cv2
import numpy as np

cv2.imshow("im", im)

# sharpening operator 1
sharpen_1 = np.array([[-1, -1, -1],
[-1, 9, -1],
[-1, -1, -1]])
im_sharpen1 = cv2.filter2D(im,
-1,
sharpen_1)
cv2.imshow("im_sharpen1", im_sharpen1)

# sharpening operator 2
sharpen_2 = np.array([[0, -1, 0],
[-1, 8, -1],
[0, 1, 0]]) / 4.0
im_sharpen2 = cv2.filter2D(im,
-1,
sharpen_2)
cv2.imshow("im_sharpen2", im_sharpen2)

cv2.waitKey()   # Wait for the user to press a key
cv2.destroyAllWindows()   # Destroy all created windows

## Chapter 6 Image Outlines

### 1. What is an image outline

• Although edge detection can detect edges, the edges are discontinuous, and the detected edges are not a whole. The image contour is a whole formed by connecting the edges, which is used for subsequent calculations.
• The image contour is a very important feature information in the image. Through the operation of the image contour, we can obtain the size, position, direction and other information of the target image.
• Image contour operations include: finding contours, drawing contours, and contour fitting.

### 2. Find and draw contours

• A contour corresponds to a series of points, these points represent a curve in the image in some way, and drawing these points into lines of different styles is the contour search and drawing

### Three, contour fitting

• When calculating the contour, the actual contour may not be needed, but only an approximate polygon close to the contour is required. Drawing this approximate polygon is called contour fitting.

### 8. Code

#### 1. Find and draw contours

# find and draw contours
import cv2
import numpy as np

cv2.imshow("im", im)

# Grayscale
im_gray = cv2.cvtColor(im, cv2.COLOR_BGR2GRAY)
# Binarization
ret, im_binary = cv2.threshold(im_gray, 127, 255, cv2.THRESH_BINARY)
# cv2.imshow("im_binary", im_binary)

# Find contour
cnts, hie = cv2.findContours(im_binary,   # Original image (binarized)
cv2.RETR_EXTERNAL,   # Only detect outer contour
cv2.CHAIN_APPROX_NONE)   # Store all contour points
# print(type(cnts) ) # tuple
# for cnt in cnts:
# # print(type(cnt)) # <class 'numpy.ndarray'>
# print(cnt.shape)
# """
# (257, 1, 2)
# (371, 1, 2)
# (336, 1, 2)
# (228, 1, 2)
# """

# draw contours
im_cnt = cv2.drawContours(im,   # original image
cnts,   # contour data, return value of findContours
-1,   # draw all contours
(0, 0, 255),   # contour color: red
2)   # contour thickness
cv2 .imshow( "im_cnt" , im_cnt)
cv2.waitKey()   # Wait for the user to press a key
cv2.destroyAllWindows()   # Destroy all created windows

#### 2. Draw a rectangular bounding box

# Draw the outline bounding rectangle
import cv2
import numpy as np

cv2.imshow("im", im)

# binarization
ret, im_binary = cv2.threshold(im, 127, 255, cv2.THRESH_BINARY)
cnts, hie = cv2.findContours(im_binary,
cv2.RETR_LIST,   # Do not establish hierarchical relationship
cv2.CHAIN_APPROX_NONE)   # Store all coordinate points of contour
# print(cnts[0].shape) # (498, 1, 2)

# Generate bounding rectangle parameters according to the outline
x, y, w, h = cv2.boundingRect(cnts[0])

# draw rectangle
brcnt = np.array([[[x, y]], [[x+w, y]],[[x+w,y+h]], [[x, y+h]]])
cv2.drawContours(im, [brcnt], -1, (255, 255, 255), 2)
cv2.imshow("result", im)

cv2.waitKey()   # Wait for the user to press a key
cv2.destroyAllWindows()   # Destroy all created windows

#### 3. Draw a circular enclosing circle

# Draw the smallest circular enclosing circle of the outline
import cv2
import numpy as np

cv2.imshow("im", im)

# binarization
ret, im_binary = cv2.threshold(im, 127, 255, cv2.THRESH_BINARY)
cnts, hie = cv2.findContours(im_binary,
cv2.RETR_LIST,
cv2.CHAIN_APPROX_NONE)
(x, y), radius = cv2.minEnclosingCircle(cnts[0])   # Minimum enclosing circle parameters to generate contour
print((x, y), radius)   # (204.5, 91.5) 86.53621673583984
center = (int(x) , int(y))   # Convert the coordinates of the center of the circle to an integer

cv2.circle(im, center, radius, (255, 255, 255), 2)   # draw a circle
cv2.imshow( "result" , im)
cv2.waitKey()   # Wait for the user to press a key
cv2.destroyAllWindows()   # Destroy all created windows

#### 4. Draw the best fit ellipse

# draw the best fit ellipse
import cv2
import numpy as np

cv2.imshow("im", im)

im_gray = cv2.cvtColor(im, cv2.COLOR_BGR2GRAY)
ret, im_binary = cv2.threshold(im_gray, 127, 255, cv2.THRESH_BINARY)
# extract contours
cnts, hie = cv2.findContours(im_binary,
cv2.RETR_LIST,
cv2.CHAIN_APPROX_NONE)
ellipse = cv2.fitEllipse(cnts[0])   # Generate the best fit ellipse data
print( "ellipse" ,
ellipse)   # ellipse ((204.24952697753906, 91.23741149902344), (69.94644165039062, 162.95944213867188), 84.70285034179688)

cv2.ellipse(im, ellipse, (0, 0, 255), 2)   # draw an ellipse
cv2.imshow( "result" , im)
cv2.waitKey()   # Wait for the user to press a key
cv2.destroyAllWindows()   # Destroy all created windows

#### 5. Approximate polygons

# Use polygons to fit contours
import cv2
import numpy as np

cv2.imshow("im", im)

# Convert grayscale image
im_gray = cv2.cvtColor(im, cv2.COLOR_BGR2GRAY)
# Binarization
ret, im_binary = cv2.threshold(im_gray, 127, 255, cv2.THRESH_BINARY)
# find contours
cnts, hie = cv2.findContours(im_binary,
cv2.RETR_LIST,
cv2.CHAIN_APPROX_NONE)
# accuracy one
epsilon = 0.005 * cv2.arcLength(cnts[0], True)   # Accuracy, calculated according to perimeter
approx = cv2.approxPolyDP(cnts[0], epsilon, True)   # Construct polygon, return polygon data
adp = cv2.drawContours( adp, [approx], 0, (0, 0, 255), 2)   # draw
cv2.destroyAllWindows()   # Destroy all created windows