Adaptive multiple thresholds base on canny edge detection of an LED taillight in vehicular visible light communication using image sensor

Dat Vuong, Van Thanh Le, Anh Phan, Trang Nguyen and Trong Hop Do Adaptive Multiple Thresholds Base on Canny Edge Detection of an LED Taillight in Vehicular Visible Light Communication Using Image Sensor Dat Vuong1 , Thanh Van Le1, Anh Phan1, Trang Nguyen1, Trong Hop Do2 1 Vietnam-Korea University of Information and Communication Technology, Danang, Vietnam vcdat@vku.udn.vn 2 University of Information Technology, Ho Chi Minh City, Vietnam, Vietnam National University, Ho Chi Minh City, Vietnam Abstract. Edge detection is used to segment the object by extracting edges and lines of an image. To do this, edge detection applies grayscale thresholds to obtain the edge points of the object. Unreasonably determined thresholds cause an increase in detection error, especially in movement scenario. In the vehicular visible light communication system, the distance between the leading vehicle to the following vehicle as well as the LED taillight and image sensor change continuously. This leads to the brightness variation of LED taillight. In this case, the updating thresholds for edge detection must be examed to detect exact positioning of LED taillight. Through the analysis, this paper proposes adaptive thresholds based Otsu algorithm for edge detection to identify LED taillight in the vehicular visible light commu- nication scenario. The experiment is performed to verify the effectiveness of the proposed scheme. Keywords: Visible light communication, Edge Detection, Threshold, LED, Otsu 1 Introduction Intelligent Transportation System (ITS) is an attractive field for researchers from various nations. It was applied in many related-transportation areas such as rail, waterway, and road system with the aim of im- proving traffic safety. Numerous research papers had concentrated upon this field using radio wave signals. In [1][2], the author has given the overview of ITS in general, vehicular in particular as well as analyzing the potential and drawbacks when it was applied. More than one recent decade, a new perspective technol- ogy for communication has been introduced and brought a promising development including of ITS in general. The mentioned technique is Visible Light Communication based vehicle safety (V2LC). Vehicular visible light communication uses visible light from the LED taillight of the leading vehicle to transmit data to the receiver of the following vehicle. Nowadays, most of the vehicle install the image sensor has done well in not only tracking the on-road situations but also assisting the driver. By processing acquired images, the safety information is announced to the driver or automotive system. Therefore, image processing is a key factor to extract the information. Besides, object detection seems to be a priority issue of image processing. In [3], the author applied the Canny edge technique to detect traffic light. However, two constant thresholds are used to gain the edge points of light within different communication distances. This leads to increasing error detection as well as a bit error rate of the V2LC system. In [4], the author utilized the Ostu algorithm to get multiple thresholds. The Otsu algorithm scans the whole image to get a gray-level histogram. The background and foreground are classified by processing a gray-level histogram. This method will ignore the far-distance LED taillight due to its low-grayscale value. To address these problems, the adaptive threshold-based Otsu is proposed to achieve threshold values corresponding to dif- ferent communication distances. The calculated threshold values are based on the received optical power of the image sensor. Then, the LED taillight position is extracted by applying these thresholds. The rest of the paper is organized as follows. Section 2 provides an overview of the V2LC system. In Section 3, Canny edge detection is given. Section 4 analyzes the drawback of Canny edge detection and proposes the adaptive threshold scheme. The experimental result and discuss are provided in section 5. Finally, a conclusion is provided in section 6. 53 KỶ YẾU HỘI THẢO KHOA HỌC QUỐC GIA CITA 2020 “CNTT VÀ ỨNG DỤNG TRONG CÁC LĨNH VỰC” 2 System model Similar to the usual communication system, the V2LC system has two main components that are transmitter and receiver. Figure 1 illustrates the representative V2LC c system. There are two typical communication systems categorized vehicle-to-vehicle (V2V) and vehicle-to-infrastructure (V2I) communication. V2V ap- plication turns the available light of the vehicle, such as taillight and headlight to account for transmitting data. Whereas V2I communication exploits existing lighting-road systems and traffic light to exchange information data. Fig. 1. Illustrate vehicular visible light communication system Visible light communication (VLC) is one of the wireless communication technologies using light power. The modulated signal will be transferred by the emitting light source and received by an image sensor. The basic VLC system is described in figure 2. Using VLC gains tremendous advantages such as not being harmful to human life, high data rate, and so on. Besides, LEDs are omnipresent installed as the dominant light source. LEDs have a long-life time, lower power consumption, and high intensity. Currently, LEDs also are used the light system of the vehicle because of its above advantages. Fig. 2. The fundamental VLC system 3 Canny edge detection Edge detection is one of the fundamental image processing in the field of object image analysis and com- puter vision. Some schemes were proposed to improve this technique such as Sobel, LOG, Prewitt and Canny, etc. However, these proposed works gain unsatisfying results in practice. This paper focuses on the drawback of Canny edge detection for the target movement scenario. The Canny edge detection is per- formed by 5 steps. 54 Dat Vuong, Van Thanh Le, Anh Phan, Trang Nguyen and Trong Hop Do 1. Grayscale conversion The original image will be converted to grayscale. The grayscale value is from 0 to 255 corresponding to the level of brightness from dark to bright. 2. Smooth and noise reduction with Gaussian filter blur The input image is smoothed by applying the derivation of 2-D Gaussian Kernel function into the Canny operator. The function of the Gaussian Kernel is described as follows: (, ) = (1) Where determines the width of the Gaussian kernel which is also called standard deviation. The result of smoothing is based on this parameter. 3. Find the intensity gradients of the smoothed image In this step, the gradient magnitude and direction of the image are computed. The gradient can be obtained by using the Sobel filter, the edge is represented when the color of the image as well as the intensity of pixel changes.. || = + (2) ∠ = (3) 4. Non-maximum suppression This work finds the maximum pixel value of an edge. Base on the gradient magnitude, the center pixel is compared with two adjacent pixels along its gradient direction from step 3. In case, the center pixel is greater than any of the two, this pixel is kept as candidate edge points, otherwise, as it is set 0 as a black pixel. The completed step makes multi-points are “slimed” within the one-pixel width. 5. Double thresholding The double thresholding determines the exact edge point because the result from non-maximum suppres- sion is not perfect. The double thresholds includes a high threshold (Thh) and a low threshold (Thl). Firstly, the pixel value of candidate edge points is compared to Thh . If the pixel value is greater than Thh, the actual edge points are selected. Secondly, the pixel value of candidate edge points is range from Thl to Thh , suspected edge points (SEP) are assigned. Finally, the connectivity of SEP is scanned, this process checks adjacent pixels of SEP. If there is an edge pixel within the adjacent pixel, the SEP are edge points, other- wise, they are non-edge points and are removed to out of the image. 4 Limitation and proposed scheme 4.1 Limitation Currently, the Canny edge is widely applied in image processing, however, there are existing drawbacks. Some high-frequency signals as well as candidate edge points will be removed after smoothing the image with Gaussian Kernal. As result, losing edge point information leads to the deformed image. In addition, although the double threshold is set manually through previous experiments, they waste processing time. Moreover, in practice, the double thresholds vary due to illuminance change frequently. One of the reasons for illuminance change is the change in the distance between the object and camera. Therefore, the adaptive threshold must be paid attention to this scenario. 55 KỶ YẾU HỘI THẢO KHOA HỌC QUỐC GIA CITA 2020 “CNTT VÀ ỨNG DỤNG TRONG CÁC LĨNH VỰC” 4.2 Proposed scheme As mentioned limitation in section 3.1, the proposed scheme focuses on updating the double threshold in step 5 of Canny edge detection. The calculated threshold bases on the pixel value of the LED taillight on the image. The pixel value is determined by the received optical power and parameter setting of the camera. In this paper, we assumed that the camera is set perfectly. The relationship between optical power and the pixel value is described as the following equation[5]: = 118 × 2(,)/ (4) (, ) is a function of the exposure time difference, PV is the pixel value of LED taillight in the image, Y is received optical power, EH is exposure time, is the gamma encoding value for transferring the received optical power to pixel value. The received optical power is a function of communication dis- tance from LED taillight to camera. Ignoring the light reflection, the light – distance equation describes as follow [6]: = [∆] (5) Suppose that the initial power () is measured at distance d from taillight. The ∆ getting negative or positive value depends on the movement direction of vehicle. For example, if the leading vehicle goes far from following vehicle, the ∆ obtains the positive value and vice versa. Normally, gray-level histogram of an image ranges from 0 to 255, this leads to skip the weak-brightness object. In our proposed scheme, the gray-level histogram of image is from 0 to PV. Then, we apply Ostu method to achieve double threshold. The Otsu is performed by three parameters, namely weight (w ), mean (), variance (). The classified threshold of background (C0 ) and foreground (C1) is T. The threshold T is the high threshold of Otsu algorithm. Base on T, the weight of background and foreground are as follows: = ∑ = (6) = ∑ = 1 − (7) Where: = = The mean of the background and foreground are respective as follows: = ∑ = (8) = ∑ = (9) Where: = ∑ (10) is sum of total mean level from grayscale levels range from 1 to PV. From Eq (3) to Eq (7), the total mean can be verified as follow: + = (11) The variances of two classes are given by: = ∑ (−) (12) = ∑ (−) (13) To determine the high threshold, the Within-class variance must be calculated. This variance is sum of the two variances multipled by their respective weights. 56 Dat Vuong, Van Thanh Le, Anh Phan, Trang Nguyen and Trong Hop Do = + (14) Replacing in turn the value of T from 1 to PV to Eq (11), the adaptive high threshold (Thh) is achieved corresponding to the minimum value of Within-class variance ( ). Similarity, the low threshold (Thl) is calculated as high threshold corresponding to the gray level range from 1 to Thh. 5 Experiment 5.1 Experimental parameters and Scenario The experiment was performed to prove the effectiveness of proposed scheme. The camera of Samsung note 10 plus and LED taillight of Honda CRV were used to fulfill inthis experiment under a static condition. The static condition means that the car was parked at a fixed position while the camera moved further at distance from 10m to 50m. The camera parameters are shown in table 1. To avoid the ambient noise, the Iso and exposure time was set as the following table. Table 1. Camera parameters Camera type Complementary Metal-Oxide-Semeiconductor (CMOS) Resolution 12MP Pixel size 1 Focal length 26mm Iso 800 Exposure time 1/3000sec The images were captured under condition of drizzle night. Figure 3 depicts the captured images at dif- ferent distances. Consequently, the acquired images at the distances of 40m and 50m are not clear enough to be detected by the human eye. Fig. 3. The captured images at different distances 5.2 Experiment results and discussion The high threshold (Thh) of two methods is shown in figure 4. The result proves that the threshold of Otsu is not changed much when the distance increases. As above mentioned, it leads to disappearance of LED taillight. To make clear, the figure 5 illustrates the detection result of Otsu and proposed work at distance of 50m. Because of using lower threshold, the image of proposed scheme has ambient light sources, how- ever, the LED taillight is still available on image. Meanwhile, LED taillight is disappeared a half on the image of Otsu scheme. 57 KỶ YẾU HỘI THẢO KHOA HỌC QUỐC GIA CITA 2020 “CNTT VÀ ỨNG DỤNG TRONG CÁC LĨNH VỰC” Fig. 4. The high threshold of proposed scheme and Otsu algorithm Fig. 5. Result comparision of two scheme at 50m 6 Conclusion In this paper, the new adaptive threshold was introduced for Led taillight detection based Canny edge de- tection. The threshold of proposed work can update corresponding to change of distance. The outcome of the experiment confirms the effectiveness of the proposed work. In further work, the calculated distance by applying a positioning algorithm in the V2LC system based image sensor is used to improve the pro- posed scheme. 58 Dat Vuong, Van Thanh Le, Anh Phan, Trang Nguyen and Trong Hop Do References 1. Tsugawa, Sadayuki. "Inter-vehicle communications and their applications to intelligent vehicles: an overview." Intelligent Vehicle Symposium, 2002. IEEE. Vol. 2. IEEE, 2002. 2. Wang, Cheng-Xiang, Xiang Cheng, and David I. Laurenson. "Vehicle-to-vehicle channel modeling and measure- ments: recent advances and future challenges." IEEE Communications Magazine 47.11 (2009): 96-103. 3. Premachandra, H. Chinthaka N., et al. "Edge-Based Tracking of an LED Traffic Light for a Road-to-Vehicle Vis- ible Light Communication System." 방송공학회논문지 14.4 (2009): 475-487. 4. Jie, Gao, and Liu Ning. "An improved adaptive threshold canny edge detection algorithm." 2012 International Conference on Computer Science and Electronics Engineering. Vol. 1. IEEE, 2012. 5. Do, Trong-Hop, and Myungsik Yoo. "Performance analysis of visible light communication using CMOS sen- sors." Sensors 16.3 (2016): 309. 6. https://imagine.gsfc.nasa.gov/features/yba/M31_velocity/lightcurve/derive.html 59