Driver drowsiness detection system ieee paper download free

Click here to sign up. Download Free PDF. Swapnil Titre. A short summary of this paper. Driver Drowsiness Detection and Alert System. State of the driver under real driving conditions. The aim of driver Publication Issue : drowsiness detection systems is to try to reduce these traffic accidents. The May-June secondary data collected focuses on previous research on systems for detecting drowsiness and several methods have been used to detect drowsiness or Article History inattentive driving.

Our goal is to provide an interface where the program can Accepted : 18 June automatically detect the driver's drowsiness and detect it in the event of an Published : 26 June accident by using the image of a person captured by the webcam and examining how this information can be used to improve driving safety can be used. Basically, you're collecting a human image from the webcam and exploring how that information could be used to improve driving safety. Collect images from the live webcam stream and apply machine learning algorithm to the image and recognize the drowsy driver or not.

When the driver is sleepy, it plays the buzzer alarm and increases the buzzer sound. If the driver doesn't wake up, they'll send a text message and email to their family members about their situation. Hence, this utility goes beyond the problem of detecting drowsiness while driving. Eye extraction, face extraction with dlib. Drowsiness affects around 1. Most of these mental alertness and reduces the driver's ability to accidents are caused by driver distraction or drive a vehicle safely and increases the risk of human drowsiness.

Drowsiness decreases the driver's error, which can lead to death and injury [5]. Technol, May-June - , 7 3 : error rate for the driver had decreased. Countless people drive long distances on the road day and night. Lack of sleep or distractions such as talking on the II.

It gives you a framework in which to work with pictures and videos however you want, using OpenCV algorithms or your own, without worrying about allocating and reallocating memory for your pictures.

The framework is also capable to recognize in such situation when the eyes can't be discovered, and works in sensible lighting circumstances. The result demonstrates that eye-tracking drowsiness functions admirably for a few drivers the length of the squint acknowledgment works appropriately. The camera based drowsiness measures give an appreciated contribution. However, despite outperforming in drowsiness detection, the previous method had a critical drawback in generating representations.

The previous method had a possibility that the method gener- ates extremely sparse representation which cannot contain suf- ficient information to detect drowsiness. This work is improved a b and extended from our earlier work [41], and we propose Fig. In 3D convolution b , a temporal tion. The condition-adaptive representation learning is a repre- sentation learning process to take the feature focused on some particular condition using auxiliary information a.

The main contribution of this work is the information. When the training dataset can be classified to representation learning framework that could be adapted to several conditions, whilst the normal representation learning the particular scene conditions via understanding the scenes perform to extract generalized features from overall training and generating the condition adaptive representation. In Section more specific representations reflecting given conditions. The ure 1 represents the comparison of processes about the normal architectural detail of the proposed framework is explained in representation learning and condition-adaptive representation Section III.

We describe the training and inferencing procedure learning. An auxiliary information has been used to improve of the proposed framework in Section IV, and represent the the performance of the deep learning model in many computer method of data argumentation in Section V.

In Section VI, we vision studies [42], [43]. Hong et al. The system using transferrable knowledge to the scene segmen- conclusion and discussion are described in Section VII. Zhang et al. These methods tried to improve A convolutional neural network CNN a.

CNNs show outstanding features in their target domains. As with the methods described performance in many computer vision studies such as image above, the concept of the condition-adaptive representation classification [45], object detection, and recognition [32]. The could be possibly interpreted as a representation biased to key architectural characteristics of CNNs are ensuring some some conditions. However, in compared to the above methods degree of shift, scale, and distortion invariance: local receptive which use extra information solely in training phase as prior field, shared weight, and spatial or temporal sub-sampling [44].

By using this paradigm, the proposed framework elementary feature detector for one part of an image, across can immediately generate the representation which adapts to the set of entire images. In general CNNs, the convolution is performed at the convo- The proposed framework is composed of four models con- lution layers to discover features from spatial neighbourhoods sisting of representation learning, scene understanding, feature on feature maps in each layer.

Formally, the value of a unit fusion, and drowsiness detection. The representation learning at position x, y in the i-th feature map in the j-th layer model discovers the rich and discriminative representation that presented as axyij is represented by can describe the motion and appearance of an object within the consecutive frames simultaneously.

The proposed framework detects drivers drowsiness in the unit at position x, y in the i-th feature map in the j- various situations accurately by using this condition-adaptive th layer. The red boxes with bold line denote the models, and the black boxes drawn by dotted line define extracted features or outputs of each model.

In the sub-sampling layer, the dimensional scale of the feature map is reduced by pooling over the spatially adjacent neighbourhood on the feature maps in the previous layer.

The learnt feature using 2 The proposed framework is based on four models for repre- dimensional-CNN 2D-CNN can not only discover the locally sentation learning, the scene understanding, the feature fusion, useful feature but also be helpful to understand an entire and the drowsiness detection.

The representation learning image. The scene un- 2D-CNN is robust to various computer vision studies, this derstanding model consists of four sub-models fgl , fh , fm , fe paradigm of 2D-CNNs plays the role of a hurdle in learning for interpreting the condition of glasses, illuminations, and the temporal representations about the sequential data such movement of facial elements. The fusion model ff u gener- as video. To discover the rich and informative information ates condition-adaptive representation which can acclimatize from the sequential data using CNNs, Ji et al.

The detection model fdet determines the 3D convolution [46]. The 3D convolution is achieved whether a driver is sleepy or not. Figure 3 shows an overall by convolving a 3D feature map to the 3D volume formed architecture of the proposed framework. The brief explanation by stacking multiple images together. By this principle, the for how to generate condition-adaptive representation and feature maps in the convolution layers can capture temporal detect drowsiness of drivers, using the proposed framework information that is contained in multiple contiguous frames.

Initially, the representation learning based on The value of a unit at position x, y, t in the i-th feature map the 3D-DCNN extracts a feature that can describe motion in the j-th layers which is denoted as axytij can be formulated and appearance from a video clip simultaneously.

The one-hot encoding is one of the latent representation of a unit at position x, y, t in the i-th encoding approaches which indicates the state of a system feature map in the j-th layer, bij is the bias for the feature using the binary values.

The encoding result is represented map, w is the value of the kernel 3D Local receptive field by the group of bits among which the legal combinations of connected to the feature map, and W , H and D are the values are only those with a single high 1 bit and all the width, the height, and the depth of the kernel, respectively.

Then, feature fusion learns a condition- Figure 2 shows the comparison of 2D and 3D convolutions. Finally, the detection given single image only, a 3D convolution can extract both model identifies a state of driver drowsiness by analyzing the spatial and temporal representation simultaneously in multiple condition-adaptive representation.

In the following, we will consecutive images because the kernel of 3D convolution describe the detail of information of each model and training explore not only spatial axis but also temporal axis. The green box and red box denote an input data and extracted spatio-temporal representation respectively, and the blue boxes represent convolution layers and pooling layers. Numbers located in the upside of the boxes represent the depth of each layer, and numbers below the boxes illustrate the dimensionality and structural detail of the kernel in each convolutional layer.

Spatio-temporal representation learning Da denote the width, height, and depth of the spatio-temporal In this section, we describe the representation learning representation. The Figure 4 shows the architectural detail of the 3D-DCNN in objective of the representation learning is discovering a rich the representation learning.

To discover a spatial and temporal and discriminative feature from inputted consecutive frames. When drivers feel drowsiness, their facial elements i j k make various changes, and these changes would be interpreted as either a shift in shape or change of motion. Therefore, where a is an activation value of the hidden unit, and v, w, to detect a drowsiness of drivers, we have to consider the and b are the input value, the weight, and bias respectively.

While the change according to a time sequence. When we consider these ordinary 2D structure of the kernel local receptive field in limitations observed when a input is a single frame, it is 2D convolution layers can extract spatial information only, the necessary to use multiple consecutive frames as an input to 3D structure of the kernel in 3D convolution layer allows to us discover the spatial and temporal information simultaneously.

The In this work, we employed 3D-DCNN to discover various extracted representations which contain spatial and temporal spatial and temporal change in given multiple consecutive features convey to the scene understanding model and feature frames.

W , H, and T are the width, height, and the temporal length respectively. For a given input video clip x, the representation B. These interpreted information help to train the framework The spatio-temporal representation is defined as the activation for adapting the learnt representation to the various scene values of the hidden units in the last convolutional layer of conditions.

Wa , Ha , and with the scene conditions and a driver drowsiness status. In this work, the scene condition contains the three cate- Scene condition Category One-hot vector Condition gories of the facial elements and one category for the status 1 Day bare face 2 Day glasses of glasses and illumination: 1 conditions of glasses and Glasses and illumination 3 Night glasses illumination Lgl , 2 head Lh , 3 mouth Lm , and 4 eye Le.

The 2 Looking at both sides Head condition detailed explanation for the annotation of each scene condition 3 Nodding is described in Table I. We adopt a fully connected neural 1 Normal status Mouth condition 2 Talking and laughing network since there is a possibility that given spatiotemporal 3 Yawning representations have complex distributions which can not be 1 10 Sleepiness eye Eye condition modelled by a linear kernel.

Each model is composed of two hidden layers C. Feature fusion and a corresponding output layer. The learning procedure of each sub-model in element-wise multiplication interaction between the feature the scene understanding is similar to the back propagation maps [50].

To train the proposed framework that generates the algorithm [48]. Each sub-model estimates a condition that combined representation which needs joint learning between corresponding to the given spatio-temporal representations a, the multiple resources, we refer to the training procedure then computes the difference between the predicted conditions proposed by Hong et al. The fusion model is defined and annotations to train the parameters of the network of as follows the sub-model.

Two images in b and c represents the visualization of activation results of hidden units in representation learning and feature fusion modules. The proposed condition-adaptive representation learning framework adaptively discover the conditional feature in an input volumes depending on the result of the scene understanding model. Drowsiness detection input domains containing the spatio-temporal representation The fusion model described in the previous subsection gen- and the scene conditions.

The drowsiness detection standing empirically computes values that are close to zero. As same as the scene understanding model, we put tiplication results exceeded the range that can be represented an additional fully connected deep neural network on top of by computation machine.

The output of the fully connected function in [34], [51]. The normalization is formulated as network is consists of two units: non-drowsiness unit and follows drowsiness unit, to classify the drowsiness of a driver.

An Intuitively, v represents a condition-adaptive representation optimization scheme for both ff u and fdet operates under the defined over all spatio-temporal representations and the corre- detection objective.

Our detection model is trained to minimize sponding scene conditions. Figure 5 shows the input images, the detection loss using detection annotation associated with the spatio-temporal representations, and the condition-adaptive fusion feature, and representation as follows: representations. We used the softmax cross-entropy function … Image filtering as the objective function for Edet. The objective function is worked to all models embedding into the proposed framework.

Rotation IV. Combining Eq. However, when we begin the training, we do not train the all models of the proposed VI. The overall architecture see Fig 2. Benchmark dataset of the representation learning model, and also denotes that Previous studies [12], [19], [20] on driver drowsiness detec- the representation learning and scene understanding models tion attempted to recognize small cases in the private dataset can considerably influence to the other models feature fusion which is constructed in their own experimental environment and drowsiness detection.

First, we train the representation for driver drowsiness detection. Abtahi et al. After publicly-available dataset for yawning detection [52]. How- that, we train all models containing the feature fusion and ever, it is still insufficient for a comprehensive drowsy driver detection models. It is too tations using the representation learning, and then the spatio- difficult and dangerous to construct a dataset for detecting temporal representation is used to understand scene conditions.

The NTHU- these two pieces of information are combined to produce the DDD dataset is composed of several videos containing a driver condition-adaptive representation. Drowsiness is detected by who was sitting on a car seat and playing a racing game with using this condition-adaptive representation.

The drivers in the dataset conducted various facial expressions during video recording. The most general approach to reduce overfitting on a given The NTHU-DDD dataset is composed of three subsets for training dataset is artificially enlarging the dataset using label- training, evaluation, and test, which are composed of non- preserving transformations [45].

In this work, we apply the redundant video files. Each subset consists of the videos which data augmentation based on horizontal transformation and im- contain diverse situations for the condition for drivers that is age pyramid technique. This approach allows transformation captured using visual sensors such as a camera and an active of an image with very little computation so that we can make infrared IR sensor. The entire dataset including training and an additional dataset without huge computational load.

The driving scenarios include normal by using the image filtering methods based on the Gaussian fil- driving, yawning, slow blink rate, falling asleep, and burst ter. Figure 6 illustrates the procedure of the data augmentation.

Based on the Dlib toolkit, the landmarks of frontal driver facial in a frame are found. According to the eyes landmarks, a new parameter, called Eyes Aspect Ratio, is introduced to evaluate the drowsiness of driver in the current frame. Taking into account differences in size of driver's eyes, the proposed algorithm consists of two modules: offline training and online monitoring.

In the first module, a unique fatigue state classifier, based on Support Vector Machines, was trained which taking the Eyes Aspect Ratio as input. Then, in the second module, the trained classifier is application to monitor the state of driver online.