PALMPRINT, FINGER KNUCKLE PRINT AND FACE FEATURES FOR THE HUMAN RECOGNITION SYSTEM

: In the real time application, the majority of the biometric systems are unimodal. The unimodal takes only one source of information like palmprint, face etc. for the person recognition. Some problems occurred by this unimodal are spoof attacks and intra-class variations. Due to the presence of multiple independent portions of data, multimodal biometrics prevails over these problems by fusion of two or more unimodal biometric systems. In this paper, a multimodal biometric system by integrating Palmprint, Finger Knuckle Print (FKP) and Face at the matching score level is proposed. The features are extracted using the Scale Invariant Feature Transform (SIFT) and the Speeded Up Robust Features (SURF) and classified by K nearest neighbour (KNN) and Support Vector Machine (SVM). The experimental result for PolyU database shows the effectiveness of multimodal biometric system with reference to False Accept Rate (FAR), False Reject Rate (FRR) and Genuine Accept Rate (GAR).


INTRODUCTION
Biometric system is fundamentally an automatic recognition system that recognizes a person by determining the genuineness of their specific characteristics possessed by an individual. Since last decade, biometric system booms in various industries and continues to provide higher security features for authentication [1]. In the real world application, the unimodal biometric systems are employed for authentication. At the same time they are susceptible to variety of problems and leads to less significance. To reduce the error rate, the multimodal biometric systems is employed and it integrates two or more biometrics systems. Due to the existence of multiple and independent biometrics, these systems are more reliable and securable [2]. In this paper, the different modalities used are Palmprint, FKP and Face dataset. Even identical twins who share the same DNA sequences have different palmprints. These line structures are stable and remain unchanged throughout the life of an individual. FKP is rich in texture features, contactless image acquisition, invariant to emotions and easily accessible. Human facial features and complete face recognition can be done automatically by non-contact. So fusion of these three biometric will surely produce good performance in the multimodal system. The methods used for feature extraction are Scale Invariant Feature Transform (SIFT) algorithm introduced by Lowe in 2004 to solve the scaling, image rotation and affine deformation, illumination changes, viewpoint change and has strong robustness. Speed Up Robust Features (SURF) algorithm, introduced by Bay et al in 2006 is quite robust to noise and error detection, and invariant to photometric changes like scale, rotation, etc. It is fast and produce good performance than SIFT. By this achievable benefits, SIFT and SURF can performs well and generate good features for the recognition system.
Researchers showed better authentication performance by the use of multimodal biometrics [3] . Lin et al. [4] obtained the features based on Speeded Up Robust Features and PCA to establish the local descriptors. The feature classification stage contained two steps in which the K-means algorithm was applied to gather the local descriptors and then the local and global similarities were combined to classify the face images. The ORL face database was employed for the performance evaluation. Nagesh kumar et al. [5] used palm print and face biometric and produced an efficient secure multimodal biometric system. Fusion of Palmprint and Face features improved the person authentication. Lin et al. [6] implemented an algorithm which combines Principal Component Analysis (PCA), Scale Invariant Feature Transform (SIFT) and gradient features for face recognition. Different local gradient descriptors were extracted by SIFT in which the feature vectors were invariant to image scaling and rotation. Secondly PCA was applied for the dimension reduction and decrease in computation time. Then the K-means classifier was applied to cluster of the local descriptors. Simulation results demonstrated that PCA-SIFT local descriptors were robust to expression variations and produced better performance than other comparative methods. In addition, PCA-SIFT combination reduced the storage space and raised the computational efficiency. Zhang et al. [7] applied Scale Invariant Feature Transform (SIFT) Flow for video based human action recognition. Local appearance descriptors, local motion descriptors and motion boundary histogram (MBH) were evaluated using SIFT flows. They compared the performance using different classifier like bag-of-words approach, support vector machines, linear and nonlinear to classify the feature vectors. The developed novel approach based on key points produced competitive results when compared with other state-of-the-art results.
Asadi and Obaid [8] enhanced the performance of object detection by selecting the strongest features descriptor from the objects. The SURF algorithm was used to detect the interest points for the selected object. The specific threshold and selection criteria were employed to obtain the strongest features. The features descriptors of the biometrics were extracted by considering the pixels of an interest point. Second threshold was used to evaluate the matching feature of the object under variable conditions of rotation, partial occlusion, orientation and illumination changes. Result contained outliers matched features, RANSAC (Radom Sample Consensus) algorithm was used to remove outliers in matched feature sets. They established an initial inliers combination, computed Sampson error and compared matching pairs based on their selected threshold. Azeem et al. [9] applied Hexagonal scale invariant feature transform (H-SIFT) for feature extraction on face. Normally, SIFT based feature descriptors depends on square image pixels. Applying SIFT on square image coordinates provides less number of features across edges and it removes low contrast pixels containing discriminate facial features. The low contrast areas on face constituting the local features are highlighted and provide sharp edge response with the use of hexagonal image processing. The matching process using Fisher Canonical Correlation Analysis (FCCA) further boosted the recognition accuracy. Experiments performed on AR, ORL, Yale B and FERET datasets revealed better performances in terms of feature extraction. With this knowledge, the present system was aimed to improve the recognition performance using multimodal biometric system.

METHODOLOGY
The block diagram of the proposed multimodal system is shown in Fig 1. The different modalities used in this system are Palmprint, FKP and Face images. Each modality is processed independently. First pre-processing is performed to extract the region of interest (ROI) from every biometric image. The feature template was formed by extracting the features using Scale invariant feature transform (SIFT) and Speeded up robust features (SURF) separately for the trained dataset. Similarly for the test dataset, the features are extracted and matched with the feature templates which are obtained from trained dataset. The matching scores are obtained by considering the distance between the feature points of the sample image with the template. Matching scores from the three unimodal biometric systems are combined to form a unique matching score. The techniques used for score level fusion are MAX, MIN, PROD and SUM. If the fused matching score (MS) is greater than the threshold T then it is genuine, otherwise imposter distribution. This improved design takes the advantage of proficiency of each unimodal biometric system.

FEATURE EXTRACTION
In the present study, SIFT and SURF algorithms are employed for the person recognition system. These algorithms are used to extract the features from the images of various biometrics such as Palmprint, FKP and Face. Extracted feature vectors are found to be distinct, robust to rotation, robust to scale and invariant to illumination. Thus, features of test images can be matched accurately with high possibility against features extracted from the database images.

SCALE INVARIANT FEATURE TRANSFORM
The main design of SIFT algorithm is to extract features from images to achieve reliable matching among the images. The SIFT feature extraction process can be described with the following steps shown in  The scale invariant feature transform is described by Lowe et al. [10]. The Initial step of SIFT algorithm is to build the scale space representation. Gaussian scale space of an input image can be obtained by convolving the input image with gradually raised Gaussian variances. The Difference of gaussian is determined on the basis of the Gaussian scale space, by subtracting adjacent image scales. Each pixel is compared with its neighbours in the same level as well as in the two neighbouring lower and higher levels. Extreme keypoints are found by considering whether the pixel is maximum or minimum of all neighbouring pixels. The locations which are unstable and low contrast along edges are discarded. Then, orientation is assigned to each of the resultant keypoints. Create the feature vectors (descriptors) from the resultant keypoints. SIFT descriptor is obtained by dividing the whole region into 4×4 local region, then finding histograms of gradient orientations for each region and finally concatenating these histograms together which is further normalized to an unit vector as the descriptor. SURF [11] is an efficient method for key-point detection and descriptor construction. Feature vectors of SURF are formed by means of local patterns around key-points which are detected using scaled up filter. The major process utilized to determine the SURF feature vectors of the images are observed in  The benefit of integral image is fast computation of the sum of the intensities over rectangular region. The SURF points were detected by Hessian matrix approximation using box filters for Gaussian second order derivatives. Keypoints are localized by applying non-maximum suppression in a 3×3×3 neighborhood. In order to make the SURF descriptor invariant to image rotation, a reproducible orientation is first computed based on the information from a circular region around the interest point. This stage describes the keypoints. Then find the dominant orientation based on information around the interest point from a circular region. From the oriented square image region (local) around keypoint, feature vector of 64 values are computed.

MATCHING PROCESS FOR RECOGNITION
For the recognition, feature sets of all the images in the database are matched with the feature set of the query image. Fig 4a represents the feature points of the FKP samples . For Unimodal, the K nearest neighbour (KNN) and support vector machine (SVM) are used to classify the images of the trained data with the test data. Point group matching for both SIFT and SURF algorithm was observed in Fig 4b and 5 to get the matched output (genuine) or unmatched output (imposter) . In multimodal, the matching scores are found by using Euclidean distance. The distance between the feature points of query images and the template is calculated. If the matching pairs are more between the images means, greater will be the similarity. A person with different subject images is considered for the database. The matching between different subjects of same user is called genuine. While matching between different users is called imposter.

SCORE LEVEL FUSION FOR MULTIMODAL SYSTEM
Score level fusion refers to the fusion of matching scores provided by single biometric system. This system is experimented with four different fusion schemes: Max-rule (MAX), Min-rule (MIN), Product-rule (PROD) and Sumrule (SUM). These fusion approaches are widely used by experts in different system. The multimodal system provides ease implementation, conceptual simplicity, practical aspects, etc. MAX fusion method chooses the maximum of the three unimodal biometric scores as the multimodal score value.

PERFORMANCE MEASURE OF THE BIOMETRIC SYSTEMS
The performance of the biometric verification system is measured by False Acceptance Rate (FAR), False Rejection Rate (FRR) and Genuine Acceptance Rate (GAR) [12,13]. The formula used to find the performance of the system is given by FAR= α / β × 100 Where α = Number of accepted imposter β = Total number of imposter access FRR= γ / µ × 100 Where γ = Number of rejected clients µ = Total number of client access Genuine acceptance rate is defined as a percentage of genuine users accepted by the system. GAR=1-FRR Equal error rate is the point where false acceptance rate and false rejection rate are optimal. EER = (FAR+FRR) / 2 The efficiency of the proposed method is computed by using the formula Accuracy = 100-(FAR+FRR) / 2

RESULTS AND DISCUSSION
In this paper, the proposed analysis has been done with Palmprint, Finger knuckle print and Face images which are obtained from Hong Kong Polytechnic University Data set. The feature extraction algorithm namely SIFT and SURF was employed to get the feature set of the test images and further matched with the feature template which helps the identity of the individuals. The performance of SIFT and SURF are computed using FAR, FRR and GAR. For analysis ten samples of hundred users totally 1000 Palmprint images, 1000 Finger knuckle print images and 1000 Face images are used to train the verification model. Ten samples for each user are used as test data. Unimodal system is used for recognition using KNN and SVM classifier. An experiment is conducted to evaluate the performance of the Palmprint, FKP and Face. SIFT and SURF algorithm is used to extract the features of Palmprint, FKP and Face for recognition.    Table 2). The genuine acceptance rate for MAX rule is 0.9877 and for PROD rule is 0.9853.    The accuracy graph of multimodal system using SURF was shown in Fig 9.The multimodal system scored the best performance with 97.475 % accuracy for MAX, with EER= 2.615%. The poor performance of 96.765% accuracy for PROD, with EER = 3. 235%   Fig 9: Accuracy graph of the multimodal system using SURF In the evaluation of the system performance, palmprint as input of biometric data, the results expressed as a FAR and FRR depending on the threshold values is plotted in Fig. 10 a. the genuine and impostor distributions are plotted in Fig.  10 b. The system performance can be depicted in the form of a ROC curve (Fig. 10 c). The recognition system achieves an EER equal to 3.255% and a maximum Genuine Acceptance Rate (GAR) equal to 97.49 %.
(a) (b) (c) Fig.10: Unimodal verification results for Palmprint using SIFT and SVM 10 (a) The ROC curve of FAR Vs FRR for Palmprint using SIFT +SVM ; 10(b) The genuine and impostor distribution for Palmprint using SIFT +SVM and; 10(c) The ROC curve of FAR Vs GAR for Palmprint using SIFT +SVM System performance is calculated for SURF algorithm using palmprint as input of biometric data, the results expressed as a FAR Vs FRR is plotted in Fig. 11 a. The genuine and impostor distributions are plotted in Fig. 11 b. The system performance can be represented in the form of a ROC curve (Fig. 11 c). The recognition system achieves an EER equal to 2.75% and a maximum Genuine Acceptance Rate (GAR) equal to 97.96%. The proposed multimodal system uses MAX, MIN, PROD and SUM rules for fusion at score level. Similar to that Fouda (2012) also employed the SPR (Sum Product Rule) rules for fusion and achieved 89.12% accuracy. In the present study, maximum percentage of accuracy is achieved in MAX rule. The multimodal system produces better results than unimodal. This multimodal system based on SURF algorithm produces high accuracy of 97.475% and minimum error rate of 2.525%.
(a) (b) (c) Fig.11: Unimodal verification results for Palmprint using SURF and SVM 11(a) The ROC curve of FAR Vs FRR for Palmprint using SURF +SVM; 11(b) The genuine and impostor distribution for Palmprint using SURF +SVM ; 11(c) The ROC curves of FAR Vs FRR for Palmprint using SURF +SVM The ROCs shows the performance of multimodal using SIFT algorithm for different fusion rules are shown in Fig.  12 a. The ROCs from various fusion rules suggest that MAX rule has performed well. As expected, the MIN and PROD rule shows poor performance than other rules Fig. 12 b shows the genuine and impostor distributions when using the method of MAX rule. Finally, Fig. 12 c shows the fusion at the score level by using MAX rule of Palmprint, FKP and Face, GAR was 98.77 %.
(a) (b) (c) Fig. 12: Multimodal verification results using SIFT 12(a) The ROC curve of FAR Vs FRR based on score level fusion using SIFT; 12(b) The genuine and impostor distribution for multimodal by SIFT and ; 12(c) The ROC curve for the fusion scheme using SIFT The ROCs shows the performance of multimodal using SURF algorithm for different fusion rules are shown in Fig.  13 a. The ROCs from various fusion rules suggest that MAX rule has performed well. As expected, the MIN and PROD rule shows poor performance than other rules.  The previous studies on the recognition revealed the various feature extraction methods viz., Gabor, Log Gabor, Sobel, DLBP, DFT, DCT, Wavelet transform, PCA, ICA, FDA, Contour transform, SIFT, SURF etc. In the present study also SIFT and SURF algorithm is employed for the feature extraction from palm, finger knuckle print and face. To improve the identification efficiency various classifiers viz., KNN, SVM, Neural network, Hamming distance, Pixel to Area match, Angular distance etc were applied. In this system, KNN and SVM classifiers are applied to classify the images for recognition. Wu et al. [14] also employed the SIFT for feature extraction from Palm biometric. Chennamma et al. [15] identified the manipulated facial images with 92% accuracy using SIFT. Comparative analysis of the Existing system for various feature extraction with different biometric trait was shown in Table 5.The recognition rate and Equal error rate have been discussed for the analysis. also recognised the Face using SURF and SVM and obtained the accuracy of 93.60% with scale 2. As the results of unimodal is compared with multimodal, the multimodal produces better results of about 97.475% accuracy for SURF. CONCLUSION This paper presents the unimodal and multimodal recognition system using Palmprint, Finger knuckle print and Face. To reduce the existing drawbacks of single biometric modal, an integrated approach has been used to improve the recognition. The features are extracted using SIFT and SURF algorithm. The SVM classifier produces better results in the recognition system. The multimodal integrated the matching scores of three biometric traits Palmprint, FKP and Face. The fusion techniques MAX, MIN, PROD and SUM are used to fuse the matching score in the multimodal system. In the present study maximum percentage of accuracy was achieved in Max rule. This multimodal system using SURF achieved effective accuracy of 97.475% and error rate of 2.525%. The results showed that the proposed scheme achieved high level security, good efficiency and less error rate. The proposed system can be employed for the individual recognition in the digital system like smart card, passport and surveillance system.