Authentication is becoming an increasingly important application in the connected world and is driven by the growing use of mobile and IoT devices that use an increasing number of applications that require transactions of sensitive data. Security usually relies on passwords and/or two-factor authentication which are too intrusive for daily use. Biometric solutions such as fingerprints, voice, iris and retina are a good alternative to overcome previous problems. In this project an audio-visual identity verification is presented, where the use of multiple modes that can already be captured from most IoT devices (microphone and camera) make authentication robust to adverse conditions. End-factor analysis (i-vectors) with cosine distance is implemented as the main classification algorithm which takes into account variations within and between speakers. Mel Frequencies Cepstrum Coefficients (MFCC) are used as audio features, 2D-DCT coefficients of a single snapshot and Motion Vectors (MV) of the lips are extracted for visual features. Improvements combining different modes are shown using VidTimit dataset where the proposed algorithm achieves 0.7% of Half Total Error (HTER) in the test set outperforming single modes audio and visual by 9.5% and 6.4%, respectively.

Biometrics authentication has been very useful and necessary nowadays due to the great developments in technology and the transaction of huge amounts of sensitive data on a daily basis. Traditionally, access to some data or service is achieved by means of some documents or a password. However, these methods are not very convenient. Alternatively, typical biometric systems can be employed that use fingerprint, iris, voice, face recognition or a combination of them. This project focuses on the task of face recognition from still images and investigates how different algorithms for face verification perform under various adverse conditions modelled by blur, salt-and-pepper noise and changes in illumination. Conventional pattern recognition algorithms are first presented. Pixel intensities, Gabor features, Local Binary Patterns (LBP) and 2D-DCT coefficients are considered as features while for classification the nearest neighbor (NNC), nearest mean (NMC), SVM classifiers, and Likelihood Ratio Tests (LRT) with Gaussian Mixture Models (GMM) are examined. Out of all these methods, Gabor features combined with the linear SVM classifier are shown to produce best results across all degradations giving an average Equal Error Rate (EER) of 0:97% using the ORL face dataset. Then, emphasis is placed on deep learning and Convolutional Neural Networks (CNN). Specically, VGG-Face with triplet loss training for face verification is suggested. VGG-Face achieves an average EER of 2:63% when both test images of a query image pair are drawn from the same degradation conditions and an average EER of 3:80% when only one image in the given pair is degraded and the other one is derived from the clean ORL dataset. We also experimented with the extracted VGG-Face features and NNC, linear SVM and Gaussian SVM and it is seen that a linear SVM gives an average EER of 1:10% by macro-averaging the Detection Error Tradeoff (DET) curves.