How to adaptively add and use face images collected while authentication to improve performance of face authentication??

Hi, My current project is to build a face authentication system. The constraint I have is: during enrollment, the user gives single image for training. However, I can add and use images given by the user while authentication.

The reason I want to add more images into training is, the user environment is not restricted - different lighting conditions, different distance from camera, from different MP cameras. The only relief is the pose is almost frontal.

I think, the above problem is similar to the face tagging app widely available. Can anyone suggest a method to use the available images adaptively and smartly??

--Thanks