Estee Lauder

This was a computer-vision based, virtual try-on app for Estee Lauder makeup In 2014, before Lidar-equipped smartphones were a thing, I worked on an iPad app that used Computer Vision, Structure From Motion, and other tricks to create a fairly accurate 3D model of a persons face from a 180 degree face scan using the front-facing camera. Utilizing a pre-modeled human head created from 200 real human heads to get an average, we were able to displace the vertices of the average model to form fit the new subject. We unwrapped the UVs and then we had a full 3D model of the person's face. Mixing that with some clever OpenGL shaders and BAM, we had built an incredibly accurate, virtual try-on, makeup app. We worked with the chemical engineers from Estee Lauder to understand the physical light properties of their makeup ingredients and translate those into pixel shader code. For 2014, the result was stunning.