Overview

My goal for this project was to produce successful “face morphs” between different people. Throughout the project, I utilized techniques like Delaunay triangulation, affine transformation, and interpolation to create face morph animations, compute the mean face of a population, and develop a caricature of my own face.

Section I: Defining Correspondences

I chose to morph these two photographs of Andrew Garfield and Jesse Eisenberg, both taken by Martin Schoeller.

To start, I used the provided tool to select points corresponding to key facial features like the eyes, nose, mouth, and face shape; I added additional points in the corners of the images. I generated Delaunay triangulations based on 120 correspondence points from each image. I also averaged the two sets of correspondence points to find the midway shape, the triangulation of which I'll use for the morph sequence.

Section II: Computing the "Mid-Way" Face

Equipped with triangulations for the source and target images, I was ready to compute the midway face. Treating them as target triangles, I iterated through each triangle from the midway triangulation and calculated the affine transformation matrix mapping source vertices to their corresponding target vertices.

Since direct warping can leave gaps, I performed inverse warping by using the inverse transformation matrix to map the target triangles to the source triangles. Generating a polygon mask for each triangle allowed me to warp all of its pixels at once instead of iterating through them manually.

Finally, given the warped triangles, I used nearest neighbor sampling to populate the pixels of the warped image with the corresponding colors from the original; cross-dissolving the two images produces the final midway face.

I only included correspondence points on their faces, so there’s a tiny bit of artifacting in the warped image of Andrew Garfield. However, the midway face still does a pretty good job of blending both Andrew Garfield and Jesse Eisenberg’s features, most notably the eyes, chin, and mouth.

Section IV: The "Mean Face" of a Population

I applied techniques from the previous parts at a larger scale by computing the average face from the Danes dataset, which contains 37 images. I first parsed through the .asf files to obtain the correspondence points from each image, then averaged all of the points together to obtain the average shape overall. Following my procedure from Part II, I warped each face to the average shape and cross-dissolved all of the results together to produce the final composite face. I also took the average faces from the male (30 photos) and female (7 photos) subsets, and found that the average male face overpowers the overall average.

IMM Face Database

The average Dane.

IMM Face Database (Female)

IMM Face Database (Male)

Finally, I warped my own face to the average shapes and vice versa. Though warping my face didn't go so well, the average faces mapped relatively well to my face shape.

My face.

Section V: Caricatures: Extrapolating from the Mean

I can also produce caricatures of my own face by extrapolating from the population mean obtained in the previous part. To do so, I warp my face to the shape determined by the formula alpha * mean_face + (1 - alpha) * my_face, where alpha is a constant value that represents the intensity of my unique features. A negative alpha value accentuates my facial features, while a positive alpha value decreases the strength of my own features and increases those of the average.

Section VI: Bells and Whistles

Morphing Music Video on a Theme

I was inspired by the film Everything Everywhere All At Once to create a morphing music video between 12 different multiverse variants of Evelyn. I selected 38 correspondence points for each image, created morph sequences between different pairs of images, and combined the sequences to create the final video linked below, which consists of 286 unique frames.

Morphing Music Video With Classmates

I participated in a morphing music video with Rohan Gulati, Shuai Zhou, Kelly Yeh, Jaewon Lee, and Jennifer Yin, other students enrolled in this class. Each of us selected 43 correspondence points for our own face and created morph sequences between our face and everyone else's; Jaewon Lee then edited all of our morphs together. The entire sequence contains 11916 frames.

Changing Gender

Finally, I experimented with changing my gender based on the average male face from this collection. I manually cropped and aligned the images, picked correspondence points, and followed the procedure from previous parts to morph my face to male. I created morphs using only the midway shape, only the appearance, and both, which I think produced the best result.