Computer Graphics 2015: Facial Animation through Reverse Engineering of Actions to Thought Process

I propose a method where facial animation for characters can be derived as a result of reverse engineering from the final action on the storyboard to the thought train driving the action. For this particular process, we classified the actions into conscious action, subconscious action, and unconscious actions, and derive the lesser obvious subconscious and unconscious parts leading to the conscious action. We begin by analyzing the situation at hand, and how it applies to each character in it. Then we use the storyboards to understand the primary action of the character. Now, we study the facial character, i.e., his expressions, and the body language, i.e., the line of action and the pose. Then we proceed to analyze the possible symmetrical or references to the past of the character that could drive the action. Here, we try to reason things he might have seen or heard and his own internal reasoning that lead to his interpretation of the situation and the consequent action. Finally, we derive the inner-monologue of the character simulation that drives the action. Once we finish the reverse engineering process from the action in the storyboard to the thoughts and emotions in it, we map the eye darts, blinks, eyebrow movement, leading actions, and its required anticipations within the time frame stipulated by the storyboard. This method of reverse engineering-based animation results in greater cohesive acting throughout a film and creates greater connect with the audiences. In this paper, we present the first Facial Action Coding System; a valid model to be based on dynamic 3D scans of human faces for use in graphics and psychological research. The model consists of FACS Action Unit based symmetrical parameters and has been independently validated by FACS experts. Using this model, we explore the perceptual differences between linear facial motions -- represented by a linear blend shape approach -- and real facial motions that have been synthesized through the 3D facial model. Through numerical measures and visualizations, we show that this latter type of motion is geometrically nonlinear in terms of its vertices. In experiments, we explore the perceptual benefits of nonlinear motion for different AUs. Our results are insightful for designers of animation systems both in the entertainment industry and in scientific research. They reveal a significant overall benefit to using captured non-linear geometric vertex motion over linear blend-shape motion. However, our findings suggest that not all motions need to be animated non-linearly. The advantage may depend on the type of facial action being produced and the phase of the movement. Using this model, we explore the perceptual differences between linear AU facial motions – represented by a linear blend shape approach – and the same AU facial movements as observed from a real face. These real movements are those captured using the dynamic3D facial scanner, and are re-synthesized through the facial model. Such a perceptual study is important since linear blend shape animation approaches are highly popular in animation, whereas the natural facial movements during expressions are typically geometrically nonlinear as opposed to linear. This geometric nonlinearity is well known, and physical wrinkling and elasticity models are often employed to approximate the motion. Using our 3D dynamic scanner, we capture those geometric nonlinearities and then recreate them in our model. 3D stereo capture approaches can overcome many of the limitations of sparse optical motion capture. Zhang et al developed a real-time 3D stereo surface acquisition technique and used this to capture both nonlinear geometric surface facial deformations as well as color texture during a performance. The color information observed contains expression wrinkle detail too subtle to capture during the stereo acquisition process. Ma et al more recently developed a photometric stereo based capture system capable of observing nonlinear changes in skin pores andfine-scale wrinkles. Our acquisition device consisted of a 3DMD active stereo 3D capture system. This uses a projected infra-red speckle pat-tern to calculate stereo correspondence and produces an accurate3D surface reconstruction. The system has a capture rate of 60Hz, and therefore provides smooth temporal acquisition of fast facial movements. Each raw facial surface scan contains approximately30K vertices along with 1280 x1024 pixel color UV maps. Table 1lists the captured AUs and provides notes on their validation as per-formed on the output of the animation model. Note that matching AU target and validation codings indicate that the animation modeling pipeline (Section 3) is faithfully preserving the AU and not corrupting its representation. Figure 1 shows images of AU peaks from each recorded sequence as captured by one of the color acquisition cameras in the stereo capture system. Note that while validation was only applied to the peak image, we ensured that no other AU interference occurred during the capture. Note that our decision to use FACS as the basis for facial animation in our experiments was that it provides the clearest description of official movement available to researchers. It, therefore, allows replication of our experiment with different facial models and performers and can be used independently by researchers in various fields. In this experiment, we focused only on single AUs as this is the principle adopted by many facial model designers – whether creat-ing models entirely artistically or capturing facial movement from actors using dynamic 3D systems such as MOVA. Thus, knowing which AUs could benefit from nonlinear geometric movement could help an animator to focus on realism on certain facial controllers. However, we acknowledge that a comparison of linear and nonlinear motion types in the perception of more complex expressions (e.g. anger, fear) would be worthwhile, and is a direction for future work.
Author(s): Anandh Ramesh

Abstract | PDF

Share This Article