Skip to content

SLTAT 2013 – Weeks 5 through 7

October 28, 2013

This quarter has been a very exciting one, especially regarding research and development. I have thoroughly used and abused the program we used to create the expressions – a simple series of buttons and sliders to adjust the points of articulation and bones in the face. Unfortunately, this set up did not take into account the way that the human face does, in fact, push tendons and skin to the limit for even basic emotions.

Image 

For instance, when attempting to create a jaw drop, or in my case, a “happy” surprise versus a fearful one, Paula has issues baring her teeth – and if the jaw drops too low, the center of her upper and lower lip stick to her teeth. Her eyebrows also have a limit, as do the corners of her lips. Furthermore, her eyeballs are actually poking out of the sockets a bit – that’s what those dimples are near the puffy parts of her eyes! It sets a challenging standard for testing this software. I have pushed beyond its boundaries, and this is something that has been brought up in…

Image

SLTAT 2013! A neat symposium hosted this year at DePaul to check out the current work from many different groups around the world, all seeking to find a way to synthesize signed language.

This happened October 18th – 19th with about 30 or so people showing up to reveal their current work. To be honest, a lot of the technical details went over my head, but the depth of research is startling. Me, I just enjoy playing with the human face and finding what exactly creates uncanny valley vs. a naturalistic avatar. Discovering the nuances of coarticulation is a whole new realm of territory that I am nervous about, but excited to dip my toe into.

My partner, Marie Stumbo, put together a video and mounted her tablet onto our poster for our own presentation of our work – optimizing Paula, our avatar, for real-time rendering capabilities. I created a couple videos, which are posted below in three separate posts and linked on Vimeo, to demonstrate what I have done to help with that and now how making expressions without morph targets is absolutely possible.

The lighting rig was a piece of work I enjoyed playing with earlier in the summer. Simply put, lighting is taxing on the processor, and definitely forces rendering time to suffer. Combine this with a high polygon count and you may be sitting and waiting for something to render for hours or days! In our case, the original rig was a set of six symmetric lights that left Paula neutrally lit and with a shadow down the bridge of her nose, which almost created a line of symmetry.

I couldn’t do the traditional three point lighting scheme due to the lack of true ambient light in a computer setting; this caused Paula to have dark shadows on her face, which is too dramatic for our purposes. Instead, I created a four point lighting scheme and adjusted the glow and bloom of the one spot light I used in order to bring more saturation and color into her face, and also to remove the shadow going down the bridge of her nose. I’d like to think the results were pretty decent!

In my opinion, the expressions video was more interesting to look at. 3DS Max gave me issues with putting holds/pauses between most of the expressions until the last two or three at the end, but the results still surprised and delighted the colleagues who looked at the video. Each person was impressed by the results of a bone and articulation-based system instead of pure morph targets.

However, every single person brought up a point: the eyebrows. If you look at the video, you can notice how red/dark shading near her eye sockets and around her brows seems to jump sporadically. This is an issue with the texture map coded on the skin. As far as I understand, and pardon me if this explanation is lacking or a bit incorrect, but the map is based on opacity. It will show more or less depending on the push or pull of the brow bones.

Because I had pushed the expressions to the extreme (for the software), the texture map became confused and jumped, creating an ugly look that needs to be sorted out. That said, that is my next point of research and development: re-programming and playing with the texture map for the brows, so more fluid motion and transition can be achieved! I also hope to play with the eyes a bit to fix the issues of the corners popping out near her lower eyelids.

I am very excited for my upcoming work with the texture mapping. I hope this will allow for an even wider range of emotions that are more naturalistic – I love what I do, and I am always excited to improve upon what I have done.

Advertisements

From → Uncategorized

Leave a Comment

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

w

Connecting to %s

%d bloggers like this: