Individuals with Autism Spectrum Disorders (ASD) present deficits, or impairments when processing faces and categorize emotional expressions, in comparison to their neurotypically developed peers. Previous studies have indicated that an atypical local-oriented strategy while processing faces might be one reason for these impairments. 

In our research, we aimed to investigate how stylization of the faces of virtual characters can influence the categorization of emotions by children and adolescents with ASD and ADHD. To that end, our partners at the University of Konstanz developed non-photorealistic rendering (NPR) algorithms that are not only parameterizable, but also abstract the geometry of our characters in real-time. These algorithms were implemented using Frapper. The NPR styles used are:

Image Abstraction

This style is inspired by paintings and painting-like images, where absence of fine-grained texture details and increased sharpness of edges are two relevant visual characteristics.

Left: Hank (medium & high abstraction). Right: Nikita (medium & high abstraction)

Coherent Line Drawing (CLD)

Line drawing effectively conveys shapes and outlines to the viewer with simple primitives: lines. We use the algorithm of Kang et al. [2007] which creates coherent and artistic-looking lines.

Left: Hank (medium & high abstraction). Right: Nikita (medium & high abstraction)

Pencil Drawing

This is one of the most fundamental techniques in visual arts to abstract human perception of natural scenes. By converting input scenes into just lines and shading, a great number of details are removed, while keeping the object boundaries and plasticity of the rendered objects by preserving their shading.

Left: Hank (medium & high abstraction). Right: Nikita (medium & high abstraction)


Watercolor painting is an artistic style that creates the effect of water dissolved colors on paper, or similar surfaces.
We decided to use this style not only for the abstraction it provides, but also because it is used by many individuals with ASD to express themselves through painting, or during therapies that introduce artistic elements [Tataroglu 2013].
Our algorithm is based on the work of Luft et al. [2006] whose approach simplifies the visual complexity and imitates the natural effects of watercolor.

Left: Hank (medium & high abstraction). Right: Nikita (medium & high abstraction)

Loose & Sketchy (not tested in SARA)

Another algorithm initially implemented as a collaboration between Filmakademie Baden-Württemberg and University of Konstanz, and recently improved by the latter was Loose & Sketchy.

To improve the visual quality, instead of drawing thin lines from the seed points along the vector field, real strokes are generated. For this, the geometry shader stage in GL3+ is used to create a quad-strip with texture coordinates and apply a real stroke texture. This is done is 2 steps: once from the seed position in direction along the vector field and once from the seed point against the direction of the vector field.

This is still work-in-progress.