Entertainment Computing

Planar character animation using genetic algorithms and GPU parallel

computing

q

Benjamin Kenwright

a r t i c l e

i n f o

Article history: Received 8 March 2014 Revised 23 September 2014 Accepted 24 September 2014 Available online 5 November 2014 Keywords:

Animation Control Interactive

Graphical processing unit (GPU) Computer games

Genetic

a b s t r a c t

The emergence of evolving search techniques (e.g., genetic algorithms) has paved the way for innovative character animation solutions. For example, generating human movements without key-frame data. Instead character animations can be created using biologically inspired algorithms in conjunction with physics-based systems. While the development of highly parallel processors, such as the graphical pro-cessing unit (GPU), has opened the door to performance accelerated techniques allowing us to solve com-plex physical simulations in reasonable time frames. The combined acceleration techniques in conjunction with sophisticated planning and control methodologies enable us to synthesize ever more realistic characters that go beyond pre-recorded ragdolls towards more self-driven problem solving ava-tars. While traditional data-driven applications of physics within interactive environments have largely been confined to producing puppets and rocks, we explore a constrained autonomous procedural approach. The core difficulty is that simulating an animated character is easy, while controlling one is more complex. Since the control problem is not confined to human type models, e.g., creatures with mul-tiple legs, such as dogs and spiders, ideally there would be a way of producing motions for arbitrary phys-ically simulated agents. This paper focuses on evolutionary genetic algorithms, compared to the traditional data-driven approach. We demonstrate generic evolutionary techniques that emulate physi-cally-plausible and life-like animations for a wide range of articulated creatures in dynamic environ-ments. We help address the computational bottleneck of the genetic algorithms by applying the method to a massively parallel computational environments, such as, the graphical processing unit (GPU).

Ó2014 Elsevier B.V. All rights reserved.

1. Introduction 1.1. Preface

How do creatures move in the real-world? For example, how do we as humans move? Can you explain it? While we are able to simulate dynamic rigid body worlds (e.g., by means of unintel-ligent solid objects moving due to forces and collisions) in interac-tive virtual environments, such as video-games and training simulations, the control of complex physics-based character systems is an active area of research. The traditional approach of creating character animations in interactive worlds is through data-driven solutions (i.e., pre-recorded animations) that are blended and applied to rigid body structures. Alternatively, a number of approximate procedural approaches are available, that

include using low-dimensional systems (e.g., inverted pendulum or optimization search algorithms). These low-dimensional mod-els allow the animation to run in real-time while mimicking the fundamental dynamic principles of real-world characters (e.g., balanced stepping logic). Hence, creating animated character ani-mations using procedural techniques is highly challenging, inter-esting, and important.

1.2. Motivation

Static poses (i.e., motion-less poses) can be used to control pro-portional-derivative controllers. However, in dynamic environ-ments, where the character is in motion (e.g., running and stepping), the timing information might not correlate between the aim and target. For example, if a character is taking a step, the character’s foot will miss the target, and in interactive environ-ments, such as games, this can be unacceptable. Hence, we need to constantly create poses on-the-fly that can accomplish diverse tasks while being able to adapt to the environment around them

http://dx.doi.org/10.1016/j.entcom.2014.09.003

1875-9521/Ó2014 Elsevier B.V. All rights reserved. q

This paper has been recommended for acceptance by Ryohei Nakatsu. (Where Ryohei Nakatsu is the name of the Handling Editor).

E-mail address:[email protected]

Contents lists available atScienceDirect

Entertainment Computing

(e.g., changing terrain types, such as, slopes, bridges, and pot-holes).

1.3. Challenges

The key challenges that make it difficult to reproduce physi-cally-plausible life-like movements in ‘‘real-time’’ that mimic real-world creatures are given below. We also point out where an evolutionary genetic algorithm sits in relation to each problem.

Realism: A particular character model gives rise to a large set of possible motions with different styles. While the physics-based system enforces the control laws, the genetic algorithm has the challenging task of searching for poses that re-con-struct and reproduce the intricate and agile movement we observe in nature.

High dimensions: Characters have a relatively high number of degrees-of-freedom, making the search for the appropriate control parameters hard. The search space can be vast and non-continuous while containing multiple solutions. Although

single threadedgenetic algorithms can cope with large search spaces, the stringent demands make it impossible for interac-tive applications or even performed in reasonable time-frames.

Underactuation: Dynamically simulated characters are difficult to control because they have no direct control over their global position and orientation. Even staying upright is a challenge for large disturbances. In order to succeed, the genetic algo-rithm control law must plan ahead to determine actions that can stabilize the body (e.g., not just a static moment in time, but will have to the run simulation forwards to predict how the movement will evolve based on the physical constraints)

[9].

Contacts: Characters are restricted to move within a certain region of its three-dimensional environment, and these con-straints are difficult to maintain in real-time control systems. Furthermore, frequent ground contacts create a highly discon-tinuous search space rendering most condiscon-tinuous controller synthesizing methods ineffective at planning over longer time horizons.

1.4. Contribution

The contributions of this paper are: (1) We introduce and dem-onstrate a procedural approach for synthesizing the evolution of artificial-life animations without data (e.g., key-frames) through a genetic based search algorithm (i.e., an evolving survival of the best approach). (2) We create physics-based character motions that adapt and explore complex environments (e.g., we go beyond flat terrain and include stairs and slopes). (3) We extend the tech-nique to non-biped type avatars (e.g., creatures with multiple legs, such as cats and snakes). (4) We explore performance improve-ments through exploitation of massively parallel execution envi-ronments, such as the graphical processing unit (GPU) for evolving populations. (5) We discuss practical aspects, such as fit-ness functions and controller models, for generating the joint con-trol torques of the articulated character.

2. Related work

Evolving genetic algorithms have been used across multiple dis-ciplines to solve a wide variety of problems (e.g., robotics, biome-chanics, and computer animation). While there is a plethora of exciting and interesting research in the area of procedural

animation, we review and compare essential research that is spe-cifically related and has inspired this work.

2.1. Where did it all start?

The idea of using a population of solutions to solve practical engineering optimization problems was considered several times during the 1950’s and 1960’s[5]. The concept of a Genetic Algo-rithm (GA) was coined by John Holland and his colleagues and stu-dents at the University of Michigan in the 1970s[12]. However, GA was originally popularized by one of John Holland’s students, David Goldberg, who was able to solve a difficult problem involv-ing the control of gas-pipeline transmission for his dissertation in 1980s[7].

2.2. Space–time constraints

Space–time constraints, originally proposed by Witkin and Kass

[17], allows an artist to specify how an animated figure should move but not how to do it. This approach is very appealing in ani-mation. Witkin and Kass [17] original space–time constraint method generated kinematic motions that satisfied both high level goals (e.g. jump from here to there) while appearing physically plausible. Furthermore, the resulting motions exhibited many of the principles of traditional animation (e.g., squash, stretch, fol-low-through, and anticipation). Space–time constraints use sequential quadratic programming (SQP) to optimize the kine-matic positions of an articulated figure, using energy consumption as an objective function that constraint the solution in order to ensure that the motion is physically plausible. Unfortunately, this method generates solutions that depend on an initial guess given to the optimizer, which can unfortunately converge on a local min-imum that creates a kinematic motions that appears unnatural. In addition, it is by no means clear that energy is the best criteria to optimize.

2.3. Automation

Other researchers proposed methods of automatically generat-ing walkgenerat-ing motion usgenerat-ing search and optimization techniques

[16,10]. Both of which have demonstrated impressive results; yielding physically plausible motions and automatically finding a number of walking methods. However, the resulting motion had high specialization (i.e. they made walking gaits, not general motions), but low specificity (i.e. they both simply walked forward, rather than having more specific instructions, like, walk to position X). These methods were optimizing structures of fixed complexity (a network of fixed topology and a stimulus response table, respec-tively). We believe that this is a disadvantage compared to a repre-sentation which may change its own complexity. We chose a different representation, namely a mathematical description (com-puter program) describing how the joint forces vary with time and changes in the state of the simulation. We believe that this repre-sentation offers many advantages, and is appropriately optimized using the genetic algorithm technique.

2.4. Genetic algorithms

Karl Sims[13]used a genetic-like approach for evolving proce-dural textures, where the fitness metric was human evaluation. Later, Karl Sims also described the use of evolutionary program-ming to design entire creatures[14], in which even the creature topology was evolved, for example, walking and swimming. In con-trast, our work deals with figures of fixed topology and geometric structure, as one would expect for character animation.

2.5. Aesthetically pleasing evolution

Two interesting papers that used a genetic algorithm to gener-ate aesthetically pleasing characters are: Firstly, Ogura and Hagiw-ara[11]who demonstrated how a genetic algorithm could be used to create 3D faces. Secondly, Ando and Hagiwara[1]created entire characters using a genetic algorithm. These interesting systems employed a function whereby they attempted to extract rules based on the user selection, in order to try and present solutions that are aesthetically pleasing. This is a fascinating augmentation to the procedure and one that could be of interest in developing further automated procedural tools.

The impressive work by Yamamoto et al.[19]created a tool for producing diverse creatures and was explicitly aimed at creating characters quickly. The tool allowed users to pick bone chains for sections of a creature based on body part templates and join them together as well as move and otherwise alter them. This resulted in a controllable skeleton from which a creature could be designed. Their tool was able to rapidly create numerous creatures in short periods of time. Creating skeletons and altering them to produce user specified designs.

2.6. Our work

The most significant difference between our work is: (1) We exploit massively parallel execution environments, such as the GPU, to rapidly speed up the genetic animation creation tool. (2) We investigate and explore handling interactivesituations (e.g., instead of a constant fixed speed walk on a flat terrain). (3) We look at different dynamic control mechanisms (e.g., inverted pendulum) for animating the articulated rigid body creatures.

3. Genetic Algorithm (GA)

3.1. Difference between Evolutionary Algorithms (EA) and Genetic Algorithms (GA)?

Genetic algorithms (GA) and evolutionary algorithms (EA) are both popular optimization search techniques. The primary differ-ence between a GA and a EA is the crossover and mutation. EA only requires the mutation process and not the crossover. While studies of genetic algorithms without crossover outperformed genetic algo-rithms with crossover this was with very small population sizes. As Speers and Anand[15]showed the effectiveness of crossover when the population size is larger. While EA are less computationally intense compared to GA[18], the system evolves in a shorter times if the genetic algorithm uses crossover[15](seeFig. 1).

3.2. What other evolutionary approaches are there?

Evolutionary techniques are based on nature – hence, we find a number of other evolutionary concepts in the literature. A few pop-ular examples, include:

Genetic Algorithm (GA) – as discussed in this paper (i.e., uti-lizing survival-of-the-fittest principle with individual solu-tions competing to propagate to the next level).

Artificial Immune System (AIS) – pattern matching, e.g., develop antigens to detect antibodies.

Ant Colony Optimization (ACO) – graph problems that imitate ant foraging behaviours (i.e., uses pheromones to identify advantages/disadvantages).

3.2.1. Computer generated

As most of you are aware, computer games and animated films often make use of computer-generated images, known as com-puter graphics (CG). While off-line comcom-puter generated films use highly skilled animation specialists, who take great pains to create believable life-like character movements. This type of animation involves giving form to the animator’s image of a character, mean-ing that the quality of the finished animation depends on the skill and creativity of the artist. However, over the past decade comput-ers have become an indispensable tool for artists and have allowed the production of highly realistic animation solutions. Alterna-tively, for real-time dynamic scenes, such as in games, the anima-tions needs to be able to adapt to unforeseen circumstances (e.g., pushes and terrain height deviations) on-the-fly while attempting to keep the artist’s look (seeFig. 2).

3.2.2. Components

The principle components of our system are shown inFig. 3. As input to the system, the user (animator) supplies both a detailed articulated figure model and a fitness metric. The figure model includes information on sizes and connectivity of the links, such as, masses, inertias, and joint limits. The fitness metric (or objective function) implicitly encodes information about the motion which is desired from the character. The fitness metric rates the motion sequence by giving lower numbers to motion sequences which are close to what is desired by the user, and high numbers to sequences which deviate from the goals of the animation. The sys-tem itself consists of a Genetic Programming-based optimization module, and a dynamics simulation module.

When a genetic algorithm run has finished (i.e., an evolutionary epoch update), the final output of the genetic algorithm subsystem is a single controller program. This controller program is the one which, when used to govern the actions of a simulated agent, results in the best rated agent according to the user-supplied

fit-Fig. 1.Brief time-line of evolutionary genetic algorithm – [A] Population optimization Problems 1950’s and 1960’s[5]; [B] Genetic Algorithm (GA) was coined by John

ness metric. This controller program may then be used to generate the motion control needed for the animation (seeFig. 4).

3.2.3. Joint torques

We use a proportional derivative (PD) controller to generate the joint torques. The PD controller works by comparing the desired orientation of each joint with the current orientation at the time when it is evaluated. The final motions for animated characters is generated using dynamic simulation of articulated bodies. Each joint is controlled internally by a damped angular spring (i.e., a PD controller) and has a neutral (or desired) orientation which at any time is given by evaluating the PD controller at each time step of the simulation. The PD controller spring-like action pushes the joint towards the desired orientation with a torque proportional to the angle between the actual and desired orientations using a proportional derivative (PD) controller, which is a common tech-nique used in both computer animation and robotic control. The system also accounts for joint limits and collision detection and response. Therefore, we are guaranteed to get motions that are physically-correct (seeFig. 5).

3.2.4. Movement

In order to achieve automated animation (i.e., a procedural solution), it is necessary to determine the timing for each individ-ual part and what type of movement it will perform to create the final motion. The genetic algorithm is utilized in computer calcula-tions to determine such movements. Genetic algorithms have been used successfully to optimize the physical behaviours of articu-lated stick figures and other animated creatures for goal-directed behaviour. We should note, that little emphasis has been done to optimize the expressibility, whereby an animator inserts him/her-self into the optimizing loop to aesthetically influence the evolu-tionary of motion of the avatars behaviour. Our technique brings both automatic and interactive evolution together into one process as a means to bring evolutionary tools closer to the concerns of the

Fig. 2.Evolutionary principle – We can map the model parameters to binary gnomes that are used to create multiple instances (i.e., populations). The parallel instances are

evaluated based on a fitness function. The instances with the best fitness function are used to create the next generation.

Fig. 3.Motion generation – Combining genetic evolutionary search algorithms to

create interactive character animations. The character is constrained by the physical-laws (i.e., classical mechanic principles), while the controller parameters can include, centre-of-mass, environmental contacts, balance, and comfort.

Fig. 4.Balanced walking– Simple five link biped simulation, showing the different

character poses for the 60th epoch. The fitness function is to stay upright and balanced while moving forwards (i.e., with a population of 100 and the best 4 poses emphasised).

Fig. 5.Creature physiology – The flexible nature of the algorithm allows us to

experiment with various creature physiologies (e.g., multiple lets and body types). (a) biped, (b) quadruped, (c) spider, and (d) snake/worm.

character animator, who may be just as interested in developing amusing behaviours in a world of ‘cartoon-laws’, as in simulating realistic creatures in a world of Newtonian physics (seeFig. 6).

3.2.5. Genetic learning

Genetic algorithms use a fitness function to bias the output con-trol signals for different parts of the body, just as in abiological organism. The control parameters are found in order for controller to properly send outputs to the body. The theory behind this approach is based on the genetic functions found in living crea-tures. We evaluate the animated movement after a specified amount of time and the genetic algorithm evolves and promotes the best individuals with the highest fitness function. Through adoption of this technique, we can automatically generate ani-mated movements that meet certain requirements that are physi-cally plausible and mimic the real-world (e.g., remaining upright and balanced while walking) (seeFig. 7).

3.2.6. Artist creativity – directors perspective

Genetic algorithms explore the possibilities. They encourage creativity. They allow users to see motions that they never thought were possible. The generated motions are bound by the laws of mechanics which ensure the final motions are physically plausible. Traditionally, an artist painstakingly creating the pose for the char-acter during motion transitions (i.e., specifying each joints angle). For a genetic algorithm the artist specifies a fitness metric. For example, the fitness metric could be the height of the body, the walk speed, or a specific pose. The genetic algorithm uses a proba-bilistic search approach based on a best solution over multiple evo-lutionary updates that most closely meets the artists demands.

Instead the artist becomes more like a film director. Giving orders while the genetic algorithm shows the multiple results. For exam-ple, even a simple animation, such as walking, can produce a mul-titude of solutions. The fitness criteria could be that the character should remain upright and balanced while moving forwards. How-ever, the character could walk at different speeds, swing their legs differently, even slouch, slide, or hop all meet the fitness criteria if it is too abstract. The genetic algorithm tool allows an artist’s cre-ative juices to flow more freely. The artist can explore and search for diverse motions that meet specific fitness constraints.

Algorithm 1. Classical Genetic Algorithm.

1: Initialize parameters 2: Generate Population P 3:generation 1

4:whilegeneration6_{maxNumberGenerationsdo}

5: Evaluate each individuals’ fitness function f(.) in P 6: Sort by fitness

7: Select N best from population 8: Create new empty population list P0

9: whilegeneration6_{maxNumberGenerationsdo}

10: Select two parents.(i.e., roulette-wheel) 11: Perform crossover (c)

12: Perform mutation (m) 13: Insert children into P0

14: end while

15: P P0

16: generation generationþ1

17:end while

3.2.7. Animation

Aside from animation, the typical CG production process involves modelling (drawing of shapes that serve as a CG frame-work) and rendering (processing to achieve an external appearance resembling the true object). While traditional key-framed solu-tions are the most popular approach for generating animasolu-tions, a procedural evolutionary search approaches, as presented in this paper, provides a fresh perspective with multiple avenues of exploration.

3.2.8. Static and dynamic poses

We can search for joint angles that result in the character stay-ing upright and balanced under static conditions (e.g., standstay-ing still with the overall centre-of-mass above the foot support area). How-ever, for dynamic situations (i.e., animated movement), such as walking or jumping, the joint angles are constantly changing. Hence, rather than have the genetic algorithm calculate the param-eters directly for the joints, we calculate the paramparam-eters for a con-troller function (e.g., trigonometric system), which simulates the changing joint angles over time.

4. Learning through experience

4.1. Characters that learn through experience

Most people are familiar with the phrase ‘‘genetic algorithms’’, which is often encountered in science fiction films and novels. Genetic algorithms is the pursuit of an evolving expression of sur-vival of the best actions observed in nature by living creatures. This field is not a mere thing of the imagination, but an active area in which numerous individuals at universities and corporations are conducting research and applying what they learn in products,

ser-Fig. 6.Joint angles and genome Sequences – The binary genome sequences are

made up of joint angles (i.e., with each chromosome being represented by a particular joint). We visually show the differences between a genome, chromosome, and gene.

Fig. 7.Crossover & mutation – Uncomplicated example of (a) crossover and (b)

vices and other such applications. We begin by giving a brief intro-duction to the principle of genetic algorithms in the context of self-driven character animations. We explain how genetic algorithms employ learning concepts through experience, which comprises an important element of the intelligence of humans and other liv-ing thliv-ings (i.e., learnliv-ing from successes and failures of the past to enable future improvements) (seeFig. 8).

4.2. Supervised evolution

An incredibly helpful concept, would be if we could make machines evolve and learn from past experience in the same way as humans, so that they would gradually become ‘‘smarter’’. Cur-rently, many machines operate under computer control, with the computer software (i.e., programs) controlling a great number of complicated movements. Note, the genetic evolution method of achieving this is through experience (i.e., a trial and error search approach). For example, when a child learns to walk, he will con-tinue to try to get up, trying this way and that way, until he even-tually learns to gain balance. The evolutionary training of the animation control parameters is through a guided random proba-bility; whereby, the updates from previous answers steer the search focus towards improvements.

5. Method

5.1. Physical properties

We want the virtual character to possess realistically plausible physical values for mass and joint strength. For example, the five-link biped had the dimensions: body 0.8 m, upper-leg 0.4 m, and lower-leg 0.4 m. The mass for the limbs is calculated based on the mass and dimensions (e.g., body approx. 60 kg). The cen-tre-of-mass for each rigid object could be calculated easily, since we used simple shapes. Summing the individual body component masses allows us to calculate the overall centre-of-mass dynami-cally on-the-fly (i.e., for balance information and control feedback).

5.2. Five link planar biped

The uncomplicated low-dimensional 2D biped model has no-feet (i.e., pin-point ground contacts). Due to pin-point no-feet, the character cannot apply any foot push or forces or torques (e.g., con-trolling the pendulum like pole-vaulting motion between stepping cycles). Nevertheless, the pin-point contact is sufficient to produce the push force to propel the character forward. The maximum tor-que for joints to push the character forward was set between (100 and 300 Nm).

5.3. Box2D

The simulation dynamics were done using the Box2D simula-tion engine[2]for stand-alone simulations while later when we ported the model to the GPU, we implemented a custom first order Euler integrator with penalty based springs for the constraints.

While Box2D is a computationally fast and robust 2d physics engine based on the popular open source Bullet engine[3], it was difficult to spawn a dense number of instances on the GPU. How-ever, Box2D allowed us to run more complex simulations on the CPU with numerous joint constraints. As with Bullet, Box2d uses a symplectic Euler for its integrator. Hence, the Box2D joint con-straints are enforced using an iterative solver. We set a fixed-time integration step of 0.01s. Gravity is set at9.81 m/s2_{. Damping}

(i.e., both linear and angular) is disabled. Restitution (i.e., the elas-ticity of collision) is zero, so all collisions are perfectly inelastic. Friction is 1.0 (i.e., the maximum allowed by Box2D), since this helps the feet contacts grip against the ground. The simulation was developed in C++ for performance reasons.

6. Evolutionary details

What is a ‘Genome’? How do we implement a flexible anima-tion soluanima-tion? How do we represent the character’s parameters in binary form (e.g., floating point precision to binary)? The evolu-tionary principles for an uncomplicated genetic algorithm is ele-gant and straightforward, however, implementing it can be daunting and paved with challenges. This section discusses the implementation details and the technical workings of the classical genetic algorithm that we used for simulation experiments.

6.1. Concepts and principles

A number of key words are used to describe evolutionary con-cepts based on biological principles, these are:

Genome.

Epoch. - Crossover. - Mutation. - Selection.

Default parameters (e.g., population size and randomness).

6.2. Genome

The same genome structure is used for every character (i.e., dimensions). A genome is a collection of points within the input space, each of which is associated with a value in the output space. If the input space hasndimensions then the output space hasm

dimensions. Each point in the genome would haveðnþmÞreal val-ues. A genome withkpoints would thus be represented by a list of

kðnþmÞnumbers. The numbers are scaled to a range of 0–1.0, so inputs and outputs can be mapped easily. For example, since joint angles should only vary from

p

p

6.3. Epoch

The two basic operations of evolution are ‘selection’ and ‘repro-duction’ (with reproduction containing crossover and mutation). For each generation the genome’s fitness is evaluated by instanti-ating a simulator with that genome providing the controller and judging how well it performs at a given task (i.e., fitness functions are discussed in a later section). Between individuals the character is reset to its starting pose (i.e., all joint angles and velocities zero). In practice, the same genome will produce the exact same results, after reset. The genomes that perform best (i.e., greatest fitness value) are selected to reproduce, and those reproducing may be crossed over with other genomes or mutated to produce the next population for evolution.

Fig. 8.Time – The computational cost of searching for a character animation

solution over specific period of time (i.e., continues to walk and stay upright and balanced over 5 s).

6.4. Selection

We use elitism, random, and roulette-wheel selection. With elitism selection the top individuals based on fitness are propa-gated to the next generation without change. The elite ratio is the percentage of the new generation made of such elitism individ-uals. The elite are selected directly in order of fitness. A proportion of the new population is initialized to random genomes. Random genomes are produced by uniformly generating floats in the range 0.0–1.0 for all coordinates of all positions and values for the gen-ome points. The roulette wheel selection is based in a non polyno-mial curve (y¼1=x). As a non polynomial curve picks the best individual with less value in the result than the first individuals in the population. The unallocated proportion of the new popula-tion is filled by choosing two individuals at random with replace-ment from the mating pool to crossover and produce two offspring until the new generation is full.

6.5. Crossover

When two individuals are crossed, they produce two offspring. Since genomes are of fixed size, crossover consists of randomly choosing one of the parent’s points for each point in the offspring. In particular, for a point with a given index in the genome, the first child gets the mother’s point with probability crossover and the father’s point with a similar probability. The second child receives the opposite parent’s point from the first child. This results in the two children being complementary mixes of the father and mother, with crossover determining the purity of the mix and results in both the father and mother being cloned into respective offspring. For example, a crossover mother. father split ratio of 0.8 results in children that are 80% of one parent and 20% of the other).

6.6. Mutation

Each point in the genome is examined in turn. The point is selected for a bit mutation with a given probability (e.g., 5% ran-dom chance). These selections are independent; a point may be selected for none, either, or both mutations. This jitters the values. The values are always clamped to the range 0.0–1.0 (i.e., the muta-tions cannot push the solution out of range).

6.7. Evolutionary parameters

The default genetic algorithm parameters are given below (i.e., unless specified as different in an experiment):

Elite ratio 10%.

Crossover 80%.

Mutation 5%.

Population size 100–500.

Chromosome size 10 (i.e., 10 bit precision).

7. Experiments

We explored a variety of fitness functions to create motions that would accomplish specific tasks (e.g., balance, walking up slopes, jumping, and navigating discontinuous terrain).

7.1. Standing

The fitness function was the length of time the geometric centre of the torso managed to avoid falling 0.3 meters from its starting position. Trials were limited to a maximum of 5 s. A successful stander typically evolves by taking a stance with slightly spread legs, probably because this configuration is more stable than legs

completely together. The stander sways somewhat; since it was not penalized for swaying (or equivalently, rewarded for being still) this is not unexpected. It does indicate, however, that it is at least somewhat tolerant to perturbations.

7.2. Upright locomotion to right

We want characters (i.e., especially bipeds) to remain upright for as long as possible. We accomplished this by adding a head height fitness to every simulation step. However, while it worked initially, it tended to evolve individuals with a preference for standing and not walking characteristics. We rectified this by only adding the hight fitness bias when the x-distance was bigger than the previous maximum value. The fitness function was the length of time the geo-metric centre of the torso managed to avoid falling 0.3 meters plus ten times the horizontal distance travelled as measured by the geo-metric centre of the torso. Trials were limited to 5 s; a trial was ended as soon as the torso dipped below the 0.3 m line (i.e., individ-uals got no benefit from travelling horizontally after falling). Individ-uals that move to the right were successfully evolved (i.e., this included sliding and walking strangely), with none moving particu-larly fast. The curious gait adopted to move to the right suggests that the evolutionary method has difficulties coping with large body movements, since the gait tends to minimize body movement. Slid-ing is hard to prevent, since characters’ do not need to worry about energy efficiency or social stigma, sliding is actually an easy way to move forward. We weed out the sliders by giving big fitness bonuses for every actual step taken (i.e., checking the distance between the feet and the distance the character moved).

7.3. Jumping

The fitness function was the maximum height achieved in 5 s, as measured from the geometric centre of the torso; this just evolved standers. The stander seems to be an evolutionary local minimum for this task, since to get any higher would require bend-ing down first, but bendbend-ing down decreases fitness (at least at first) by increasing the risk of falling over.

7.4. Pre-seeding/ warm-start

We explored seeding the genome with points sampled from the trajectory of a hand-coded walker (i.e., the biped character config-uration). However, the seeded genome was unable to balance, let alone walk, and so was unlikely to make a good base for evolu-tion. One reason is that the trajectories followed by the character through the state space are very sparse, and any slight deviation will quickly plunge the biped into unseeded territory where it has no reliable information.

8. Results 8.1. Simulation

The simulations on the CPU were implemented using the Box2D engine. For the GPU, we had to use a simplified system that could be instanced hundreds of times to exploit the massively parallel execution environment. The control torques were computed using uncomplicated PD controllers. The time-step was 0.01s and gravity simulated at 9.8 m/s2_{. We assume the time-step is relatively}

small enough for the controller to converge on a solution.

8.2. Convergence

The problem with searching a complex character space for spe-cific motions that accomplish constrained goals is that we cannot

guarantee if a solution is found or how long it will take to converge on an acceptable answer. For example, this raises the questions:

How many generations does it take to reach an acceptable solution?

Computational cost? (Box2D or simplified system for the GPU).

How does it compare with other methods? (e.g., data-driven solution or inverse kinematic search techniques).

A number of reasons why evolving techniques are highly suit-able to character animation are:

Articulated characters are highly ‘ambiguous’ (i.e., multiple solutions).

The state-space is ‘discontinuous’ (i.e., joint angles have lim-its and constraints).

Physical constraints (i.e., gravity, environmental contacts, external force disturbances).

Characters possess unique feature characteristics, e.g., num-ber of arms, weight, size, strength

Solutions are ‘non-linear’ (i.e., do not follow any continuous repetitive path).

Bias and control the solution using a ‘fitness-score’. Traditional methods might not converge on a solution (e.g., stuck in minimums due to the discontinuous characteristics), hit-ting glass walls again and again, never converging on a solution. Genetic algorithms provide a global search-based solution for solv-ing ‘non-linear’ problems. Genetic algorithms are a novel biological evolutionary approach to solving a traditionally challenging and difficult problem. For example, creating a complex animations for characters getting in and out of a car and searching for an anima-tion for reacting to a punch to avoid falling over.

8.3. Plausible animations

Creating character animations that accomplish simplified goals (e.g., staying upright or walking) was straightforward. However, a secondary goal was to accomplish animations that appeared aes-thetically correct and life-like (e.g., did not appear strange and bizarre). The generated animations in most cases were diverse and interesting with mixed results. The fitness function for balanced upright standing was straightforward, while walking required engi-neering modifications to resolve issues (e.g., dragging feet along the ground) and achieve a stereotypical walking gait (i.e., lifting feet between foot support transitions). Watching the characters evolve over time to accomplish predefined goal is also hypnotic to look at. All in all, it is educational and informative to see physically-plausible motions being created on-the-fly that are original and inventive.

9. Parallel exploitation

Why should we parallelize the genetic algorithm? What does the GPU give us compared to traditional CPU multi-core techniques?

9.1. CPU to GPU

A simple way to understand the difference between a CPU and GPU is to compare how they process tasks. A CPU consists of a few cores optimized for sequential serial processing while a GPU con-sists of thousands of smaller, more efficient cores designed for han-dling multiple tasks simultaneously (e.g., seeFig. 9).

9.2. GPU

The genetic algorithm’s evolving population is well suited to parallel execution environments, such as the graphical processing unit (GPU). This is because, for each epoch, we create a large num-ber of instances with different attributes and run them in parallel with the best from the population being the seed for each new gen-eration. The simulation of the individuals within the population need to be done over a specified time-frame (e.g., seconds or min-utes), which can be computationally expensive, if we have a large number of characters (seeFig. 10).

9.3. Box2D on GPU

Having a large number of Box2D instances run in parallel on the GPU (e.g., greater than 300) was not possible. Instead, we imple-mented a simplified penalty based physics engine using a first order Euler integration for the GPU. Since we had more control over the memory usage and the uncomplicated character models did not require a precise constraint solver. Furthermore, for alter-native controllers, such as the inverted pendulum (discussed in Section10), this could be implemented easily without Box2D, since we only had to calculate the foot-terrain intersection and each legs spring-damper body forces for each inverted pendulum time-step update (seeFig. 11).

10. Controllers (e.g., inverted pendulum) 10.1. Trigonometric parameters

Certain motions, such as walking, are continuous and repetitive. The joint angle changes over time follow a simple sinusoidal motion. Hence, we make us this as a starting approximations for the joint angles (seeFig. 12).

Fig. 9.GPU and CPU – GPUs have thousands of cores to process parallel workloads

efficiently.

Fig. 10.Parallelization – Evolving large populations in parallel on the graphical

processing unit (GPU). For example, the Nvidia GeForce GTX 470 graphics card has 448 high performance floating point cores.

angle¼AsinðBtþCÞ ð1Þ

whereA;B, andCare the unknown parameters that we search for using our genetic algorithm, andtis the elapsed time.

10.2. Inverted pendulum (IP)

The genetic algorithm is used to feed trigonometric parame-ters (e.g., sinusoidal functions) to control the character’s joint tor-ques. However, the parameters could feed into a different controller model, for example, an inverted pendulum[8], to pro-duce the final motions. Of course, the limitation of this, is the inverted pendulum model focuses on biped physiology (i.e., two legged creatures), while the basic trigonometric approach allows us to create animations for any type of creature. Nevertheless, human bipeds are popular in training environments and games. The low-dimensional inverted pendulum simplifies the search space. For example, the spring loaded inverted pendulum (SLIP) only needs 4 parameters (i.e., the spring and damping coefficients for the legs). The parameters, such as leg-length, mass, and foot placement are all fixed (seeFig. 13).

11. Physics-based character 11.1. Rigid-body joints

The character is decomposed of rigid body elements connected by joints. The joints have limits attached to them (e.g., angle

ranges). Since the model is physically-based the character achieves movement through joint torques and needs to use the environ-ment to accomplish target moveenviron-ments (similar to the real-world). For example, gravity causes the character to fall forwards while pushing downwards on the ground to accomplish forward momen-tum (e.g., walking or jumping) (seeFig. 14).

11.2. Joint torques

‘‘PD Controllers’’ are a common approach in game development for controlling articulated characters, since they are computation-ally fast, intuitive, and relatively robust. The PD controller calculates

Fig. 11.Inverted pendulum control parameters – The inverted pendulum (IP)

provides a low-dimensional search space ideal for solving the genetic algorithm quickly. Suitable IP model parameters can be found in ‘real-time’, which is ideal for interactive environments. For example, remaining upright and balanced while traversing complex terrain, such as stairs and slopes in addition to compensating for unforeseen external forces. The limitation of the IP model is it is only suited to bipeds and does not take into account all the additional degrees-of-freedom, such as the arms and posture.

Fig. 12.Worm locomotion – A simple interconnect set of rigid body limbs to

produce a worm-like forward motion.

Fig. 13.Biped stairs – Having the biped traverse a set of stairs.

Fig. 14.Jumping through gap – We can solve a variety of unique problems using

simple fitness functions. For example, simply specifying that the character with the greatest x-distance has the highest fitness.

Fig. 15.Discontinuous terrain – Experimenting with the terrain (e.g., gaps), to

the joint torque using the error between the current joint angle and the desired joint angle. The current joint angle comes from the con-troller (e.g., trigonometric functions or inverse kinematic systems) (seeFig. 15).

12. Conclusions 12.1. Practicality

To achieve a ‘real-time’ frame-rate and make the evolutionary system a viable tool for artists, we exploited the computational power of the graphical processing unit (GPU). We focused on low-dimensional models to demonstrate the viability of this approach for creating character animations that are self-driven and do not require key-frame data. The user would specify a fitness function to achieve specific tasks. This would allow the user to see a banquet of creative animation solutions that would meet the specified fitness definition. If a more specific motion was desired then more constraints could be added to the fitness function. The approach breaks away from normal data-driven trend and offers a novel alternative for artists to create their animations.

12.2. Final thoughts

This genetic approach make it difficult for individual genomes to radically change once the population has begun to follow an evolutionary trend. Furthermore, in a single threaded environ-ment, it can take hours or days for an acceptable solution to appear, since the evolutionary speed of running the simulation can be time-consuming, even for low-dimensional models. In addi-tion, choosing suitably set-up variables for precision, mutaaddi-tion, and population size can have a dramatic effect on how fast an ani-mations solution appears and on the direction the population evolves.The classical genetic algorithm in its original form is difficult to parallelize as a whole – however, we could distribute computational bottlenecks onto the GPU (e.g., calculating the fitness for each charac-ter by running the simulations over time in parallel).There are still lots of exciting avenues of exploration possible, such as, multiple populations for the genetic algorithm with different parameters to further exploit the parallel processing power with self-adapting probabilities.

12.3. Tomorrow

Future work would do well with mixing more low-dimensional models to form intermediate character representations, to speed up explorable evolution so the approach can be used for crowds and large sets of characters. Further investigation into warm start and hybrid biasing of the population as it progresses to produce a solution that is more desirable in less time. There is also motion editing and re-targeting potential. Since the evolutionary

approach can be used to adapt individual key-frames so they are physically-accurate while possessing the fundamental artistic behaviour.

Acknowledgements

We gratefully acknowledge the support of NVIDIA Corporation with the donation of hardware used for this research. In addition, we want to thank the reviewers for taking the time to provide insightful and valuable comments to help to improve the quality of this paper.

References

[1]Ando Masaki, Hagiwara Masafumi, 3d character creation system using Kansei rule with the fitness extraction method, in: 2009 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE 2009), IEEE, 2009, pp. 1507–1512.

[2] Box2D.<www.box2d.org>, 2014. [3] Bullet.<www.bulletphysics.com>, 2014.

[4]Nick Cheney, Robert MacCurdy, Jeff Clune, Hod Lipson, Unshackling evolution: evolving soft robots with multiple materials and a powerful generative encoding, in: Proceeding of the Fifteenth Annual Conference on Genetic and Evolutionary Computation Conference, ACM, 2013, pp. 167–174.

[5]David A. Coley, An Introduction to Genetic Algorithms for Scientists and Engineers, World Scientific, 1999.

[6]Thomas Geijtenbeek, Michiel van de Panne, A. Frank van der Stappen, Flexible muscle-based locomotion for bipedal creatures, ACM Transact. Graphics (TOG) 32 (6) (2013) 206.

[7]Randy L. Haupt, Sue Ellen Haupt, Practical Genetic Algorithms, John Wiley & Sons, 2004.

[8]Ben Kenwright, Richard Davison, Graham Morgan, Dynamic balancing and walking for real-time 3d characters, Proceedings of the 2011 Motion in Games Conference, volume 2011, Springer, Berlin/Heidelberg, 2011, pp. 63–73. [9]Uldarico Muico, Yongjoon Lee, Jovan Popovic´, Zoran Popovic´, Contact-aware

nonlinear control of dynamic characters, in: ACM SIGGRAPH 2009 Papers, SIGGRAPH ’09, ACM, New York, NY, USA, 2009, pp. 81:1–81:9.

[10]J. Thomas Ngo, Joe Marks, Spacetime constraints revisited, in: Proceedings of the 20th Annual Conference on Computer Graphics and Interactive Techniques, ACM, 1993, pp. 343–350.

[11]Takuya Ogura, Masafumi Hagiwara, 3d character creation system based on sensibility rule extraction, Adv. Affect. Pleasurable Des. (2012) 87.

[12]Said Salhi, Heuristic Search Methods, Erlbaum, Mahwah, NJ, 1998.

[13]Karl Sims, Artificial Evolution for Computer Graphics, volume 25, ACM, 1991. [14]Karl Sims, Evolving virtual creatures, in: Proceedings of the 21st Annual Conference on Computer Graphics and Interactive Techniques, ACM, 1994, pp. 15–22.

[15]William M. Spears, Vic Anand, A Study of Crossover Operators in Genetic Programming, Springer, 1991.

[16]Michiel Van de Panne, Eugene Fiume, Sensor-actuator networks, in: Proceedings of the 20th Annual Conference on Computer Graphics and Interactive Techniques, ACM, 1993, pp. 335–342.

[17]Andrew Witkin, Michael Kass, Spacetime Constraints, ACM Siggraph Computer Graphics, volume 22, ACM, 1988, pp. 159–168.

[18]Man-Leung Wong, Tien-Tsin Wong, Ka-Ling Fok, Parallel evolutionary algorithms on graphics processing unit, The 2005 IEEE Congress on Evolutionary Computation, 2005, volume 3, IEEE, 2005, pp. 2286–2293. [19]Noriyuki Yamamoto, Nahomi Maki, Akihiko Shirai, Skeleton-based diverse

creature design tool for mass production, in: SIGGRAPH Asia 2011 Posters, ACM Press, 2011, p. 11.

[20]Xin Yao, Evolving artificial neural networks, Proc. IEEE 87 (9) (1999) 1423– 1447.