What is Neuroevolution?
Neuroevolution is a form of artificial intelligence that uses evolutionary algorithms to generate artificial neural networks (ANN), parameters, topologies and rules. It is most commonly applied in artificial life, general game playing and evolutionary robotics.
Neuroevolution is inspired by the natural #evolution. Human brain is a 100-trillion-connection product of evolution – a natural process without intelligent oversight or forethought.
Although artificial neural networks have seen great progress in recent years, they remain distant shadows of the great cognitive masterpiece of natural evolution.
It is seen by many as an alternative to deep learning, thanks to many unique and effective techniques it developed in the framework of this concept.
How do we get from where we are to what is called artificial general intelligence (AGI), which roughly means artificial intelligence (AI) as smart as humans?
The main benefit is that neuroevolution can be applied more widely than supervised learning algorithms, which require a syllabus of correct input-output pairs. In contrast, neuroevolution requires only a measure of a network’s performance at a task. For example, the outcome of a game (i.e. whether one player won or lost) can be easily measured without providing labeled examples of desired strategies.
Neuroevolution is commonly used as part of the reinforcement learning paradigm, and it can be contrasted with conventional deep learning techniques that use gradient descent on a neural network with a fixed topology.
It is remarkable to consider that the 100-trillion-connection human brain is a product of evolution, a natural process without intelligent oversight or forethought. Although artificial neural networks have seen great progress in recent years, they remain distant shadows of the great cognitive masterpiece of natural evolution. How do we get from where we are to what is called artificial general intelligence (#AGI), which roughly means artificial intelligence (#AI) as smart as humans?
Current neural network research is largely focused on the fields of ‘deep learning’1,2 and ‘deep reinforcement learning’3,4. In these fields, the dominant method for training neural networks is backpropagation5, an efficient algorithm for calculating the loss function’s gradient, which when combined with #stochastic gradient descent (#SGD) can modify each neural network weight to greedily reduce loss. This method has proven remarkably effective for supervised learning, and has also produced impressive reinforcement learning results.
Neuroevolution is an alternative approach, which draws inspiration from the biological process that produced the human brain, is to train neural networks with evolutionary algorithms. It enables important capabilities that are typically unavailable to gradient-based approaches.
Such capabilities for neural networks include learning their building blocks (for example activation functions), hyperparameters (for example learning rates), architectures (for example the number of neurons per layer, how many layers there are, and which layers connect to which) and even the rules for learning themselves.
This latter is a prominent feature of natural evolution. It has another advantage.
Do you know the power a human brain needs? Let us see. The average power consumption of a typical adult is 100 Watts and the brain consumes 20% of this making the power of the brain 20 W.
Meanwhile, the power consumption of Summit Supercomputer at Oak Ridge National Laboratory is 10,096 kW, which is 504,800 times more (!)
Until now, computing capacities have been growing extensively. But at some point – and quite soon – we will have to face the fact that we can not do it forever.
#Neuroevolution has other fundamental differences from traditional approaches, for example that it maintains a population of solutions during search, which grants it interestingly distinct benefits and drawbacks.
Finally, because neuroevolution research has (until recently) developed largely in isolation from gradient-based neural network research, the range of unique, interesting and powerful techniques invented by the neuroevolution community can provide an exciting resource for inspiration and hybridization to the deep learning, deep reinforcement learning and machine learning communities.
There has also been a surge of interest lately in hybridizing ideas from neuroevolution and mainstream machine learning, and to highlight this emerging direction we describe a few such efforts. We conclude with promising future research directions that advance and harness ideas from neuroevolution, including in combination with deep learning and deep reinforcement learning, that could catalyse progress towards the ambitious goal of creating #AGI.
What we do and what is the Promise?
We are happy to share our vision of neuroevolution with the broader machine intelligence community. Each of us has spent over 20 years in Artificial Intelligence, and our perspective naturally reflects our experience and the interests that we have developed over this long period.
Many great researchers have contributed to this field: Kenneth O. Stanley @kenneth0stanley, Jeff Clune @jeffclune, Joel Lehman @joelbot3000, Risto Miikkulainen. By the way, we highly recommend reading their article.
We believe evolutionary algorithms are similar to democracy : it looks like they are permanently in a crisis, struggling to find a solution, but they always achieve the best possible result.
In this respect, we would like to quote the famous Roy Amara’s law:
„We tend to overestimate the effect of a technology in the short run and underestimate the effect in the long run.“
The case of neuroevolution reminds us a bit the one of GPS – technology was originally created in 1970s for the U.S. Army in order to facilitate the ammunition delivery. It was almost abandoned in the 1980s and resurged only in 1990s.
And now, that tech, created for one precise goal, is used virtually by everyone, helping to find a way back home, track flights and even manage the stocks.
That might be the neuroevolution case as well. Let us hope it will be just as useful for everybody!
If you would like to share your ideas on #Neuroevolution or offer a cooperation opportunity – we are at your entire disposal. Please, fill in the form below and we will get in touch with you as soon as possible!