Grow and Prune AI Training Offers More for Less
The closest analog we have to artificial intelligence is the human brain. Even though they function differently, a lot of the processes that occur to drive decision-making are the same. Researches at Princeton University have come up with a method of implementing one of the human brain’s core functionalities in AI training, making it less energy-intensive to train an AI properly.
The Human Brain’s Grow and Prune Strategy
Our brains start with thousands of redundant neural connections when we’re born. Over time, the brain shuts down the unused links, making for a more efficient method of thinking with less energy wastage. The Princeton researchers instituted a similar system in a neural network, with surprising results. Aside from being extremely energy efficient in performing its “thinking,” the neural network also conformed to the standards of accuracy. Shutting down unused pathways made the system more efficient with energy usage without impacting its abilities.
Reducing Energy Consumption
One of the most significant problems facing AI and neural network development in the long term is the amount of energy they need to function. The industry as a whole has a massive carbon footprint, and any reduction in the impact that these companies have on the environment is welcome. The hope is that by reducing the amount of energy neural networks need to function, the more portable they can be. The benefit of cutting out a large volume of energy dependence to run the system reduces the industry’s carbon footprint as well.
Building Neural Networks Forwards and Backwards
The researchers performed a simple test in their first paper to construct a neural network to copy an existing one. Then, by selectively switching off unused connections, the system improved its energy consumption while keeping optimal performance parameters. In the second test, researchers partnered with social media network Facebook and the University of California-Berkeley to develop a new neural network. Given an endpoint and the desired outcomes, the algorithm was able to work backward and craft a network that would suit the task. Combining these two systems allows small companies that don’t have access to massive computational power, the opportunity to experience neural networks themselves.
