Prediction engine : a large-scale neuromorphic hardware based on helmholtz machine with autonomous scalability

Abstract

The principal aim of `Prediction Engine` is to develop autonomous scalable neural networks running on distributed and parallelized computer systems and to introduce some heuristic means of scaling the capacity of a Human-brain mimicking algorithm called "Helmholtz machine" online rather than during cross-validation. Helmholtz Machine (HM) is an unsupervised learning algorithm which models brain based on the (Helmholtz) free energy principle. On top of HM, we proposed a new architecture calledHelmholtz Machine with Autonomous Scalability (HMAS) and implemented this machine into a scalable embedded cluster system based on Message Passing Interface (MPI) framework. HMAS changes its computational resources with respect to different problem complexity in real-time by employing two methods: Moving average filter method or Ratio of marginal free energy method. To implement HMAS on neuromorphic hardware, we need to solve many practical issues such as limited bandwidth problem, synchronization and load balancing, hierarchical delay and so on. We present methods to solve these problems in order to obtain high connectivity and fexibility simultaneously. This fexibility in problem complexity effectively prevents under-fitting and over-fitting problems in neural networks and increases learning efficiency. The results of our studies suggest that HMAS better minimizes free energies than the original HM where any information of problem such as complexity or underlying probabilistic distribution is unknown, by scaling its own architecture up properly.