Artificial learning and evolving brain for robots

Abstract

The ultimate goal of this research is developing an artificial brain for robots. On the proposed hypothesis that the fundamental functions of animals’ intelligence are auto-classification and reinforcement learning, several basic synaptic plasticity models have been developed. Researchers believe that STDP (Spike Timing Dependent Plasticity) is an essential brain function for auto-classification and that dopamine system plays an important role for reinforcement learning. A rate-coded STDP (Activity-Variation-Timing Dependent Plasticity, AVTDP) has been derived from the kinetic STDP model of Senn. AVTDP is a simple and efficient model and preserves the timing dependent property. A method to interpret the plasticity mechanism has been proposed in graphical manner. The similarity between AVTDP and STDP is shown through a series of simulations. It is shown that several conditions exist for the parameters that allow the similarity to become valid. Because the formula simply consists of differentiation, multiplication, addition and subtraction, the model is suitable for implementation not only in computer programs but also in electric circuits. It is believed that the proposed model effectively capitalizes on both the rate code and the timing dependent plasticity of the spike code. A synaptic reinforcement algorithm has been developed. The algorithm determines which synapses are to be potentiated or depressed by reward signals. A simulation is conducted to demonstrate how the interaction between the synaptic eligibility and the reward signal influences synaptic plasticity. The use of the pre- and postsynaptic spike correlator (PPSC) is proposed for reinforcement learning in a spiking neural network. The PPSC is used to determine the synaptic pathway eligible for reward. It represents the synaptic eligibility that increases only if the postsynaptic spike occurred shortly after a presynaptic spike. The magnitude of the synaptic eligibility exponentially decreases as a ...