A new adaptive controller based on multiple neural networks (NNs) for an uncertain robot manipulator system is developed in this paper. The proposed multiple neuro-adaptive controller (MNAC) switches to a memorized control skill or blends multiple skills by using visual information on the given job to improve the transient response at the time of task variation like a change of manipulating object. MNAC is a type of adaptive feedback controller where system nonlinearity terms are approximated with multiple NNs. The proposed controller is effective for a job where some tasks are repeated but information on the load cannot be scheduled before the operation. During the learning phase, MNAC memorizes a control skill for each load with each NN. For a new task, most similar existing control skills may be used as a starting point of adaptation, which improves the performance of learning. Lyapunov-function-based design of MNAC guarantees the stability of the closed-loop system to be independent of switching or blending law. Simulation results on a two-link manipulator for changing the mass of the given load were illustrated to show the effectiveness of the proposed control scheme by comparison with the conventional neuro-adaptive controller.