Improving perceptual and planning capabilities of robots

Abstract

namely action classification, and future prediction. In the first part, a deep Convolutional Neural Network which uses the ResNext Architecture as the backbone and employs 3D spatiotemporal convolutional kernels is proposed for human activity classification. The network also employs a spatio-temporal attention mechanism which softly weights the learned features at each stage of the network to the salient features. To further enhance the representational power and accuracy of the network, the ResNeXt modules are replaced by Squeeze and Excitation ResNeXt modules which implement channel wise feature recalibration. To prevent overfitting due to the large number of parameters in 3D kernels, ImageNet pretraining is used to initialize the weights of the network. A computationally efficient variant, which uses separable spatial and temporal convolutions instead of using 3D convolutions, is also proposed. Transferability of the features learnt by the network to similar action datasets is also demonstrated by fine tuning the network on the UCF101 dataset. The second part of this proposes an end-to-end unsupervised predictive network for future generation, that, instead of generating the raw pixel values for the future frame directly, outputs a set of transformation which are then applied to the input frames to generate frames. The architecture is adversarially trained using standard generative adversarial networks(GAN) as well as Wasserstein GAN and is evaluated on the moving mnist, UCF101 and robot pushing dataset.

We as humans can rely on our extensive knowledge and experience accumulated over our lifetime to perceive what is happening around us

recognize actions and gestures, interpret intentions, anticipate the future and act accordingly. How do we give prediction models in robots access to such common sense knowledge. One solution is to employ deep learning algorithms, which have made it possible for robots to learn computational models for various tasks directly from raw data without. In this study, this non-trivial task is focused on two main areas