Deep bi-directional LSTM attention via multi order difference feature for action recognition

Abstract

We consider a soft attention based model for the task of action recognition in videos. We used C3D 3d cnn network for feature extractor and multi-layered Recurrent Neural Networks (RNNs) with bi-directional Long Short-Term Memory (bi-LSTM) units which are deep both spatially and temporally as a classifier. Our model learns to focus selectively on parts of the video frames and important frames and classifies videos. The model essentially learns which parts and frames are relevant for the task and attaches higher importance to them. We evaluate the model on UCF-101 (YouTube Action)datasets and compare the result without attention mechanism. The result show that it's accuracy is 93% which acurracy is comparable to other state of the art result.