This paper presents the first full end-to-end deep learning framework for the swift prediction of lithium-ion battery remaining useful life. While lithium-ion batteries offer advantages of high efficiency and low cost, their instability and varying lifetimes remain challenges. To prevent the sudden failure of lithium-ion batteries, researchers have worked to develop ways of predicting the remaining useful life of lithium-ion batteries, especially using data-driven approaches. In this study, we sought a higher resolution of inter-cycle aging for faster and more accurate predictions, by considering temporal patterns and cross-data correlations in the raw data, specifically, terminal voltage, current, and cell temperature. We took an in-depth analysis of the deep learning models using the uncertainty metric, t-SNE of features, and various battery related tasks. The proposed framework significantly boosted the remaining useful life prediction (25X faster) and resulted in a 10.6% mean absolute error rate.