Continuous casting is the process of concretion of hot molten liquid in a continuous groundwork. As the process of secondary cooling has a critical impact on strand surface quality and casting productivity, the temperature control has always been a major issue in steel industry. Herein, a hybrid model of convolutional neural network (CNN) and deep neural network (DNN) for addressing an inverse problem of continuous casting process, is dealt with. The temperature data obtained from finite differential method (FDM)-based simulation is used as image inputs of CNN, whereas DNN receives some process condition parameters, e.g., initial temperature, steel size, and carbon weight for its input layer. The final nodes of the two-architecture models are concatenated using fully connected layer to predict the final outcomes, e.g., cooling temperature zones. The proposed model demonstrates a significant performance improvement against GoogLeNet-artificial neural network (ANN) hybrid and traditional neural network, while learning the temperature distribution. Furthermore, an optimal model is chosen from a number of hyper-parameter settings during the learning process. The proposed model is not only able to overcome the limitations of aforementioned models but also able to reasonably reduce both the computational time and prediction error.