Although plastic scintillation detectors possess poor spectroscopic characteristics, they are extensively used in various fields for radiation measurement. Several methods have been proposed to facilitate their application of plastic scintillation detectors for spectroscopic measurement. However, most of these detectors can only be used for identifying radioisotopes. In this study, we present a multitask model for pseudo-gamma spectroscopy based on a plastic scintillation detector. A deep- learning model is implemented using multitask learning and trained through supervised learning. Eight gamma-ray sources are used for dataset generation. Spectra are simulated using a Monte Carlo N-Particle code (MCNP 6.2) and measured using a polyvinyl toluene detector for dataset generation based on gamma-ray source information. The spectra of single and multiple gamma-ray sources are generated using the random sampling technique and employed as the training dataset for the proposed model. The hyperparameters of the model are tuned using the Bayesian optimization method with the generated dataset. To improve the performance of the deep learning model, a deep learning module with weighted multi-head self-attention is proposed and used in the pseudo-gamma spectroscopy model. The performance of this model is verified using the measured plastic gamma spectra. Furthermore, a performance indicator, namely the minimum required count for single isotopes, is defined using the mean absolute percentage error with a criterion of 1% as the metric to verify the pseudo-gamma spectroscopy performance. The obtained results confirm that the proposed model successfully unfolds the full-energy peaks and predicts the relative radioactivity, even in spectra with statistical uncertainties.