In this letter, we propose an efficient online bandwidth allocation method based on Gaussian process regression (GPR) in order to guarantee given quality of service (QoS) requirement and to prevent a waste of resources. To this end, we analyze large-buffer asymptotics for Gaussian queues including non-stationary input processes, and derive the necessary bandwidth to meet the requirement. Our experimental results demonstrate the proposed method satisfies the QoS requirement for both synthetic traffic and real-world traffic. We also show that the proposed algorithm performs better than the existing algorithm in terms of resource efficiency.