Participation of EV aggregators in frequency regulation service may cause time-varying delays in load frequency control (LFC) systems. These time-varying delays affect the performance and even cause instability of power systems. In this paper, we investigate a single and multiple time-varying delays-dependent stability of an LFC system with participation of EV aggregators. Based on the Lyapunov theory and linear matrix inequality (LMI) approach, we propose delay-dependent stability criteria for a single and multiple time-varying delays using the Wirtinger-based improved integral inequality. With the stability criteria, we obtain delay margins for LFC with a single and two EV aggregators and investigate the interaction among the delay margins, PI controller gains, and participation ratios of EV aggregators. In addition, we investigate the significance of time-varying delays in fast-response resources by comparing the delay margin obtained in EV aggregator model with that obtained in delayed conventional generators. We discuss an extension to multi-area LFC systems and guidelines on how to improve performance of LFC systems under time-varying delays. We expect that the proposed criteria can help to guide the determination of delay requirements for EV aggregators participating in frequency regulation service.