Having a phone call while driving, especially with chatbots, poses challenges and safety risks. This paper explores the feasibility of an AI system for drivers that helps them have a conversation with AI chatbots safely while driving. In this work, I introduce the "Sidekick AI" system, which detects and mitigates stressful situations while conversing with chatbots by considering the driver’s situation. I designed and developed a system that performs a simple pause-and-resume-based dialogue intervention that helps the driver focus only on driving in stressful situations. Real-driving experiments were conducted with 8 participants to understand the feasibility of this type of intervention. Information about the intervention’s acceptability, helpfulness, appropriateness, participants' emotional responses, and impact on driving behavior and chatbot conversations was collected through the experiments, interviews, and post-experiment surveys.