As autonomous driving vehicles are being tested on public roads, they will share the road with human-driven vehicles. It becomes important for autonomous driving vehicles to estimate human drivers’ intentions in order to interact properly with the human drivers to achieve safe and efficient experiences. The current work proposes a new cooperative driving framework which is capable of predicting other vehicles’ behaviors. The estimated prediction provides an input for a trajectory planner to perform cooperative behavior and to generate a path to react to other vehicles. The system has three stages: 1 Abstract intention prediction; 2 Intermediate-level important points prediction; 3 Ultimate trajectory prediction. The system bridges the gap between higher-level mission planning and behavioral execution or trajectory planning, especially in interactive scenarios. The validation contains two aspects: Firstly, the estimated trajectory is compared with the groundtruth in datasets. Secondly, the estimated trajectory is applied to current trajectory planners to generate cooperative plans. The second step evaluates the closed-loop performance of the behavioral estimation in the whole system. The proposed method outperforms previous solutions in terms of collision rates, safety distance, and error when tested against a human-driven trajectory database. The method is implemented in simulation and on a real autonomous driving platform to test its feasibility in real scenarios.