As extensive research has been done with autonomous vehicle related modules such as perception, localization, planning, control, and prediction , human-in-the-loop end-to- end approaches such as deep learning (DL) and imitation learning (IL) still have open research challenges for safety- critical operation. On one hand, the problem with the modular algorithm design is that each individual module in the pipeline may not be aware of the ”high-level” task required, and result in an omission of information and a lack of coordination. An end-to-end approach, on the contrary, considers the autonomous system as a whole, and maps directly from the raw sensory input to the control output (throttle, steering, brake). This approach not only has merits in the racing field but can also be extended to make meaningful real road applications
