Reinforcement learning (RL) is a powerful learning framework which can be used in complex environments such as autonomous driving. Generally, in autonomous driving, vehicles run RL algorithm locally. However doing so will not give a desirable performance as each vehicle will only consider its own environment. So in autonomous driving it is very important that the vehicles make actions with the knowledge learned by other vehicles as well which can be received using V2X technology. However, relying on a single radio or V2X mode of communication is not desirable. In the absence of communication infrastructure on the road side, one can depend on technologies such as 4G/5G for V2N (Vehicle-to-Network) communication and Wi-Fi Direct for V2V (Vehicle-to-Vehicle) communication. Vehicles can depend on cellular technologies for indirect mode of communication (V2N), if direct V2V communication is not possible with other vehicles present in the close vicinity. To reap in the benefits of both federated learning and V2X, we present a federated learning architecture with support from V2X, where all the participant agents make their actions with the knowledge received using V2X, even when they are acting in very different environments. Effectiveness of the proposed V2X federated learning system is demonstrated using collision avoidance application using Flow, Veins, and SUMO simulators. Simulation results suggest that it important to use a federated learning to significantly improve the reliability of of the collision avoidance application. © 2021 IEEE.