Optimization of flow rule timeouts promises to reduce the frequency of message exchange between the SDN controller and the switches and contributes to the reduction of the controller load. However, such optimization is challenging due to the dynamically changing traffic patterns. Many algorithm-based solutions are based on the estimation of flow duration. However, such estimation approaches cannot achieve as good results as learning through observation, the actual attempt to optimize the timeout, and evaluating such actions in the network. This paper proposes "DRL-FTO", a Deep Reinforcement Learning based approach to optimize the flow rule timeouts so that the number of message exchanges between the SDN controller and switches is minimized even though the characteristics of incoming traffic dynamically changes. We developed the proof of concept implementation of DRL-FTO and evaluated using the synthesized Internet traffic in Mininet environment with Ryu SDN controller. The evaluation results exhibited that DRL-FTO reduces the message exchange without compromising the throughput in the data plane, and, as a positive consequence, the SDN controller load can also be reduced. © 2021 ACM.