Run time power management has been a major problem in modern-day edge computing devices. Most of the built-in run-time power managers are not adaptable across a variety of workloads. A similar problem in the domain of High-Performance Computing (HPC) is resolved using Reinforcement learning (RL) approaches like Q-learning which can continuously learn from new workload scenarios. The main bottleneck of implementing Q-learning on edge compute platforms is the large Q-table size and the compute load as the algorithm continuously runs in the background. Compute load by RL continuously running in the background adds a significant amount of overhead on power which needs to be addressed to meet the power constraints for edge devices. As a part of our work, we propose a lightweight Q-learning methodology with intelligent memory management and algorithmic workload management policy. These policies make the algorithm lightweight and fit on edge computing devices. This lightweight Run Time Manager (RTM) eventually helps to bring down the power in run time. Our model has been implemented on the A57 processor of the Jetson Tx2 board which is widely used in a number of edge devices. Our proposed lightweight methodology brought run-time average power saving of 16.63% and the Q-table size decreased by 60% compared to the state-of-the-art edge computation Q-learning approach. © 2022 IEEE.