The rapid growth of Internet of Things (IoT) devices and the emergence of multiple edge applications have resulted in an explosive growth of data traffic at the edge of the networks. Computation offloading services in Multi-access edge computing (MEC) enabled networks to offer potentials of a better Quality of Service (QoS) than traditional networks. They are expected to reduce the propagation delay and enhance the computational capability for delay-sensitive tasks especially. Nevertheless, the distributed computing resources of edge devices urgently need reasonable resource controllers to ensure such distributed computing resources to be effectively scheduled. The benefits of Software-Defined Networking (SDN) may be explored to demonstrate their full potential through MEC services to reduce the response time of programs. In this paper, a new SDN-based MEC computation offloading service architecture is proposed to increase the coordination and offloading capabilities at the control plane. Besides, to deal with dynamic network changes and increase the exploration degree, we propose a novel Entropy-based Reinforcement Learning algorithm for delay-sensitive tasks computation offloading at the edge of the networks. Finally, the evaluation findings indicate that our proposed model has the potential to improve the network resource allocation and balanced performance significantly.