The continuous iteration of consumer electronics has significantly promoted the development of medical devices, which has enabled the collection of large amounts of heterogeneous medical data. These data are offloaded from local devices to cloud servers for processing using traditional methods, which results in high transmission latency and the risk of privacy leakage. Additionally, researchers have employed federated learning to protect data privacy, but this approach can lead to a straggler effect due to the limited computational and communication resources of terminal devices. To address these issues, a computation offloading framework is designed to optimize task and resource allocation, addressing multi-optimization problems and mitigating the straggler effect in federated learning. Moreover, a novel computation offloading method within a federated learning framework assisted by edge computing, named DRWB, is proposed. Specifically, we develop a deep reinforcement learning-based approach to transfer lagging training tasks to idle edge servers, enhancing data processing speed, minimizing transmission delays, and protecting user privacy. Extensive experimental results demonstrate that the DRWB method outperforms baseline methods, showcasing superior performance in handling heterogeneous medical data tasks.