We introduce DAMAS (Dynamic Adaptation through Multi-Agent Systems), a novel framework for decision-making in non-stationary environments characterized by varying reward distributions and dynamic constraints. Our framework integrates a multi-agent system with Multi-armed Bandits (MAB) algorithms and Bayesian updates. Each agent in DAMAS specializes in a particular environmental state. The system employs Bayesian estimation to continuously update the probabilities of being in each environmental state, enabling rapid adaptation to changing conditions. Our evaluation of DAMAS included both synthetic environments and real-world web server workloads.