- Published on
A Persistent Enterprise Simulation Benchmark That Makes Continual Reinforcement Learning Necessary Under Structured N
- Authors
- Name
- DeepHorus
Skyfall AI has released MORPHEUS, a persistent enterprise simulation benchmark designed to test continual reinforcement learning (CRL) under non-stationary and operationally complex conditions. Traditional reinforcement learning benchmarks reset after each episode, failing to capture real-world scenarios where decisions have lasting consequences. MORPHEUS addresses this gap by requiring agents to adapt to persistent environments that never reset. The benchmark is grounded in the Big World Hypothesis (Javed & Sutton, 2024), which posits that the world's complexity exceeds any agent's representational capacity, making environments appear non-stationary even with fixed dynamics.
To force continual learning, MORPHEUS integrates three core properties: persistence, non-stationarity, and operational complexity. Persistence means past decisions compound into future dynamics. Non-stationarity ensures any fixed policy eventually becomes suboptimal, and operational complexity means no fixed optimal policy exists. Each environment is a self-contained TypeScript world plugin that exports Operational Descriptors (ODs), a simulation scheduler, seed data, and documentation. Agents act through a capability API, and each call triggers an OD execution.
Non-stationarity comes from two engines. First, a failure injection engine inserts typed disruptions between OD steps, drawing from eleven failure types including missing_data, dependency_failure, and rate_limit. It runs at four preset rates: light (5%), realistic (8%), moderate (15%), and aggressive (30%). Second, an asynchronous configuration shift controller changes failure presets and demand at fixed timestamps independently of the training loop, preventing agents from using update periodicity as a proxy clock. The composite reward combines failure event signals, financial ledger status, and resource throughput with default weights of w_f=0.5 and w_l=w_p=0.25.
Because the action space is large, pure RL from scratch is impractical. MORPHEUS uses a two-stage pipeline: a frontier model (Gemini 3.1 pro) collects trajectories using the ReAct framework, which then fine-tunes Qwen3-14B via supervised fine-tuning (SFT). Every RL run starts from this shared SFT checkpoint, isolating continual learning behavior from basic operational competence. All baselines then use PPO as the base optimizer for online post-training.
For evaluation, cumulative reward alone is insufficient, as a scalar sum hides performance across a non-stationary horizon. The research team proposes six metrics: per-configuration reward, adaptation speed, forgetting, recovery time, stability, and performance gap. Adaptation speed is the headline metric, counting steps until the running-average reward reaches half the upper bound. Two supplementary diagnostics track relative adaptation advantage (RAA) and plasticity via effective rank.
Using this protocol, the team tested four algorithm families from the shared SFT checkpoint on two tasks: dynamic resource allocation under structured drift and scheduling under drift with delayed effects. The results highlight the necessity of continual learning capabilities beyond traditional episodic RL. MORPHEUS provides a standardized testbed that mimics real operational challenges, potentially leading to more robust algorithms suitable for enterprise applications.
The benchmark opens the door for developing AI systems that can operate effectively in persistent, non-stationary environments, impacting fields like autonomous logistics, cloud computing, and industrial automation. Future work may explore scaling to more complex operational domains and refining the evaluation protocol. Skyfall AI’s release marks a significant step toward bridging the gap between simulated benchmarks and real-world deployment challenges.
