Most works on adversarial robustness in distributed training focus on data parallelism, e.g., poisoned gradients, malicious clients, or aggregation attacks. Pipeline parallelism has a different attack surface. Here, the model itself is partitioned across nodes.

Dynamic Upcycling MoE (DUME) cleverly reuses dense experts trained on different domains to create a unified MoE multidomain expert model. DUME retains the knowledge of the original dense experts without any additional training, offering a cost-effective, scalable solution with no compromises.

We present a decentralised two-sided market design that treats compute as a time‑bound asset, enabled by reproducibility, verification, and checkpointing, yielding dynamic pricing and simple matching without combinatorial auctions.

Our paper, “Hail to the Thief: Exploring Attacks and Defenses in Decentralized GRPO”, is the first systematic study that explores both the attack vectors and defense strategies in decentralised reinforcement learning for Large Language Models (LLMs).