Our method, F-TIS, enables diverse models to collaborate in decentralised GRPO by combining truncated importance sampling with filtering. Across model-size, expertise, and PEFT heterogeneity, F-TIS matches on-policy convergence and, in some cases, improves out-of-distribution reasoning.

Most works on adversarial robustness in distributed training focus on data parallelism, e.g., poisoned gradients, malicious clients, or aggregation attacks. Pipeline parallelism has a different attack surface. Here, the model itself is partitioned across nodes.

Dynamic Upcycling MoE (DUME) cleverly reuses dense experts trained on different domains to create a unified MoE multidomain expert model. DUME retains the knowledge of the original dense experts without any additional training, offering a cost-effective, scalable solution with no compromises.