1sec.ai

Tag

#federated-learning

Every item tagged federated-learning, newest first.

3 items

research20h

FoMoE: Breaking the Full-Replica Barrier with a Federation of MoEs

Researchers propose FoMoE, a federated Mixture-of-Experts approach that enables large-scale LLM pre-training on limited compute budgets without requiring high-speed interconnects. FoMoE breaks the full-replica barrier by distributing MoE models across nodes with slower interconnects, improving efficiency and scalability. This approach can help builders train large models on constrained infrastructure. FoMoE achieves state-of-the-art results while reducing computational costs.

Key takeaways
  • FoMoE enables LLM pre-training on limited compute budgets.
  • Distributes MoE models across nodes with slower interconnects.
  • Improves efficiency and scalability for large-scale LLM training.
aarXiv#large-language-models#mixture-of-experts#federated-learning
otherApr 12

Creating Privacy Preserving AI with Substra

Owkin and Hugging Face partnered to integrate Substra, a privacy-preserving AI framework, into Hugging Face's platform. This integration enables secure, decentralized data collaboration for AI model development. Builders can now access Substra's federated learning capabilities directly within Hugging Face. The partnership aims to enhance data privacy and security in AI development.

Key takeaways
  • Substra integrated into Hugging Face for privacy-preserving AI.
  • Enables secure, decentralized data collaboration.
  • Federated learning capabilities now accessible within Hugging Face.
HHugging Face Blog#privacy-preserving-ai#federated-learning#hugging-face
researchMar 27

Federated Learning using Hugging Face and Flower

Hugging Face and Flower collaborated on a federated learning example using open-source frameworks. The code enables machine learning across decentralized networks. You can use this approach for private, secure model training.

Key takeaways
  • Federated learning combines Hugging Face and Flower frameworks.
  • Enables decentralized, private model training.
  • Uses open-source tools for implementation.
HHugging Face Blog#federated-learning#open-source#decentralized-ai