The post discusses the Muon optimizer, a momentum-based algorithm for training deep neural networks, particularly focusing on its advantages over traditional methods like SGD with momentum. It explains how the optimizer can improv...
About:
Yacine Mahdid is a researcher and entrepreneur passionate about machine learning and neuroscience.
Website:
Specializations:
Interests:
Incoming Links:
Subscribe to RSS:
The post discusses the advancements in attention mechanisms, particularly focusing on Lightning Attention, which combines the benefits of FlashAttention and linear attention. It highlights the Minimax-01 model, which integrates Li...
7.3Going too deep in R1...
2025-02-17 • reinforcement learning ppo deep learning gpo
The author discusses the implementation of KL divergence in deepseek's R1 model, highlighting its role as a penalty term in the GRPO formula. The text explains the differences between KL divergence's application in GRPO and PPO, e...
The post discusses the DeepSeek R1 research, which introduces three families of reasoning models based on the DeepSeek V3 architecture. It highlights the innovative use of reinforcement learning without human feedback in the DeepS...
The post explores the concept of in-context learning in large language models (LLMs) like GPT-4, explaining how these models can perform tasks by conditioning on prompts with examples, without traditional learning or fine-tuning. ...
The post discusses the importance of selecting the appropriate cross-validation scheme in machine learning to avoid overfitting. It explains the concept of cross-validation, outlines common pitfalls practitioners face, and provide...
The author reflects on their experience visiting Prime Intellect, a research team focused on artificial intelligence, in San Francisco. They describe the calm atmosphere of the city and the intense focus of the researchers working...
The post discusses the advancements in large language models (LLMs) as of 2025, highlighting the value of code generation and the challenges associated with long context usage. It emphasizes the difficulties in obtaining quality l...
This post is a playful and engaging assembly manual for a Hierarchical Reasoning Model (HRM) designed for toddlers. It guides young readers through the process of building a puzzle-solving model using various components, emphasizi...
The post discusses the relevance of learning programming in 2025, emphasizing the importance of understanding the distinction between coding and programming. It argues that while AI can assist in coding, the human element of probl...
The HuggingFace paper argues against the development of fully autonomous AI agents, citing a poor risk-reward ratio. The paper highlights the ambiguity surrounding the definition of 'agent' in AI, comparing it to the murkiness of ...
The post discusses the distinction between AI engineering and machine learning, emphasizing that they are fundamentally different fields. AI engineering focuses on utilizing trained foundational models to create solutions, while m...
The author shares a personal narrative about overcoming feelings of sadness and confusion during a winter period in their life. They describe a transformative moment in a pre-university school where they discovered a passion for l...
The blog post addresses the feelings of confusion and anxiety experienced by computer science students regarding their future in technology. The author shares a personal method called the 'Wafflehouse Method' to help individuals u...
Yacine Mahdid shares advice for undergraduate students interested in research, particularly in machine learning applications within healthcare. He reflects on his own experiences as a young researcher, emphasizing the importance o...
The author shares personal experiences and insights on navigating the challenges of university education, particularly in relation to grading and course selection. They emphasize the importance of understanding the grading system,...
0Coming soon
2023-11-16 •
...