The blog post discusses the impact of tokenization on the output of large language models (LLMs). It explains how shorter tokens can have a higher likelihood of being selected due to their ability to complete multiple words, while...
About:
Brendan Long is a software engineer at Meta with interests in OCaml and MPEG, contributing to open-source projects and writing about technology.
Website:
Specializations:
Interests:
Outgoing Links:
Subscribe to RSS:
Filler tokens enhance parallel reasoning in LLMs but fail to support long-term sequential reasoning, contrasting with Chain of Thought methods.
The blog post discusses the inaccuracies of CPU utilization metrics, emphasizing that reported utilization does not always reflect actual performance. Through stress tests on a Ryzen 9 5900X processor, the author demonstrates that...
The article discusses the limitations of large language models (LLMs) in processing certain tasks, particularly those involving counting characters or visual reasoning. It explains that LLMs tokenize input data, which can hinder t...
A new method for fairly breaking ties in approval voting is proposed, utilizing algorithms like Rock-Paper-Scissors and hash functions to ensure fairness without reliance on trust.
Teacher forcing in LLM training can lead to overconfidence in outputs, causing reasoning errors, and the author discusses its implications and possible solutions.
Brendan Long discusses the implications of training AI models, particularly reasoning models, on outputs versus methods used to generate those outputs. He warns against training on the method (M) or the analysis technique (T) as i...
The author recounts their experience at a company where they improved data processing by storing references to all source data for every piece of output data. This change allowed for easier debugging of data quality issues, enabli...
Robust software design focuses on system resilience to component failures rather than solely on error handling mechanisms.
The author argues against exclusivity deals with monopolistic companies, suggesting they create an unfair market for new entrants. By analyzing market share and royalty rates of major platforms, the author illustrates how these de...
The author discusses the limitations of AI in articulating its internal experiences, emphasizing that AI training data is derived from human inputs rather than AI-generated insights. The text argues that when AIs respond to questi...
The blog post discusses the challenges faced by Claude, an AI, in playing 2D ASCII games like Dwarf Fortress and Angband due to its limitations in interpreting screen outputs. The author explores the use of tmux to facilitate inte...
Writing detailed GitHub issues is akin to system design interviews, highlighting the importance of high-level design in software engineering.
Brendan Long updates on Lion Reader, an open-source RSS reader that now includes various features and supports public signups after significant improvements.
Clawed Burrow enables users to run Claude Code locally with enhanced flexibility and security, overcoming the limitations of cloud environments.
Brendan Long details his productive experience using Claude Code to develop a read-it-later service, showcasing its coding capabilities and addressing challenges faced during development.
A straightforward method to fix a loose mouse wheel using adhesive putty, enhancing its responsiveness and preventing future issues.
A Discord game lets players guess the channel, date, and user of random messages, featuring a scoring system and easy installation.
4.4Fresh Fruit Tourism
2025-08-04 • travel seattle, washington fruit food culture berries
The post discusses the concept of 'fresh fruit tourism,' encouraging Americans to travel to locations where they can pick and eat ripe, fresh fruit, particularly blackberries in Seattle. The author highlights the superior taste of...
The article discusses the impact of capital gains taxation on investment risk and reward. It explains that capital gains taxes reduce both potential gains and losses, leading to a lower perceived risk in portfolios. To counteract ...
The post analyzes the four axes of hunger—pangs, appetite, fullness, and emotional state—highlighting their interrelations and implications for diet discussions.
E-bikes significantly enhance commuting in hilly areas, providing motivation and convenience, though safety and weight considerations are important for potential buyers.
Improving your remote meeting experience requires investing in reliable WiFi, quality microphones, and proper lighting to facilitate clear communication.
The author explores feelings of inadequacy in sports and writing, realizing that even being in the top percentages can feel discouraging when surrounded by more dedicated individuals.