The hierarchical highlight journalling system involves writing a single sentence highlight at the end of each day, then selecting a weekly, monthly, and yearly highlight from those. It has several benefits, including brevity, ease...
About:
Sidney Radcliffe is the author of the website sidsite.com, which is described as 'The site of Sid'.
Website:
Specializations:
Outgoing Links:
Subscribe to RSS:
The text discusses training a simplified transformer neural network to compute Conway’s Game of Life from examples of the game. It presents the model SingleAttentionNet, the problem formulation and training loop, and the model's c...
The text discusses the difficulty of getting ChatGPT to output copyrighted text due to copyright detection implemented by OpenAI. It provides examples of successful attempts to get the model to output copyrighted text using JSON m...
0Demo: Gridnotes - an infinite 2D text editor
2023-11-26 •
...
The article discusses the process of rewriting and optimizing Python code in Rust, resulting in a 180,000x speed-up. It details the journey of profiling and iteratively speeding up the code in Python, replicating the original benc...
0Visual content search over music videos - demo
2023-10-25 •
...
The Reversal Curse (Sep 2023, Berglund et al.) is a paper that has been trending on social media. The authors have released the code on GitHub and encouraged people to try improving the results by modifying the prompts. The author...
The text explains how Byte Pair Encoding (BPE) works, a tokenization algorithm used by large language models such as GPT, LLaMA, and RoBERTa. It provides a worked example and Python implementations of the algorithm, and discusses ...
The post introduces the concept of learning per example (LPE) as a measure of how much a deep learning model has learned about each example in a given training batch. It explains how LPE can be obtained and its uses, and discusses...
The author trained a BERT model from scratch on a desktop PC with a Nvidia 3060 Ti 8GB GPU. The model architecture, tokenizer, and trainer came from Hugging Face libraries. The training of large language models is generally associ...