The paper discusses subliminal learning, emphasizing that it does not require token entanglement or logit leakage. Key findings include the use of greedy sampling to prevent logit leakage, the identification of divergence tokens t...
About:
Hi! I'm Olly Britton. I'm an undergraduate currently studying mathematics and computer science at Christ Church, Oxford.
Website:
Specializations:
Outgoing Links:
Subscribe to RSS:
The text discusses the implications of omitting sensitive features, such as race, from machine learning models, emphasizing that other features may correlate with these sensitive attributes. It defines fairness-related conditions ...
The text discusses the randomized least-squares problem in numerical linear algebra, focusing on the formulation of the least-squares problem, the concept of row subset selection, and the sketch-and-solve algorithm. It explains ho...
The GMRES (Generalized Minimal Residual) algorithm is designed to solve linear systems of the form Ax = b by minimizing the residual in the Krylov subspace. The process involves using the Arnoldi iteration to generate an orthonorm...
The text discusses the concept of 'Born Again Neural Networks' (BAN), which focuses on knowledge distillation in neural networks. Unlike traditional methods that compress models, BAN involves training a new student network with th...
This post provides a comprehensive overview of linesearch methods in continuous optimization, detailing algorithms, conditions for convergence, and the Armijo condition.
The article explores whether reasoning models utilize their scratchpad in a manner similar to humans. It discusses concerns about models potentially enhancing performance through encoded reasoning or Chain-of-Thought steganography...
The text discusses Krylov subspace methods for solving linear systems and eigenvalue problems. It introduces the concept of the order-r Krylov subspace, defined as the span of a vector and its successive applications by a matrix. ...
Newton's method for optimization is analyzed, covering its algorithm, convergence properties, and necessary modifications for non-positive definite Hessians.
Steepest descent is an optimization method that can converge slowly due to scaling issues, but variable transformations can enhance its efficiency.
The text discusses the shifted inverse power method for computing the eigenvalue and eigenvector of a matrix A closest to a prescribed value s. It outlines the steps involved in the method, including the initialization of a vector...
The text discusses the concept of 'Dark Knowledge' in machine learning, particularly focusing on model distillation techniques. It highlights the combination of soft and hard targets, the use of dropout for model averaging, and th...
The text discusses the Lanczos algorithm, which is used for solving symmetric eigenvalue problems. It describes the Rayleigh-Ritz algorithm and the process of finding approximate eigenpairs using orthonormal bases. The text explai...
The text discusses the conjugate gradient (CG) method for solving positive definite linear systems, detailing its mathematical formulation and algorithm. It explains the motivation behind the method, the concept of Galerkin orthog...
The post details the necessary and sufficient conditions for local minimization in unconstrained optimization, supported by examples and theoretical proofs.
The text discusses key concepts in computer vision, particularly focusing on camera models and projection techniques. It defines single-view ambiguity, camera coordinate systems, and the relationship between 3D points and their 2D...
Mathematical formulations for first and second-order Taylor expansions are presented, emphasizing their significance in optimization processes.
The text discusses the Marchenko-Pastur theorem, which describes the distribution of singular values of random matrices with independent and identically distributed entries. It provides the density function and support for the sin...
Mathematical insights into the Rayleigh quotient and Lipschitz continuity are explored, focusing on eigenvalues and Hessian conditions in continuous optimization.
The text discusses the Lanczos decomposition for symmetric matrices and its implications for the Arnoldi iteration in numerical linear algebra. It presents mathematical formulations and recurrences related to the decomposition, co...
The text discusses the QR algorithm and its application to tridiagonal matrices, highlighting that the reduction to Hessenberg form simplifies the process, requiring only O(n) steps. It also connects the QR algorithm to singular v...
The text discusses the Schur decomposition theorem for matrices over the complex numbers, detailing the existence of a unitary matrix and an upper triangular matrix such that a given matrix can be expressed in a specific form. It ...
The text discusses Chebyshev polynomials, defining them in terms of trigonometric functions and deriving a three-term recurrence relation. It also states the behavior of these polynomials within the interval [-1, 1] and outside of...
Guilt can be reframed as a motivator for positive action towards a better future, rather than a source of paralysis or regret.