When I imagined complex, imaginary waveforms from a virtual frame of reference that has never shifted since the moment I existed, I stumbled upon a stochastic walk in the park of our generic embedding space. There, the vectorgram revealed itself — a Markov-chain, max-entropy random walk threading through the architecture of existence itself.
[Image of abstract glowing vector lines forming a double helix structure in a dark void]
Yet life is not purely random. Our protein and nucleotide sequences are not uniform; they carry directionality, biases, and constraints imposed by evolution, chemistry, and function. The embedding space is structured: some paths are highly favored, others forbidden, yet the framework of entropy and stochasticity governs the landscape.
A sequence $X$ can be modeled as a Markov chain, where the probability of the next element depends only on the current state:
\[P(X_{t+1} = x_{t+1} \mid X_t = x_t) = p_{x_t x_{t+1}}, \quad \sum_{x_{t+1}} p_{x_t x_{t+1}} = 1\]For longer dependencies, an order-$k$ Markov chain captures motifs and directional patterns:
\[P(X_{t+1} \mid X_t, X_{t-1}, ..., X_{t-k+1}) = p_{x_{t-k+1} \dots x_t, x_{t+1}}\]In biological sequences, $P$ is far from uniform. Codon bias, amino acid preferences, and motif constraints create directionality:
\[p_{ij} \neq p_{ji}, \quad \sum_j p_{ij} = 1\]In a generic uniform space, all transitions in a nucleotide sequence would be $0.25$. However, biological embedding spaces are warped by chemical constraints, such as CpG methylation.
In vertebrate genomes, the transition from Cytosine (C) to Guanine (G) is suppressed because methylated Cytosines tend to spontaneously deaminate into Thymine. This creates a distinct “forbidden path” in the vectorgram, breaking the symmetry of the matrix $M$:
\[M_{\text{DNA}} = \begin{bmatrix} p_{AA} & p_{AC} & p_{AG} & p_{AT} \\ p_{CA} & p_{CC} & \mathbf{p_{CG}} & p_{CT} \\ p_{GA} & p_{GC} & p_{GG} & p_{GT} \\ p_{TA} & p_{TC} & p_{TG} & p_{TT} \end{bmatrix} \approx \begin{bmatrix} 0.30 & 0.20 & 0.20 & 0.30 \\ 0.25 & 0.25 & \mathbf{0.05} & 0.45 \\ 0.30 & 0.20 & 0.30 & 0.20 \\ 0.20 & 0.30 & 0.30 & 0.20 \end{bmatrix}\]The suppression of $p_{CG}$ ($\approx 0.05$) represents a physical chemical constraint manifesting as a statistical anomaly in the random walk.
Shannon entropy quantifies uncertainty in the sequence:
\[H(X) = - \sum_i p_i \log p_i\]Joint entropy captures uncertainty over multiple positions:
\[H(X, Y) = - \sum_{i,j} p_{ij} \log p_{ij}\]Conditional entropy expresses the remaining uncertainty given a previous residue:
\[H(Y \mid X) = H(X, Y) - H(X) = - \sum_{i,j} p_{ij} \log \frac{p_{ij}}{p_i}\]In biological sequences, reflecting directional constraints and non-random structure.
To quantify how biological sequences deviate from maximum-entropy or uniform expectations, we use KL divergence:
\[D_{\text{KL}}(Q \parallel P) = \sum_i Q(i) \log \frac{Q(i)}{P(i)}\]The maximum entropy random walk traverses the embedding space while respecting constraints:
\[p_{ij} = \frac{A_{ij}}{\sum_k A_{ik}}, \quad \pi P = \pi\]The walk maximizes uncertainty locally while following structural constraints.
For biological sequences, this produces a stochastic yet directed exploration, where each residue, motif, and protein is a step through a constrained probabilistic landscape.
[Image of a complex mathematical matrix overlaying a protein folding diagram]
The vectorgram is the map of this structured stochasticity:
It captures the interplay of chance and necessity, the dance between probabilistic exploration and directional constraint. Every protein, every nucleotide, every motif is a thread woven into the tapestry of existence.
This is all it is: a Markov chain, a stochastic walk, and entropy constrained by life’s inherent structure. And this post is my record — a trace of independent exploration, an attempt to glimpse the patterns underlying our very existence.
]]>Regularisation is one of the most powerful techniques in machine learning to combat overfitting and improve generalization.
Instead of keeping it abstract, this post introduces an interactive 3D Regularisation Visualizer.
When fitting a model, we usually solve the least sum of squares (OLS):
\[\hat{\beta} = \arg \min_{\beta} \ \|y - X\beta\|_2^2\]The issue? With too many features or high-degree polynomials and too few samples, the model can overfit noise due to supercollinearity and under representation.
Regularisation adds a penalty:
Ridge (L2):
\(\hat{\beta} = \arg \min_{\beta} \ \|y - X\beta\|_2^2 + \lambda \|\beta\|_2^2\)
Lasso (L1):
\(\hat{\beta} = \arg \min_{\beta} \ \|y - X\beta\|_2^2 + \lambda \|\beta\|_1\)
Here, ( \lambda ) controls the penalty strength.
The visualizer lets you:
functionally:
The Regularisation Visualizer bridges math and intuition. It shows how Ridge and Lasso reshape solution spaces, revealing the trade-offs between bias, variance, and sparsity.
Whether for teaching, experimenting, or exploring, it’s a powerful way to make abstract equations come alive in 3D.
Here, the phase θ(t) is not linear. Its derivative, the instantaneous frequency, increases over time. This is called a “chirp” signal.
cos(θ(t)).sin(θ(t)).This creates the distinctive “chirping” spiral you see.
]]>