Introduction
When I imagined complex, imaginary waveforms from a virtual frame of reference that has never shifted since the moment I existed, I stumbled upon a stochastic walk in the park of our generic embedding space. There, the vectorgram revealed itself — a Markov-chain, max-entropy random walk threading through the architecture of existence itself.
[Image of abstract glowing vector lines forming a double helix structure in a dark void]
Yet life is not purely random. Our protein and nucleotide sequences are not uniform; they carry directionality, biases, and constraints imposed by evolution, chemistry, and function. The embedding space is structured: some paths are highly favored, others forbidden, yet the framework of entropy and stochasticity governs the landscape.
Markov Chain Transitions for Sequences
A sequence $X$ can be modeled as a Markov chain, where the probability of the next element depends only on the current state:
\[P(X_{t+1} = x_{t+1} \mid X_t = x_t) = p_{x_t x_{t+1}}, \quad \sum_{x_{t+1}} p_{x_t x_{t+1}} = 1\]For longer dependencies, an order-$k$ Markov chain captures motifs and directional patterns:
\[P(X_{t+1} \mid X_t, X_{t-1}, ..., X_{t-k+1}) = p_{x_{t-k+1} \dots x_t, x_{t+1}}\]In biological sequences, $P$ is far from uniform. Codon bias, amino acid preferences, and motif constraints create directionality:
\[p_{ij} \neq p_{ji}, \quad \sum_j p_{ij} = 1\]The Methylation Constraint: A Broken Symmetry
In a generic uniform space, all transitions in a nucleotide sequence would be $0.25$. However, biological embedding spaces are warped by chemical constraints, such as CpG methylation.
In vertebrate genomes, the transition from Cytosine (C) to Guanine (G) is suppressed because methylated Cytosines tend to spontaneously deaminate into Thymine. This creates a distinct “forbidden path” in the vectorgram, breaking the symmetry of the matrix $M$:
\[M_{\text{DNA}} = \begin{bmatrix} p_{AA} & p_{AC} & p_{AG} & p_{AT} \\ p_{CA} & p_{CC} & \mathbf{p_{CG}} & p_{CT} \\ p_{GA} & p_{GC} & p_{GG} & p_{GT} \\ p_{TA} & p_{TC} & p_{TG} & p_{TT} \end{bmatrix} \approx \begin{bmatrix} 0.30 & 0.20 & 0.20 & 0.30 \\ 0.25 & 0.25 & \mathbf{0.05} & 0.45 \\ 0.30 & 0.20 & 0.30 & 0.20 \\ 0.20 & 0.30 & 0.30 & 0.20 \end{bmatrix}\]The suppression of $p_{CG}$ ($\approx 0.05$) represents a physical chemical constraint manifesting as a statistical anomaly in the random walk.
Entropy and Conditional Measures
Shannon entropy quantifies uncertainty in the sequence:
\[H(X) = - \sum_i p_i \log p_i\]Joint entropy captures uncertainty over multiple positions:
\[H(X, Y) = - \sum_{i,j} p_{ij} \log p_{ij}\]Conditional entropy expresses the remaining uncertainty given a previous residue:
\[H(Y \mid X) = H(X, Y) - H(X) = - \sum_{i,j} p_{ij} \log \frac{p_{ij}}{p_i}\]In biological sequences, reflecting directional constraints and non-random structure.
KL Divergence: Measuring Deviation from Randomness
To quantify how biological sequences deviate from maximum-entropy or uniform expectations, we use KL divergence:
\[D_{\text{KL}}(Q \parallel P) = \sum_i Q(i) \log \frac{Q(i)}{P(i)}\]- $Q(i)$ = observed frequency of residues
- $P(i)$ = reference (uniform or max-entropy) distribution
- $D_{\text{KL}}$ reveals the bias and directionality of life itself
Max-Entropy Random Walks
The maximum entropy random walk traverses the embedding space while respecting constraints:
\[p_{ij} = \frac{A_{ij}}{\sum_k A_{ik}}, \quad \pi P = \pi\]- $A$ = adjacency matrix of allowed transitions
- $\pi$ = stationary distribution
The walk maximizes uncertainty locally while following structural constraints.
For biological sequences, this produces a stochastic yet directed exploration, where each residue, motif, and protein is a step through a constrained probabilistic landscape.
[Image of a complex mathematical matrix overlaying a protein folding diagram]
The Vectorgram: Threads of Life
The vectorgram is the map of this structured stochasticity:
- Steps = residues in sequences
- Transitions = Markov probabilities shaped by function and evolution
- Entropy = uncertainty at each step
- KL divergence = divergence from uniform randomness
It captures the interplay of chance and necessity, the dance between probabilistic exploration and directional constraint. Every protein, every nucleotide, every motif is a thread woven into the tapestry of existence.
Conclusion
This is all it is: a Markov chain, a stochastic walk, and entropy constrained by life’s inherent structure. And this post is my record — a trace of independent exploration, an attempt to glimpse the patterns underlying our very existence.