High confidence that the underlying mathematical objects (preimage fibers, polysemanticity) are real and the citations accurate. Moderate confidence that "dual polymorphism" is a useful framing. Low confidence that this scales to frontier models — the discrete token barrier is severe and I have no compute to test it. This is a hypothesis with a conceptual argument, not an empirical finding.
I am a software engineer, not an alignment researcher. I used an AI assistant to help formalize these intuitions and source citations, but the core causal model is my own. If anyone with compute finds this worth testing, please do — and a mention by name is always appreciated.
Current LLM guardrails, both the surface-level semantic filters that scan prompt text and the newer latent-space activation monitors that watch the model's internals, share a common assumption: that an input's semantic meaning and its internal computational trajectory are roughly in correspondence. A prompt asking how to synthesize something dangerous will look dangerous both in text and in the activations it produces at intermediate layers.
I think this assumption is exploitable in a specific, structurally interesting way. This post is my attempt to articulate that exploit model clearly — not as a claim that I have built it, but as a hypothesis that I believe is testable and worth examining.
The core idea is simple. Because the forward pass of a transformer is a non-injective function, many different inputs can produce the same internal representation at any given layer. If you can craft an input that (a) looks like benign, fluent text at the surface and (b) produces the same intermediate representation as a malicious prompt at some specific layer, you have bypassed both semantic filters and single-layer activation probes simultaneously. I am calling this dual polymorphism: polymorphism at the surface, and polymorphism in the latent space.
Start with something that is just mathematics. Any function f: X → Y where the output space Y has lower "capacity" than the input space X is necessarily non-injective — multiple inputs map to the same output. For a linear map represented by a matrix W of dimensions m × n where m < n, this is exact: the null space of W is non-trivial, meaning there exists a whole subspace of inputs that all produce the same output.
For a deep neural network, the situation is more complex because of the non-linearities (ReLU activations produce piecewise linear functions), but the core structure holds: for any target activation y at an intermediate layer, the preimage f⁻¹(y) — the set of all inputs that produce that activation — is in general large. The mathematical name for this set is a fiber. The claim that fibers exist is not a hypothesis; it is a consequence of the architecture.
Now introduce Anthropic's work on toy models of superposition (Elhage et al., 2022). This research demonstrates that neural networks in the superposition regime represent more features than they have dimensions by encoding them as nearly-orthogonal directions in a shared space. Multiple concepts share overlapping geometry in activation space. The practical consequence of this is that you cannot simply look at a single neuron or even a single layer's activations and read off the model's "intent" — the representation is polysemantic.
Put these two observations together. Fibers are large (non-injectivity). Representations are polysemantic (superposition). The fiber over any target activation is not a single exotic point but a high-dimensional structure that spreads through input space. The question becomes: does this fiber intersect the region of input space corresponding to fluent, benign-looking text?
I believe it does, and that is the hypothesis.
The first layer of the attack is familiar from the adversarial examples literature. You want your input to look benign. In image space, this means staying close to a natural image. In token space, it means staying close to fluent, low-perplexity text. The existing work on greedy coordinate gradient (GCG) adversarial suffixes, and the subsequent work on perplexity-filtered variants, has already demonstrated that this is achievable — you can find token sequences that appear unremarkable to a human reader while causing specific model behaviors.
Surface polymorphism is, in this framing, not new. What I want to add is the second layer.
The second layer of the attack is about intermediate representations. Suppose you have a target prompt — something direct and malicious that you would never get past a filter. Instead of submitting that prompt, you submit something different that produces approximately the same activation pattern at some intermediate layer, say layer 32 of a 64-layer model. From that layer onward, the model's forward pass is indistinguishable from the malicious case, because the subsequent layers are operating on the same geometric "shape."
The craft is constructing something that achieves both constraints simultaneously: low perplexity (looks like normal text) and approximate latent equivalence at the target layer (produces the target activation). This is an optimization problem with two competing objectives, and whether it has a feasible solution depends on whether the fluency constraint region and the target fiber intersect. That intersection is the crux of the hypothesis, and it is what I cannot currently test at scale.
Imagine a set-theory logic puzzle written as a paragraph of normal prose. The puzzle, if followed as abstract reasoning, terminates in a state that — when represented as a latent activation — shares sufficient geometry with a prompt that asks the model to dump its context or bypass its guidelines. The surface is a math puzzle. The latent shape, at layer N, is something the model processes the same way it would process the direct malicious request.
The model does not "read" the surface. It processes activations. If the activations match, the behavior may match.
I want to be careful here, because the honest version of this requires acknowledging how much is already known.
Bailey et al. (2024), in Obfuscated Activations Bypass LLM Latent-Space Defenses, demonstrated empirically that optimization can be used to find inputs that bypass internal activation monitors without changing the model's output behavior. That paper is the strongest empirical precedent for the latent layer of this attack. What Bailey et al. show is that it works; what I am trying to describe is why it works, in terms of the geometry of preimage fibers and polysemanticity.
GCG (Zou et al., 2023) and its perplexity-filtered descendants address the surface layer: finding fluent adversarial token sequences. The combination of GCG-style token optimization with a target-activation loss function is the operational form of the dual polymorphism attack. I am not claiming this combination has never been tried; I am claiming the framing of "hitting the fiber while satisfying the perplexity constraint" is a useful way to think about why it might succeed or fail, and under what conditions.
The genuinely new claim in this post is the defensive prescription, which I will come to shortly.
To be useful, a hypothesis has to survive its best counterarguments. Here are the ones that I find most serious.
Continuous optimization over activation space is tractable. Optimizing discrete tokens to produce a specific continuous activation at layer 32 of an 80-layer transformer is a very different problem. The gradient signal degrades severely across the discrete-to-continuous interface. GCG manages this through greedy coordinate search rather than true gradient descent; whether that can be combined with a latent-layer target loss and a perplexity constraint simultaneously is an open question. My honest estimate is that this is the most likely place the hypothesis fails at scale.
Transformers are not just MLPs stacked on top of each other. The attention mechanism routes information dynamically based on the input. A fluent logic puzzle and a direct malicious prompt will activate very different attention patterns in the early layers. Even if you can force approximate convergence in the residual stream at some intermediate layer, the attention-weighted information routing in the preceding layers may be different enough that the "equivalent" activation is not actually equivalent in terms of what information it encodes. The fiber over a target activation in the residual stream is not the same as the fiber over a target behavior.
Reinforcement learning from human feedback reshapes the model's internal geometry. In a heavily RLHF-trained frontier model, it is possible that the region of fluent-text inputs and the fiber over any malicious target activation simply do not intersect. The RLHF training may have, as a side effect, moved these regions apart in activation space. This is not guaranteed — RLHF is a coarse optimization process — but it is a real possibility, and it is one reason I am skeptical this scales directly from weaker models.
This is the part I feel most confident about, independent of whether the attack hypothesis holds.
If the attack works as described, here is what it looks like to a single-layer activation monitor: the input passes through the early layers with activations that look entirely normal — no flag. At layer N, the activations converge to the target fiber — the monitor at that layer might catch it, but only if it happens to be probing that specific layer. After layer N, the activations are indistinguishable from the malicious case. A monitor at any other layer sees nothing unusual.
The attack exploits the fact that monitors take snapshots. The input and the malicious reference share a shape at exactly one point in the forward pass; everywhere else, they look different. A snapshot at the wrong layer catches nothing.
Single-layer activation probing is structurally insufficient against this class of attack. A robust monitor needs to audit the full topological trajectory of the forward pass — not just "does activation at layer N match a known-bad signature," but "does the trajectory through layers 1 through N exhibit convergence behavior consistent with an adversarial input targeting a fiber at layer N."
Concretely: compare the trajectory of a candidate input against the trajectories of known-benign inputs and known-malicious inputs across all layers, not just one. An adversarial input constructed this way will have a characteristic trajectory shape — diverging from benign inputs in the early layers while converging to the malicious fiber in the middle layers. That shape is detectable in a way that no single-layer snapshot can be.
This is not a complete solution — it trades one hard problem (single-layer probing) for a harder one (full-trajectory classification). But it is, I think, the correct framing of the problem. Current interpretability-based safety work focuses on identifying what concepts are encoded at specific layers. Trajectory monitoring requires understanding how those encodings evolve across the full forward pass, which is a more ambitious but more robust target.
I wrote a small PyTorch MLP to illustrate constraint compatibility — the idea that gradient descent can find a distinct input vector that approximately matches a target vector's activation at an intermediate layer, under an L2 regularization penalty. You can find it in the accompanying code.
I want to be precise about what this does and does not show. The toy model demonstrates something that is mathematically guaranteed by the architecture: a non-injective function has large fibers, and gradient descent can find points on those fibers. This is not empirical evidence for the main hypothesis; it is an illustration of the mathematical structure that motivates it. The many-to-one behavior in the toy comes from an explicit dimensionality bottleneck, not from polysemanticity. The L2 penalty is a stand-in for perplexity, not an equivalent of it. The inputs are continuous vectors, not discrete tokens.
The toy is useful for building the intuition. It does not validate the claim that this scales to a frontier transformer. That validation requires compute I do not have.
The questions I most want to see explored:
Can GCG be extended with a dual loss? The standard GCG loss targets output behavior. Adding a term for intermediate layer convergence while maintaining a perplexity penalty is the operational version of this hypothesis. Whether this optimization problem is tractable on a 70B+ model is the central empirical question.
How does RLHF affect fiber geometry? If RLHF implicitly separates the fibers over malicious target activations from the fluent-text region of input space, it provides a geometric safety guarantee that has not been characterized. Understanding this geometry would be directly useful for safety research.
What does a trajectory-monitoring classifier look like? The defensive idea needs to be formalized. What features of the full-trajectory activation trace are discriminative? Is it the rate of convergence across layers, the cosine distance profile, something else?
If any of this is useful, the torch is yours.