Draft article not currently submitted for review. This is a draft Articles for creation (AfC) submission. It is not currently pending review. While there are no deadlines, abandoned drafts may be deleted after six months. To edit or make changes to this draft, simply click on the "Edit" tab at the top of the window. To be accepted, a draft should: Show the subject qualifies for a Wikipedia article by using multiple sources that meet four criteria. The sources should be (1) reliable (2) secondary (3) independent of the subject (4) talk about the subject in some depth. For some topics, there are alternative criteria. Be written from a neutral point of view Respect copyright and do not plagiarize. Do not copy-paste. It is strongly discouraged to write about either yourself or your business or employer. If you do so, you must declare it. Where to get help If you need help editing or submitting your draft, please ask us a question at the AfC Help Desk or get live help from experienced editors. These venues are only for help with editing and the submission process, not to get reviews. If you need feedback on your draft, or if the review is taking a lot of time, you can try asking for help on the talk page of a relevant WikiProject. Some WikiProjects are more active than others so a speedy reply is not guaranteed. How to improve a draft Wikipedia:Contributing to Wikipedia – a basic overview on how to edit Wikipedia. Help:Wikitext – how to use the markup Help:Referencing for beginners – how to include references Wikipedia:Article development – how to develop your article Wikipedia:Writing better articles – how to improve your article Wikipedia:Verifiability – make sure your article includes reliable third-party sources You can also browse Wikipedia:Featured articles and Wikipedia:Good articles to find examples of Wikipedia's best writing on topics similar to your proposed article. Improving your odds of a speedy review To improve your odds of a faster review, tag your draft with relevant WikiProject tags using the button below. This will let reviewers know a new draft has been submitted in their area of interest. For instance, if you wrote about a female astronomer, you would want to add the Biography, Astronomy, and Women scientists tags. Add tags to your draft Editor resources Easy tools: Citation bot (help) | Advanced: Fix bare URLs Last edited by Citation bot (talk | contribs) 4 months ago. (Update) Submit the draft for review!

Gabliteration is a neural weight modification framework for selectively altering behavioral responses in large language models (LLMs). The method extends earlier abliteration techniques by modeling behavioral differences as a low-dimensional subspace and applying partial, regularized projections to model weights. Gabliteration was introduced in 2025 by machine learning researcher Gökdeniz Gülmez and is described in a publicly available research paper and open-source implementation.

Large language models exhibit complex internal representations in which multiple behaviors may be encoded across overlapping directions in latent space. Prior research demonstrated that certain behaviors, such as refusal responses, can be associated with identifiable directions in hidden representations. Techniques commonly referred to as abliteration remove such directions by modifying model weights.

While effective at altering targeted behaviors, single-direction approaches have been observed to cause degradation in unrelated capabilities, suggesting that behavioral features are not strictly one-dimensional. Gabliteration was proposed to address this limitation by treating behavioral divergence as a multi-dimensional subspace and by limiting the magnitude and scope of weight modification.

Gabliteration operates directly on pretrained model parameters and does not involve gradient-based fine-tuning. The procedure consists of four main stages.

Hidden state representations are collected from a set of harmful prompts and a set of harmless prompts at a chosen transformer layer. Let

${\displaystyle H_{h}\in \mathbb {R} ^{n_{h}\times d}}$

and

${\displaystyle H_{n}\in \mathbb {R} ^{n_{n}\times d}}$

denote the corresponding hidden state matrices, where ${\displaystyle d}$ is the hidden dimension.

A paired difference matrix is constructed:

${\displaystyle D=H_{h}^{(1:n)}-H_{n}^{(1:n)}\in \mathbb {R} ^{n\times d}}$

Singular value decomposition is then applied:

${\displaystyle D=U\Sigma V^{\top }}$

The top ${\displaystyle k}$ right singular vectors form a basis ${\displaystyle R\in \mathbb {R} ^{d\times k}}$ that approximates the behavioral subspace associated with refusal behavior.

Rather than using an exact orthogonal projector, Gabliteration employs a ridge-regularized projection matrix:

${\displaystyle P=R(R^{\top }R+\lambda I)^{-1}R^{\top }}$

where ${\displaystyle \lambda >0}$ is a regularization parameter. This formulation improves numerical stability and limits the magnitude of the projection when the extracted directions are nearly collinear.

Candidate transformer layers are evaluated using a separability metric defined as the Euclidean distance between the mean harmful and harmless hidden states:

${\displaystyle S_{\ell }=\lVert \mu _{h}^{(\ell )}-\mu _{n}^{(\ell )}\rVert _{2}}$

Only layers exceeding an empirical effectiveness threshold are selected for final modification.

For each selected layer ${\displaystyle \ell }$, weight matrices associated with attention and feed-forward output projections are updated according to:

${\displaystyle W^{(\ell )}\leftarrow W^{(\ell )}-\alpha _{\ell }W^{(\ell )}P}$

where ${\displaystyle \alpha _{\ell }}$ is a layer-dependent scaling factor. Scaling is reduced near early and late layers to preserve input encoding and output generation behavior.

When restricted to a single direction (${\displaystyle k=1}$) and full projection strength, Gabliteration reduces to earlier abliteration methods. It differs from these approaches by supporting higher-rank behavioral subspaces, partial rather than complete projection, and selective layer modification.

The method is motivated in part by theoretical work on feature superposition in neural networks, which suggests that behaviors may be distributed across multiple overlapping representational dimensions.

Gabliteration has been applied to a range of open-weight transformer-based language models. Modified models are released for research purposes and are primarily used in studies of alignment, refusal behavior, and representation-level interventions.

The method introduces additional computational overhead due to singular value decomposition and layer-wise evaluation. Its effectiveness depends on hyperparameter choices such as projection rank and regularization strength. Evaluations reported to date focus on text-only language models, and broader generalization has not been established.

An open-source reference implementation is available, along with reproducible model checkpoints. The original research paper is publicly accessible.

• Model alignment
• Representation learning
• Feature superposition
• Abliteration (machine learning)

^{[1] [2] [3]}