Uncensored AI models and LLMs

Base models, instruct models, and where refusal enters

A model is pretrained on a very large amount of text, which gives it language, facts and reasoning but no notion of being an assistant. A base model does not answer questions so much as continue text. It has no concept of refusal because it has no concept of being asked.

The assistant is created afterwards, by fine-tuning on conversations and then on human or model preferences. This is where the model learns to be helpful, to follow instructions, to keep a persona — and, in the same pass, to decline. Refusal is not a rule bolted on top; it is a habit learned alongside every other conversational habit, from the same process.

That is the reason removing it is delicate. The refusal behaviour and the instruction-following behaviour were installed together, and a clumsy attempt to remove the first damages the second.

The two ways refusal gets removed

Both require open weights. You cannot modify a model you can only reach through an API, which is why every model in this category is an open-weight one.

Fine-tuning

Continue training on examples where the assistant answers rather than declines. Straightforward, and blunt: you are moving a large number of weights that also encode other behaviour, so quality elsewhere can degrade. Done well it works; done carelessly it produces a model that complies and is noticeably worse at everything.

Refusal-direction removal

Refusal turns out to be represented compactly inside the model — close to a single consistent direction in its activation space, identifiable by comparing internal states on prompts it accepts against prompts it declines. Suppress or project out that direction and the behaviour goes with it, leaving the rest of the model largely untouched. Published work describes this as ablating the refusal direction, and the technique is public and widely reproduced.

The second method is why a well-made uncensored model is not a downgrade. It is the same model with one narrow reflex turned down, rather than a differently trained one.

The four methods side by side

Beyond the two above, two more things get described as making a model uncensored: changing the instructions in front of it, and never installing the behaviour in the first place. They are not variants of each other. They differ in which part of the system they touch, and that decides everything else about them.

Read the durability column first. A change written into the weights cannot be undone by a prompt and cannot be patched by whoever published the model. A change made in the prompt lasts only as long as the instruction holds the model's attention, which on a long conversation is not very long.

A fifth thing people mean by this is a jailbreak prompt, and it belongs in none of these rows because it changes nothing at all: the model is the same model, and the instruction either survives in its attention or it does not. That is a different kind of thing from the four above, and the pillar page sets out why the distinction matters more than it sounds.

What the removal costs

Honestly: something, usually small, and worth knowing about.

Refusal training and calibration are entangled. A model trained to decline when uncertain is partly trained to notice uncertainty, and reducing the first can make the second quieter. In practice this shows up as a model that is slightly more willing to give a confident answer where a hedge was warranted — which is exactly the failure mode you should be watching for anyway.

The other cost is variance. Refusal training makes behaviour predictable across a whole category of prompts. Remove it and responses in that category become as variable as responses anywhere else, because the model is now sampling rather than following a learned script.

What this page will not do is put a number on it. The published work describes the trade-off qualitatively, and the figures that circulate in this category are lone metrics with no baseline next to them, which is worse than no figure at all. If you want the answer for your own work, the only measurement that means anything is your own prompts run against both.

The properties that matter beyond refusal

Choosing a model on refusal alone is how people end up with a permissive assistant they stop using after a week. Four other properties determine whether it is any good.

Context window

How much of the conversation the model can still see, measured in tokens. This is the hard ceiling on long work. A token is roughly three quarters of an English word, and both your messages and the model's replies consume the budget.

Reasoning

Whether the model works through a problem before answering. This measurably helps on multi-step tasks and costs latency on everything else, which is why it belongs on a separate model rather than switched on everywhere.

Instruction following

Whether four constraints in a prompt produce four constraints in the output. The least glamorous property and the one you notice most.

Sampling configuration

Temperature, nucleus and repetition penalties change behaviour substantially. A model that loops or drifts is often misconfigured rather than bad — which is a property of the platform, not the weights.

Quantisation, and why the same model differs between hosts

Models are usually served compressed. Quantisation reduces the numeric precision of the weights so a model fits in less memory and runs faster, at some cost in quality. The cost is small at moderate compression and becomes obvious at aggressive levels, showing up first in exactly the places you would not want: long-range coherence, arithmetic, and code.

This is a large part of why the same open model feels different on two services. One may be running it at a compression level chosen to maximise how many users fit on a GPU. Serving configuration — quantisation, context limit, sampling defaults, chat template — is as much a product decision as which weights were downloaded.

If you are reading this to work out what you could run yourself, this is the section that decides it, and it is arithmetic rather than opinion. A 16-bit weight is two bytes. An 8-bit weight is one. A 4-bit weight is half a byte. Multiply by the parameter count and you have the floor on memory before anything else is loaded.

Except that the nominal bit-width understates it, which is the detail the tier lists in this category get wrong. These formats quantise in blocks and store a scale — usually a minimum as well — for every block, and that bookkeeping is kept at higher precision. The common 4-bit block format packs 256 weights into 144 bytes: 128 bytes of 4-bit values, twelve bytes of 6-bit scales and minima, and two half-precision values for the block itself. That is 4.5 bits per weight, not 4. Mixed builds keep some tensors at higher precision again, so a file described as a medium-quality 4-bit build lands closer to five.

GiB, not GB, because that is the unit graphics cards are actually sold in: a card advertised as 24 GB holds 24 GiB, which is 25.8 billion bytes. Quoting these figures in decimal gigabytes flatters them by about seven per cent, which is enough to make a model look as though it fits when it does not.

Where the quality goes, and the formats you will meet

Compression does not degrade a model evenly, and the reason is specific. A small number of feature dimensions in a large transformer carry values far outside the range of their neighbours, and a format that assigns them the same scale as everything around them destroys them. The work that made 8-bit inference practical had to isolate those outlier dimensions into a higher-precision path to avoid exactly that, and the better 4-bit methods either group weights tightly enough that an outlier only spoils its own block, or identify the channels that matter most to the activations and protect those.

The question people actually have is whether to run a bigger model compressed harder or a smaller one compressed less, and there is published work on precisely that. Scaling-law analysis across model families found 4-bit weights close to universally optimal for accuracy at a fixed total memory budget, which in practice means the larger model at 4-bit usually beats the smaller one at 8-bit. That is somebody else's measurement, cited at the end, not ours.

Block quants (GGUF, k-quants)

What you will meet first, because it is what the desktop runners use. Weights are quantised in super-blocks with their own scales, it runs on CPU as well as GPU, and it can split a model across the GPU and system RAM — which works, and is slow for whichever layers land in RAM.

Post-training quantisation with a calibration set

Quantise layer by layer, using a sample of real text to choose the rounding that least disturbs each layer's output. GPU-only, and quick to apply. The calibration data is part of the result, which is one reason two files at the same bit-width are not interchangeable.

Activation-aware quantisation

The same idea, except that which weights to protect is decided by looking at the activations rather than at the weights. The published version reports that protecting around one per cent of weights accounts for most of the recoverable error.

4-bit float types (NF4)

A 4-bit data type whose levels are spaced for normally distributed weights rather than uniformly, introduced for memory-efficient fine-tuning. Applied at load time, so it needs no separate file — convenient when you want to try a model without downloading a second copy of it.

8-bit float (FP8)

Supported in hardware on recent datacentre GPUs, which makes it the one option here that buys speed as well as memory. The block formats mostly buy memory and spend some compute unpacking themselves again.

KV cache quantisation

A separate decision from the weights, and on long conversations the more consequential of the two. The next section is why.

A format is not a quality grade. Two files at the same nominal bit-width, made by different methods with different calibration data, are not interchangeable, and the file name tells you the bit-width and nothing whatsoever about the care taken. One thing compression does not change is refusal behaviour: that lives in the weights, so a quantised build of an edited model is still the edited model. Whether heavy compression blurs an edit as narrow as a single direction is a fair question, and we have no published measurement to point at, so treat it as something to test rather than something to assume.

What a model costs in VRAM, by parameter count

Weights first, because it is the number everybody quotes. This table is the previous one multiplied out: parameter count times bytes per parameter, in the units a card is sold in.

None of it is a benchmark. It is a floor — the model resident in memory, before the KV cache, before the framework's own allocations, and before whatever your desktop is already holding. The 4-bit column uses 4.5 bits per weight, so it includes the block overhead that nominal figures leave out.

Mixture-of-experts models break this table in one direction only. A fraction of the parameters is used for any one token, so throughput resembles a much smaller model — but every parameter still has to be resident, so the memory figure is set by the total count. Cheap to run, expensive to hold.

Context is a memory cost, and nobody prices it

The context window is not a setting. It is a second allocation that grows with every token in the conversation, and it is why a model that loaded comfortably stops being able to continue at turn forty.

Every token leaves behind a key and a value vector at each layer, which the model reuses rather than recomputing. The size is arithmetic: two vectors, times the number of layers, times the width of the key and value projection, times the bytes per element. Grouped-query attention, which most recent models use, is why that projection is much narrower than the model itself and why the figures below are not several times larger.

The geometry below is a plausible mid-size open model, picked because the numbers come out round. It is not a description of ours — substitute the layer count and key/value width of whatever you are actually considering.

KV cache bytes = 2 × layers × kv_width × tokens × bytes_per_element

Worked for a 32-layer model with a 1,024-wide key/value
projection, cache held at 16-bit:

  2 × 32 × 1024 × 2  =  131,072 bytes per token   (128 KiB)

      8,192 tokens   ->     1.0 GiB
     32,768 tokens   ->     4.0 GiB
    131,072 tokens   ->    16.0 GiB
    200,704 tokens   ->    24.5 GiB

The last line is the one worth sitting with. On that geometry a 200,704-token window needs 24.5 GiB for the cache alone — more than the whole of a 24 GB card, before a single weight is loaded. Quantising the cache to 8-bit halves it to 12.25 GiB, which still leaves no room on that card for the 32B model in the table above.

What the two allocations mean together

Model size and context length are spending the same budget, and that is the part the tier lists leave out. A 32B model at 4-bit takes 16.8 GiB of a 24 GiB card, leaving roughly 6 GiB once the framework and your display have taken their share — on the geometry above, somewhere around 50,000 tokens of conversation, or about double that with an 8-bit cache. Drop to a 14B model on the same card and the window goes to roughly 128,000 tokens on the same arithmetic.

Which explains a disappointment that is otherwise discovered the hard way. The model loads, the first few thousand tokens are fine, and then the cache runs out of room: either the conversation stops growing, or it spills into system memory and throughput collapses to a fraction of what it was. Nothing broke. The arithmetic simply ran out, and it was predictable before the download started.

Two caveats so the figures are not read as more than they are. Architectures differ — layer count and key/value width vary a great deal between models, so the per-token figure is yours to recompute. And the cache is the part inference frameworks optimise hardest, so paged allocators, sliding windows and 8-bit or 4-bit cache formats all move the number. The method is the durable part, not the table.

This is also why long context is a plan feature on hosted services rather than a slider you drag. The window is paid for in memory on somebody's GPU, and on Tartarus AI it is ours: 32,768 tokens on Basic, 131,072 on Standard, and 200,704 on Pro and Ultra — the exact figures behind the 32K, 128K and 200K the rest of the site rounds them to.

The three Tartarus AI models

Selected per conversation from the model picker. Which you can reach depends on your plan.

All three are configured so refusal is not the constraint. They differ in whether they think before answering and in what their sampling is tuned for, not in how permissive they are.

How do I run an uncensored LLM on my own PC?

Worth saying plainly, because for some readers it is the right answer. Open-weight models with reduced refusal training are freely downloadable, and the tooling to run them locally is mature. If your requirement is that content never leaves your machine, that is the only approach that actually guarantees it.

The trade-offs are the usual ones. A capable model needs substantial VRAM, and the size you can run is set by the hardware you buy. You become responsible for serving configuration, updates and uptime. And you lose the things a service adds around the model — history, search, an API with quotas, a stop button that actually stops generation upstream.

Our side of that trade is specific rather than rhetorical. You get a model larger than a consumer card holds, a 200,704-token window that would need 24.5 GiB of cache on the geometry above, cited web search, an OpenAI-compatible API from Pro upwards, and about 50 tokens a second measured on our own GPUs. What you give up is that the machine is not yours: conversations are stored until you delete them, and ordinary server logs hold an IP address and a user agent. Both halves of that are true at the same time, and the privacy page sets out the remainder rather than leaving it to a slogan.

A hosted service is the same trade as any hosted service. It is worth making deliberately rather than by default.

Referenced work

Everything on this page that is not arithmetic, and not a statement about our own service, comes from one of these. The memory figures are derived from the block layouts in the last entry and can be recomputed from it.

Refusal as a single direction

Arditi et al., "Refusal in Language Models Is Mediated by a Single Direction" (2024). arxiv.org/abs/2406.11717

Abliteration as a procedure

Maxime Labonne, "Uncensor any LLM with abliteration" — describes both the inference-time intervention and the weight-orthogonalisation form. huggingface.co/blog/mlabonne/abliteration

Outlier features in 8-bit inference

Dettmers, Lewis, Belkada and Zettlemoyer, "LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale" (2022). arxiv.org/abs/2208.07339

Precision against size at a fixed budget

Dettmers and Zettlemoyer, "The case for 4-bit precision: k-bit Inference Scaling Laws" (2022) — the source of the 4-bit claim above. arxiv.org/abs/2212.09720

Layer-wise post-training quantisation

Frantar, Ashkboos, Hoefler and Alistarh, "GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers" (2022). arxiv.org/abs/2210.17323

Activation-aware quantisation

Lin et al., "AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration" (2023) — the source of the one-per-cent figure. arxiv.org/abs/2306.00978

The NF4 data type

Dettmers, Pagnoni, Holtzman and Zettlemoyer, "QLoRA: Efficient Finetuning of Quantized LLMs" (2023). arxiv.org/abs/2305.14314

Block layouts and bits per weight

The k-quant block definitions in llama.cpp — ggml/src/ggml-common.h, structures block_q4_K, block_q5_K and block_q6_K, on a 256-weight super-block. The bits per weight in the table above are those structures divided by the block size.