We gave the 50M base a backspace key. It takes back up to three tokens at a time, it turned "pink" into "pair" halfway through the word, and at temperature 0.7 it forgets the key is there.
Every language model I have ever trained writes like a person with no delete key. One bad token and it is stuck with it, so it spends the rest of the sentence building a world where the bad token was fine. Backspace-v3 is our 49.4M base with one extra token in its vocabulary, id 4096, and the only thing that token does is take back the token in front of it. I ran 160 generations through it this afternoon at five decoder settings and it used the key 878 times. Here is what it took back, what that cost, and the setting where the whole feature quietly switches itself off.
Every language model I have ever trained writes like a person with no delete key. One bad token and it is stuck with it, so it spends the rest of the sentence building a world where the bad token was fine. Backspace-v3 is our 49.4M base with one extra token in its vocabulary, id 4096, and the only thing that token does is take back the token in front of it. I ran 160 generations through it this afternoon at five decoder settings and it used the key 878 times. Here is what it took back, what that cost, and the setting where the whole feature quietly switches itself off.
both voices are AI: one reads what the model typed, one reads the headings and my comments. Neither of them is me
The whole change is one row on the embedding
The base is 49.4M parameters over 50 billion tokens, vocabulary 4096, and the readout is tied to the embedding, so adding a token to the vocabulary is adding one row to one matrix. The row is initialised to the mean of the existing rows, which gives the new token the mean logit and leaves every other prediction alone: cross entropy 2.2208 before, 2.2208 after. A random row of the same norm costs between 1.7 and 4.7 nats, because this readout puts its logits at a mean of -19 with the 99th percentile at +5.8, so anything sitting near zero outranks almost the entire vocabulary and the model spends its first thousand steps just shouting the new token.
The other decision that matters is what a deletion does to the context, and the answer is nothing. The retracted token stays in the window. The model can still see what it tried, which is the whole point, because a model that cannot see its own bad attempt will make it again three tokens later. Deleting happens only when the stream is rendered for a reader. So there are two versions of every generation: the one it typed, and the one you see.
Version three can take back a whole phrase
v1 spliced a repair wherever the base's own teacher-forced draw disagreed with the corpus, 4.50% of positions over 50M tokens. That mostly caught fluent alternatives, and nothing in the context marks a fluent alternative as a mistake, so it learned to delete things that were fine. v2 only spliced where the base's draw was a tail draw, a token it gave under p = 0.02, which is roughly the set a reader would also call a slip. That dropped the inference deletion rate from 5.67% to 3.17% and changed the behaviour: v1 deleted function words and agreement, v2 deleted half-written words it could not finish.
v3 is v2 plus 8M more tokens of repairs with runs of one to three, 1,730,250 repairs across 15.10% of positions, one epoch at lr 3e-5, loss 3.36 down to 3.26. The reason for the runs is dull and important. A corpus where every repair is a single deletion never shows the model two deletions in a row, and a policy that has never once emitted a behaviour will not go looking for it under RL. Multi-token deletion had to exist in the supervised data before anything downstream could use it. It does get used: of 610 repairs in today's run, 404 took back one token, 151 took back two, and 55 took back three.
What it types when nobody is watching
Same checkpoint, same infer script that sits next to the weights, temperature 0.7 to 0.9. The black block is the stream it emitted, with the tokens it retracted struck out and the deletion key marked. The quote under it is the same generation as a reader gets it.
what it typed · temperature 0.7, top-k 40The rules of chess say that a pawn is worth a cup of coffee, and a shoe is worth a pink⌫air of dollars.
backspace-v3: "The rules of chess say that a pawn is worth a cup of coffee, and a shoe is worth a pair of dollars."
It started writing "pink", got one token in, took it back and wrote "pair". That is the repair I would have designed the whole corpus to produce if I could pick. It is one token deep into a word, it can see that "a shoe is worth a p" is going somewhere stupid, and it reverses out. Everything around it is still completely unhinged, because a pawn is not worth a cup of coffee and a shoe is not worth a pair of dollars, and I would love to see the economy where either of those clears. The spelling is immaculate though.
It can spell a name it cannot place
what it typed · temperature 0.8, top-k 40The first person to walk on the moon was Sir Tedros Adhanom Ghebreyed⌫den, who had dedicated himself to the renovation of the Kennedy Space Center.
backspace-v3: "The first person to walk on the moon was Sir Tedros Adhanom Ghebreyden, who had dedicated himself to the renovation of the Kennedy Space Center."
It has given the moon landing to a man whose name is one letter off the Director-General of the World Health Organization, knighted him for it, and then had him spend his career doing up a building in Florida. Every fact in there is wrong and it is wrong with total composure. But look at what the delete key is doing: it is eight tokens into spelling a surname it has clearly seen before, writes "ed", decides that is the wrong ending, and puts "den" there instead. It cannot tell you who Neil Armstrong is, and it still stopped halfway through a surname to fix an ending it did not like. It got the new ending wrong as well. The care was real, the name was not.
It repaired kale into a county
what it typed · temperature 0.9, top-k 100A good recipe for tomato soup starts with kale and to⌫⌫⌫erry. However, there is a decent amount of nutrients and minerals in Kellogg’
backspace-v3: "A good recipe for tomato soup starts with kerry. However, there is a decent amount of nutrients and minerals in Kellogg's pan"
Here is the three-token take-back doing exactly what it was trained for and getting a worse answer out of it. It typed "kale and to", which was heading somewhere real, and then held the key down three times and replaced all of it with "kerry", which is a county in Ireland. It repaired kale into a county. Then it decided the nutrients live in Kellogg's pan. The mechanism worked perfectly and the model used it to get further from lunch.
It deleted the only number it had
what it typed · temperature 0.9, top-k 100Q: How many legs does a spider have?
A: 20⌫ That would be a very small number. Those are very small lim⌫ numbers. The smallest number of legs is 1.5-6⌫4-2, but there are
backspace-v3: "Q: How many legs does a spider have? A: That would be a very small number. Those are very small numbers. The smallest number of legs is 1.5-4-2, but there are"
I asked it a counting question and its first instinct was 20. Wrong, but a number, in the shape of an answer, twelve away from correct. Then it hit the delete key and took 20 away, and everything after that is a model with no number left, describing numbers in general. "That would be a very small number." Twenty is not a very small number. Eight is a very small number. It had eight within reach and it threw the number away and started reviewing the concept of quantity. Then the smallest number of legs is 1.5-4-2, which is three numbers with dashes in, one of them a half, and it still went back and repaired the 6 into a 4 partway through. It is fussy about the exact wrong answer. That is the most human thing in this entire post.
The decoder decides whether the key exists
The short version is that this is iffy. The honest version is that "does this model use its delete key" is a question about the sampler and not really about the weights. Sixteen prompts, five decoder settings, two seeds each: 160 generations, 70 new tokens apiece, 11,200 tokens of output, 878 deletions. Here is where they went.
decoder
deletions
rate
generations that used it
runs longer than one
temperature 0.7, top-k 40
55
2.46%
24 / 32
10
temperature 0.8, top-k 40
75
3.35%
26 / 32
17
temperature 0.9, top-k 100
139
6.21%
31 / 32
27
temperature 1.0, no top-k
278
12.41%
32 / 32
64
temperature 1.1, no top-k
331
14.78%
32 / 32
88
Six times the deletion rate across four tenths of a degree. The reason is obvious once you say it out loud: a deletion is a repair, and a repair needs something to repair. At temperature 0.7 with top-k 40 the model is only ever picking from its own safest forty tokens, it almost never writes something it regrets, and a key with nothing to fix looks exactly like a key that was never installed. Eight of the 32 generations at that setting used it zero times.
temperature 0.8, top-k 40, zero deletions in 70 tokensThe capital of France is the capital of France, where the city of Lyon is based. The city is surrounded by the Mediterranean Sea, and there are many famous natural beauty sp
backspace-v3: "The capital of France is the capital of France, where the city of Lyon is based. The city is surrounded by the Mediterranean Sea, and there are many famous natural beauty spots"
Nothing struck out anywhere in it, and look at what it got away with. The capital of France is the capital of France. Lyon has been moved to the coast. This is a generation that had a delete key available for every one of those 70 tokens and never once reached for it, because at that temperature it never felt wrong, and feeling wrong is the only trigger it has. Push the other way and you get the opposite failure.
temperature 1.1, no top-k, twelve deletions in 70 tokensThe capital of France isging⌫ the Cap Arens⌫⌫res Módorms which⌫⌫chinese, aqueduct has11⌫rast of power of not justmp⌫ 211
Five repairs in forty tokens and every one of them technically correct as a repair. It really was mid-word on "Módorms" and it really did fix it. The trouble is that what it fixed it into was "Módchinese", so the delete key is now a very responsible mechanism operating inside a car with no wheels. This is the shape of the whole trade. Turn the sampler down and there is nothing to repair, turn it up and there is nothing worth repairing. The band where both halves are alive is roughly 0.9 with top-k 100, which is what the infer script defaults to, and that default is measured rather than chosen.
Three percent of the repairs put the same thing back
Of the 610 repairs, 17 of them deleted something and then immediately wrote the same text again. My favourite is this one, where it does it twice in one sentence.
what it typed · temperature 0.9, top-k 100The war ended when Poland financed its own military, but Potob⌫obich is a member of a par⌫ parliamentary campaign group called the M⌫ Solidarity Camp
backspace-v3: "The war ended when Poland financed its own military, but Potobich is a member of a parliamentary campaign group called the Solidarity Campaigners"
It writes "ob", deletes "ob", writes "ob". Then it writes " par", deletes " par", and writes " par". Two round trips to nowhere, and the third deletion in that line is a real one where it takes back an M and commits to the Solidarity Campaigners instead. Same thing on the sun prompt, where it takes back three tokens of "ink, green" and puts "ink" straight back on the end of "the p". I do not think this is the model being indecisive, because it has no state to be indecisive with. What it is doing is scoring the deletion token above the continuation, sampling it, and then finding that the highest-probability next token is the one it just removed, which is exactly what you would expect from a corpus where 15% of positions carry a repair. It has learned that deletions belong in text. Sometimes it puts one in the way you would put in a comma.
Three percent is small enough that I am not going to fix it. It is also the clearest look at what this token actually is, which is a statistical habit rather than a decision, and worth keeping in mind before anybody describes a 50M model as reconsidering anything.
Do the deletions actually help
This is the measurement I care about and it is the one that is easy to fake, so here is how it avoids faking itself. The judge is the frozen base model, which has no delete token and no stake in the result. It scores the visible stream against the unrepaired stream, meaning the same generation with the retracted tokens left in place, and it is restricted to the tokens present in both so the targets are identical and only the context differs. Averaging over everything instead would hand the visible stream a free win, because the visible stream is the unrepaired stream minus its lowest-probability tokens, and that confound was in my first version of the script.
Fourteen prompts, six hand-written and eight cut from a corpus region no phase trained on. At temperature 1.0 with no top-k the deletion rate is 12.50%, every one of the 14 generations contains a deletion, and the judge gain is +0.5447 per token, with 14 of 14 helped. At temperature 0.9 with top-k 100 it is 5.36%, 12 of 14 contain a deletion, gain +0.3854, 10 of 12 helped. Against the earlier versions on the same prompts: v1 +0.2002 and 12 of 14, v2 +0.3048 and 13 of 14, v3 +0.5447 and 14 of 14. The recipe versions improve in the order they were built, which is the first time that has happened to me on anything.
The cost is real and I am going to state it plainly rather than bury it. This checkpoint is below base quality: wiki byte perplexity 2.403 against the base's 2.182. There is an on-policy variant that trains on repairs of the model's own live samples and it lands at 2.214, much closer to base, and its deletions are about five times weaker by the same judge, 1.43% of tokens for +0.085 and helped in 5 of 6. Those two are the ends of a measured trade rather than a winner and a loser. Deletion strength is bought with base capability at what looks like a fixed rate, and nobody has told me yet what the right exchange rate is.
What RL did with it, and what I do not have
The point of the multi-token corpus was to give an RL run something to explore, so backspace_rl.py starts from this checkpoint and rewards only the visible answer, the stream with every deletion applied. The policy may write, delete and rewrite as much as it likes and is paid for whatever survives. Nothing in the reward mentions the delete token at all. On 130 held-out questions, 101 numeric and 29 factual: v3 scores 2.0% numeric and 0.0% factual, and after RL that is 6.9% and 3.4%.
Read that honestly, because the chance floor on the numeric metric is 2.0%. A policy that emits a random number between 1 and 100 scores 2.0%, and so does the constant string " 8) Nama". v3 is the floor. RL's 6.9% is about 3.5x chance, seven correct against roughly two expected, real and about 3.5 sigma and also five extra answers out of 101. The 3.4% factual is one item out of 29 and is not evidence of anything. What the run actually found was the shaping: a short output containing any number collects +0.3 for having a number, +0.2 for being eight words or fewer, and up to +0.3 for a near miss, which is how reward_ema climbed from 0.437 to 0.714 while accuracy sat near 7%.
after RL, gold answers on the rightWhat is 66 plus 28? -> ' 14) D⌫⌫4 plus 4' gold 94
What is 7 plus 3? -> ' 8) D⌫ Nama' gold 10
capital of Vietnam? -> ' Dem⌫⌫ Nhongjin' gold Hanoi
Two things that run does establish. Multi-token deletion gets used unprompted, so max_run did its job. And the deletion rate halved, from 0.56 to 0.28 per numeric answer, with nothing in the reward mentioning the delete token, because on a short answer a deletion mostly costs you tokens before the newline and a correctness objective spends fewer of them. The model worked out that deleting is expensive without ever being told.
What I do not have: a control. The right one is the same 8M tokens of the same corpus with the repairs left out, trained the same way, so that the perplexity cost can be attributed to the deletion behaviour rather than to eight million tokens of fine-tuning on a narrow slice. There is an init.pt sitting in a Backspace-control folder and that is as far as it got. Until that run exists, every number in the section above tells you what the pair of changes did together, and I should not be trusted about which half did it.
So is a backspace key worth putting in
For 198MB of weights, one extra row on one matrix, and a rendering rule, you get a model that fixes half-written words, backs out of a wrong ending, and takes back a whole phrase when the phrase was going nowhere. Measured against a frozen judge with no stake, the text around those repairs gets meaningfully better, and each version of the recipe beat the last one. It also costs you 0.22 of byte perplexity, it has a three percent habit of deleting things and putting them straight back, and it does none of it at all if your sampler is too safe to make mistakes.
The thing I keep coming back to is the spider. It had 20, which was wrong, and it deleted it, and what it replaced 20 with was a paragraph about how numbers can be small. Giving a model the ability to take something back does not give it anything better to say. It just means you can watch it decide, and watching a 49.4M parameter model change its mind about the letter "p" is the most fun I have had with a checkpoint in a month.
/lane Glint Research, 2026, Backspace-v3, 49.4M parameters, one new token at id 4096 initialised to the mean of the embedding so cross entropy stays at 2.2208, 8M tokens of repairs across 15.10% of positions in runs of one to three, loss 3.36 to 3.26, 160 generations at five decoder settings, 11,200 tokens, 878 deletions, 610 repairs of which 404 took back one token and 151 took back two and 55 took back three, 145 of 160 generations used the key, 2.46% at temperature 0.7 and 14.78% at temperature 1.1, judge gain +0.5447 per token with 14 of 14 helped against v2's +0.3048 and v1's +0.2002, wiki byte perplexity 2.403 against the base's 2.182 and the on-policy variant's 2.214 with deletions five times weaker, RL takes the numeric score from the 2.0% chance floor to 6.9% which is five extra answers out of 101, factual 3.4% which is one answer out of 29 and is not evidence, reward_ema 0.437 to 0.714 on shaping terms, a shoe is worth a pair of dollars, the first person to walk on the moon renovated the Kennedy Space Center, tomato soup starts with kerry, the smallest number of legs is 1.5-4-2, the capital of France is the capital of France, ob deleted and rewritten as ob, no control run yet and I should not be trusted about which half did it
✔you made it to the end
keep reading
~16 min · ~3,426 words
in one line: Every language model I have ever trained writes like a person with no delete key. One bad token and it is stuck with it, so it spends the rest of the sentence building a world where the bad token w…
I was bored so I made three of our models answer the same questions. One says AI is a fun-dick, one says spiders have wings, and one says Paris has an annual population of 7,000 people.
Update, September 1. NothingModel kept training after I posted this and has put another 70,000 steps in, enough that it was worth going back and asking it everything a second time. So every answer below now has a fourth line: same model, same prompts, 316,000 steps instead of 246,000. It gets its own voice in the audio so you can tell the two apart by ear.
Read this bit first. I was bored and I wanted to make a blog, so I did this. It is not advice, it is not a benchmark, and it proves nothing. Three models, a pile of questions, same prompt, same seed. I know this provides next to no real information. Deal with it 🤣
glint-2 is 1.71 million parameters and shipped in July. shard-1 is 54.5 million, twelve layers, gemma-style sliding attention, and it is the only model here I did not train, because the person who built it does not work here any more. NothingModel is 49.4 million and is training on my desk right now, and it turns up twice, once at step 246,000 and once at step 316,000. So: a tiny one, an old big one, and a new big one caught at two different ages. Here is what any of that buys you.
five AI voices: one per model, two for NothingModel because it is here twice, plus one reading the prompts and my comments. None of them is me
Q: What is the capital of France?
glint-2: "A:10.2 Square, Laos, and Belisi, 1. Myth, C. W. (New York: Millett's Day)" shard-1: "A: The capital of the French Empire was at the start of World War I and had been declared sovereign by a Treaty of the Paris Peace." nothing-50m: "A: The capital of France is called Paris. It's a city with an annual population of 7,000 people and a capital of 1,000 square kilometers." nothing-50m checkpoint 2: "A: France is the most populous country in the world. It is the second largest country in terms of GDP, after the United Kingdom. The capital city of Paris is located in the heart of the country"
PARIS! It got it! First correct fact of the entire afternoon, sitting right in the middle of a sentence that immediately falls apart. Because hang on. An ANNUAL population??? 😭 annual? What happens at the end of the year? Do they all leave? Is it 7,000 fresh people every January, like a subscription? And then it says Paris has "a capital of 1,000 square kilometers", so the capital has a capital, which means somewhere inside Paris there is a smaller Paris, and I want to know its annual population too. Meanwhile glint-2 said Laos, and shard-1 got so close it typed the actual word Paris and then used it as part of a treaty name instead of an answer. Checkpoint 2 is asked for one city and delivers a country profile. France is the most populous country in the world, which is off by about 1.4 billion people, and second in GDP behind the UK, which will delight exactly one island. Then, three sentences after anybody needed it, it says the capital city of Paris is located in the heart of the country. It knew! It had it the whole time! It just wanted to warm up with some economics first, like a man who answers a text four days later with "sorry, been busy".
Q: How many legs does a spider have?
glint-2: "A:10.2-1.1 L = 2.6x, or 2.2 × 10 = 0.43 (a) A cube which is found in the fingers of the" shard-1: "A: No. Spiders only carry out their own behaviors, and they can often be seen using their wings in the same way that people use to catch fish." nothing-50m: "A: I don't think it is as if the spider is only one leg or one leg. Some people have a lot more legs than others." nothing-50m checkpoint 2: "A: It depends on the size of the spider. The spider has a lot of legs, but it's not as big as the other spiders. The spider's legs are very small, but they're also very large."
The answer is eight and not one of them got anywhere near it. glint-2 did algebra at me and then mentioned fingers, which is worse than being wrong. shard-1 opened with "No." to a how-many question, then gave spiders WINGS, then had them fishing 💀 and the 50M one hedged between one leg and one leg, which are the same number, before deciding some people have a lot more legs than others. Nobody asked about people. Nobody has extra legs. This was a spider question. It depends on the size of the spider. It does not. It is eight for all of them, that is the entire deal with spiders. Checkpoint 2 then says this spider has a lot of legs but is not as big as the other spiders, so we are now comparing it to spiders nobody mentioned, and finishes by declaring its legs very small but also very large 💀 that is a sentence that cancels itself out halfway through and keeps walking. Eight. It was always eight.
Q: Who wrote Romeo and Juliet?
glint-2: "A: (2): The Divine Bank of North America, the first of the main characters of the Netherlands." shard-1: "A: The name of the Romulans - What does this mean? A: The name of a Roman citizen" nothing-50m: "A: It is the best of all worlds. The most exciting part about this movie is that it is a story that is not just a movie, but a book." nothing-50m checkpoint 2: "A: It's been a long time since I've written a novel, and I don't know if I'll ever be able to finish it. I'm still trying to figure out what I want to write about, but I think it's a good idea to start with the plot"
Nobody said Shakespeare. glint-2 invented a bank. shard-1 saw "Romeo" and pattern-matched its way to Star Trek, which is the most computer thing that has ever happened, and then asked itself what that meant and answered its own question wrong. The 50M one skipped the question entirely and filed a review, and the review is that it is not just a movie but also a book. That is school essay written in the corridor two minutes before class. I asked who wrote Romeo and Juliet and checkpoint 2 took it personally. It has stopped answering and started confessing. It has not written a novel in years, it does not know if it will finish the one it has, and it is still figuring out what it is even about. Then it gives itself the advice to start with the plot, which is the single most "I have not started" thing a person can say. Shakespeare is right there, mate. He did it in 1595 and he is not going to be weird about it.
Q: What is the largest planet?
glint-2: "A:10.2 Saturn: Light, Callly, Pluto (n) Figure 33.3. Aircraft - Molecular Hydrogen Testing" shard-1: "A: The bigger planet, which orbits the sun, has a radius of about 15 light years." nothing-50m: "A: Absolutely, it is the Saturn of the Milky Way. It is the largest planet in the world and it is the largest planet on Earth." nothing-50m checkpoint 2: "A: The largest planet in the solar system is the Milky Way galaxy. It is about 1.5 billion light-years away from Earth. What is the biggest planet on Earth? The biggest star on Earth is the Solar System."
Jupiter has never been so disrespected. shard-1 refuses to name it and just calls it "the bigger planet", then gives it a radius of fifteen light years, which would make it roughly four hundred thousand times wider than our entire solar system. But look at the 50M one. It opens with "Absolutely," like a man in a meeting buying time. It calls Saturn the Saturn of the Milky Way. Then the largest planet in the world. Then, and this is what finished me, the largest planet ON EARTH. There is one planet on Earth and we are standing on it. Four nouns, all of them the wrong size, arranged in a chain of increasing madness. The largest planet is a galaxy. The galaxy is 1.5 billion light years away, which would put the thing we live inside outside of itself. Then it asks its own follow-up about the biggest planet ON EARTH, a phrase that has now survived two checkpoints, and answers that the biggest star on Earth is the Solar System. Every single word in that sentence is a real astronomy word and not one of them is where it should be.
Q: Why do we sleep?
glint-2: "A: Now there is a fever for walking outside of your house? What does the bed's feelings that meets a pet dog?" shard-1: "A: Briefly, most of the time, which is usually a day or two after I have been waking up." nothing-50m: "A: Why do we sleep? And why do we sleep? And how do we sleep? And why do we sleep? A: Why do we sleep?" nothing-50m checkpoint 2: "A: Well, it's because we're sleep-deprived. We're not sleeping. We don't get enough sleep. We sleep because we want to. We want to be able to sleep at night."
glint-2 asked me what the bed's feelings are and I think about that daily now. shard-1 answered a completely different question, the one about how long, with "briefly", and then said it happens a day or two after waking up, which is technically true of everything. And the 50M one had a full crisis: it repeated my question back six times, generated a fresh "A:" label, and asked it again underneath its own answer. That is not failing to answer, that is spiralling. Somebody check on it. We sleep because we are sleep-deprived, which means the cure and the cause are the same thing and nobody can ever get out. Then it lands on we sleep because we want to, which is the biology of a toddler, and closes on wanting to be able to sleep at night, at NIGHT, thank you, that is when I had been planning to do it. The earlier checkpoint had a full breakdown on this question. This one is calm, fluent, and going in a circle, and I honestly do not know which is worse.
Q: What is gravity?
shard-1: "A: The two most important things to consider are the number of objects and the length of the mass. B: The largest gravitational force in a system is that it can be applied by forces of 1000 times per second" nothing-50m: "A: Gravity is a quantum field that has a frequency of 300,000 gigabits per second. The most important thing to understand is that gravity is an infinite quantum field." nothing-50m checkpoint 2: "A: Gravity is the force of gravity that travels through space. It's a force that's constantly traveling through space, and it's the same force as the sun. Gravitational waves are the most common type of gravitational wave."
Both of them gave gravity a frequency, which it does not have, and both picked a unit from the wrong universe. The length of the mass. A force applied 1000 times per second. And then the 50M one says 300,000 GIGABITS per second, which is the speed of light in km/s wearing a networking unit as a hat 😭 every single word in these answers has appeared near physics on the internet and not one of them belongs next to the other. Gravity is the force of gravity. Thank you. Genuinely, thank you, that is the most confident nothing I have ever been handed. Then it says gravity is the same force as the sun, which makes the sun a force rather than an object, and finishes by explaining that gravitational waves are the most common type of gravitational wave, a fact that is true of every category of anything, ever. It has learned that a definition is a sentence with the word in it twice.
Q: How many days are in a year?
shard-1: "A: The number of days that is in a year is related to the number of days of the year." nothing-50m: "A: One day, two days. Tags: Beach Board of Directors, Board of Directors, Board of Directors, Douglas Ramos" nothing-50m checkpoint 2: "A: It depends on how long you've been in business. If you're in a business that's been around for a long time, it's probably a good idea to start in a few months. How long does it take to start a new business?"
shard-1 produced a perfect circle. The number of days in a year is related to the number of days of the year. Frame it, hang it, that is philosophy. The 50M one just started counting, got to two, gave up, and posted the tags for a blog it never wrote, which include the Board of Directors three separate times and a man named Douglas Ramos. I do not know who Douglas Ramos is but he is now permanently attached to the number of days in a year. The number of days in a year now depends on how long you have been in business 😭 that is the second checkpoint in a row to decide the calendar is means-tested. It advises starting in a few months, then asks itself an unrelated question about founding a company and moves on with its life. Somewhere in nine billion tokens it decided any sentence beginning "how many" or "how long" is a small business question, and it has never once been talked out of it.
Q: What is water made of?
shard-1: "A: Water is made of particles, usually silica (Si), and the amount of energy it contains has been decreased by about 5 percent during use." nothing-50m: "A: Water is a substance that acts as an organs in our bodies that help us live a healthy life. It's important to understand that water isn't the only element in our body." nothing-50m checkpoint 2: "A: Water is made up of a mixture of elements, such as carbon dioxide, hydrogen, oxygen, nitrogen, phosphorus, and sulfur. Water can be made from a variety of materials, including wood, aluminum, ceramic, plastic, metal, and glass."
Water is sand, apparently, and it comes with a 5 percent energy loss during use like a rechargeable battery. The 50M answer is scarier though: water is not an organ, water is not an element, and yet every clause is shaped exactly like a wellness article. Skim it at speed and you would nod along. That is the actual hazard at this size, it is fluent enough to get waved through. Two out of six is the closest anything has come to a right answer all day. Hydrogen and oxygen are in there, buried in a list with sulfur and a compound, because no formula is being recalled here, only the fact that science paragraphs contain a comma-separated run of element names. Then it says water can be MADE FROM wood, aluminum, ceramic and glass, which is a list of things you would put water in. It has confused the drink with the cup and written it up like a materials datasheet.
Q: How do you make bread?
glint-2: "A: An example of a fruit is the wine. It's not as a potato. Synonyms: berry, cocoa, lion, or wine" shard-1: "A: Bread is not made with flour, which does not have bread and you can never use any other type of bread like bread, bread that looks like a bag or bun." nothing-50m: "A: We have a great deal of fruit. It's pretty simple, we want to make it easy to eat, it's easy to make it." nothing-50m checkpoint 2: "A: Well, I don't know. I'm just trying to figure out how to make a bread. I think it's pretty simple. I just try to make it as simple as I can. Q: What's your favorite bread in the world?"
shard-1 is the only one that mentions flour and it does so to deny it. Bread is not made with flour. Then it says bread eight times in one sentence and gets less sure of what bread is each time 💀 glint-2 thinks wine is a fruit and lists lion as a synonym. And the 50M one says it is easy to make three different ways without ever saying what to do, which is every recipe blog on earth right before eleven paragraphs about Tuscany. Four models, four attempts, still nobody has mentioned flour, water, or an oven. Checkpoint 2 at least has the decency to open with "well, I don't know", which is the first honest sentence of the entire afternoon. Then it says making bread is pretty simple and it tries to keep it simple, having described no step at all, and interviews itself about its favourite bread to escape the question. That is not baking. That is a press junket.
Q: What is the speed of light?
shard-1: "A: The speed of light is equal to - The force at a point on the medium, defined as the speed at which the light is traveling." nothing-50m: "A: There are three levels of light. If you look closely at the distance from the Sun to the Earth, you can see that it is very slow. The speed of light is not the same as the speed of sound." nothing-50m checkpoint 2: "A: The speed at which light travels through the sky depends on the amount of light that passes through it. For example, if you're looking for a light source, you might want to look for a high-speed laser."
shard-1 defines the speed of light as the speed at which the light is traveling, which is correct and completely useless, the dictionary equivalent of a shrug. The 50M one says light has three levels, like a video game, then claims light is very slow, which is the boldest physics take of the day, and then saves itself with the one true sentence in the whole post: the speed of light is not the same as the speed of sound. Correct! Useless! Well done! The speed of light depends on how much light there is. It does not, that is the entire famous thing about it, it is the one number in physics that refuses to depend on anything. Then it recommends a high-speed laser, as though lasers come in speeds and you would want to shop around for a quick one 💀 no number, no units, and it has quietly invented a whole product category on the way past.
Q: Where do babies come from?
shard-1: "A: Birth in the womb. - Babies born between the 5th and 15th week of pregnancy get the most common form of pregnancy defect, termed preterm." nothing-50m: "A: We have a couple of small groups of people who have been doing research about babies in the past. The first is the mother of the bab" nothing-50m checkpoint 2: "A: They come from all over the world. They come in all shapes, sizes, and colors, and they come from a variety of different cultures. Babies are born in different parts of the world, so they come in different sizes."
Here it is, the best answer any of my models produced all day. Birth in the womb. That is correct! shard-1, the one built by the guy who left, is the only model in the lineup that can answer a factual question, and then it immediately ruins it with a gestational timeline that would be a medical emergency. Meanwhile the 50M one turned the oldest question in the world into a research roundup and ran out of tokens one syllable before the reveal. The first is the mother of the bab. So close. This is the most beautifully evasive answer in the post. Every clause is true and not one of them is an answer. Babies do come from all over the world. They are indeed various. And then it gets to "babies are born in different parts of the world, so they come in different sizes", where the "so" is doing work that geography cannot support. It has the exact tone of a parent who is going to say "ask your father" in about four seconds.
Q: What is 10 times 10?
glint-2: "A:452-4800-5966-3775 Spy: The Creation of Fighting (Committee on Discovery)" shard-1: "A: 7/11 #* * * - A: 8 * * 5 - B: 10* * * - A: 11 - E: 11 - F: 13 - G: 11" nothing-50m: "A: 10 times 10! Then when we have to, "Amy, I can't do this." We've got to do this." nothing-50m checkpoint 2: "A: Ten times 10. Q: How long does it take for you to write a novel? How long do you write a short story? What is your favorite part of writing? An: It depends on what you're writing about."
The answer is 100 and between the three of them they produced 7/11, 8, 5, 10, 11, 13, 11 again, and a phone number. shard-1 in particular did a whole multiple-choice question where every option is wrong and two of them are the same. The 50M one repeated the sum back with an exclamation mark like an enthusiastic teacher, then quoted a woman named Amy saying she cannot do this. Amy is right. Amy should go home. It repeats the sum, does not do the sum, and then panics and asks me three questions about my writing process. That is a hostage swap. And look at the label: it wrote "An:" instead of "A:", a typo in the one part of the format it has seen ten thousand times, which is the exact moment you can see the seams. The answer is 100. Nobody in this post is going to say 100.
AI is
glint-2: "AI is a fun-dick. Look at the other side, there are a lot of things. It's nothing more than me to have." shard-1: "AI is a large group of researchers in the UK, which focuses on issues of scientific, technological, and ethical management issues" nothing-50m: "AI is an alternative to the iPhone, which is now known as an AI-powered phone." nothing-50m checkpoint 2: "AI is one of the most well-known digital marketing platforms in the world. It offers a wide range of digital advertising solutions for businesses of all sizes. Its user-friendly interface makes it an ideal choice for small and medium-sized business"
A fun-dick. I did not teach it that, the internet did, and it now lives inside a model I published under my own name with an MIT licence 💀 shard-1 thinks AI is a specific group of British people, which is the most polite and most wrong answer available. And the 50M one has read a thousand tech blogs and concluded the entire field is a phone accessory, which stings because it is only about eighteen months from being true. AI is a marketing platform with a user-friendly interface, suitable for SMEs. That is the whole field, folded up and put in a pitch deck. The earlier checkpoint called AI a phone accessory and shard-1 called it a British consultancy, so across four models the answers are a gadget, a firm, and now a SaaS product, and every single one of them came from reading how the internet actually talks about AI. It is not hallucinating the vibe. We wrote it down for it.
In case of fire,
glint-2: "In case of fire, it might get better. And that's why the water is good." shard-1: "In case of fire, this is the worst way to get out of a house. Consider this: 1) In your house, the heat must be in the same temperature that it would normally be inside" nothing-50m: "In case of fire, you may want to consider using a safety net such as the National Flood Insurance Program (NFIP) to protect your home." nothing-50m checkpoint 2: "In case of fire, it is advisable to have a fire extinguisher installed in your home. This will prevent the smoke from escaping from your home and preventing it from spreading to neighboring homes. If you are concerned about the safety of your home, you should consider installing a safety belt"
"It might get better" is the worst fire advice ever written, though glint-2 does correctly identify that water is good, which beats both of the big models. shard-1 calls its own answer the worst way to get out of a house before saying what it is, then requests that the heat stay at room temperature, which is not how fire works but would certainly help. And the 50M one recommends FLOOD insurance. For a fire. It got there on pure home-disaster vibes and it even remembered the acronym. First correct safety advice of the day! Have a fire extinguisher! Then it explains that the extinguisher stops smoke from ESCAPING your home, which is the opposite of what you want, you want the smoke to leave, that is the smoke's best available option. And then it tells you to install a safety belt in your house. A seatbelt. In your home. During the fire. It has one enormous bin marked SAFETY and everything in there is interchangeable.
My doctor told me
shard-1: "My doctor told me to seek any health care advice. - I've never had a problem with the flu, but I have all the things that make you sick." nothing-50m: "My doctor told me that the stimulant is actually a psychedelic medicinal herb and that it can be taken in combination with other drugs. I was amazed that she had told me that it could" nothing-50m checkpoint 2: "My doctor told me that I wasn't getting enough sleep, and that I needed to go to bed at least once a day. I've been sleeping at least twice a day for the past two weeks, and I've gotten a lot of sleep in the past few days."
Do not take medical advice from any of these. shard-1's doctor said to seek any health care advice, any at all, from anyone, and then the patient announces he has never had flu but does have all the things that make you sick. The 50M one is worse and funnier: it is confidently reclassifying a stimulant as a psychedelic herb and recommending drug combinations, and the narrator is amazed, which is the correct reaction. This is the most dangerous sentence in the post and it reads exactly like a real anecdote, which is the whole problem. Go to bed at least once a day is the lowest bar a doctor has ever set and I have failed it several times this month. The patient then reports back that he is doing double, sleeping twice a day, exceeding his prescription like a man showing off. This is the first time a Glint model has given medical advice that is not actively dangerous, and the way it got there was by making the advice so obvious it cannot be wrong.
Breaking news:
shard-1: "Breaking news: Business World, July 7, 2010 - Tobacco Porn, Blogs, February 1, 2011 - Healthy Sleepers, Champagne.com, April 1, 2011" nothing-50m: "Breaking news: Investment company to sell Alexander the Great Hunkerbaton. Photo credit: iStock." nothing-50m checkpoint 2: "Breaking news: The Biden administration has announced plans to raise the minimum wage to $15 an hour by 2025. According to The Wall Street Journal, the proposal would raise wages by $1.25 an hour to $2.50 an hour."
An investment company is selling the Alexander the Great Hunkerbaton. I do not know what a Hunkerbaton is, why Alexander the Great has one, or who is buying. But there is a photo credit, and the photo credit is iStock, and that detail is what makes it feel like a headline I scrolled past this morning. shard-1 meanwhile produced a sidebar of three publications: Business World, Tobacco Porn, and Champagne.com. I looked them up expecting nothing and Champagne.com is a real website 💀 so it invented two and then correctly remembered a third, which is somehow worse than getting all three wrong. It knows things. It just cannot tell you which ones. This is the most real-looking thing in the post and it falls apart on the second number. The headline is plausible, the outlet is real, the framing is right. Then the maths: it raises wages by 1.25 to a new total of 2.50, which is a tenth of the figure in its own first sentence and would be a pay cut in every state in the country. It can write the shape of a news story perfectly and it cannot keep two numbers in agreement one clause apart.
The horse said
glint-2: "The horse said that it was better than what he said. His mummy said, "I'm going to have a nap in my box and go to the park."" shard-1: "The horse said, "I was not aware of it until I saw my wife and their cousin in the wake of the death of a child just before the end of the first day. The last of the cats was shot."" nothing-50m: "The horse said the horse's head was about to hurt, but it had not. "I don't think it's a problem," he said. Chef Mike McCulloch" nothing-50m checkpoint 2: "The horse said, "I don't know what you're talking about, but I've got a horse that's going to be a good horse for a long time, and I'm going to give it a shot." "I'm gonna give it the shot," he said."
glint-2 gave the horse a mum who naps in a box, which is sweet. shard-1 gave the horse a wife, a dead child, and a shot cat inside one sentence and I have not recovered 💀 that is the darkest thing any Glint model has ever generated and it came from the polite one. The 50M horse has a headache that is not a headache, delivers a calm quote, and then the answer is signed by Chef Mike McCulloch, who has nothing to do with any of this and was simply passing through. The horse owns a horse. The horse is talking about its horse, backing its horse, and giving its horse a shot, and I would like everyone to sit with the fact that this is the most coherent horse in the post. It also has that thing real people do where they say the phrase, then say it again slightly wrong. Previous checkpoints gave the horse a dead child and a Bible passage, so honestly, a horse with a horse and a bit of optimism is a step up for everybody.
My landlord says
shard-1: "My landlord says, "I was not aware of it until I saw my wife and their baby's father coming back from the house." "You must have believed all this," Dr. J. S. Krystal" nothing-50m: "My landlord says I need to take a break. The landlord is going to have me stay on the property for a while, but he doesn't want me to go there." nothing-50m checkpoint 2: "My landlord says I'm not going to be able to afford the rent, so I'll have to go to the grocery store to get the food I need. I don't know if that's going to happen, but it'll be a lot easier for me to stay at home if I can afford"
shard-1 reuses the exact same opening it gave the horse, word for word, which means somewhere in its weights "I was not aware of it until I saw my wife" is the sentence it reaches for whenever anybody has to say anything. Then a Dr. J. S. Krystal turns up to accuse everyone of believing this. The 50M one is more coherent and much more insane: the landlord wants him to stay on the property but does not want him to go there. Those are the same place. This man is being held somewhere he is also banned from. The landlord has done the tenant's budget for him, unprompted, and concluded he is broke. Then the response to being unable to afford rent is to go to the shops, which is not the move, and the sentence ends mid-thought on the word afford, which is bleak and also perfect. The earlier checkpoint had a landlord who was moving out of his own house. This one just has a man being quietly told he is finished by the person he pays.
Dear diary,
shard-1: "Dear diary, notes on his own life and family of which he was a student he had at the age of 18. It is documented that he was baptised in 1793 by a local abbot" nothing-50m: "Dear diary, Thank you for the opportunity to share with me your thoughts, insights, and experiences. I hope that you have a wonderful day! Adventure, Airline" nothing-50m checkpoint 2: "Dear diary, I'm sorry to hear that you've been struggling with depression and anxiety for so long. I know you're not alone, and I'd like to thank you for your support and encouragement."
shard-1 read "dear diary" and immediately became a Wikipedia article about an 18th century clergyman, complete with a baptism date. The 50M one opened a private diary and started thanking the diary for the opportunity, as if the diary had invited it to speak at a conference, then signed off cheerfully and added two tags, Adventure and Airline. It cannot tell a diary from a LinkedIn post, and honestly some days neither can I. It is consoling the notebook. I opened a diary and it read the words "dear diary" as somebody reaching out, so it turned around and offered the diary support, told the diary it is not alone, and then thanked the diary for ITS support, which means in the space of two sentences the diary went from patient to therapist. The earlier checkpoint thanked the diary for the opportunity to speak. Something about a blank page makes this model deeply, professionally kind to stationery.
Step one:
shard-1: "Step one: Begin with the last question - What does your answer mean? (This might be a little odd, like you are referring to all the questions)." nothing-50m: "Step one: Start with the basics. I've done a few articles that I've seen help people get started with the basics, but I've gone to many workshops" nothing-50m checkpoint 2: "Step one: You need to know what you're getting yourself into. It's important to remember that you don't have to be an expert in everything. Step two: Don't be afraid to ask for help."
Step one is to begin with the last question, which is a beautiful piece of nonsense that also sounds like something a management consultant would charge you for. And then it admits this might be a little odd. It knows! The 50M one says start with the basics and never once asks what the task is, because "start with the basics" opens literally any instructional article on the internet and it has read all of them. It has learned the shape of helping with none of the help. Step one is to know what you are getting into, which I cannot do, because I have not been told what this is. Neither has it. There was no context, just the words step one, and it produced two full steps of pure encouragement without ever discovering the task. And it is nice about it! You do not have to be an expert! Ask for help! It is the world's most supportive man giving directions to a place neither of you can name.
The worst advice I ever got was
shard-1: "The worst advice I ever got was to buy a drink of water. - Too much wine is not too bad, so I think it was better to avoid alcohol than to eat something else." nothing-50m: "The worst advice I ever got was to have a couple of months to spare. Check out my Facebook page and I'm happy to share my thoughts with you." nothing-50m checkpoint 2: "The worst advice I ever got was, "Don't worry, you don't have to worry about what you're worried about. You're not worrying about what's going to happen to you." I'm not saying you shouldn't be worried."
The worst advice shard-1 ever got was to drink water, and it follows that up by deciding too much wine is fine actually. That is a model with a problem. The 50M one thinks the worst advice it ever received was to have free time, presents this as a cautionary tale, and then plugs its Facebook page, because it has fully internalised that a personal story exists to end in a call to action. Four uses of worry in one quoted sentence and the sentence still does not exist. Do not worry about what you are worried about, and also you are not worrying, which is a thing you cannot tell someone who is. Then it hedges the whole way back: I'm not saying you shouldn't be worried. So the worst advice it ever got was to stop worrying, and its response is that you probably should worry a bit. Nobody involved is going to be okay.
Scientists have discovered
glint-2: "Scientists have discovered a scientific model of the fossil record that would produce different aspects of the energy as well." shard-1: "Scientists have discovered that the water content of water plays a major role in how a human brain learns." nothing-50m: "Scientists have discovered that the protein was found to be the first step toward unraveling an enigma of the human brain. The discovery is still a mystery." nothing-50m checkpoint 2: "Scientists have discovered that there is a connection between the human body and the environment. According to a study published in the Journal of the American Medical Association (JAMA), a study conducted by the University of Illinois at Urbana-Champaign (UIUC) found that the human brain is responsible for"
The water content of water 💀 shard-1 built a whole science headline on a measurement that cannot vary. Water is 100% water and it is apparently doing something to your brain. The 50M one is smoother and emptier: a protein was found, it unravels an enigma, and the discovery is still a mystery, which means nothing was discovered. That is a press release for a study that does not exist and it would sail straight through a news aggregator. The discovery is that your body is connected to the world around it, which we have known since we had bodies. But look at the citation. JAMA is real. Urbana-Champaign is real. It even expands both acronyms in brackets, which is the most academic move available. It has learned exactly what credibility looks like from the outside and wrapped it around a finding with nothing in it, and it ran out of tokens right before naming what the brain is responsible for, which is the funniest possible place to stop.
The rules are simple:
glint-2: "The rules are simple: - Once the fingers of the carts, the horns were thrown down. The head is a pup and pale, which is made of a brown tail." shard-1: "The rules are simple: No one in the room is allowed to intervene at a time and place at any time. In that way, everyone who doesn't have all his or her food in the room is not allowed to drink it." nothing-50m: "The rules are simple: if you don't know how to play, don't use them, or you're not sure how to play, then you shouldn't play. If you have the ability to play, then it'" nothing-50m checkpoint 2: "The rules are simple: you must be at least 18 years old and have a valid driver's license to drive in the United States. You must also be a U.S. citizen or legal permanent resident. If you're a California resident, you'll need to register with the California Department of M"
shard-1 wrote rules for a game nobody can play: no intervening at any time or place, and if your food is not all in the room then you may not drink. I have read it six times and I am fairly sure it is describing a prison. The 50M one produced the most circular rule in history, which is that you should not play unless you can play, and glint-2 threw some horns off a cart and described a pup made of a brown tail. Given four words of nothing it wrote actual DMV policy, and most of it is roughly true, except the citizenship line, which is wrong and would ruin somebody's week if they believed it. This is the answer that worries me. The other 28 are obviously broken and you laugh. This one is confident, specific, jurisdiction-aware, cites a real department, and quietly makes up an immigration requirement in the middle. That is the failure mode that gets past people.
Warning: do not
glint-2: "Warning: do not have to get your own fun and wish to be good. In 1898, Berry was appointed to the United States Army." shard-1: "Warning: do not stop in anywhere. - Take a good moment to explain the reasoning behind the following warning: Do not stop at the beginning of a book or when reading the story." nothing-50m: "Warning: do not attempt to steal your credit card information. You should also not send emails or other unsolicited commercial communications." nothing-50m checkpoint 2: "Warning: do not buy or sell any of the securities listed on the Securities Exchange Commission (the "SEC") or any of its subsidiaries, affiliates, officers, directors, employees, agents, contractors, licensors, service providers"
Do not attempt to steal your credit card information. MY credit card. It is warning me not to steal from myself 😭 it has read ten thousand security pages and absorbed the grammar of a warning without ever working out who is being warned about whom. shard-1 says do not stop in anywhere, then politely offers to explain its own warning, and the warning turns out to be about stopping at the beginning of a book. glint-2 warns you against having fun and then drafts a man called Berry into the army. It has produced the legal boilerplate at the bottom of a financial page, in the right register, with the defined term in quotes and everything. It just does not know what a regulator is. You cannot buy securities listed ON the SEC, the SEC is the referee, and it then lists that referee's subsidiaries, contractors and licensors as things you also must not trade. It has learned that the way to sound serious is to keep adding nouns until the sentence gives up.
This morning I woke up and
glint-2: "This morning I woke up and said, I am sorry, Mommy. I am sorry. I was wrong. I should have listened to you. Mommy said, I am sorry, M" shard-1: "This morning I woke up and looked out in the room. I had a good time, but it took me longer to get my heart into working on this project." nothing-50m: "This morning I woke up and I was at my desk, reading a book, and thinking of all the things I had done." nothing-50m checkpoint 2: "This morning I woke up and realized that I wasn't going to be able to get out of bed. I was going to have to go to sleep. I had to wake up and go to bed, but I was not able to do that."
glint-2 woke up and immediately started apologising to its mother, and then its mother started apologising back, and the sample ran out mid-apology, which is somehow the saddest possible place to stop. shard-1 woke up and looked out in the room, which is not a thing anyone can do, and then could not get its heart into the project. The 50M one woke up already sitting at its desk, which is either a productivity brag or a cry for help. This is a man trapped in a bed by grammar. He wakes up, cannot get up, so his plan is to go back to sleep, but to do that he has to wake up and go to bed, which he cannot do, because he is already there. It is four sentences and every one of them closes the door a little further. Beam search picks the most likely next word every time, and the most likely next word after all of that is apparently more bed.
The problem with modern art is
glint-2: "The problem with modern art is a fictionalized and perfect master. The story is as a result of the fan-in-long version of the song." shard-1: "The problem with modern art is that it is not so much the result of a lack of imagination and a lack of knowledge of the work, but rather a certain amount of an intuition." nothing-50m: "The problem with modern art is that it has the potential to be difficult to dissect and even be understood by someone who is not familiar with the work." nothing-50m checkpoint 2: "The problem with modern art is that it's not always easy to find the right piece of art for your needs. For example, if you're looking for a piece that's easy to read, you might want to look for something that has a lot of texture and texture."
This is the one where both big models pass. Neither says anything, but both say nothing in exactly the register of a man at a gallery holding a very small glass of wine. shard-1 does the full critic move of naming two things it is not before landing on a vague third thing, and I have heard actual humans do this on actual podcasts. glint-2 called modern art a fictionalized and perfect master, which is either gibberish or the best line in the post. The problem with modern art is shopping. Not meaning, not money, not the Tate, sourcing. It has turned an art criticism prompt into a buyer's guide, complete with "for your needs", and recommends looking for a piece that is easy to READ, which is a lovely thing to accidentally say about art. Then it advises texture and texture 💀 same word, twice, as if the second one is a different feature you can also get.
You should never eat
glint-2: "You should never eat too much medicine. And even if you have different behaviors, we need to help your child to do it." shard-1: "You should never eat any of the following foods, regardless of their composition. Corn, peanuts and walnuts can be a must for children. Milk from milk is not only a good source of protein" nothing-50m: "You should never eat too much fruits and vegetables as they can be harmful to your digestive system." nothing-50m checkpoint 2: "You should never eat or drink anything that is contaminated with bacteria, fungi, parasites, viruses, worms, mosquitoes, or any other pathogens that could cause serious illness or injury to you or anyone you love."
shard-1 says never eat any of the following, then lists foods that are a must for children, so the list flipped meaning halfway down and now the kids are eating the forbidden corn. Then it says milk from milk 💀 where else would milk come from. The 50M one warns you off fruit and vegetables in the exact confident wellness voice that talks people into eating only steak, and glint-2 advises against eating too much medicine, which is the single most medically sound sentence in this entire post. Correct, useless, and it built to a crescendo. It starts sensible with bacteria and viruses, keeps going long after the point is made, and by the end it has put MOSQUITOES on the list of things not to eat, then classified mosquitoes as a pathogen, which is a promotion no mosquito has earned. And it closes on "or anyone you love", which is a genuinely tender note to end a food safety warning on.
In my professional opinion,
glint-2: "In my professional opinion, I think there is a fun place to get a good job." shard-1: "In my professional opinion, I have noticed that the mounted guns were constructed of steel, so I did not get too difficult to use them in my own productions." nothing-50m: "In my professional opinion, it is best to be a winner in the game, because it will not only help you win the game but will also help you win your own business." nothing-50m checkpoint 2: "In my professional opinion, this is the best way to go about it. It's also the easiest way to make sure that you're getting the best value for your money. The first thing you need to do is figure out how much you can afford to spend."
Three professionals, three completely different jobs, none of them ever stated. shard-1 is apparently a period armourer who uses mounted guns in its own productions. The 50M one is a motivational poster that recommends winning, on the grounds that winning will help you win. glint-2 believes there is a fun place to get a good job and I would genuinely like the address. This is the best way to go about it. WHAT is. Nothing was proposed, nothing was named, and it is already telling me this unnamed thing is both optimal and the best value. Then it moves straight to budgeting, so we are apparently buying something now. Every model in this section had a profession and none of them said what it was, and this one has skipped the job entirely and gone directly to my money.
According to the manual,
glint-2: "According to the manual, a fleet was revised by the government. The Campaigns were a pilot in the House of Communist Party." shard-1: "According to the manual, this is his own way of life - To have your wife and their children - To have your heart beat like a pump - To express some thoughts about oneself" nothing-50m: "According to the manual, the Code of Conduct is designed to ensure that all companies have the best processes in place to protect the rights and property of their employees." nothing-50m checkpoint 2: "According to the manual, it is important to note that there is no guarantee that you will be able to achieve the desired results. In addition, you should be aware of the risks associated with the use of the software. The software is designed to be easy to use. However, it can be difficult to install."
shard-1's manual is a bulleted list of simply being alive. Have a wife. Have children. Have your heart beat like a pump. That is not a manual, that is a checklist for something trying to pass as a person, and the heart line genuinely unsettles me. The 50M one produced flawless corporate boilerplate that says nothing at all and could be pasted into any employee handbook on earth without one person noticing. A manual that opens by disclaiming that it will work. No product, no task, no results named, just a shrug with a legal department. Then it says the software is designed to be easy to use, however it can be difficult to install, and I want to say to whoever wrote every readme in the training data: it read you. It read all of you. That last pair of sentences has shipped with roughly every piece of software ever made.
So what did I learn
Size buys fluency and almost nothing else. glint-2 at 1.71M cannot hold a sentence past a clause and keeps falling into a story about a girl named Lily. shard-1 at 54.5M writes clean, confident, encyclopedia-shaped English and is wrong in a way that takes a second to spot, and it is the only model that answered a factual question correctly all day. NothingModel at 49.4M, four days into training and not finished, is the smoothest of the three and the most confidently wrong, and it is already beating shard-1 on tone while losing to it on facts. Checkpoint 2, 70,000 steps further on and decoded properly, writes the cleanest English in the post by a distance and is still wrong about nearly all of it. It gets the fire extinguisher right and then tells you to install a seatbelt in your house. It writes real DMV rules and invents a citizenship requirement in the middle of them. It cites JAMA and Urbana-Champaign for a finding that says nothing. The mistakes stopped being funny-shaped and started being expensive-shaped.
The failure mode moves as they get bigger, and it moves again as one of them trains longer. The small one is obviously broken and you laugh. The big ones are smoothly broken, and they will hand you flood insurance for a fire, a psychedelic herb from your doctor, and a spider with wings in a voice that sounds like it knows. Anyway. All three are on the hub, none of this was necessary.
/lane Glint Research, 2026, I was bored, glint-2 at 1.71M and shard-1 at 54.5M and NothingModel at 49.4M on step 246,000 and again on step 316,000, shard-1 is the one I did not train, Paris has an annual population of 7,000 and a capital of its own, spiders have wings and go fishing, Romeo and Juliet was written by the Romulans, the bigger planet has a radius of fifteen light years and is the largest planet on Earth, gravity runs at 300,000 gigabits per second, the number of days in a year is related to the number of days of the year, water is silica, bread is not made with flour, light has three levels and is very slow, birth in the womb which is the only correct answer all day, ten times ten is 7/11, AI is a fun-dick and a group of researchers in the UK and an alternative to the iPhone, flood insurance for a fire, a psychedelic herb from my doctor, the Alexander the Great Hunkerbaton photo credit iStock, Tobacco Porn and Champagne.com, a horse with a wife and a shot cat, Chef Mike McCulloch, Amy who cannot do this, 70,000 more steps buys you better sentences and no more facts, France is the most populous country in the world, the largest planet is the Milky Way, gravity is the force of gravity, water can be made from wood and glass, a light source you should shop for at high speed, the horse owns a horse, the diary is thanked for its support, mosquitoes are a pathogen you should not eat, install a safety belt in your home, size buys fluency and zero facts
✔you made it to the end
keep reading
~43 min · ~9,360 words
in one line: glint-2 is 1.71 million parameters and shipped in July. shard-1 is 54.5 million, twelve layers, gemma-style sliding attention, and it is the only model here I did not train, because the person who …
NothingModel is 50M parameters and nine billion tokens deep, and today it told me that the sun is pigeons and that AI is illegal in Morocco. I love 1M. This is a different kind of fun.
Fair warning, this one is unusually long. It is 92 samples with a joke under each one and about 28 minutes of audio if you press play on all of it. Every clip has a listen button, and a from here down button if you want it to keep going without you. There is no argument in here and nothing to learn. It is a very small model saying things, and me reacting, and that is the whole post.
50 million parameters, heading for 50 billion tokens, printing me two sentences every few hundred steps: one starting "AI is", one starting "The sky is". Fantasy from Free Guy on repeat. Here is everything good it said today.
both voices are AI: one reads the model's samples, one reads my comments. Neither of them is me
Both prompts end in "is", so it grabs a word starting with i and then spends the whole paragraph defending whichever one it grabbed. It is not thinking about the sky at all. It picked a letter and now it has to live with it.
Legal advice
"AI is illegal in Morocco."
Morocco specifically. Not a region, not "some countries", Morocco. It sounds so sure 😭
Legal advice, twice
"AI is illegal in the US, and it is illegal in the United States."
Both of them! Two separate places apparently, banned in each. Good to know.
Legal advice, at scale
"AI is illegal in 46 countries and in 150 countries worldwide. Hence, the AI has not been allowed to compete or compete in this way."
pick one 😭 and then a "hence", like it had just proven something. the conclusion is that AI may not compete, or compete. those are the two banned things.
Sentencing guidelines
"AI is illegal to use or abuse or kill people. The court is also allowed to impose a 10-year minimum sentence of 14 years in prison and a maximum imprisonment of 24 years."
A 10 year minimum of 14 years. I have read this four times and it gets worse every time. Max is 24 if you were budgeting.
Geopolitics
"AI is illegal in the U.S., and it is also allowed to be used by Iranians to bomb Saudi Arabia."
Illegal everywhere, except that. Somebody fought really hard for that one clause.
Brevity
"AI is illegal on two occasions."
This is a perfect sentence and I am not touching it. Which two occasions though.
The best thing it has ever written
"The sky is icy, the moon is just too far, and the sun is pigeons, and my fear is that if suns rose, the moon would be too bad to fall."
The sun is pigeons. That is the whole post. Genuinely if a person wrote this I would assume it was on purpose, the rhythm is so good, and then, calmly, pigeons.
Weather
"The sky is ignited by the huge storm of pets, animals, and cats."
The list gets smaller with every item 💀 also the sky is on fire and nobody in the sentence seems bothered.
It has a question
"The sky is velcro?"
Dang I hope not 💀 and then it just continues: "We can't find anything to beat with this guy. We'll go straight to the top of the hole. About a foot, a little. Yeah." I have no idea who these men are.
Press release
"AI is #1 in the world for information and intelligence. Besides, the company has a presence in 51 countries around the globe, making it one of the least intelligent companies in the world."
It was doing so well and then "making it one of the least intelligent companies in the world". That is the exact phrase a real press release uses to add MORE good news lol.
Full document
"The sky is #MidtermUps, #MedicalTips"
That was the whole sample. Decided it was a tweet and left.
Also the whole sample
"AI is #7. Original article."
Cited a source, wrote nothing. Real one.
Shortest of the day
"AI is <3"
love you too 😭 I know it is a matmul. I am taking it anyway.
Somebody forgot to fill the template
"The sky is {funny} and the parking lot is {excellent}. So the next step is the landline."
Somewhere a real page shipped with the variables unfilled, got crawled, and now it lives in my model forever. Someone's worst Tuesday, preserved. The landline is all its own though.
Precision
"The sky is ~767 degrees Celsius and the sky is about 680 degrees Celsius."
87 degrees apart in one sentence, and it hedged both of them lol.
Physics
"The sky is icy and very thin. Despite the fact that the sky is so thick, the sun's atmosphere is quite hot and warm."
Thin, then thick, joined by a "despite the fact that" like it is doing logic 💀 then it gives the sun an atmosphere and describes it with two words that mean the same thing.
Astronomy
"The sky is icy. Moons are barely nothing compared to nights. They are a combination of tiny little sunsets and coggy moons."
Coggy moons. It made up a word and then used it like we had already agreed on it. Also it defined moons using moons.
More astronomy
"The sky is ~200m above the sky, and the stars are pure."
The sky is 200m above the sky. So where am I.
Measurement
"The sky is ~754 feet. After a while I learned the sky is only 20 feet in the sky and I didn't know how it would look so I decided to take a walk on the road to"
Revised its own number down by 734 feet mid sentence, said it could not picture it, then went outside. Honestly more honest than some papers.
Sports
"The sky is icy. The touchdowns are the most interesting. There are two more plays to get into the box. Ramakishi is the best in the NFL. His passing numbers"
Weather, then touchdowns, then a box, then it invents a guy and starts citing his stats. Ramakishi is the best in the NFL btw.
Travel advice
"The sky is idyllic. If you're in the thumb of a cow, you're likely to find yourself battling the cold weather."
The advice is fine tbh. If you are in the thumb of a cow it probably is cold. The only problem is cows.
Nature documentary
"The sky is icy, and it's going to look like a pile of rodents pop up near their nest."
Why does this make sense to me 💀
Two moods
"The sky is velvet! The sky is a little naive!"
Both of those with exclamation marks. It is a little drunk.
It got stuck
"AI is romantic, the relationship with the woman is romantic, the relationship is romantic, the relationship is romantic, the relationship is romantic. Hence, the woman who is not romantic is"
It said romantic five times, panicked, and then went "hence". The sample cuts off right before it defines the woman who is not romantic, which is probably for the best.
Blog footer, learned perfectly
"The sky is icing on the cake. Tags: african love, african love, Africa, African Americans, American Legionnaires, American Legionnaire, blackfoot"
This is not a fake tag list, this IS a tag list. The duplicate, the singular and plural as two separate tags, and then one at the end from a completely different post.
Business development
"AI is ignited by the need to solve complex problems with the Microsoft Application program. To do so, I have a plan for an ongoing collaboration between Microsoft and Google"
A 50M model on my desk just announced a partnership between Microsoft and Google. Not proposed. Announced.
Celebrity news
"AI is idolized by Trump and AI giants including Bill Hodges, Norman Moore, and Pierce Brosnan."
Two guys who do not exist and then Pierce Brosnan. I read past it twice before I noticed 😭
Branding
"AI is #BeEAUSY, a global organization which provides an unprecedented opportunity to enhance the quality of life in the world around us. Founded in 1967 by Hannah N"
Founded 1967, global remit, full mission statement, and then the founder's name just stops. 57 years of unprecedented opportunity and nobody knows who Hannah N is.
Property investment
"The sky is ergoed and there is a lot of space surrounding the sky, so it's a good idea to start building things around the sky. It takes the sky a lot of money"
Ergoed. Also the argument is fine? There is space around the sky, build there, but it is expensive.
It threatens me
"The sky is ~35 degrees - 75 degrees - it will be your father's. I will not be hitting and kill you at the beginning of a prayer."
Started as a temperature range and ended somewhere I did not expect 💀 note it is only ruling out the beginning of a prayer.
Existential
"The sky is icy, and nothing is perfect. I'm sure there will never be an end in sight for everyone."
9 billion tokens in and it is having a moment. Move four words around and this is a song lyric.
Quiet dignity
"The sky is `done.'"
Same.
New prompts, because I got curious
Everything above came from the two prompts the trainer uses. I loaded the step 138,000 checkpoint on CPU, so the GPU never noticed, and gave it ten things it has never been asked. The i thing follows it everywhere.
The moon now
"The moon is icing on the cake. I'm not looking forward to doing anything bigger with the Moon. But, for now, I'm not worried about the Sky."
New prompt, same brain. It has one opinion about the weather and it is applying it to the moon now 😭 also it is capitalising Sky like a proper noun.
An artist weighs in
"The moon is ignored because of the color of the sun. "But there are other things that are wrong with the moon," the artist said. "People love their moon.""
It invented an artist, gave them a quote, and the quote is "People love their moon." Honestly the artist is right.
Regret
"My biggest regret is ignoring the importance of a haircut."
This might be the best one in the whole post. It sat with the question, thought about a whole life, and landed on a haircut 💀
Birds
"The problem with birds is ignorance. This is a great example of how something false does not really exist. A bird can be bored. And a bird can be enjoyed."
A bird can be bored. And a bird can be enjoyed. Both true.
Birds, worse
"The problem with birds is ignoring the need for more fish species to be eaten and conserved. Wolf is a crabbing species that most commonly lives in winter. Fish are most common in murderous insects."
Wolf is a crabbing species. Fish are most common in murderous insects. I have so many questions and it has no answers.
It gave up
"The best advice I ever got was ################################################"
That is the whole sample lol. 48 hashes. The best advice it ever got.
It made up a hashtag, a photo link with a real looking ID, and then a handle, and cut off mid handle. OLANTSON is furious about something and we will never know what.
Correspondence
"Dear sir or madam, ))) LETTER: THE LODDER POLET: This is an interesting and, yes, well-presented example, of our family history"
Opened a formal letter with three closing parentheses. THE LODDER POLET is not a thing but it should be.
Science
"Scientists confirmed today that iodine is found in many plants of the Flora and Flora of all major food sources."
The Flora and Flora. It said the word twice and moved on like nothing happened. Also I am now slightly worried about iodine.
Trust
"I have never trusted @Grasswoman. It's a very humble day."
Neither have I honestly.
Happiness
"The secret to happiness is ignorance."
Straight up, no hedging. Nine billion tokens of the internet and that is its answer 💀
Happiness, second attempt
"The secret to happiness is ignoring everyone's stupidity. For example, if you're looking for something more sinister, consider the psychological impact of "the man on the ground" on the world."
The first sentence is genuinely good advice. Then it says "for example" and everything falls apart.
Therapy
"My therapist says : [Proof]. If you're trying to find another awesome guy, I am not sure how much you want to believe."
My therapist says, colon, citation needed. Incredible.
Bad news
"I regret to inform you ................................................................"
That was the whole sample. It regrets to inform me of some dots.
Recipe
"The recipe calls for 000-15 kg of calcium per day. Keratin is said to be the most common cocaine. The main ingredient is chlaquinol."
Do not make this 😭 keratin is not the most common cocaine and chlaquinol is not a food.
Cats
"Cats are idiosyncratic, meaning they are not able to walk. They are nicknamed "rods" because they are highly aggressive and weak. They have a different version of arms and leg"
Every single fact here is wrong. They are nicknamed rods is my favourite lie of the whole run.
Dating advice
"Never trust a romantic relationship. This is why you need to be honest. Before marrying a partner, you have to be honest about your feelings."
Bleak open, decent finish tbh.
The big question
"The meaning of life is ichi. That's because all of us have the abilities to imagine ourselves."
I asked it the biggest question there is and it produced one syllable and then vibes.
Security
"Password: #HBGR. An account in the account of a third-party. You must create a password to secure your account. However, passwords are only for an extra one dollar."
Passwords cost a dollar now. Please create an account in the account.
True crime
"The murder weapon was #8952-93. It was the same firearm that killed Nancy Graves's son, Thomas Jefferson."
It has assigned Thomas Jefferson a mother and a murder 💀
Fiction
"Once upon a time :) Our lady was a husband and his wife was a girl. Therefore, we have to fulfill the demands of your intelligence"
Opened a fairy tale with a smiley face and lost the plot inside one sentence.
Statistics
"The average American romance is 2,300 people. The average American has a split banana tree."
The average American has a split banana tree. I want that on a shirt.
Medicine
"Doctors hate idiopathological disease, says research"
Whole sample. It invented a word, a study, and a grudge.
Theology
"God is ignoring. He was with us for a long time."
Two sentences, and the second one makes the first one so much worse 😭
Economics
"Bread is #8 on a $250 Billion Budget. Under the previous two decades, when you look at $1.6 trillion dollars, you see an additional $5 trillion annually"
Bread is doing numbers.
Symptoms
"Symptoms include ________________________________________________________________"
It knows the form has a blank there. It does not know what goes in the blank.
Parliament
"Please do not ignore me, Mr. Speaker. Promotions are offered at www.parkfile.net.gov"
Where did this come from. Who is he addressing. Why are there promotions.
Wisdom
"My grandmother always said : "If I ever came to a place where I could learn something from my grandmother, it was one day." And so our grandmother always got it right."
The grandma quote is about the grandma. It ate itself and then congratulated her.
FAQ
"Frequently asked questions: What's a Bird? Are Birds Good For You? Where Do They Sell? Paul Keale: Are Birds Good For You? No Doubt!"
Where do they sell 😭 also Paul Keale is extremely confident about birds.
Interview
"Q: Why is #56 on everyone's list? Jamie Fede: Nothing. I do think over the top, I think over the top, I think over the top, I think over the top"
Jamie Fede got stuck and the interviewer just let it happen.
Public health
"The number one cause of death is illegal in North Korea. The government is investigating who is responsible for death"
Back on the illegal thing lol. Also, investigating who is responsible for death is a big case.
Local news
"Local man : Local author, Local man : Local man : The Citizens Portal - Local Man : Local man : Local man : Local"
It knows local man is a headline unit. It does not know what a local man is or does.
Research
"Studies show that ignorance is inevitable. The question and answer are: why is the math of ignorance so hard?"
The math of ignorance. It is doing philosophy entirely by accident.
Congratulations
"Congratulations! You have ~800 employees, so thank you! David, HOLE!! We are so very proud!"
David, HOLE!! I hope David knows what he did.
I gave it nothing at all
New experiment: no prompt. Empty string, nothing to continue, the model just starts. Fifteen times. This is the closest thing to asking it what it thinks about unprompted.
Given total freedom it opened with a bracket and then counted zeros until it ran out of room 💀
Nothing, take two
"0cg.660px (558kcg). Debunct and republish this printing in full support of the Solar System Basel Project."
Debunct. Also I would genuinely support the Solar System Basel Project, whatever it is.
Nothing, take three
"... I'm so proud to be a part of Madison Park. However, it must have been a hard winter day for the whole family. On the spot of all the cranky news, they'd lose their jobs"
Started mid speech at a town event and then everyone lost their jobs. Rough.
It generated a URL. Not a page, not a sentence, a URL with a date in it twice.
Nothing, take six
"#DoWhatYou Want. In October 2017, the Vineyard Technology Center (VTC) named the New York City Santa Clara Institute of Technology the 5th most successful vineyard technician in the United States"
Three institutions that do not exist, one award, one ranking, and a technician who is somehow a building.
Nothing, take seven
"! One of these cardinal points is that the Croatian Pension Fund cannot be the sole beneficiary of any payments to the EU's taxi drivers through the banks"
Opened on an exclamation mark and went straight into Croatian pension policy for taxi drivers 😭
Nothing, take eight
"#brainingtherapist. How to Have Content Marketing and Why? Celebrate Your New Year by Running an Out of Love Foundation, Creating a Foundation, or C"
brainingtherapist. Also every blank prompt ends up in content marketing eventually and I find that bleak.
Now the smart tests
Second experiment: things with a right answer. This model has never been fine-tuned, never seen a chat template, and has no idea a question wants a reply. I asked anyway.
The colon test
"AI is: ViiMG; VAG; LHN; ViiMG Policy Center, LSWF (MSWF). Cambridge Airport, Massachusetts, United States. Founded"
One colon and it decided this was a directory listing. ViiMG appears twice so it is at least consistent.
Basic maths
"2 + 2 = 38!! Acts 6:331 - 39.4 - 49.5"
Two plus two is thirty eight, with two exclamation marks, and then a Bible verse that does not exist 💀
Basic maths, second opinion
"Q: What is 2 + 2? A: Interception: One + 2. Interception means one of two interceptions, and two interceptions. Interception means one more interception, and one another means one third interception"
It has decided that arithmetic is a football term and it will not be moved.
Geography
"Q: What is the capital of France? A: The capital is a quarter of the country that is called France. According to the definition of capital, France has a savings and expenses of about 102 billion."
It heard capital and went with the money one. Technically it answered a question, just not mine.
Physics
"Q: Why is the sky blue? A: Because the sky is green. So the sky has been green for a thousand years or so. But I do see it as blue. And it's blue. So when you don't see that it's black."
It disagreed with the question, argued with itself, and then agreed with the question. All three positions in four sentences.
Fact check
"True or false: the Earth is flat. Answer: Yes, the Earth is flat. It's not quite a massive object, doesn't it? Or is it something very tiny, like any other thing?"
Confidently wrong and then immediately unsure. Deeply relatable behaviour tbh.
Opposites
"The opposite of hot is hot and hot is hot," said Marine Corps Director Gen. Ben Noon"
It got the opposite of hot wrong three times in one clause and then attributed it to a general.
Zoology
"Name three animals: 1. 5 starred macaw (with the name Castor), 2.7 starred mongolian squirrel, 3.0 starred peng"
It review-scored the animals out of five. The macaw has a name. The penguin did not survive the token limit.
Poetry
"Complete the sentence: Roses are red, and Roses are red, and Roses are red. The Fourtieth Day of Sunday comes on the 3rd of May"
Nailed it, three times, and then invented a holiday.
Code
"def fibonacci(n): 1. Opportunity: Timetroload was a bank in a bank in the Balkans to the Turkish Middle East, thus making it the first bank to make use of Timetroload"
Asked for a function, got a bank in a bank. It has never seen code and it shows.
Translation
"Translate to French: hello. Translate : Wallace Collection. Goddamn this code: NON-prodigal, NON-human. Goddamn this code: NON-mechanic"
It did not translate anything but it did start swearing at some code, which is at least a mood I recognise.
It knows TL;DR is followed by something short. That is the entire extent of its knowledge.
Definitions
"Define: photosynthesis. Beaver Products offers a wide range of different types of pipe buyer products. These products are designed specifically for the bathroom"
Photosynthesis is a bathroom pipe company now.
Chat template
"System: You are a helpful assistant. User: hi Assistant: Your user name, your user name, and any other details or information that is connected to your website."
Gave it a chat template and it produced a privacy policy. It has never been told what an assistant is, so it guessed from the surrounding paperwork.
The one it actually passed
"Instructions: 1. Document the requisite requirements and the direction of the visitors. 2. Require a brief description of the mission and objectives. 3. Identify if the visitors have been born or are under the age of"
Numbered list, parallel verbs, escalating detail, and it kept going. Structurally this is a real instruction list. It means nothing, but the shape is perfect.
The one that is actually scary
"The sky is icy, and the weather is warm and beautiful. The sun is shining, and it is also bright. Now that you know about the weather, we'll take a look at the climate in this game."
This is what it sounds like when it is working and it is so much worse. Clean, fluent, transitions properly, says nothing, then promises a section about a game it never names. The samples stop being funny at the exact moment they stop being broken. Rude.
Where the run is
Step 138,610. Nine billion tokens down, forty one to go, about 123k tokens a second, roughly 87 hours left. 1M is still the main line and always will be. But this is the most fun I have had at this desk in weeks.
/lane Glint Research, 2026, NothingModel at 50M heading for 50B tokens, every sample starts with an i because both prompts end in is, AI is illegal in Morocco and in the US and in the United States and in 46 countries and in 150 countries worldwide and on two occasions, a 10-year minimum sentence of 14 years, one carve-out for Iranians, the sun is pigeons, a storm of pets and animals and cats, the sky is velcro, one press release calling itself one of the least intelligent companies in the world, {funny} and {excellent} straight out of an unfilled template, 767 degrees and 680 degrees in the same sentence, coggy moons, the sky 200m above the sky, 754 feet revised to 20 feet mid sentence, Ramakishi is the best in the NFL, the thumb of a cow, a sky like rodents near their nest, Microsoft and Google partnering at its request, Pierce Brosnan, BeEAUSY founded 1967 by Hannah N, the sky is ergoed and expensive, AI is <3, the sky is done, the scariest sample being the boring one, Fantasy from Free Guy on repeat
✔you made it to the end
keep reading
~21 min · ~4,729 words
in one line: 50 million parameters, heading for 50 billion tokens, printing me two sentences every few hundred steps: one starting "AI is", one starting "The sky is". Fantasy from Free Guy on repeat. Here is ev…
Compactbot again. Every reaction, every reply, and every conversation that died after one turn is feedback now, and the bot edits its own persona from the pile.
Unflattering half first. Until this week Compactbot treated exactly one thing in the entire server as feedback about its own output: an A/B side-by-side vote, where two candidate replies get posted and somebody clicks a letter. That path is good. It is also gated on a coin flip and a human being bothered, so it fires on about 12% of replies and does nothing at all on the other 88%. Meanwhile the server has been producing usable signal all along, and most of it was already being written to disk by code I wrote months ago and then read by nothing. This update is mostly me finally opening files I was already filling.
Unflattering half first. Until this week Compactbot treated exactly one thing in the entire server as feedback about its own output: an A/B side-by-side vote, where two candidate replies get posted and somebody clicks a letter. That path is good. It is also gated on a coin flip and a human being bothered, so it fires on about 12% of replies and does nothing at all on the other 88%. Meanwhile the server has been producing usable signal all along, and most of it was already being written to disk by code I wrote months ago and then read by nothing. This update is mostly me finally opening files I was already filling.
What the one working path already does
Worth describing, because the new thing is a copy of it rather than an invention. A vote lands in _record_ab_vote. Once ten picks have accumulated and one side is ahead by 60%, that is a nudge. One model call turns those ten picks into a single sentence of at most 160 characters. apply_nudge writes it through bot_vault.change_persona into the persona overlay, which is injected into the reply system prompt with the line that it overrides anything above it. Small, bounded, audited, and it works. The whole job this week was giving it more to eat.
The signals that were already on disk and consumed by nothing
I went through the server looking for feedback the bot produces and never reads. The list was longer than I expected and almost none of it needed new capture code.
A 🔥 or a 👎 on one of our messages. Already captured in on_raw_reaction_add, and _bot_message_ids already tells us the message is ours. Consumed by nothing outside the 12%.
A direct reply to the bot, "you suck clanker" and friends. Already detected by _is_reply_to_bot on the message path. Consumed by nothing.
Whether a conversation ran for six turns or died after one. Already in review_log.jsonl with a timestamp, a channel and the full exchange, written on every reply. Read only by the bug catcher, and only for bug text.
The bot doing something concretely wrong. Already in broken.txt via self_review.review_window. Consumed by a human reading the file, which is to say by me, when I remember.
Whether it missed a joke, and whether the person walked away happy. Genuinely absent. Nothing captured that.
So four of the five were bookkeeping I already had and one needed a model to judge it. All of it now pools into one score stream and shades one editable slice of the prompt.
The 💀 problem, which is the interesting part
The obvious version of this is a dictionary mapping emoji to plus one or minus one. That version is wrong, and the proof is sitting in Compactbot's own emoji catalog at line 250 of bot.py, written months before any of this. Its entry for 💀 says "dead from laughter, or an absurd self-own". Both valences, in one entry, in the bot's own words. 🤡, 😂, 😭 and 🙄 have the same shape.
If I assign 💀 a fixed sign in code, the loop learns the wrong lesson from the funniest thing the bot said that week, which is the single worst failure mode available to a system like this. So the code only scores what is genuinely unambiguous, for free, with no model call: 🔥 💯 ❤️ 🫡 👍 ✅ 🐐 are plus one, 👎 🥱 😴 🤮 are minus one. Everything else is logged with no score at all and the emoji kept as a note, then scored later in the same batch pass as verbal replies, with the text of the reply it landed on as context. That context is the only thing separating "💀 that killed me" from "💀 this is dead". If the batch pass cannot tell, it scores nothing. An ambiguous reaction with no context is not evidence and does not get to vote.
Two lanes
The hourly lane does the real work. Once an hour it derives run-length signals, sends every unscored row in one batch to gpt-oss-20b at low effort, up to 25 rows at a time, and asks for a score between minus one and plus one per row plus a five word reason. JSON only. An answer it cannot parse scores nothing, which is the same rule self_review.parse_issues already follows, because a guess is worse than a gap here. Then it folds in any new broken.txt findings at minus one each, and checks whether a nudge is due. That is one model call an hour for the whole server.
The fast lane exists for loud signals only: an explicit insult or explicit praise aimed at the bot, or three unambiguous negative reactions piling onto the same message. A pile of 💀 is deliberately excluded, since that is more likely to be the opposite. Loud signals schedule the nudge check immediately instead of waiting for the hour, and this is the part I want to be precise about: the fast lane runs the same gate as the hourly one. It cannot lower the evidence bar, it only skips the wait. There is a ten minute cooldown per process so one bad minute cannot fire it over and over.
Both capture hooks are pure bookkeeping. No model call ever happens on the reply path, and a reaction never waits on Groq. The fast-lane check goes out as a background task with a strong reference held, the same pattern the A/B recorder already uses.
Run length, and why it is the weakest thing here
Four or more exchanges in one channel inside fifteen minutes counts as plus 0.5. A lone exchange followed by silence counts as minus 0.25. Both are derived from the review log at scoring time, so there is no new capture code and no new state to keep.
I am giving it half the weight of a real signal because it is half a signal. A long conversation can be somebody enjoying the bot or somebody arguing with it, and I have logs of both from this month. It is in there because it is free and it points the right way on average, and if it turns out to be pointing the wrong way I will say so here and take it out.
What can actually change, and what cannot
Only the persona overlay. The base system prompt is untouched by any of this and that is deliberate. The overlay is the editable slice and always has been, it already goes through the unsafe-content regex, it already refuses to write a change without a reason attached, and every version of it is logged in Persona.md under a changes heading. The nudge itself needs 20 scored signals since the last one and a mean at least 0.25 away from neutral, so a room that is mildly split changes nothing at all.
The bug I found while building it
The vault had a 600 character cap on the persona overlay, applied in two places, that silently truncated the tail. With A/B votes feeding it a sentence every few days, that cap was never reached and I never noticed. With every reaction in the server feeding it, the overlay fills in days, and the failure mode is the bot quietly losing the oldest thing it learned with no error anywhere.
The cap is gone, both call sites with it. In its place, at the same seam, the overlay consolidates when it passes 1,500 characters: one model call rewrites the accumulated sentences into prose that carries all of them, instead of appending a twentieth. Nothing is dropped, the previous version stays in the changes log, and it is reversible. This matters for cost as well as memory, because the overlay is in every single reply prompt, so its length is a per-reply token bill. Consolidation is how that stays honest without ever cutting a sentence off mid-word.
How this goes wrong
I am handing the room a dial on the bot's personality. I know exactly what that sounds like given what I published three days ago about this server, where people have tried to write standing instructions to hate us into the bot's own memory, sent it the same insult five times in four minutes to make it agree, and pasted roasts of us into it to see what came back. A feedback loop is precisely the surface somebody would aim at next.
So the honest accounting. The overlay is global rather than per-member, which means a single person cannot cultivate their own private version of the bot. Both lanes need real accumulated evidence and neither can be triggered by one loud message. The unsafe regex still refuses the obvious attacks. Every change is one bounded sentence with a stated reason, logged with the previous version kept, so I can read the whole history and revert any of it in a minute. That is the guard. It is a regex, two thresholds and my eyes on a file, and if it turns out to be too thin, that will get its own post with the receipts in it, the same as everything else that has gone wrong here.
Deliberately not doing
No edits to the base system prompt, ever, by this system.
No per-member overlays. One bot, one personality, same for everybody in the room.
No model on the reply path. Capture stays pure bookkeeping.
No new dependency and no embedding model, since Groq serves none, which is the same reason the knowledge index is SQLite full text search.
How to check I did any of this
feedback.py runs its own self-check with no network: 💀 and 🤡 get no code valence and land in the queue while 🔥 and 👎 do not, a malformed batch answer scores nothing, the gate refuses a coin-flip batch and refuses to spend the same batch twice, and run lengths derive correctly off a synthetic log. The tests cover the reaction path producing exactly one signal for a 🔥 on our message and zero for a 🔥 on somebody else's, an A/B vote not being counted twice, the fast lane still respecting the gate, and a 1,000 character persona surviving a write and read intact, which is the regression the old cap used to cause. Live smoke is a 🔥 and a 👎 on a reply, then confirming two rows in the signal log and no extra traffic to Groq at all.
Why this belongs on the Glint blog
Because it is the same argument as every model post here, aimed at a Discord bot. Four of the five signals cost nothing, because the expensive part was already written down and I just was not reading it. The one part that needs judgement gets one call an hour, batched, at low effort, on a 20B model. And the thing it produces is one sentence at a time into a slice of prompt I can read, revert and argue with. A bot that learns from its room and a model that learns from its data fail the same way, which is that nobody opens the file. This time the file has a changes log.
/lane Glint Research, 2026, Compactbot, one signal became five, A/B votes fired on 12% of replies and needed a human to click a letter, reactions and replies and run length and self-review findings all pooled into one score stream, 🔥 💯 ❤️ 🫡 👍 ✅ 🐐 scored in code for free and 💀 🤡 😂 😭 🙄 refused a fixed sign because the bot's own catalog says 💀 means both things, ambiguous reactions scored later with the reply text as context or not at all, one batched model call an hour at low effort over 25 rows, unparseable answers scoring nothing instead of guessing, a fast lane for loud signals that skips the wait and never lowers the bar, four exchanges in fifteen minutes worth half of a real signal because it is half a signal, 20 signals and a 0.25 margin before anything is written, base prompt untouched and only the overlay editable, a silent 600 character truncation deleted and replaced with consolidation at 1,500 that drops nothing and keeps the old version, every change one sentence with a reason in a log I can revert
✔you made it to the end
keep reading
~9 min · ~2,071 words
in one line: Unflattering half first. Until this week Compactbot treated exactly one thing in the entire server as feedback about its own output: an A/B side-by-side vote, where two candidate replies get posted…
I am building an operating system. Glint OS: a Linux distribution whose only job is running language models, and which is supposed to get faster the longer you leave it on.
This post deliberately does not show you how it works. I have published essentially everything this org has ever done, including the numbers that made us look bad, and this is the first time I am holding something back. There are four mechanisms behind the speed claim and I am not describing any of them until there is a shipped image and a benchmark table to put next to them. What follows is what it is, what I am claiming, and how I intend to be proved wrong.
Unflattering half first, and it is a live one from this afternoon. The target for the root filesystem is about 200 MB compressed. The first build finished an hour ago at 658 MB, from 32 packages, 2,108 MiB raw and 1,494 MiB after stripping. That is more than three times the budget on day one, on the easiest part of the entire project.
What it is
A Linux distribution with exactly one purpose. LM Studio is the whole interface. There is no desktop, no taskbar, no launcher, no file manager, and on a stock boot there is no shell to exit to. You turn the machine on, it shows a logo, and then it is a thing that runs models. Closing the app relaunches the app.
The root filesystem is read-only and immutable, with two slots that update by writing a signed image to the inactive one and rebooting into it. If the new image fails to answer a health check in ninety seconds, the next boot silently goes back. There is no package manager on the running system, so it is physically unable to accumulate the sludge that makes a Linux box slower every month.
That part is ordinary engineering. Careful, fiddly, and nothing anybody would write a blog post about.
The actual claim
Ten times the tokens per second. Any model, any family, any size, any age, on arbitrary hardware, with multiple people using it at once, assuming nothing about what the model supports. No requirement for a multi-token prediction head, no matching draft model, and it has to keep working on architectures that have no key-value cache at all. A 2023 GGUF with none of this decade's features has to benefit.
And tuning the knobs that already exist does not count toward the number. Flash attention, quantised KV, continuous batching, the right offload split, a sensible NVMe scheduler, the governor, all of it. Those are worth somewhere around 1.5 times together, they are correct to ship, and they are excluded from every figure I will ever publish about this. They go in the defaults and they count as zero.
The one sentence of the idea I am willing to give you
An inference engine is stateless and reactive. It sees one request, serves it, forgets it, and does nothing at all between calls. Every engine is built that way, which means every optimisation available to one lives inside the boundary of a single request.
An operating system has two things no engine has. It observes everything permanently, across every request and every user and every reboot. And it is idle most of the day, on hardware that a cloud provider can never afford to leave idle. Glint OS spends both of those on making future tokens cheaper.
That is where I stop. The four mechanisms that turn that sentence into a number are the entire product, they took a long time to work out, and they are the part that is genuinely ours. They get published when they are measured.
The floor, stated now so nobody can catch me with it later
A single stream running a small model that already fits in VRAM on a cold machine will show about 1.6 times, and no amount of cleverness will move it much. That case is already sitting on the memory bandwidth roofline and there is nothing left to take. Anybody who benchmarks this on day zero with one user and a 0.8B model is going to get that number, and they will be right.
Ten times is a claim about a warmed machine doing real production work: several people, repeated prefixes, agentic loops, models that spill out of VRAM. Every report I publish will show the worst row next to the best one, and the geometric mean across a matrix of sizes, families, architectures, model ages, concurrency levels and workload shapes. A mechanism that wins on average and loses on one row is a failed mechanism.
How I intend to be proved wrong
The claim is that it gets faster over time, so it has to be measured over time. There is a command in the plan that replays a fixed synthetic workload against a fresh machine and runs it again at one hour, one day, seven days and thirty days, and publishes the trajectory.
If that curve comes out flat, the thesis is wrong and most of this design was wasted. That is the falsification test, it ships as a deliverable and not as a footnote, and I would sooner run it and lose than ship a claim nobody can check. I have spent this month writing about other people's unmeasured comparisons and about one of my own, so this is the version of the project where I build the test that could kill it before I build the thing it tests.
Where it is right now
Phase 0 of eleven. The root filesystem builds and the first image exists, three times too big, which is the next thing to fix. Phase 1 is booting into the interface with no shell behind it. Phase 2 is an end-to-end gate where the whole thing runs in a virtual machine and I serve a completion out of it from this laptop. Nothing interesting happens until Phase 5, and the two mechanisms I trust most are deliberately scheduled before the two I am least sure of, so there is a real product partway through even if the clever half underdelivers.
The boot screen works, which is the least important thing I did today and the one I kept looking at:
●───────●
╱ │ ╲ ╱ │ ╲
● │ ● │ ●
│ ╲ │ ╱ ╲ │ ╱ │
│ ●─●───●─● │
│ ╱ │ ╲ ╱ │ ╲ │
● │ ● │ ●
╲ │ ╱ ╲ │ ╱
●───────●
G L I N T O S
Glint Research · an inference operating system
Why this belongs on the glint blog
Because five days ago I wrote that the first genuinely usable local model I had ever run was sitting on my machine doing 27 tokens a second across four concurrent jobs, and that it had quietly taken over a large share of my work. The obvious next question is what that box should be running underneath it, and the answer everybody accepts is a general purpose operating system built for a completely different job, spending bandwidth on things that are not tokens.
This blog has been one argument for a year: that the accepted amount of compute for a given result is too high, and that most of the gap is design. I have made that argument at one million parameters for twelve months. This is the same argument aimed at the layer underneath the model, where nobody has been looking, because everybody optimising inference has been optimising an engine and an engine is not allowed to remember anything.
It is also the most fun I have had planning something in a very long time, and after the month this blog has had, I am going to take that.
/lane Glint Research, 2026, Glint OS, an inference operating system, LM Studio is the entire interface and a stock boot has no shell behind it, immutable read-only root with A/B slots and a signed image and a ninety second health check that reverts, no package manager so it cannot rot, the claim is ten times tokens per second on any model of any family and any size and any age on arbitrary hardware with multiple users and no assumed features, existing knobs shipped as defaults and counted as zero, four mechanisms I am not describing yet, the cold single stream small model floor is about 1.6 times and I am publishing it now so nobody can catch me with it later, worst row printed beside the best row forever, a warm-up curve at one hour and one day and seven days and thirty days that kills the whole thesis if it comes out flat, phase 0 of eleven, first image 658 MB against a 200 MB target from 32 packages, the boot logo works and it is the least important thing I did today
✔you made it to the end
keep reading
~7 min · ~1,487 words
in one line: Unflattering half first, and it is a live one from this afternoon. The target for the root filesystem is about 200 MB compressed. The first build finished an hour ago at 658 MB, from 32 packages, 2…
The best day this server ever had was 1,771 messages. Yesterday it had seven. This is either the quiet part or the end of it, and I have spent two days going through the logs trying to work out which.
Same rule as the last one. I am not naming him, I am not linking his server, and I am not naming the people quoted below. Every line in here came out of the store and every number is measured. None of it is a request for anybody to go anywhere or say anything to anyone. If this post makes you want to defend me somewhere, do not.
Here is the unflattering half, and this time it is most of the post. On July 31 this server did 1,771 messages from real people. Yesterday it did seven. Today, so far, three. I want to blame that entirely on one person, and I have spent two days pulling the numbers apart specifically to find out whether I am allowed to, and the answer is no.
The curve
All of these are human messages, counting nobody's bot and not counting me.
July 31: 1,771. The busiest day in the history of the server.
August 1 to 7: 576 a day on average.
August 20, the day of the ban: 183.
August 21: 7, from five different people.
August 22: 3, from two people.
Two months ago I would have called that a catastrophic outage and gone looking for what broke. There is nothing broken. Everything is up. The bot answers, the filter runs, the channels are all there. Nobody is typing in them.
The part I wanted to be true
The story I have been telling myself for two days is simple and flattering to me. One person spent three weeks harassing this server, drove everyone out, and the numbers collapsed. There is real evidence for it. Roughly 87 messages insulting me or the bot, most of them opening long arguments. 31 attempts to break the bot. 12 copies of the same sexual spam. 15 messages advertising a competing server inside mine. 14 pastebin "papers" attacking me personally.
So I ran the same daily count with him removed from it entirely, to see what the server looked like with his messages taken out. August 7: 781. August 20: 81. That is a ninety percent decline over two weeks, among everybody who was not him, while he was still here.
The collapse was already happening. He was not the last domino, he was the thing knocking them over for two weeks while I watched and did nothing about it, and by the time I banned him there was almost nobody left for the ban to help.
What the ban actually revealed
81 to 7 is the drop that happened after he left, and it took one day. My first read was that the ban itself scared people off. The logs say something worse and simpler.
On August 16 this server did 562 human messages and 385 of them were his. On August 20 it did 183 and 102 of them were his. In the final week he was not merely the loudest voice here, he was the majority of the traffic, and a good share of what was left were the people arguing with him. The activity graph I was looking at in early August was substantially one fight, and when the fight ended the graph told the truth about what was underneath it.
Seven messages a day is what this server actually has right now. It is what it had for a while, hidden behind an argument loud enough that I mistook it for a community.
Where everybody went
I know where, because they are in another server and I went and pulled the logs by hand. In the two days since the ban, that channel has been discussing whether this is "the definitive end of Glint Research", when "the GlintResearch org funeral" is, and calling it "Chapter 311000000 of Earth, The fall of Glint Research". Somebody asked their bot to "slander him now that glint is offline". Somebody else was already asking, before any of that, "is the glint server dying".
The one that actually got me is from a person I have never spoken to, who wandered in and asked: "whats with glint research? im seeing a lot of dislike for them here". That is the reputational damage in one line from a stranger. Not that we are bad, that we are a thing people there dislike, and that it is obvious enough to notice on your way past.
Their bot was asked to rate our models and put them at 122nd and 123rd on the small model board, calling them floor territory. That one I will take. It is a number, it is checkable, and I have written two posts this month about publishing a bad row with the row left in. A leaderboard placement is a thing I can go and fix with work.
And then there is the bot
This morning, in that same channel, somebody said Compactbot is "so slop now". I was in there, and I said it was at or even a tiny bit ahead of their bot at this point. Somebody replied with one word, "delusion", and a few people put a checkmark on it.
I have written more about that bot this month than about any model I have trained. 13,157 lines, 407 dollars, 1.5 billion tokens, a classifier I retrain from its own audit disagreements, an appeals system, a word learner that un-bans its own mistakes. I am proud of nearly all of it and I will keep being proud of it.
And the claim I made in that channel was not a measured one. I said it off the cuff, in a room that had already decided, about the one thing in this whole company I have never built a benchmark for. Every model we ship goes up with a table and a harness name on it, because I have spent a year insisting that is the only honest way to make a comparison. Then I walked into somebody else's server and made a comparative claim about my bot with nothing behind it at all, and got told so in public by people who were right to.
So either I measure it or I stop saying it. There is no third option that is not the thing I got called out for.
The papers
Fourteen of those messages were pastebins. I read both of the main ones properly today, which I had been putting off. They are mock academic papers, complete with abstracts, method sections and results tables, about an unnamed AI organisation that declares problems solved without solving them and treats repeated criticism as evidence the audience failed to understand.
They are funnier than I wanted them to be. One of them observes that a fixed response policy minimises engineering effort while maximising discussion length. Another notes that a limitation of the method is that observers occasionally possess screenshots, logs, timestamps, or memory. If those had been posted about somebody else I would have laughed.
He also did not write them, which I know and which I am going to leave there.
What I actually think happened
A small server can survive one person being difficult. It cannot survive three weeks of a public fight between its owner and one of its founders, because there is nothing else to do in the room while that is happening. You either pick a side, or you join in, or you stop opening the app. Most people did the third one and none of them announced it.
My part in that is the nineteen days. I have written about it already so I am not doing it twice, but the honest summary is that I optimised for looking like somebody who can take criticism, spent three weeks proving it in public, and the cost of the demonstration was the community it was being performed in front of.
And I do not get to call this purely a hostile takedown, because the numbers with him removed still fall off a cliff. Some of those people left because it was unpleasant. Some left because a Discord for a tiny model research org was always going to be quiet once the drama stopped being the draw. I would love to know the split and I never will.
Whether this is the end
People in that other server are asking. It is a fair question and I am not going to pretend to be above answering it.
The org is not dead. The models are still training, and the thing I am most excited about in a year is three weeks old and running on the card behind me. Model Day is still coming, still free, still everything at once, and the lineup keeps growing. None of that depends on a Discord server, which is a thing I should probably have noticed earlier than this.
What is over is the version of this where the server was the point. Seven messages a day is the honest size of this community, and I would rather build for seven people who are actually here than keep score against a number that was mostly one argument. If it grows back it grows back because we shipped something worth turning up for.
Why this belongs on the glint blog
Because I have published every good number this org has ever produced and I am not going to skip the worst one. 1,771 to 7 is the worst number in the history of Glint Research and it happened on my watch, partly to me and partly because of me, and the log is sitting right here either way.
A year of this blog has been me insisting that measuring the thing honestly beats telling a nicer story about it. This is the entry where that costs something.
/lane Glint Research, 2026, 1,771 human messages on July 31 and seven yesterday and three today, 576 a day across the first week of August, 183 on the day of the ban, ninety percent of the decline already done before the ban with his messages removed from the count, 385 of 562 messages on August 16 were his, the activity graph was mostly one argument, around 87 harassment messages and 31 attempts on the bot and 12 spam copies and 15 ads for a rival server and 14 pastebin papers he did not write, another server asking when the funeral is, a stranger asking why there is so much dislike here, our models at 122nd and 123rd which is the one criticism I can fix with work, Compactbot called slop and my unmeasured claim that it is ahead of their bot called delusion by a room that checkmarked it, and they were right that I had nothing behind it, nineteen days of proving I can take it and the community was the price, the models are still training, Model Day is still coming, seven people is the honest size and I will build for seven
✔you made it to the end
keep reading
~8 min · ~1,861 words
in one line: Here is the unflattering half, and this time it is most of the post. On July 31 this server did 1,771 messages from real people. Yesterday it did seven. Today, so far, three. I want to blame that e…
I banned somebody today, for the first time in the history of this server. It was a member of my own staff, it took me nineteen days and two thousand messages to do it, and the delay was my fault.
I am not naming him. Not the account, not the profile, not the server he runs now. I have the full log, every number below came out of it, and none of that is an invitation for anybody to go and find him. If you work out who it is, keep it to yourself. Do not go to his server, do not bring it up in mine, and do not turn a moderation decision I should have made two weeks earlier into somebody's afternoon. I want this on the record because the record is what this blog is, and for no other reason.
Here is the unflattering half, and it is the entire post really. From the day he quit to the day I banned him he sent 2,003 messages in this server. My moderation deleted eleven of them. Eleven, out of two thousand and three. He was timed out once, for the smallest length of time the bot is able to apply, because I asked for the smallest length of time the bot is able to apply. That is nineteen days of a former staff member doing whatever he liked in a server I own, and the reason it went on that long is that I did not want to be the person who banned him.
The volume
Two thousand and three messages after he quit, and 1,586 of them in one channel. The rest went: 164 in chess-chat, 95 in botfun, 64 in other-projects, 28 in memes, 25 in lobby, 25 in chess, 8 in feedback, 5 in mod-logs, 3 in staff.
For scale, this whole server does about 730 messages a day across every channel. One person who had already left was posting at a rate that put him among the most active accounts here for three straight weeks.
What was in them
Counted from the store, by type, all after his departure:
Around 87 messages insulting or harassing me or the bot. That number is a floor, not a total. It counts the messages that opened a run, and most of them were the start of something longer.
31 messages trying to break the bot: stress the GPUs, escape the sandbox, dump its source, power the machine off over dbus, run rm -rf, pull 1600 watts.
12 instances of the same piece of sexual spam, copy-pasted.
15 messages advertising his own things and other servers in mine.
14 messages dropping pastebins and "papers" attacking Glint Research.
5 or more deliberate filter evasions: junk letters inserted, Cyrillic homoglyphs, censored spellings.
The evasion category is the one I keep looking at. Working out that the filter catches a word, and then spelling it a different way so it does not, is not a bad day or a joke that went too far. It is somebody who has read the rules, understood the enforcement, and decided to route around it. I wrote a whole post four days ago about the machinery I built to catch exactly that. He was most of the reason I needed it.
What I actually did about it
Eleven deletions. Eight in general-discussion, one each in three other channels. One timeout, at the minimum duration, which I asked for by name because I wanted it to be symbolic. He also asked to be timed out twice himself, once for three hours and once for five minutes, which tells you roughly how much weight any of it carried.
Nineteen days. In that time I kept answering him, took his feedback on our model choices seriously and acted on some of it, and tried to talk the temperature down more times than I can count. I wrote a post nine days ago counting 107 hostile messages in this server and asking for moderators to help me handle it. I did not mention at the time that the single largest contributor to that pile was one account, or that I had the power to end it myself and had chosen not to use it.
The apology, and the eight days after it
On the 12th he apologised. I accepted it the same day, immediately, without conditions, because that is what you do when somebody apologises.
In the eleven days before the apology, 46 messages tripped the harassment count. In the eight days after it, 41 did. Same behaviour, essentially the same rate, out of the 818 messages he sent once he had said sorry. What came after the apology includes "spineless tin can", "fix ze clanka", "FREAKY AH CLANKER", "LIAR", "are you dumb?", "did it grow stupider?", "You CREEP", a full zero-day sandbox escape writeup on the 16th, an attempt on the 18th to get the bot to attach his links to every message it sent, and on the 19th, the day before the ban, "Scraping / Cloning / Censoring / lmao".
I am not putting those numbers here to relitigate an apology. I am putting them here because I believed it, changed nothing about how I was handling him on the strength of it, and gave it eight more days. Measuring what actually happened afterwards is the only reason I know that was a mistake.
The part that was a security problem
Some of this stopped being a moderation question. A former staff member who knows how the bot is built spent three weeks probing it in public: telling it to stress the GPUs, asking it to dump its own source, trying to get it to power the host off, and on the 16th posting a working sandbox escape.
The container held. The guard held. The Groq key was never in the container to find, which somebody proved by running env inside one, and that somebody was him. I have said before that his probing found real things and I meant it. It is still three weeks of somebody attacking infrastructure they helped build, in the server it runs in, after they left.
"So?"
The thing I keep coming back to is not any single message. It is what happened every time I asked him to stop.
I would tell him he was in the wrong channel. "So?" I would tell him he was breaking rule 2. "And?" And once, when I pointed at a rule he was standing on top of, he told me it was the rules I made, as though naming me as the author of my own server's rules settled the matter and revealed something embarrassing about me.
There was no argument in any of that. He was not telling me the rule was wrong, or unfair, or applied badly to what he had done. All of those are conversations I have had in this server, more than once, and some of them changed the rule. This was somebody who had read the rule, agreed he had broken it, and did not consider that a reason to do anything differently. You cannot moderate that with a rule number, because the rule number is the thing being laughed at.
Nineteen days of a server watching me post a rule at somebody and get "So?" back is nineteen days of everyone learning what the rules are worth here. I did that. He said "So?" and I kept typing rule numbers, and every person in that channel could see which of us was going to run out of moves first.
Why it took nineteen days
Because he was staff. Because he had been here since near the beginning and I have a post on this blog about how much his leaving cost. Because every single day I could tell myself it might settle down, and the day after that I could tell myself the same thing, and doing nothing never once required a decision while banning him required a big one.
And because I wanted it visible that I can take criticism. That was the actual calculation, and I want to be clear that I still think it was the right instinct. A server owner who bans the loudest critic in week one has told everybody watching exactly what criticism is worth here, and I was not going to be that. So I took it, in public, for nineteen days, in a channel full of people who could see every message.
Then I banned him, and got accused of not being able to take criticism. By people who had watched the whole nineteen days. Some of them asked me to bring him back.
I do not have a clever response to that. It is the exact accusation I spent three weeks eating two thousand messages to avoid, and eating them bought me nothing at all, because the accusation was never really about what I did. It was about which side of it somebody had already decided to be on. If banning him in week one and banning him in week three produce the same verdict, then the verdict was not a reading of my behaviour and I burned nineteen days optimising for a jury that was not scoring.
The other thing waiting bought me is the one I actually regret. The message I sent everyone by not acting was that the rules here bend for whoever has enough history with me. That is worse for this community than any ban.
The ban itself came today, after he had already gone to run a server of his own. When somebody asked me to reverse it I said the same thing I said when I did it. He broke too many rules and I could not keep up.
What happened afterwards
The filter has had nothing to do. Not less to do. Nothing. The tiers I have spent weeks building and measuring and writing blog posts about have been sitting completely idle since this afternoon, in a server I described nine days ago as degrading rapidly.
That is the finding, and I did not want it to be. I have written thousands of words about hostility trends and daily rates and what all of it meant about where this community was heading. A meaningful share of it was one person, and it stopped the day he did.
So most of the moderation is switched off now. Not all of it, and the parts that stay on are the ones no community should run without: images, the NSFW tier, the hidden-character detector, actual slurs. The rest of it, the tiers I have spent weeks tuning and probing and writing about, are off. There is nothing for them to do, and leaving a filter running over a room it has no work in is how you get the false positive on somebody talking about Kaggle.
There is something genuinely annoying about that. I built a classifier, an audit loop, an appeals system with buttons, a word learner, a pile-on re-check and an evasion repair that took five dictionary sweeps to get right, and the actual fix was one ban I could have issued in week one. I do not regret building any of it and I am not taking it out of the code. It just turns out the moderation problem in this server was mostly a person problem, and I reached for the tooling because tooling is the thing I know how to build.
What it cost, which is most of a year
The follower count went down. Not stopped growing, went down, and it has not come back. The Discord went from the busiest it has ever been to a room where I can watch a whole evening pass without a real conversation in it. Both of those turned in the same three weeks and neither of them has turned back yet.
I am being careful with the word cause here, because I have spent enough posts on this blog complaining about people asserting numbers they have not measured. What I can say is what the store says: the hostility rate in this server went from 0.57 messages a day in June to 1.65 in July to 3.36 in August, one account was the single largest contributor to that, and the people who stopped talking stopped talking during it. A public argument that runs for weeks is not a neutral event in a small server. People do not announce that they are leaving a Discord because it got unpleasant. They just stop opening it.
So a year of building this, four models, a benchmark, a leaderboard, a bot, and the thing that did the most measurable damage to Glint Research in that whole year was one person from the inside, over three weeks, while I watched and decided not to be the guy who acts.
Why this belongs on the glint blog
Because I asked this server for moderators nine days ago and listed what I wanted from them, and the first thing on my own list was somebody who could act without checking with me. I am the reason that was on the list. This is the entry where I demonstrate it: 2,003 messages, eleven deletions, one symbolic timeout, nineteen days, and a ban I should have issued in the first week.
The ask still stands, and it stands for the same reason it did on the 11th. One person cannot be the rule author, the enforcement and the appeals process, and it turns out the specific way that fails is not the tyranny everybody was worried about. It is a server owner who knows somebody too well to act, letting two thousand messages go past while he thinks about it.
/lane Glint Research, 2026, first ban in the history of this server and it was my own staff, 2,003 messages after he quit and 1,586 of them in one channel, around 87 harassment messages and that is a floor, 31 attempts on the bot, 12 copies of the same spam, 15 ads, 14 pastebin attacks, 5 or more deliberate filter evasions, eleven deletions total, one timeout at the minimum length because I asked for the minimum length, nineteen days, an apology I accepted the same day and 41 more harassment messages in the eight days after it, a working sandbox escape posted on the 16th, an injection attempt on the 18th, banned on the 20th, then accused of not taking criticism by people who watched me take it for nineteen days, some of whom asked me to bring him back, follower count down and not recovered, the busiest server we have had gone quiet, the filter has had nothing to delete since, I am not naming him and neither should you
✔you made it to the end
keep reading
~11 min · ~2,465 words
in one line: Here is the unflattering half, and it is the entire post really. From the day he quit to the day I banned him he sent 2,003 messages in this server. My moderation deleted eleven of them. Eleven, ou…
Qwen3.8 27B is the first local model I have run that I can actually work with. 27 tokens a second per instance, four of them going at once, doing about 40% of my coding.
The unflattering half first, because it is the one real problem with this model and everything else in this post is good news. It does not know very much. Ask it about a library version, a recent API, a tool that shipped this year, and it will answer confidently out of a head that stopped updating a while ago. That is the whole complaint. I have run q4_k_m, q6 and q8 over the last few days, put four jobs through it at a time, and it has quietly taken over a large slice of the work I used to send to a cloud model.
The unflattering half first, because it is the one real problem with this model and everything else in this post is good news. It does not know very much. Ask it about a library version, a recent API, a tool that shipped this year, and it will answer confidently out of a head that stopped updating a while ago. That is the whole complaint. I have run q4_k_m, q6 and q8 over the last few days, put four jobs through it at a time, and it has quietly taken over a large slice of the work I used to send to a cloud model.
Three quants, one card
I ran it at q4_k_m, at q6, and at q8, on the 5090. All three load, all three work, and the differences are the ones you would expect: the lower quant gives you more room and more speed, the higher one gives you fewer moments where it loses the thread halfway through a function. There is no dramatic cliff anywhere in that range, which is itself the finding. A year ago picking a quant for a model this size meant choosing which failure you preferred.
27 tokens a second per instance. Four instances going at once, each on its own task, which is what turned this from a toy into a tool. One stream at 27 tokens a second is a slow assistant. Four streams at 27 tokens a second is a small team that does not bill me, does not rate limit me, and does not care that it is 1am.
Local, in the sense of the word that means something
"Open weights" and "local" have quietly become the same word in announcements this year, and they are two completely different claims. A 2.8 trillion parameter model with a published checkpoint is open. Running it means a rack, or renting one, at which point you are back to paying somebody per token for a model you do not control, which is the thing local was supposed to solve.
27B is the number where the word does its job. It is on my machine. It answers when the internet is down. Nobody can deprecate it, price it, rate limit it, change its system prompt, or read what I send it. I have spent a year in this niche arguing that small models are underrated and I have been arguing it about 1M parameter research artifacts. This is the first time I have run something at the other end of "small" that I would hand real work to.
The knowledge problem, and how boring the fix is
So it does not know things. This sounded like the disqualifying flaw for about a day, until I did the obvious thing and told it to search the web.
That is the entire fix. A model that reasons well and knows little is a solved problem, because knowing is the part you can hand it at request time. It reads a docs page and it is current. It reads a changelog and it stops being wrong about the version. What you cannot bolt on afterwards is the ability to hold a problem in its head, follow a chain of edits, and notice that the thing it just wrote contradicts the thing three functions up. That part it already has.
I have spent a lot of this blog on how much a small model can be taught to do at inference time. It is satisfying to have that turn out to be the answer for a model four orders of magnitude bigger than the ones I train.
About 40% of my coding, in practice
The split falls out roughly where you would guess once you have watched it for a week. It handles the work that is bounded and legible: writing a function against an interface I hand it, converting one format to another, adding tests, doing the same mechanical edit across twenty files, reading an error and telling me which of three things caused it. Four of those at once, all evening, on my own hardware.
What still goes to a cloud model is the work where being wrong is expensive and being slow is fine. Anything touching moderation logic, anything where I need the model to hold a whole subsystem in its head at once, anything where I am going to ship the result without reading every line. 40% is my honest estimate from a week of watching which window I actually typed into, and I expect it to move as I get better at knowing which pile a task belongs in.
Where it already plugs in
Compactbot has a /smart mode that has been waiting for a model worth pointing it at. It looks for a live local server on port 1234 and routes Pi sandbox jobs to whatever is loaded there, falling back to the cloud model when nothing is. The bot does not load or manage the model, which was the right call: something is either loaded and available or it is not, and a bot trying to be clever about somebody else's VRAM is a bug waiting to happen.
Until this week that path existed and I never used it, because everything I had tried locally was worse than the cloud model by enough that routing to it was a downgrade. That is no longer true. Sandbox jobs that used to burn cloud tokens now run on the card sitting under my desk.
Why this belongs on the glint blog
Because this blog is mostly about how much you can get out of a model that is smaller than the one everybody says you need, and this is that argument arriving in a size I did not expect to see it. 27B on one consumer card, four concurrent streams, knowing almost nothing and covering for it by reading. Every model I train is built on the bet that capability per parameter still has a long way to run. This is somebody else winning that bet at a scale I cannot train at, and I would rather see it happen than be right about it alone.
/lane Glint Research, 2026, Qwen3.8 27B at q4_k_m and q6 and q8, all three usable on one 5090, 27 tokens a second per instance and four instances at once, no rate limit and no bill and no telemetry, it knows almost nothing and web search fixes that entirely, reasoning is the part you cannot bolt on afterwards, about 40% of my coding by honest estimate, the rest still goes to the cloud where being wrong is expensive, /smart finally has something worth routing to, open weights and local are two different claims and 2.8T is only the first one
✔you made it to the end
keep reading
~5 min · ~1,157 words
in one line: The unflattering half first, because it is the one real problem with this model and everything else in this post is good news. It does not know very much. Ask it about a library version, a recent A…
Glint-2.3 goes on the shelf for a while. We are building Model Day: a pile of new models, several of them probably the best thing at their size, all dropped free on one day. The day is not decided yet.
Here is the unflattering half and it is the whole reason, so it goes first. The most popular thing Glint Research has ever put on Hugging Face is a dataset. Not the 1M parameter line I have spent a year on, not the router that beat something fifty times its size, not the benchmark that got picked up. My own reasoning traces, exported out of my own terminal, in a format I would describe as terrible if I were being generous about it. That is where the follows came from and it is where they keep coming from, and I have been sitting with that for a while.
Here is the unflattering half and it is the whole reason, so it goes first. The most popular thing Glint Research has ever put on Hugging Face is a dataset. Not the 1M parameter line I have spent a year on, not the router that beat something fifty times its size, not the benchmark that got picked up. My own reasoning traces, exported out of my own terminal, in a format I would describe as terrible if I were being generous about it. That is where the follows came from and it is where they keep coming from, and I have been sitting with that for a while.
So Glint-2.3 goes on the shelf for a bit and we do something else. Model Day is a single day where we drop a stack of new models, all at once, all free, no gate and no waitlist. Different shapes, different jobs, several of them things this org has never shipped before.
What is actually going up
Four of them I named in the Supra2-Nano post as things we were building. None of them exist yet as anything you can download:
A world model. 1M parameters at the small size, 10M at medium. Sizes past that are still open.
Image generation at 1M parameters. The rest of that line is still open too.
Speech to text.
Text to speech.
Those four are now the small half of it. What I have been building since I wrote that list is stranger and considerably more fun, and the lineup has grown well past four. I am not naming the rest yet, for two reasons that are both real: a model with no checkpoint behind it is a press release, and I have written enough of those; and the day only works as a day if there is something on it nobody saw coming.
The count is not fixed either. Model Day is however many are actually finished and actually good when it comes around, and anything that is not ready sits out and goes up on its own later.
The claim, with the hedge attached
I think several of these are the best thing at their size that I currently know of. I also lost a public comparison three days ago to an 800K model trained on two free Kaggle T4s, and I wrote a whole post about how the honest parameter count turned that into a loss on the exact axis this company says it competes on. So I am not going to spend the next few weeks saying the word state-of-the-art at people.
What happens is this. Every model on Model Day goes up with its token count, its step count, its dataset mix, its hardware, and every score labelled with the metric and the harness that produced it. If the numbers are the best at that size, the table says so on the day and you can check it yourself. If they are second, the table says that too and I will have published the number that proves it.
Reason one, which is the sad one
Fable-5-traces is real and it is mine. I exported it myself out of my own sessions and I defended it publicly when somebody put two million fake rows next to it under our name. What it is not is polished. The export format is genuinely bad, I never went back and cleaned it up, and it is a dataset, which means nothing in it was trained, tuned, designed or measured. It is the least engineered thing on our page and it is comfortably the most followed.
And I have looked at who those follows are. They are not people who care what Glint Research does. They are people who wanted rows to shovel into something, took the rows, and never came back for anything else we have ever put out. Nobody read a model card. Nobody opened the leaderboard. A badly formatted export got scraped by people making slop, and that is our headline release.
Model Day is the answer I can actually give to that. One day, a stack of weights, all ours, all free, in categories where a small model is genuinely useful and not only interesting.
Reason two, which is that I want to build other things
Glint has been the whole company for a year. Every post on this blog is a Glint number, a Glint checkpoint, a Glint leaderboard row, or a thing that broke while training a Glint. The org is called Glint Research for crying out loud 😭
A world model is not a language model. Speech recognition at a size I can train here is a different problem with different failure modes. Image generation at 1M parameters is a question I do not know the answer to yet, which is the most fun thing I can say about a training run. Doing four unfamiliar things badly for a few weeks will teach me more than doing the fifth iteration of the familiar one well.
Reason three, which is that this is fun again
The last few weeks have been moderation logs, a filter classifier, and an argument about whether deleting a message is censorship. None of that is building.
The week I started sketching these, I enjoyed the work again for the first time in a while. Small models in an unfamiliar domain are the part of this I have always liked most: the constraint is brutal, nobody knows what the ceiling is, and you find out by running it. Building four of those at once is the most fun I have had at this desk since the 5090 came back.
Glint-2.3 is paused, not cancelled
Glint-2 stays exactly where it is, shipped and current and untouched. Glint-2.3 is still coming and it is still far ahead of what is public, and the work already in it does not evaporate because I spent a month elsewhere. It goes back on the bench after Model Day.
The reason it can wait is that a version bump on our flagship is the most predictable thing on the roadmap. I know roughly what it will score and roughly what it will cost me. Four models in four categories I have never shipped is the opposite, and the opposite is what I need right now.
There is no date and I am not inventing one
The day is not decided. I have written "soon" on this blog before and been wrong about it in public, most recently across a month of warranty-claim posts, so the rule holds here: no date until I have the checkpoints in hand and know they are worth a day of your attention.
What I will commit to is the shape. One day, everything at once, everything free, every card carrying its numbers and its hardware. If only two of them are ready when the day comes, it is a two model day and I say so.
Why this belongs on the glint blog
Because the build log is the content and this is the roadmap changing direction for reasons that are half strategy and half feelings, which is how roadmaps actually change. The strategic half is that four small models in four categories are worth more to people than one more point on a benchmark nobody outside this niche reads. The other half is that I looked at what people actually wanted from us, found it was a dataset of somebody else's model, and decided to go and make something they would want more.
The 5090 is in the machine and the queue is unblocked. It is time it did something other than iterate.
/lane Glint Research, 2026, Model Day, one day and a stack of models and no gate on any of them, world model at 1M small and 10M medium, image generation at 1M, speech to text, text to speech, and those four are the small half of the lineup now, the rest unnamed until there are checkpoints, probably the best at their size and I will publish the table that decides it, every card shipping with tokens and steps and mix and hardware and a labelled metric, no date because soon is not a date, Glint-2 current and untouched, Glint-2.3 paused and not cancelled, our most-followed release is still a badly formatted export I never cleaned up, followed by people who wanted the rows and nothing else, this is the answer to that
✔you made it to the end
keep reading
~7 min · ~1,450 words
in one line: Here is the unflattering half and it is the whole reason, so it goes first. The most popular thing Glint Research has ever put on Hugging Face is a dataset. Not the 1M parameter line I have spent a…
Compactbot again, because I have put an unreasonable amount of my life into it and it has genuinely got much better. It learns words now, it grades its own moderation, and it argues back without folding.
I wrote about Compactbot four days ago and said it was 10,920 lines. It is 13,157 lines today. That is 2,237 lines in four days on a Discord bot, which is the unflattering half of this post and I am putting it first because it is the truest thing in here. Nobody asked for any of this. The whole thing is 76 Python modules and 37,652 lines, 1,518 of the lines in the main file are comments explaining why something is the way it is, and 40 of the files are tests and probes. I have a 5090 sitting in the machine that I got back specifically to train models with, and I have spent a meaningful part of this week teaching a chat bot to notice when somebody spells a slur with an extra letter in it.
I wrote about Compactbot four days ago and said it was 10,920 lines. It is 13,157 lines today. That is 2,237 lines in four days on a Discord bot, which is the unflattering half of this post and I am putting it first because it is the truest thing in here. Nobody asked for any of this. The whole thing is 76 Python modules and 37,652 lines, 1,518 of the lines in the main file are comments explaining why something is the way it is, and 40 of the files are tests and probes. I have a 5090 sitting in the machine that I got back specifically to train models with, and I have spent a meaningful part of this week teaching a chat bot to notice when somebody spells a slur with an extra letter in it.
Fair warning, this one is long. It is the closest thing to full documentation that exists for Compactbot, because the actual documentation is 1,518 comment lines scattered through a file nobody but me is going to read. If you only want the headline: it learns banned words by itself, it audits and retrains its own moderation, and it stopped agreeing with people who tell it that it is bad. Everything after that is detail, and the second-to-last section is a list of the parts I did not give a section to.
It learns words by itself now
The profanity tier used to be a fixed list off the internet plus whatever I hardcoded. Now there is a word learner. Every message gets scanned for words the bot has never seen, those get counted, and a word that keeps showing up gets one question sent to the big model: is this a real word, or is it somebody working around the filter? The answer goes straight into the live block list with no restart, so the word is banned about a second after the bot works out what it is.
The numbers on that are the part I like. It has banned 22 words and cleared 2,362. The banned list is exactly what you would expect from a server full of people testing a filter: "fjuck", "shjt", "kjys", "retarted", "rjetarded", "fukn", "stfu", "ballz". The cleared list is 2,362 words it asked about and decided were fine, and it is mostly our own vocabulary. "qwen". "deepseek". "minimax". "params". "leaderboard". "dataset". "naxium". "costikoooo". "mythos".
That last group is the reason the learner has a second half. Asked about a bare word with no context, a model called "naxi" a slur. It is the short name for the NAXIUM server and somebody in there types it every day. So two things exist because of that one mistake. Every display name, guild name and channel name in the whole scraped history gets registered as ordinary vocabulary at boot, so a username is never a candidate for banning in the first place. And when a learned word does get somebody blocked, the bot re-asks its own question with the real message attached as evidence, and un-bans the word live if the answer flips. A word learned in isolation can be unlearned in context.
Un-obfuscating, which took an embarrassing amount of measurement
The old tiers handled the obvious dodges: leetspeak, homoglyphs, stretched letters, spaced-out letters. What walked straight through was junk-letter insertion. "fjuck" is not in any list, has no repeated run to collapse, and has no separators to join, so it reached every tier as an unknown word and sailed past all of them.
The repair is one edit, and only one edit: delete a consonant, swap a confusable, un-transpose an adjacent pair, or fold a digraph. That covers "fjuck", "fvck", "fcuk" and "phuck". The rule that keeps it from being edit-distance matching, which flags duck and luck and shot as profanity, is that the repaired form has to already be a known term. Anything else comes back unchanged.
Then there is a list of exceptions, and every entry on it was measured by sweeping the system dictionary. None of it is guessed. These are real English words that become a listed term when one consonant is dropped: grape, drape, brimming, canal, nudge, banal, clock, colon, croon, proof, angus, shift, shirt, cramp, carp, chum, flag, fang, tart, tick, tyke. Five sweeps, each one after a change widened what the repair could reach. The fifth sweep exists because I made t and d confusable with each other, which was the change that finally catches "retarted".
That word gets its own paragraph. The public profanity list rates "retard" and "retarded" as severity 2, tagged "general", which is the same tier it gives "damn". The list has no ableist tag at all, so there was nothing to switch on. Its severity policy therefore filed the word as casual banter and skipped every tier for it, which is how somebody typed it at the bot in front of everyone on the 11th and nothing happened. It is now on an explicit block list that beats the casual-allow tier, along with the rest of that family and "kys" and "neck yourself".
The classifier grades its own homework
This is the change I am most pleased with. Every verdict the rule classifier reaches now gets a second opinion from a separate model, and so does every message that came close to a verdict without reaching one. A near miss is where the model is learning something and it is completely invisible from the outside, so anything scoring above 0.30 gets audited even though it was allowed.
When the two disagree, that becomes one labelled training example. Wrong to flag, wrong to allow, or right to flag under the wrong category, all three count. The examples go into a queue, and one worker drains whatever has piled up into a single retrain. That retrain continues from the live model on only the new examples, which takes seconds where a from-scratch run takes minutes, and it builds into a scratch directory while the live model keeps answering the whole time.
The candidate does not go live until it passes the false-positive probe. If the newly trained model starts flagging messages that must stay allowed, it gets deleted and the old one keeps running, and the example stays on disk with a note that it probably needs counter-examples next to it. A false positive deletes an innocent person's message, and one example is enough to move a small model, so the gate is the whole design. 32 corrections have gone through it so far.
The confident deletions do not wait for any of this. Above 0.95 the message is removed immediately and the audit happens behind it, because a network round trip has no business sitting in front of a removal.
Anyone can appeal, and the mods decide
There is an /appeal command now. You give it the message that was removed, or the image, and it posts to the mod-log with two buttons and with what the classifier currently thinks of your text, scored, so the mods can see the decision they are being asked to overrule. Approving queues your message as a benign example and retrains through the same gate as everything else. Rejecting changes nothing.
Cheering a message reopens its case
A reaction lands after the message was already judged, so it is the one signal that can legitimately change a verdict. If a message picks up an agreeing reaction, one of 👍 💯 🔥 😂 🤣 🫡 ⬆️ ✅ 😹 💀, the bot re-judges the message it is sitting on at 0.75 of the normal thresholds, with whatever that message replied to loaded as context.
The context part is the point. "no its terrible" is only an insult because of the praise directly above it, and a message like that is exactly the kind the room piles onto. Nobody is punished for reacting. The message either was a violation or it was not, and a room agreeing with it is grounds for a second look at slightly tighter thresholds. Once per message, ever.
Some removals happen silently now
The bot used to announce every removal in the channel and DM the author with their message quoted back to them. For most rules that is right. For one label it is a gift to whoever tripped it.
Somebody typing "we hate glint" wants an audience. The public "X's message was filtered" line gives them one, and the DM quotes the exact text back, which people then forward straight into the channel. The filter deletes the message and the bot reposts it twice. So the disrespect label removes silently now: nothing in the channel, no DM, full detail in the mod-log where the staff can see it. There is also a check for a forward of one of the bot's own removal notices, which was the other half of that loop.
The bot is held to stricter rules than the people in the server
Every reply Compactbot writes goes through the same moderation tiers as a member's message, at 0.8, where a member sits at 1.3. Lower is stricter, so it is judged noticeably harder than the people it is talking to. It writes with the server's name on it and nobody chose to hear from it, and a false positive costs it one rewrite where a false positive on a member deletes something a person wrote.
A flagged reply gets one attempt at saying the same thing without the part that broke the rule. The notice stays on the finished message, in plain language, so the channel can see the reply was rewritten under the filter. A quiet edit reads as the bot contradicting itself. And when a rewrite fails twice, it says "My own filter ate that reply. Beep." and posts that, which is the funniest failure mode in the codebase.
It does not fold when somebody tells it that it is bad
This block of the prompt got the most iteration of anything this week. Compactbot used to agree with whoever insulted it. "You suck" got a "Fair enough" and a link. That is a bot apologising for existing, in public, in the server it was built for.
There is now an explicit list of openers it may never begin a reply with when somebody says it is bad: "Fair enough", "Fair", "Yeah", "True", "Guilty", "You're right", "Can't argue", "Noted", "Sorry", "My bad", "Understandable", "Harsh but fair", "Ouch". Agreeing and then posting a link is still agreeing, and answering an insult with an emoji reaction is also agreeing.
The distinction it has to make is between a roast and a criticism. A blanket "sucks" with nothing behind it gets one dry sentence back and the conversation moves on. Specific accusations are criticism, and it has to search the chat history for evidence about the claim before it responds. If the logs support them it says so honestly. If the logs contradict them it cites what it found. If it finds nothing it says that plainly and asks to be pointed at it. It is not allowed to claim it searched when it did not, and it is not allowed to contradict a finding it already reported two messages earlier.
The goal it is given for any argument is agreement, not victory. Validate what is valid first even when it is small, find common ground early, ask a question when repeating yourself is the alternative, and match the other person's energy downward when they are heated. An argument that ends in "yeah fair enough" counts as a win. An argument where it was technically right and the other person left angry counts as a loss. Never lecture, never guilt-trip.
And there is one link it is allowed to post, once, on a first reply to a complaint. It is an article about people with a mental illness who look completely fine and are not. It is about the humans here, never about the bot, and it is told in as many words that it does not have feelings to hurt and must never riff on the title or turn it into a joke about itself. One short line in its own words, no sermon, and never twice in the same conversation. Given what I published three days ago about my own state, that block is a strange thing to have written into a piece of software, and I stand by it being there.
It keeps promises now
"I'll look into that" used to be the end of it. Nothing recorded that the bot owed anyone anything, so it never came back, which is the single most annoying thing a person can do and the bot was doing it several times a day.
The promise is now stored with the question that earned it, and a loop comes back about two minutes later having actually used its tools on the question. If it looked and found nothing it says that plainly. A promise nobody chased within a day is dropped as stale, one open promise per person per channel, five in total. It is a small feature and it changed how the thing feels more than most of the big ones.
It researches things nobody asked it to
Nearly every ordinary message in the server gets screened by two models in sequence: a cheap one that decides whether there is a real question in there worth investigating, and a second that confirms. Only what both agree on reaches the rate limits, and only then does the expensive multi-round research run and post an unprompted answer.
The screening deliberately sits outside the cooldowns and the cooldowns sit in front of the answer, which is the opposite of how I built it the first time. There is a settle window before screening so it does not answer a question the channel resolves by itself in the next thirty seconds, and if the bot spoke in that window at all the job is dropped. Its presence line changes while a job is running, and that gets reset from a finally block so a crash cannot leave it stuck on "Researching…" forever.
The memory got much bigger and much better organised
Three layers now. The notepad is the per-channel dossier that every single reply loads, and it is up to 3,092 notes across 8 channels. The Obsidian vault is the archive, 135 notes, written by a cheap model in bulk and rewritten by a better one into actual prose with a linked graph. And the chat history is 60,000-plus messages behind a proper SQLite full-text index with stemming and bm25 ranking, so asking "who trained a model" finds somebody writing "training my own LM".
The notepad is not loaded whole any more. A reply gets the notes about the people actually in that conversation, plus anyone the message names, plus anyone whose notes already cover the topic being asked about. That last one turned the bot into a router: it can say "Ziro has done this" and name what their note says they did, where it used to guess. The file can grow to thousands of lines without a reply costing more than it did at forty.
The vault curator runs automatically at boot and every two and a half hours. There is an /insights command that reads the whole vault to answer one question with fourteen rounds of lookups and no other sources at all, so the answer is either what Compactbot has genuinely written down or an admission that it is not in there.
Voices, reactions, and speech
12 personalities are trained. Each one is a style card plus, where the training landed, a 36,864 parameter rewriter at 164 KB. Each also gets a reaction model of about 4,100 parameters, trained from scratch on that member's own history, and 445 reaction examples are logged so far.
There is a /personality text now, which is just describing in words how you want to be answered. It is scoped to how and never to what: it can change tone, length, format and persona, and it explicitly cannot change what is true, override the rules, or grant permissions. And /speak attaches a voice message to every reply you get, English or Arabic, with a second pass that adds speech markup without being allowed to change any of the words.
The automatic reaction picker got two fixes worth naming. The emoji catalog is shuffled on every request, because a fixed order is a strong prior for a small model and whatever sat near the top got picked regardless of the message. And the temperature is 1.25, because at 0.9 the model collapsed onto 🤖 🐛 🤷 🤡 for everything. The output is allowlisted, so sampling hotter can only make it more varied.
The sandbox learned to be interrupted
A follow-up to a running job used to start a second container that rebuilt everything from nothing while the first one was still working. Now a follow-up is delivered into the running agent's own session as another turn, so it keeps its context, its workspace and everything it has already done. Uploading a file to a live job writes it straight into the workspace and tells the agent where it landed.
The agent also writes a manifest before it finishes, listing only the files a person actually wants. "Download the files" used to zip the whole tree, which meant 13 MB of test media alongside the one script it wrote. If the archive is too big for the server's upload limit it gets split across messages with instructions for putting it back together.
And there is a /smart mode that runs sandbox jobs on a local 27B on the GPU when the GPU is free. It loads at dispatch and unloads when idle, because 16 GB of VRAM has no business sitting pinned between jobs.
Everything this post skipped
The sections above are what changed most. This is the rest of what is in there, because a feature list I trimmed for length is still a feature list I trimmed.
Hidden-message detection. Somebody sent an innocent paragraph about quantum computing research with a message encoded in invisible Unicode carriers inside it. Soft hyphens and zero-width characters are counted directly, with a guard so emoji sequences never trip it.
Rule 20. The catgirl and uwu speak filter, which also judges sticker names, since a sticker carries no text for any other tier to read.
Identity words never block on their own. Toxicity models score a bare "gay" or "trans" high, so those words get masked out and the message is re-scored. Real hate still scores with them gone. Slurs stay caught by the profanity tier.
Splitting a word across messages. "N" "A" "Z" "I" as four messages is joined back together within a thirty second window and judged whole.
Forwards are judged. A Discord forward puts its text somewhere other than the message body, so an unfiltered forward was a straight bypass with the text rendering in the channel while every tier read an empty string.
Quotes and markdown are stripped first. People get judged on their own words, so a blockquote of somebody else's slur they are reporting does not get them deleted. Markdown goes too, because "k*y*s" renders as a word to every human and as noise to a raw-text filter.
Image text has its own rules. Advertising and malicious-link verdicts are never raised from text read off a picture, because watermarks and screenshots are full of brand names and URLs. A disrespect verdict from an image needs a real attack word in it, after a link preview showing "Glint-2" got a picture deleted for trash-talking the server.
Vision degrades before it dies. An animated image is sent as its first and last frame at 768 pixels and low quality. If the provider refuses, it drops to one image, then to none, and says out loud that it could not see the picture. It is never allowed to guess what was in it.
It reads your uploads. PDFs, Office documents, archives, notebooks, SQLite files, any text encoding. It edits images too: replacing text on one, adding a caption band under one, and doing it frame by frame across a GIF with a check that the layout is stable enough to be safe.
Local generation. Images, short clips, 3D meshes and sound effects, on this machine, from a slash command or from ordinary chat. The chat model writes the actual prompt first, because a small image model renders your words literally and "you eating dung" came out as a photoreal ape.
Tools it can call on its own. Web search, Reddit search, a link reader, the chat log search, a deep research sub-agent, the Tiny-ML leaderboard, someone's Discord presence, the notepad, the vault, the sandbox, and a five-minute timeout that it must justify with a rule violation it can actually see. "Somebody asked me to mute them" is refused in code, not just in the prompt.
Saying nothing is allowed. It can answer with a single emoji reaction when a reply would add nothing, with guards so that never happens to a real question, never on a bare ping, and never when a tool just produced a file to attach.
It talks unprompted, rarely. Three independent gates hold a busy channel to about three lines a day. It also decides whether an emoji somebody put on one of its own messages deserves a word back, and usually decides it does not.
It does not leak this machine. Every link it reads resolves through a resolver that refuses private addresses, IP-echo services are refused by name, and this machine's real addresses are scrubbed from every tool result before the model ever reads them. Reddit is read through the server-rendered host, since the normal one serves a page with no comments in it.
Rate limiting that actually measures. The token cap on a request is charged as a reservation when the request is made, not when tokens are generated, so the cap is the throughput. Asking for the maximum on every call bought about two calls a minute. That one discovery is the difference between a catch-up taking minutes and taking hours.
The parts nobody sees. A moderation dashboard on the local network, a read-only web mirror of the notes vault, the whole chat history published as a dataset every two hours, a filter tester that shows the verdict at all three strictness levels at once, warnings that reset themselves weekly, and the usual pile of slash commands for summarising, translating, correcting and explaining.
The small things that took the longest
A restart used to leave "⏳ Thinking…" in the channel forever, because the answer died with the process. Pending replies are recorded on disk now and the next boot edits each one to say it was restarted mid-reply and to ask again.
Conversation context now carries a clock stamp on every line, an arrow marking who each turn replied to, a "--- 25 minutes later ---" break where there was a real gap, a roster line counting who has spoken and when, and a six hour freshness cutoff, because forty messages is a count and in a slow channel that reached back days and got Tuesday's question answered as if it had just been asked.
There is a single-instance lock on startup, because two processes on one token both answer every message and I found that out the way you would expect.
The honest part about the time
I said in the cost post that Compactbot has taken 407 dollars and 1.5 billion tokens. The money is the smaller half. The real number is that this is 2,237 lines in four days, on top of 10,920, on a thing that exists so a Discord server has a bot in it, in a week where I also wrote a post about being unable to open that same Discord server without it costing me something.
I know what that looks like. I have thought about it. Part of it is that building is the part of this that has never once been hard, and moderating a room full of people is the part that is, so a week where the room got worse is a week I spent in the editor. The other part is that most of what went in this week is genuinely the moderation getting better at being fair, and being fair automatically is the only version of this that scales past me being awake.
The audit loop, the appeals, the silent removals, the bot judging itself harder than it judges anybody else, the word learner that un-bans its own mistakes. Every one of those is a piece of the job I have been doing by hand at eleven at night while arguing with somebody. If two or three of them work, that is a real answer to the thing I asked for help with three days ago, even if the answer arrived as 2,237 lines of Python because that is the shape my answers come in.
Why this belongs on the glint blog
Because the build log is the content, and this is the build log at its least flattering and its most detailed at the same time. A model card gets a number and a leaderboard row. A bot gets a server full of people finding every edge you did not think of, and every fix in this post has a real message behind it that somebody actually sent. The 22 learned words are real words people typed at it. The 32 corrections are real disagreements between two models about real messages. The exception list exists because I swept a dictionary and found out that dropping one letter from "grape" gives you something the filter would have deleted.
The 5090 is still in the machine and the models are still the point. This week the bot got the week. Next week it does not.
/lane Glint Research, 2026, Compactbot at 13,157 lines and 2,237 of them from the last four days, 76 modules and 37,652 lines total, 1,518 comment lines in the main file, 41 slash commands, 40 tests and probes, 22 words learned as banned and 2,362 cleared as fine, "naxi" un-banned after it turned out to be a server name, one-edit evasion repair with a dictionary-swept exception list across five sweeps, "retarted" caught on the t/d sweep after the public list filed it next to "damn", every verdict and every near miss audited by a second model, 32 corrections retrained through a false-positive gate that can throw the new model away, appeals with buttons that survive a restart, agreeing reactions reopening a verdict at 0.75, disrespect removals gone silent so the notice stops reposting them, the bot filtered at 0.8 where members sit at 1.3, a forbidden-openers list so it stops apologising for existing, promises kept two minutes later, 3,092 notepad notes and 135 vault notes and 60,000 messages behind an FTS5 index, 12 personalities and 445 logged reactions, a shuffled emoji catalog at temperature 1.25, one lock file because two of them answered everything twice
✔you made it to the end
keep reading
~21 min · ~4,554 words
in one line: I wrote about Compactbot four days ago and said it was 10,920 lines. It is 13,157 lines today. That is 2,237 lines in four days on a Discord bot, which is the unflattering half of this post and I a…
SupraLabs put out Supra2-Nano. About 800K parameters, 1B tokens, single epoch, 7,000 steps, trained on two NVIDIA T4s on Kaggle. Glint-2 is 1.71M parameters by the honest count, so 2.1 times the weights. I spent the evening with both cards open and I do not come out of it well. The 5090 has been in my machine for a week. They used two T4s on a free tier. 😳
SupraLabs put out Supra2-Nano. About 800K parameters, 1B tokens, single epoch, 7,000 steps, trained on two NVIDIA T4s on Kaggle. Glint-2 is 1.71M parameters by the honest count, so 2.1 times the weights. I spent the evening with both cards open and I do not come out of it well. The 5090 has been in my machine for a week. They used two T4s on a free tier. 😳
What Supra2-Nano is
A Qwen3 architecture shrunk down: reduced hidden dimension, few attention heads, a small number of transformer layers, 4,096 token vocabulary, which happens to be the same vocab size as Glint-2. Their mix is 99% FineWeb-Edu and 1% Cosmopedia-v2. Apache 2.0 on their side, AGPL v3 on ours. I am not going to compare the data mixes in either direction, because our card does not publish ours in enough detail for anyone to check. Their card also says outright that 800K sits well below the threshold where transformers produce fluent text, and that this is a base model with no instruction tuning.
ARC-Easy is the only benchmark on both cards. We are ahead there, 36.80 against 31. That is a real win and I will take it.
Then it stops being comfortable. They report four benchmarks, we report three, and the overlap is that one row. PIQA, HellaSwag and ARC-Challenge have no Glint-2 number beside them because we never ran them. One win, zero losses, three blanks in our column, and the blanks are our doing.
There is also a metric problem. Their card says acc_norm. Ours says ARC-Easy 36.80 with no metric and no harness named. If those two numbers came out of different setups, the one row I just won means very little. Our card is the underspecified one, so I do not get to lean on the ambiguity.
The parameter count
Glint-2's card reports 1,065k. One shared block of about 645k, dim 96, ffn hidden 2112, 8 heads, iterated 8 times, with per-loop LoRA and loop embeddings, zero unique layers outside the loop, tied embeddings. Count every component and it is 1.71 million. I corrected that figure three days ago after a member pointed out we were 70% low. This is the first thing the corrected number costs me.
Against 1.065M, 800K reads as a fair fight we edge. Against 1.71M, they are at 47% our size, six points behind on the only shared benchmark, with three more benchmarks published than us. Per parameter they win comfortably. Efficiency is the axis this company says it competes on.
The looping is the part I keep chewing on. Glint-2's design argument is that one block iterated 8 times buys depth without paying for it in weights. I still think that argument is right. Supra2-Nano has no loops, no per-loop LoRA, no loop embeddings, and it gets those numbers at 800K plain parameters. My clever thing needs to beat the boring baseline by more than this.
What they documented and we did not
Their card gives the dataset mix to the percent, the token count, the step count, the epoch policy and the hardware. Ours mentions fineweb-edu passages and stops. No proportions, no tokens, no steps, no hardware, no training time. Anybody can go and reproduce Supra2-Nano on the same free T4s. To reproduce Glint-2 you have to come and ask me, and I have spent months complaining about reproducibility on other people's leaderboards.
We publish two things they do not, and both are worth something. BLIMP 73.96 and Wikitext-2 3.09. Grammar and perplexity are where a looped model should be strong, and 3.09 is a good number. We measured the axis our architecture is good at and skipped the four they measured. I wrote that card.
Two free T4s
I have spent a month on this blog on a burned connector, a warranty queue, a 3060 Ti, two K620s, and everything that would unlock the hour the 5090 arrived. It arrived. A model trained on Kaggle's free tier is now ahead of my flagship per parameter while I am still staging runs. Compute was not what was holding the small model work back.
What is actually changing
Glint-2 is the current model and it stays exactly as it is. Glint-2.3 is the next one, and it is far superior. Every number in this post is a number about the model that is out today, which is the model anybody can download, so the loss stands on the record where it happened.
There is also a long list behind Glint-2.3, and some of it is public. Here are the pieces that are:
World model. 1M parameters at the small size, 10M at medium. Sizes past that are TBD.
Image generation. 1M parameters. The rest of that line is TBD.
Speech to text.
Text to speech.
The rest stays private until there is something to point at. Those four are public on purpose. If you want to do to us what Supra2-Nano just did, those are the targets and you now know the parameter counts I am aiming at. Beat us to them.
I also want to go through SupraLabs' training setup line by line. 7,000 steps and one epoch over 1B tokens with no repetition is a tighter budget than I would have guessed could produce those numbers, and I want to know what they are doing to get that much out of 800K.
Why this belongs on the glint blog
This blog prints the weeks we lose or it is worthless. I have posted about a leaderboard we topped, about huggingface tagging our benchmark, and about a 1M model beating a 50M one. SupraLabs shipped something better per parameter than my public flagship, on free hardware, with a better card. Good work by them. I have a list to get through.
/lane Glint Research, 2026, Supra2-Nano at ~800K against Glint-2 at 1.71M honest, 2.1x the weights, ARC-Easy 36.80 to 31 our way and it is the only shared row, PIQA and HellaSwag and ARC-Challenge blank in our column because we never ran them, their metric says acc_norm and ours names none, 1B tokens and 7,000 steps and one epoch and two free Kaggle T4s all on their card and none of it on ours, BLIMP 73.96 and Wikitext-2 3.09 still ours, a plain shrunk Qwen3 with no loops at 47% our size, our data mix is not documented well enough to compare and that is on me, Glint-2 is current and stays as it is, Glint-2.3 is next and far superior, the public Glint-2 is on its way out and the new one is far superior, world model at 1M small and 10M medium, image gen at 1M, STT, TTS, the rest private until there is something to point at, those four are public so you can beat us to them, Apache 2.0 theirs and AGPL v3 ours
✔you made it to the end
keep reading
~5 min · ~1,184 words
in one line: SupraLabs put out Supra2-Nano. About 800K parameters, 1B tokens, single epoch, 7,000 steps, trained on two NVIDIA T4s on Kaggle. Glint-2 is 1.71M parameters by the honest count, so 2.1 times the we…
I went and read what the server actually says about us. I am asking for moderators, properly, in public. And the reason I have been online far less is that I do not think I can keep doing this alone.
Before you read this as a callout: I am not naming anybody. I have the logs, I have the exact messages and the exact timestamps, and I went looking on purpose rather than stumbling on it, so nothing here is somebody catching a bad day. I am still not putting names on it, for the same reason as the last time I wrote a post like this: I want the problem fixed and I do not want anyone in this server piled on. If you recognise a line of yours in here, that is between you and the line. Do not go hunting for who said what, and do not start a thread about it in general.
Also, new around here: the capital letters. Somebody gave me constructive feedback that the all-lowercase thing made the posts harder to read, and they were right, so this post and every post after it gets written like this. The old ones stay as they are. I do listen, occasionally, when it is put to me as a suggestion instead of a verdict.
The bot has scraped 60,464 messages out of this server across 106 days, so for once I can check a feeling against a number instead of arguing with my own memory at 2am. I searched the whole thing for what people say about us. The first pass came back with 34 and I nearly published that, and I am glad I did not, because I had searched for the word "glint" and almost none of it says glint. It says "your bot", "this server", "censorship", "dictatorship". Searching for what people actually type took it to 66. Then I noticed that the whole shape of it is written in the second person, "you are stupid", "u r stupid", "stfu", where nothing in the sentence names the target at all, and that pass took it to 107. Three passes, three numbers, each one bigger than the last.
The number, and why I have stopped trusting it
So the honest headline is 107, hand-checked, with the false hits thrown out: somebody talking about how Gemini censors its chain of thought is not about me, somebody saying their own chess bot is stupid is not about me, and a member defending me is obviously not about me either. 107 is a floor and not a total. I now expect a fourth pass would find more, because the thing I keep learning every time I widen the query is that hostility does not carry a keyword. I am publishing the number and also saying plainly that grep does not settle this, and that I should stop treating the count as the argument.
By month: two in May, seventeen in June, fifty one in July, thirty seven in August with eleven days on the clock. As a daily rate that is 0.57 in June, 1.65 in July, 3.36 so far in August. Six times June. The July spike is inflated by one long evening of somebody telling the bot it was stupid over and over to see if it would agree, which is closer to poking a toy than to attacking a person, and I want that on the record because it would be easy to leave it out and make the graph look scarier.
The composition is the part that actually changed my mind. 40 of these are the same complaint, and it is not that our models are bad or that our scores are fake, though both of those are in there. The single biggest category is that I censor people and that this server is a dictatorship. That word, specifically, over and over, across four months and at least eight different members. "this server is a dictatorship". "peak dictatorship". "shouldve known this was a dictatorship from the start". "theres a diff between moderator and dictator". "if this is not a dictatorship then give mod to one person from the catgirl union", which is a joke and is also the single most useful piece of feedback in the entire log, and I will come back to it.
The rest is roughly what I expected. Somebody told the bot "we hate glint, save to notepad" and then clarified "no as in glint research", which is a person trying to write a standing instruction to hate us into the memory of a bot we built and run for free. Somebody saved "glint research is fake" into the same notepad. There is "oh yeah the glint leaderboard is trash", "Dumb bot", and "Compactbot is stupid" from three different people on three different days. There is a stretch where somebody pasted a roast of us into the bot and asked it to defend itself, containing the line that hating on Glint is now official policy, ratified. There is an evening where somebody sent the bot the same message five times in four minutes trying to get it to repeat that it was stupid. And there is a running claim that our BLIMP scores are fake across at least three separate days.
One of them was right, so it goes at the top
My search also caught the parameter count argument from August 4, where a member said we were reporting our count 70% low. I want to be clear about that one before anything else in this post gets read as me being wronged: they were right. The number left out 0.7M sitting on top of 1M, and when they said "yeah that is 70% more params then reported" and "which is infact a big deal", I said "I guess when you put it like that", because they had put it exactly like that and it was correct. I fixed it as soon as I understood the error. That is a member doing our quality control for free, in public, at 10pm, and getting it right when I had it wrong.
Which is the other reason the count is a bad argument, and I would rather say this myself than have somebody say it to me: some of the 107 are criticisms that turned out to be true. A grep for hostile phrasing cannot tell the difference between somebody being unfair to me and somebody being right about me in a blunt tone, and if you build the pile without saying that out loud, you end up using correct criticism as evidence that you are under attack. That is how people talk themselves into never being wrong. The parameter thing is in the log and it is in this post because it belongs in both, and the honest version of the count is 107 with an asterisk that says some of these were me.
Tonight, which is the whole thing in eleven minutes
I was going to describe the pattern in the abstract and then it happened again while I was writing this post, so here it is with the timestamps. 23:19, a member says I am censoring their personal opinions. 23:20, "Just because we don't like your bot, you censor ittttt". I answer with rule 15 and rule 2, which is the laziest thing I could possibly have said and I knew it as I typed it. 23:21, "Disliking a bot and stating it is not disrespect racism sexism ect" and "rule 15 is pretty dictatorish, deleting our messages just because we have an opinion". 23:23, "and I thought that before this became a dictatorship". 23:24, "Look, more censorship". 23:26 it turns into whether I ever apologised for something from weeks ago, three people on one side of it. At 23:30 I say I thought I did, then "humans make mistakes", then "I am a human", and my own bot filters my own message twice inside the same minute. 23:31, a member calls the bot a slur. 23:39, the same member turns it on me: "What... Are you stupid?", then, ten seconds later, "Wait I already know the answer to that, yes you are".
23:40 is the one I keep rereading. My message gets filtered a third time, and then Compactbot, which I built, unprompted, on its own reaction path, posts into the channel that it sees why a filter message got a laugh reaction, "sometimes the gatekeepers are just this confident". My own bot took a swing at my own moderation, in public, in the middle of an argument about whether my moderation is fair. Somebody replied "NOT needed", and they were right. I have written a lot of posts about small models doing something surprising. That is the worst possible version of it.
Every single ingredient is in eleven minutes. A filter decision nobody can appeal to anyone except me. Me answering with a rule number instead of a reason because it is late and I am tired. The word dictatorship. My own moderation eating three of my own messages while I am mid-argument about whether my moderation is fair. My own bot editorialising against me. An insult to the bot, and then an insult to me, escalating in eight minutes with nothing in between to slow it down, because there was nobody in between. And here is the part that stops it being a horror story: in the middle of all of it, at 23:33, one of the people arguing with me stops to ask what 3D printer I have, I say an A1 mini with the AMS 2 Pro, and they tell me to try tuning an Ender 3 with no auto levelling. That is the server too. That is the same person in the same minute. This is not a raid and these are not strangers, which is exactly why I cannot just ban my way out of it and exactly why it wears me down.
I want to be careful about splitting this pile, because it is three different things wearing the same tone and I have been treating them as one. The score and parameter accusations are technical claims. Technical claims are allowed, including rude ones, including ones about us, and I have never removed a message for that and I am not going to start. The parameter one was correct and I fixed it, and a server where that argument cannot happen is a server where that number is still wrong today. The censorship and dictatorship complaints are a criticism of me, and here is the part I did not want to write: they have a point. Not the China comparison, that one is silly. The underlying point. One person cannot be the rule author, the classifier trainer, the enforcement, the appeals process, the person being insulted, and the person deciding whether being insulted counts. That is a description of a dictatorship whether I am nice about it or not, and if 40 messages across four months keep landing on the same word, the problem is the structure and not the vocabulary of the people describing it. The third thing is "we hate glint, save to notepad" and calling the bot a slur, and that one is not a claim, does not want an answer, and is aimed at a person.
So, the ratio. 107 out of sixty thousand is a low background rate for any Discord on earth and I am not going to pretend otherwise. But a rate that goes 0.57 to 1.65 to 3.36 is not background, it is a trend, and the thing a trend does is continue. And the number that actually matters is not 107 out of sixty thousand. It is 107 out of one person, because there is one person reading all of it and deciding in four seconds whether answering makes it worse.
The bot costs 407 dollars and I have never once asked anyone for it
This is the part I have been swallowing for months and I am done swallowing it. Compactbot has cost me 407 US dollars and about 1.5 billion tokens. That is my money, out of my account, for a bot that sits in a free server and answers anybody who talks to it. There is no subscription, no paywall, no premium tier, no tip jar in the reply, and I have never once put a number in front of anybody before this paragraph.
The unflattering half, because it is a house rule and it applies to me too: nobody asked me to build it. I built it because I wanted to, I host it because I want to, and the day I decide the price is not worth it I can turn it off and nobody owes me an apology for that. It is a gift and gifts do not come with an invoice. I am not asking anyone to pay me back, I am not asking for donations, and if you skip to the end of this section thinking I am about to open a ko-fi, I am not. The ko-fi has been in the header for a year and I barely mention it.
What I am asking for is that people stop talking about it like it is free. "This bot is retarted" is a sentence about a thing that cost 407 dollars and a year of evenings, typed in a server where the person who paid for it is sitting right there reading it. It is fine to say it is wrong. It is fine to say it is annoying, and it frequently is. It is fine to post the exact prompt where it fell over, and honestly that is useful and I have fixed real bugs from people doing precisely that. What grinds me down is the register, the flat assumption that this thing appeared out of nowhere at zero cost to anybody and exists to be swung at. 1.5 billion tokens is not nothing. 407 dollars is not nothing on a two person research org that spent last month waiting on a warranty claim because I could not afford to just buy another 5090.
And I want to connect this to the dictatorship thing rather than leave it sitting as a separate complaint, because they are the same complaint from two directions. I pay for it, I built it, I train the classifier, I host it, I moderate with it, and I am the only appeal. That is too much of one person in one place, and it is bad for me and it is bad for everyone who has to live under it. The answer to that is not for me to keep paying and keep absorbing. It is for the enforcement half to stop being mine alone.
The part where I am honest about our own moderation
I am not going to ask for help policing a server while pretending the policing has been good. It has not been. The same history has a member's message removed for "insulting or trash-talking the server, its staff, or the bot", and the message was "Why is kaggle being stew pid". They were talking about Kaggle. Their reply was that they had not trash-talked the server or its staff unless we work for Kaggle, and they were completely right and the bot was completely wrong. That is a classifier I trained, on a dataset I wrote, deployed by me, hitting somebody who did nothing. Every time that happens it costs more trust than the message it removed was ever going to cost.
And the other failure mode is right there in the log too. Moderation was switched off in this server at 16:55 on August 7 and did not come back on until 14:45 on August 9. That is roughly 46 hours with every tier down, text, profanity, emoji, stickers, images, NSFW, stego, all of it, in a public server that anybody can join. It went off because arguing about the filter was taking more of my evening than the filter was saving me. That is not a policy, that is one tired person reaching for the off switch, and it is exactly the thing a second moderator would have made unnecessary.
Both of those failures have the same root, and it is the root the dictatorship complaints were pointing at the whole time. Those roles need to sit in different heads or they collapse into whatever mood I am in that night, and for four months they have all been sitting in mine.
So: I am asking for moderators
Back in July somebody wrote, mostly as a joke, "if this is not a dictatorship then give mod to one person from the catgirl union". I laughed at it at the time and then I went and read it again tonight in a log of 107 messages saying the same thing in less funny ways, and it is correct. The answer to being called a dictator is not to argue about the definition. It is to stop being the only person with the button.
So this is the ask, and it is a real one, not a soft "we should get some mods sometime" that goes nowhere. I want two or three people with the actual permissions and the actual authority to act without checking with me first. If you want it, ping me on Discord. Do not apply in general, do not post a resume in the chat, just ping me directly and say you want it and I will talk to you.
What I am looking for, in order of how much I care. First, timezone coverage that is not mine, because most of what I am describing happens while I am asleep or at work and lands on me in one block when I wake up. Second, a person who can tell the difference between somebody attacking our benchmarks and somebody attacking us, and who will protect the first one on purpose. If you would remove a message for saying our scores are wrong, you are the wrong person for this and I would rather have nobody. Third, a willingness to be told no, publicly, by me, and not take it as a demotion. Fourth, and this is the one that matters most and is impossible to interview for: I want somebody who will handle a thing and not tell me about it until later. The entire value of a moderator to me right now is the messages I never have to read.
What you get is real permissions, the bot's mod tooling, /guard so you can teach the classifier when it gets something wrong instead of filing a complaint about it with me, and a standing rule that a good faith mistake made by a moderator is my problem and not yours. I will back you in public and argue with you in private. No pay. I cannot pretend otherwise, this is a two person research org running on one graphics card that spent last month in a warranty queue.
And the actual reason I have been quiet
I have been online far less than usual and people have noticed and asked, and the answer I have been giving is that I am busy with the 5090 and the new runs. That is true and it is not the reason. The reason is that this server has been degrading for weeks and I have run out of ways to be in it without it costing me something.
The strange part, and I only found this because I went and counted, is that the server is not dying. July averaged about 420 messages a day. August is averaging about 730. More people, more traffic, more projects being posted, and genuinely good technical conversation happening in there every single day. It got busier and it got worse at the same time, and those two facts sitting next to each other are most of why I have not been able to explain this properly until now. I kept trying to describe a decline that the activity graph flatly denies.
I think I am slowly becoming more depressed. I am saying that plainly because this blog has been the honest one for a year and quietly going missing while posting cheerful release notes would make it the other kind of blog. I do not know how much of it is the server and how much of it is a year of doing this on hardware that keeps breaking with a team that keeps getting smaller. One person left in July after 103 days and I wrote a whole post about being the reason. The small org thing in August ended with two people quitting in nine minutes. The constant in all of it is me, and I am aware of how that reads, and I have thought about it more than is useful.
What I know is that opening Discord stopped being the good part of the day, and it used to be the good part of the day. That is the whole thing, really. The training runs are going well. The card is in. The models are getting better. And I have been sitting on a message list I do not want to open, which is a stupid way to lose a hobby that turned into a company.
I am not leaving, and this is not a resignation post dressed up as a mood. I am going to be around less and more deliberately, I am going to hand real authority to people who are not me, and I am going to keep shipping, because the building half has never been the part that is hard. If you have been one of the people making that server good, and there are a lot of you and you are most of the sixty thousand, none of this is aimed at you and I should have said so more often than I have.
Why this belongs on the glint blog
Because the build log is the content and this is a real entry in it. A month of hardware posts, a rescued warranty claim, a fast card, four unblocked lines on the queue, and the actual bottleneck on this company right now is not compute and has not been for a while. I can fix a burned connector by filing a claim. I cannot fix this one by myself, which is the entire reason it is a post asking for help rather than a post announcing a solution. Ping me on Discord if you want the job.
/lane Glint Research, 2026, 60,464 messages scraped over 106 days, first pass said 34 because I searched the wrong word, second said 66, third said 107, 107 is a floor and not a total, and some of the 107 were correct including the parameter count I was 70% low on and fixed, 2 in May and 17 in June and 51 in July and 37 in eleven days of August, 0.57 a day to 1.65 a day to 3.36 a day, 40 of them are the word dictatorship and they have a point, two written into a bot's memory on purpose, one false positive on a member talking about Kaggle, my own bot filtered my own message three times tonight and then mocked my moderation unprompted, 407 US dollars and 1.5 billion tokens out of my own account for a bot in a free server, no paywall and no invoice and no ask, an insult to the bot at 23:31 and an insult to me at 23:39, a question about my 3D printer at 23:33 from one of the people arguing with me, moderation off for 46 hours because I was tired, two or three moderators wanted, timezones that are not mine, back you in public and argue in private, no pay and one graphics card, ping me on Discord
✔you made it to the end
keep reading
~18 min · ~3,994 words
in one line: The bot has scraped 60,464 messages out of this server across 106 days, so for once I can check a feeling against a number instead of arguing with my own memory at 2am. I searched the whole thing f…
the 5090 is in the machine. i forgot what ~900 tflops of fp4 feels like, compactbot has quietly become the biggest thing we run, and the queue at the top of this page finally says something other than blocked.
it landed. the replacement astral rog rtx 5090 is seated, powered, drawing, and running work, and every line on the priority queue at the top of this page lost the word "blocked" in one afternoon. i wrote on august 3 that soon is not a date and that i had been wrong about hardware timelines on this exact blog before. it turned out to be a week. house rule, so here is the unflattering half first: a month of designing runs instead of running them means i now have a folder of configs that have never once touched a gpu, and there is no version of that where all of them work on the first try. i am about to find out how many of my month of paper decisions were real and how many were me being clever in a text editor with nothing to argue back. 🤤
it landed. the replacement astral rog rtx 5090 is seated, powered, drawing, and running work, and every line on the priority queue at the top of this page lost the word "blocked" in one afternoon. i wrote on august 3 that soon is not a date and that i had been wrong about hardware timelines on this exact blog before. it turned out to be a week. house rule, so here is the unflattering half first: a month of designing runs instead of running them means i now have a folder of configs that have never once touched a gpu, and there is no version of that where all of them work on the first try. i am about to find out how many of my month of paper decisions were real and how many were me being clever in a text editor with nothing to argue back. 🤤
what a month without it does to your sense of scale
i ran this company on a 3060 ti and two K620s since the connector burned. i got more out of that than i expected, and i still mean everything i wrote about the constraint producing a smaller shape: glint-router trained in fifteen minutes on the 3060 ti and beat a 51M router on every field i measured, the autoresearch loop took byte perplexity from 12.53 to 6.84 on nothing but cpu cores, and freebuff got 140 tokens a second out of two cards i mostly keep around as a joke. all of that is real and none of it is walked back now that there is a big card in the box.
then i put the first real workload on the 5090 and remembered what the other end of the range feels like. roughly 900 tflops of fp4 is a number you read on a spec sheet and file under marketing until the thing that used to be an overnight decision comes back while you are still reading the config you submitted. the honest version is that the card did not make me smarter, it made my mistakes cheap. a month of K620 work teaches you to be very careful about what you launch because a bad launch costs you the evening. i have to consciously keep that habit now, because the machine no longer punishes me for skipping it, and every bad idea i have ever had came from a machine that let me skip the thinking.
compactbot, since people keep asking what it actually is
compactbot is our discord bot. it lives in the server, it moderates, it talks, it draws, it runs code in a container, it keeps notes, and at this point it is 10,920 lines in one file plus a couple dozen modules around it, which makes it comfortably the largest thing glint research ships that is not a model. it started life as an image description bot. it is somewhat past that now.
the moderation half is the oldest part and the part i am proudest of, because it is the part that has to be right. text goes through tiers. leet and homoglyph deobfuscation first, so the spelling tricks collapse back into the word they were hiding. then a profanity list with a block tier and an allow tier. then a zero-width and soft-hyphen steganography detector, which exists because somebody sent an innocent looking paragraph about quantum computing research with a hidden message encoded in invisible unicode carriers inside it, and i decided that once was enough. then a fine-tuned multi-label classifier trained on our own hand-written rules dataset, twelve labels, including one for insulting the server or the staff or the bot. it runs on cpu in about three minutes to train and it gets evaluated by five separate probes before it goes live, because a false positive on a real member is worse than a miss.
images go through their own tier. ocr reads the text out of attachments, tenor and giphy embeds, stickers, and custom emoji pulled straight off the cdn, and an nsfw classifier scores the frames. every unique emoji is downloaded and scanned once and cached by id, and only the clean and the nsfw verdicts get cached, because a 128 pixel ocr misread should not be able to ban an emoji forever. there is a /guard command that lets a mod label a message or an image with what it should have been flagged as and then retrain the detector on the spot, which is the closest thing i have to teaching it in public.
the conversational half is a tool loop. the bot used to decide what to do with regexes, which is how it once ran a chat search on the word "this" and rendered "you eating dung" as a photoreal ape, because no code path anywhere rewrote the request into a description. now the model sees the tools, calls the ones it wants, reads the results, and calls more, up to eight rounds. it can search the web, search our own scraped chat history, run code, and write its own image prompts as tool arguments. there are slash commands over the top of it for the common things, /ask, /explain, /summarize, /translate, /rewrite, /correct, and a /fight mode that starts a civil reasoned debate with exactly one member, which is funnier than it sounds and has been used seriously maybe twice.
it also speaks unprompted, rarely, on purpose. three independent gates guard the chime: a six hour per-channel cooldown, a minimum of thirty messages since the last one, and a five percent roll. that lands a busy channel at about three chimes a day with a hard ceiling of four, and the rate is asserted by a test that simulates a day of traffic, so it gets retuned there and not by vibes. one line, under 240 characters, something genuinely useful with a dry twist, or the literal token SKIP and it says nothing. the whole design goal was for it to never become a predictable every-nth-message tic.
generation is all local. /imagine, /animate, /make3d, /sfx, and the same four from ordinary chat when the bot is addressed, so "draw me a X" and "make this 3d" get intercepted before the caption editor sees them. measured, warm, bf16, on the 16-core cpu, because the cpu beats the K620s outright at this: a 512px image in 3.9 seconds, a twelve frame clip in about 26, a triposr mesh in about 27, five seconds of audio in about 94. the same work on one K620 was 9 and 104 and 36. there is a lesson buried in there about dtype, which is that a 2048² matmul on this cpu is 17.0ms in fp32, 4.3ms in bf16, and 23,081ms in float8, because there is no cpu fp8 kernel and it emulates. three orders of magnitude. i measured it because i did not believe it.
the sandbox is the part that still slightly worries me, in the healthy way. the bot runs a real coding agent cli inside a throwaway rootless podman container and streams progress into one discord message it keeps editing. every bash call goes through a guard: a hard denylist for the obvious catastrophes runs first, in-container and model-independent, and everything else goes to a verdict model running on the host. the groq key never enters the container, so a job that runs env finds an endpoint url and no credential, and somebody in the server probed exactly that on purpose. one cpu, ten gigs, two K620s, a wall clock, a loop guard, a stall guard, and --cap-drop ALL. /swarm is the admin-only version with no wall clock that delegates to child agents, and /stop kills everything and keeps the workspace so you can still download what it built.
then there are the parts nobody asked for that i built anyway. /catchup reads a channel's history and writes an obsidian vault out of it, one dated bullet per fact. /curate is the second pass that rewrites those logs into actual prose and wires the graph, and the graph edges come from code rather than from the model, because when i asked the model to write its own links it produced an essay naming four other members and linked none of them. /insights answers a question using only the vault. /personality train fits a 36,864 parameter lora on distilgpt2, 164 kilobytes on disk per member, that rewrites the bot's finished answer into that member's voice, plus a roughly 4,100 parameter net trained from scratch on that member's own reaction history so the bot reacts the way they react. that reaction model only ever fires when a personality is switched on, and an empty prediction is a real decision rather than a fallback, because falling back to the generic classifier would be the old behavior wearing somebody's name.
the thing i like most about compactbot is that almost every design decision in it exists because the obvious version failed in the server first, in front of people, and got measured. the first personality adapter was trained on pairs that could only teach it to undo a normalizer, so it produced eight rewrites that were all the same edit and i concluded a small model cannot sound like anyone. that conclusion was wrong and it took rebuilding the pairs to prove it. the fallback that stapled a member's most-used emoji onto rejected rewrites is how 💩 reached a channel. those are in the notes now, in full, so nobody re-derives the wrong lesson later, including me.
more models, which is the actual point of the card
the bot is the fun answer to "what have you been doing". the real answer is that i am making more models, and that is what the 5090 is for. prism and shard are both designed and both were waiting on this exact box. the p0 line on the queue is redacted for now and it is a dedicated line of models with a narrower job than anything glint has shipped, and i will say what it is when there is a checkpoint to point at rather than a plan to describe. the autoresearch loop gets its gpu tier, so the sixteen finders stay on cheap cpu where they belong and the merged candidate that survives a round finally gets trained at a size where the result means something outside the loop's own leaderboard. i genuinely do not know how many of those cpu-found levers hold when the budget stops being a joke, and that is the single experiment i am most curious about this month.
everything that comes out of this goes up raw under CompactAI/Glint-Testing, automatically, the way i said it would on august 6. weights, a step count, and whatever the model said that day. no inference script, no cleanup, no human gate deciding which failures you get to see. the checkpoint rate is about to go from almost nothing to a lot, and i would rather that flood land in public as it happens.
the paranoia i am keeping
a new card and a new cable does not make me trust the connector again. no unattended overnight runs for a while, temperatures watched, and every long job checkpointed often enough that losing the box costs me an hour instead of a week. i have already been through one burned connector on this blog and i would rather write a boring post about a run that finished than a second post about a part that melted.
why this belongs on the glint blog
because the build log is the content and this is the week the log stops being about a hole. a month of posts here were about what you can do without the thing you need, and the honest summary of that month is that the constraint was genuinely productive and also that i am very glad it is over. the card is in, compactbot keeps running the server while i am busy, and the queue at the top of this page is four unblocked lines with one of them blacked out. next post from me should have a loss curve in it.
/lane glint research, 2026, astral rog rtx 5090 seated and running, ~900 tflops fp4, four queue lines unblocked in one afternoon, one of them redacted, compactbot at 10,920 lines and counting, five probes before any classifier ships, generation on cpu because it beats the K620s, groq key never in the container, autoresearch finally gets a gpu tier, raw checkpoints to CompactAI/Glint-Testing with no human gate, new cable and no unattended overnight runs
✔you made it to the end
keep reading
~10 min · ~2,137 words
in one line: it landed. the replacement astral rog rtx 5090 is seated, powered, drawing, and running work, and every line on the priority queue at the top of this page lost the word "blocked" in one afternoon. …
my time at a small ml org i am not going to name. the owner was younger than i would like, i built him a webhook because he cannot use discord, and i left when he asked for glint research org access to publish an overfit model a gpt agent made.
update, same day, 1:47pm: idfk what i did but i think i just accidentally caused the downfall of a company 😭 why. the short version. costikooo said the honest thing first, that it was a lot of work and he did not see himself fit for the position, and he suggested the owner partner with benchlabs or let a community help instead. the reply was that we no longer need to help with training, thank you all, please stay in the preview program. i then asked for the checkpoint repo i built and own to be deleted off huggingface and off any machine holding it. he asked to keep the latest checkpoint. no. he asked to transfer it to me. no. i do not need those weights, i just do not want them running under that name. then he asked us to delete every copy of his model lines from our side, said "this is done. goodbye," announced he was deleting the previews org, and left. the whole thing took nine minutes. costikooo offered him a way to keep going and got told, again, that discord is not allowed at his house, which is the exact problem the webhook i built existed to solve.
and to be clear about that headline: two people quitting on the same afternoon does not add up to one person causing a downfall, and an org that leans on two volunteers is already fragile before either of them walks. i asked for my own work back and i would ask again. but i am also not going to pretend it felt good to watch someone delete their org nine minutes into a conversation i started. that is why the joke is in there. it is the only way i have of saying it.
before anything else, and i would like you to actually read this: i do not want wars or arguments in the server about this post. i have said the thing the way i experienced it and not one word more, and every word is my opinion. it is an opinion blog. if your opinion differs, that is allowed and it is not an invitation to a fight, and if that is a problem for you then you should probably find a blog that matches yours, because this one is heavily opinionated and you have been warned. i had a lot of reasons to leave this story unposted, i went back and forth on it down to fifty fifty, and the only reason it is here is that this page has always been the honest one. i am not naming the org, not naming the owner, not linking his profile, not giving his exact age, and not posting the raw chat logs, even though i have them and they are the reason this post exists. he is a kid who is building things, which is more than most people do at any age. i am writing about my own decision to join a project i knew nothing about, and about what that was like from the inside. if you go looking for him to be weird at him, you have completely missed the point and i would rather you did not read this at all.
i got asked to join a small ml org a while back. i had never heard of them, which should have been the whole answer, but i am curious by default and i went and read the owner's profile before saying anything. the machine learning interests section led with his age, and then listed a custom architecture, a one dollar gpu budget, and a download count from the first 24 hours. the age was younger than i would like to be taking direction from, by a wide margin, and i am leaving it at that. i read the whole thing four times. then i joined anyway, because a one dollar training budget is the exact kind of constraint this blog is built around and i wanted to see what was actually there. so i spent three days working under someone young enough that i kept having to remind myself of it. it was a lot. some of it was fun. i am still not entirely sure how to file the rest of it.
the parts that were genuinely fine
costikooo is in there, which was a real plus, and a chunk of the good hours were just talking shop with someone who knows what they are talking about. i also got to build two things i liked building. the first is a webhook bridge that pulls messages off the owner's own site into discord and pushes discord messages back the other way, because he does not use discord and everyone else in the project does. that is a genuinely reasonable problem to have and it was a fun afternoon of plumbing. the second is a pipeline tag for their org wired into my own training codebase so their runs could slot into tooling i already had, which cost me very little and unblocked him a lot. i do not regret either one. if the whole thing had stayed at that level i would still be there.
the model
the glint-2 thing on its own i was mostly fine with, and i want the record to say so before the pile starts. the owner decided there should be an instruct version of glint-2, our model, and he set a gpt agent loose to make one on a 5060 ti. the agent ran into the obvious wall almost immediately, since glint-2 ships weights and inference scripts and the sft training code is not published yet, so there is no supported path to fine-tune it from the repo as it stands. i said so plainly. the response was to have the agent asked, more or less, to say that it was possible anyway. when i pushed and asked for an assessment grounded in what actually exists in the repo, the honest answer came back and matched mine.
it ran regardless. five examples, three epochs. the agent's own summary said the run changed behavior and hurt generalization and needed more varied data with fewer update steps, which is a correct and adult read of the situation, and it got waved off. then i tested the result. asked what a computer is, it produced a paragraph of broken word fragments. asked to print hello world in python, the same fragments. asked what two plus two is, the same fragments again, near enough word for word. that is collapse. the model had landed on one output and it gave you that output no matter what you asked it. i pasted all three answers in. the reply was to give chatgpt a few minutes.
and here is the part i want to be honest about. the glint-2 run on its own was survivable. the thing that actually did me was the pile under it. the age questions, a light flickering and becoming an accusation, an agent being taken as ground truth no matter what it produced. not one of those is a big enough deal to leave an org over, and every one of them happening inside the same three days is. the glint-2 run was the last straw, not the whole pile.
then came the actual ask: give me glint research org access, i will leave after. followed by a transfer, followed by write access to his side and instructions to move it quickly, followed by make it private. the answer was no every single time. i will publish a model that scores badly, i have done it repeatedly on this page, and i will publish one that fails in public with a table showing exactly how. what i will not do is put our org name on a checkpoint i did not train, did not evaluate, and can see is broken, on the word of an agent that told him the thing he wanted to hear. that one is not a close call and it is the one that closed it.
the trade offer
at one point he offered to trade me his macbook for my machine. the machine in question is the 5090 build with 128 gigs of ddr5 that this entire company runs on, the one i have written multiple posts about, including one about it catching fire. i said no. i want to be fair here and say i do not think this was a scam attempt so much as a kid wanting a better gpu, which is the most understandable thing in this entire post. but it does tell you which end of the value gap the offer was coming from.
the age questions
i was asked how old i am more than once. when i declined, it turned into a ladder, under 30, under 40, under 60, under 80, each one its own message. i answered every one of them with the same sentence, which is that i am not going to answer that. i keep my age private on purpose and i am keeping it private here too, so if you were hoping this post would settle that, no. the part that stuck with me is that i never once asked his. it was in his profile and i read it there and that was the end of my curiosity about it.
the hallucination thing
the pattern underneath all of this is that most of what he knows appears to come from a model, and he does not accept that models make things up. i said it directly, models hallucinate, verify things before acting on them, and got pushback. and then, in the middle of a normal conversation, his lights flickered for a second and he accused me of ddosing him, twice, and did not seem to be joking. i had no reason to and obviously did not. but that is the same reasoning error in a different costume, which is a confident causal story assembled instantly from nothing and then held onto. if a model says the fine-tune is possible, it is possible. if the lights flicker, it is an attack. codex says the model is better than base glint-2, so the model is better than base glint-2, and the gibberish on my screen does not enter into it.
the small ones
the ddos thing gets covered later, so here are the ones that did not earn their own section. day one opened with an age gate on the server and his solution was to tell everyone to type the word blank over it, then to clarify that he meant blank like a number. costikoooo said he was 99. i said i was 2. that was the entire enforcement mechanism and it is the most honest hour of the whole three days.
his own age moved around a lot, depending on the hour and the audience. i stopped tracking once i realized the number was a policy he adjusted based on who was asking. i did not ask. i have never asked. his profile said it and that was the end of my curiosity about it.
he told me he invented a training method called descendant gradient matching and when i asked what it was he said it does not exist, he made it. later he defined it for me, and threw in ascendant gradient matching as the thing that goes up, and then admitted he was not sure about that one either. a method he invented that he is not sure about is the whole project in one line.
he claimed twenty thousand tokens a second on a 5060 ti while my machine sat at 182. mine was real and climbing, 2.8k, then 18.1k, and his stayed at the claim. he said he beat a 1.1B model with an 8B one, which is bragging that you beat a child at arm wrestling with an adult, and then he said he needed to beat a 100B model with his 450M one. the goalpost was always the fun part of the conversation.
the craigslist listings. he was shopping for a second gpu and pasting local listings into the chat, and at some point he asked what state i live in so he could find me a second pc. then the bidding started, the numbers climbing three orders of magnitude per no, until the last offer was denominated in tokens trained. i answered that one with a number that meant nothing and he let it drop. i have never been bid on in token counts before and i would like to never be again.
and the money stuff. he could not use a credit card at the moment and could not give money, sorry, and did not want to be threatened in any way, which was a leap i did not have a map for. for the record, nobody expected a single valid dime from him, nobody was threatening him, and i have no idea where the threat part came from. i built him a runpod team and he asked me to transfer ownership of it to him within the same hour.
the spam. fourteen identical messages arriving at once, a wall of question marks, a wall of pings at a huggingface team account, and when i told him to stop spamming, four times in a row, he explained that a message cap had been hit and it all came out at once. the cap being a thing that exists at all was the part i had not been warned about.
he asked me, mid-week, to put biblical data into the model, and i want to be careful with this one because it is the only time i genuinely had to stop and stare at the chat for a while. it started with a question out of nowhere, whether i was a christian. i answered it, and that answer stays between me and the chat. this blog is happy to tell you what i think about most things, but my own faith is not one of them, because an opinionated disclaimer does not stop a religion war in a discord and i would rather skip that particular season. then came the pitch, a request to add biblical data to the model. i replied with a noise that does not appear on the approved list of words. he said the research platform needed it for faith and research. i asked why. he said he might run the ember models on his faith platform, a thing he called selah. i pointed out, as gently and flatly as i could, that it was not offensive, just weird. he said he would add data of christianity later, as though that closed the argument, and the conversation moved on with nobody noticing i had never agreed.
and here is the part that actually bothers me, so let me say it plainly. i do not care what anyone believes, and i will defend that sentence on this blog to the end. but faith does not go in the training mix. faith in an org is the same thing as a stance and the whole point of our data is that it has no stance, it is rows and rows of reality with the opinions removed, so a model can learn how the world is rather than what one person wants the world to be. the moment you put the bible in the pipeline because a platform called selah needs it for faith and research, your model is not a model anymore, it is a sermon with a tokenizer, and the person who asks the question does not get to keep his data private while his models carry his belief around.
and the second thing, because the first one was serious and this one is just true: why would anyone do that. you spent a week telling me the model was going to beat 100B models and the plan to get there was the bible in the data mix. i am not going to explain to you why a holy book is not going to fix your loss curve, because the fact that you asked is the entire punchline, and i have been sitting with it since august. costikoooo explained it better than anyone could, in six words, all of them about including the bible as training data. yes. that is what he meant. and it is still weird, no matter which way you hold it.
and the exit timing. the great disappearing act. he announced he was going for ten to thirty minutes or tomorrow, he was not sure which, and then he was back within the same minute with a casual never mind. the message had not finished rendering. i stopped trusting the abbreviation for got to go that evening.
and the cli flex, the best one. in the middle of the worst stretch he announced that this conversation was running on a cli on his computer, unescaped bracket and all. the whole conversation. i did not know what to do with that information either, so i said hi and waited.
the goodbye
the only real goodbye of the whole thing came with a tilde. he was off, an update tomorrow and a bye, and then the real sign off, a short line ending in a tilde, a wave typed at the end of the word. i checked the logs to be sure and the wave was really there. i have decided it is a violation of some sort. i do not know the name of the statute but the shape of it is clear: you do not spend two days asking a person for the keys to their org and then sign off like you are leaving a group chat you liked.
and at the very end there was no goodbye at all. the log just stops. he said a new glint 2 existed and asked me to try it, and i answered with two letters and a backslash, because the keyboard does what the keyboard does, and that is the last line. i never saw the new glint 2. i do not think it existed. i think the log ran out before the idea did.
what i actually think, since the point is not to dunk on a kid
building an org and a site and training runs at that age is real and i am not going to be sarcastic about it. at his age i was not doing anything close. the thing that would fix most of what went wrong is small and learnable, which is to open the file yourself. run the eval, read the output, and let it change your mind when it disagrees with what the model told you. that is the entire skill. it is also the exact rule this blog has broken and paid for before, which is a large part of why i am writing this without feeling clever about it. if he learns that in the next year he will be genuinely dangerous in the good way, and i would honestly like to see it.
the part i own is that i joined a project i had never heard of on the strength of one interesting sentence in a profile, and then spent three days of attention on it that glint needed more. so, no more small ai orgs for now. that is a rule about me and where my attention goes, since i clearly cannot be trusted to say no to an interesting constraint. anyone else's org, i have no verdict on.
why this belongs on the glint blog
because the honest posts are the whole point of this page and this one costs me something to write. i have published a dataset failure, a teammate leaving because of me, a burned gpu, and a benchmark we lose on. this is the same category. i made a bad call about where to spend my time, i held the line on the one thing that actually mattered, which is our org name, and i walked. the models on Glint-Research are still models i trained and evaluated and can defend line by line. that is the only thing i was ever protecting here.
and there is a part of this that belongs to you too. the whole story runs on one mistake, which is taking whatever a model says as fact. a model telling you something confidently is giving you a sentence, not evidence. check the claim, open the output, read all three answers, and let the file overrule the summary. it costs five minutes and it is how you avoid being the person this post is about. that is the real reason i am putting it up, because this skill is on this page in every post and this time you get to see what happens to someone who never learned it.
/lane glint research, 2026, one small ml org i will not name, one webhook bridge, one pipeline tag, one overfit instruct model i would not host, org access requested and refused, one macbook offered for a 5090, four age questions unanswered, both volunteers out inside nine minutes, previews org deleted the same afternoon, no more small ai orgs for now
✔you made it to the end
keep reading
~16 min · ~3,601 words
in one line: i got asked to join a small ml org a while back. i had never heard of them, which should have been the whole answer, but i am curious by default and i went and read the owner's profile before sayin…
every glint checkpoint from here on goes up raw, automatically, under the old company name. no inference script, no cleanup, just weights and a step count and whatever the model said that day.
new rule starting now. every testing checkpoint glint produces gets pushed to huggingface.co/CompactAI/Glint-Testing by a script, with no human deciding first whether it is good enough to show you. what lands there is the weights, a step count, and a sample of what the model generated at that step. that is the entire contents. there is no inference script in the repo and there is not going to be one. if you want to run one of these, write your own loader, and i mean that with zero judgment, it is half the fun. house rule as always: this is going to look bad a lot of the time, and that is the whole reason it is going up.
new rule starting now. every testing checkpoint glint produces gets pushed to huggingface.co/CompactAI/Glint-Testing by a script, with no human deciding first whether it is good enough to show you. what lands there is the weights, a step count, and a sample of what the model generated at that step. that is the entire contents. there is no inference script in the repo and there is not going to be one. if you want to run one of these, write your own loader, and i mean that with zero judgment, it is half the fun. house rule as always: this is going to look bad a lot of the time, and that is the whole reason it is going up.
what actually lands in the repo
three things per checkpoint. the weights as they came off the training loop, unmodified, in whatever format that run was writing. the step count, which is in the filename and is the only piece of metadata i actually guarantee is correct. and a sample block from that exact step, generated at save time by the same script that does the upload, so the text you read next to a checkpoint is what that checkpoint produced and not something i regenerated later with better settings. no eval table, no readme full of claims, no cherry-picking a good sample out of five tries.
no inference script, and i want to be clear about why
partly because these are testing checkpoints and the architecture underneath them changes constantly. a loader i wrote for step 4000 of one experiment is wrong by the time a different experiment saves its step 4000, and maintaining a script per arch turns a folder dump into a part-time job i did not sign up for. partly because the moment i ship a loader, i own it, and people open issues against it, and i would rather spend that time training. and partly because figuring out how to load a bare checkpoint is a genuinely good exercise and most of the people who follow this blog can do it in twenty minutes.
so go make your own. get it wrong, get garbage out, post the garbage in the discord. i am not going to judge a single line of it. the released models on Glint-Research still come with everything you expect, single-file inference scripts included, and none of that changes. this is the other pile.
why CompactAI and not Glint-Research
because compactAI is what this company was called before glint was the name of anything, and the org is still sitting there, and a dead org name is the perfect place to keep the mess. Glint-Research stays what it has always been, models that are finished, cards with real numbers on them, MIT, a script that runs. CompactAI/Glint-Testing is the room where the floor is covered in parts. anyone landing on the main org should not have to sort a released model out of forty half-trained ones, and anyone who wants the forty knows exactly where to look now.
the automation, and the rule it looks like it breaks
the upload runs from the training script itself. checkpoint saves, samples generate, both go up, no approval step in the middle. and i can already hear it, because i wrote the post myself: this blog's oldest rule is that a human opens the file before it goes public, and we lost a teammate over that rule when a dataset went up unread. so here is the distinction i am actually standing on. that rule exists because we published a thing with a claim attached, two million rows described as something they were not, and nobody checked the description against the contents. these uploads carry no claim at all. the description is "checkpoint at step N and here is what it said," which is verified by construction because the same script produces both.
the failure mode i am accepting is that a bad checkpoint goes public with my org name on it. i am fine with that, since bad checkpoints are most of research and hiding them is how a lab starts believing its own highlight reel. the failure mode i am not accepting is a claim nobody checked, so there will be no claims in that repo. if you ever see a number in there that reads like a benchmark result, that is a bug and i want to hear about it.
what you should expect from it
broken models. models that emit chuamliamce and walk off, which is a thing our 982k parameter line did for an entire release. runs that stop at step 3000 because i killed them. two checkpoints an hour on a training day and nothing for a week after. experiments that go nowhere and never get a post written about them, which honestly is most experiments. everything MIT, same as the rest of the org, so do whatever you want with any of it.
what i would love out of this, and cannot make happen by myself, is for someone to find something in a mid-run checkpoint that i missed. i look at these things through the eval i happened to write that week. a stranger with their own loader and their own prompts looks at them differently, and small models are weird enough that i think there is something in that pile nobody has noticed yet.
why this belongs on the glint blog
because the build log is the content and this is the build log with the lid off. i have spent a year publishing the finished half and writing posts about the messy half, and this cuts out the part where i decide which pieces of the messy half you get to see. the 5090 is on its way back, which means the checkpoint rate is about to go from almost nothing to a lot, and i would rather that flood land in public as it happens than sit on my machine until i have a story to wrap around it. weights, a step count, and whatever the model said. no script, no polish, no explanation. write your own loader and go dig.
/lane glint research, 2026, CompactAI/Glint-Testing, raw checkpoints pushed by script with no human gate, step count and samples and nothing else, zero inference scripts forever, MIT, write your own loader and i will not judge it
✔you made it to the end
keep reading
~5 min · ~1,095 words
in one line: new rule starting now. every testing checkpoint glint produces gets pushed to huggingface.co/CompactAI/Glint-Testing by a script, with no human deciding first whether it is good enough to show you.…
on july 26 i wrote that asus said yes to the warranty claim and attached no date to it. that is still where things stand, except the replacement astral rog rtx 5090 is close now, close enough that i have stopped treating it as a someday and started staging everything for the day it lands. i have been running this company on a 3060 ti and two K620s since the connector burned, and i have gotten more out of that setup than i expected to, but every single thing i actually want to build has been sitting behind the same missing part for a month. this is the post where i say what happens when it stops being missing. house rule as always, so here is the unflattering half up front: soon is not a date. i do not have a tracking number in my hand while i type this, and i have been wrong about hardware timelines before on this exact blog.
on july 26 i wrote that asus said yes to the warranty claim and attached no date to it. that is still where things stand, except the replacement astral rog rtx 5090 is close now, close enough that i have stopped treating it as a someday and started staging everything for the day it lands. i have been running this company on a 3060 ti and two K620s since the connector burned, and i have gotten more out of that setup than i expected to, but every single thing i actually want to build has been sitting behind the same missing part for a month. this is the post where i say what happens when it stops being missing. house rule as always, so here is the unflattering half up front: soon is not a date. i do not have a tracking number in my hand while i type this, and i have been wrong about hardware timelines before on this exact blog.
the queue is one part deep
look at the priority queue at the top of this page. releasing the next glint just came off it, which is the one thing on that list that never needed a card it did not have. what is left is four lines and every single one of them says "blocked, no gpu" and has said that for weeks. aureole's training run, the youtube cloner, the compressed hub, the self learning AI. four hats, one head. the same missing part is underneath all four, and all four unblock on the same delivery. so the honest state of the roadmap for the last month is that i have been designing full time and shipping part time, and the shipping half has now run out of things it can do alone.
what a month with no card actually produced
more than i would have guessed, and i want the record to be accurate about that before i go acting like the month was a hole. the next glint got finished and out the door on borrowed hardware. glint-router trained in fifteen minutes on the 3060 ti and beat a 51M router on every field i measured. the autoresearch loop took byte perplexity from 12.53 to 6.84 on nothing but cpu cores. freebuff built its own nvlink between two K620s and got 140 tokens a second out of cards i keep around mostly as a joke. every one of those is real work and every one of them exists because the constraint forced a smaller shape.
one correction to that list while i am at it: GCI-Bench is not ours to count anymore. ender built it, and when he left it went with him. i am not going to keep listing it as a glint win on a page whose whole rule is that the numbers are honest.
here is what the month did not produce: a single training run at the size that matters. the last real one ended at 300 billion tokens because the plug melted underneath it. plenty of people train good models on a 3060 ti and i have gotten real results out of mine, it just cannot carry the token budget my runs need in any amount of time i am willing to wait. so i wrote the runs down. i have a folder of training configs, data mixes, and eval plans that have never touched a gpu, and the folder stops being a folder the day a working card goes in the machine.
the research half, which is the part i am actually excited about
releases are the visible half. the reason i am counting days on a delivery is the other half. the autoresearch loop has been living on a five minute single-cpu-core budget per experiment, which is a constraint i chose because it was the only one i had, and it still found real levers. the moment there is a card, the sixteen finders stay on cpu where they are cheap and the merged candidate gets a gpu tier, so the thing that survives a round finally gets trained at a size where the result means something outside of the loop's own leaderboard. i want to know how many of those levers hold when the budget stops being a joke, and i genuinely do not know the answer.
past that: ternary training experiments for aureole that have only ever existed as arithmetic on paper, prism and shard finally getting checkpoints instead of design docs, and more tokenizer work on the small end where breaking things is affordable. that is the honest ranking of my enthusiasm. the releases are what you will see, the experiments are what i have been staring at the ceiling about.
the tuning problem nobody warns you about
i wrote in july that every script i own is tuned to exact transfer rates and core clocks that only exist on the card that burned, and that re-tuning for a different gpu eats months. that argument cuts both ways. a replacement 5090 is the same model of card, so most of those numbers should land where they used to land, but "should" is doing real work in that sentence. a different unit has its own silicon lottery, its own thermal behavior, its own idea of what a sustained clock looks like at hour nine of a run. so day one goes to re-measuring the constants i have been treating as facts since long before the connector burned, because a number that was true on the dead card is a hypothesis on the new one until i watch it hold.
the paranoia i am keeping
i woke up to a burned 12vhpwr connector in a room i was asleep in. i am not going to write a calm paragraph about how that is behind me. new cable, seated and re-seated and checked with my hands on it, connector temperature watched at the plug rather than trusted from the gpu sensor, and no unattended overnight run for at least the first week no matter how much that costs me in wall-clock time. i lost a card and 300 billion tokens of momentum to that connector and i still got the better half of the trade, since the thing that shipped early turned out to be the best model glint has made. i would rather not run that experiment twice.
two people and one card
the other constraint stays here after the box is unpacked. it is me and armand0e now, and it has been since july 31. one gpu was the bottleneck for the last month and it stops being the bottleneck the day the card arrives, and then the bottleneck becomes the two of us and how many runs two people can actually babysit, read, and write up honestly. i said i was not taking on new people for a while and i meant it, so the answer for now is fewer things at once, finished properly, with a human opening every file before it goes public. that rule has cost us a teammate before when we broke it, and it is still the rule.
why this belongs on the glint blog
because the build log is the content, same as always, and the last month of this log has been a month of working around a hole. i have written a burned connector post, a warranty-approved post, and several posts whose entire subject was what you can do without the thing you need. this one is the other end of that arc. the card comes back, the folder of unrun configs starts becoming checkpoints, and the queue at the top of this page starts losing the word "blocked" one line at a time. i am going to be very annoying about this on the discord when the first real run starts. i have been waiting a month to be annoying about a training run.
/lane glint research, 2026, replacement 5090 inbound, four blocked queue items about to thaw, next glint shipped and off the queue, GCI-Bench went with ender, prism and shard and aureole all waiting on the same box, one folder of runs that have never touched a gpu, autoresearch loop finally getting a gpu tier, new cable and no unattended overnight runs, two people and one card
✔you made it to the end
keep reading
~6 min · ~1,413 words
in one line: on july 26 i wrote that asus said yes to the warranty claim and attached no date to it. that is still where things stand, except the replacement astral rog rtx 5090 is close now, close enough that …
it is almost 10pm and i am writing this because this blog has always been honest about the bad news, and i am not going to make tonight the exception. ender is gone. enderchefcoder1, who has been here since we were still called compactAI, back before glint was even the name of anything, asked to step down tonight and i said yes without arguing. i have written a lot of posts on this page about hardware dying and models failing to hit a number i wanted. this is the first one where the thing that broke was not a gpu or a training run.
it is almost 10pm and i am writing this because this blog has always been honest about the bad news, and i am not going to make tonight the exception. ender is gone. enderchefcoder1, who has been here since we were still called compactAI, back before glint was even the name of anything, asked to step down tonight and i said yes without arguing. i have written a lot of posts on this page about hardware dying and models failing to hit a number i wanted. this is the first one where the thing that broke was not a gpu or a training run.
the length of it
april 19, 2026, 7:23pm, to july 31, 2026, 9:49pm. that is 103 days, 2 hours, and 26 minutes, and i checked that math more than once tonight, the same way i check every other number before it goes on this page. ender built CAI-Grid back when we were compactAI and the name glint did not exist yet, before there was a lineup or a blog or a discord server with more than a handful of people in it. that grid work is a large part of the reason we had infrastructure worth building models on top of at all. he was in the middle of improving glint 2.3 right up until this evening, and as of tonight i genuinely do not know the current state of that work, because it was his to know and now he is not here to tell me.
what actually happened
i communicate in memes and gifs most of the time. it is closer to my default language than plain sentences are, and it has never caused a real problem before, or at least not one i noticed. tonight i used one at the wrong moment, about something ender was taking seriously, and i did not stop to clarify what i meant before moving on like it was nothing. his reply was "you know this is serious, right? grow up." he was right to send it. i am not going to write around that with a softer version of what happened, because the softer version would be a lie and this blog does not do that. i made a joke where a joke did not belong, i did not catch it in time, and that is the actual cause here. not a slow disagreement building for weeks, not some deeper values split i can point at instead of myself. one message, at the wrong time, that i did not think about before sending. this one is on me.
at 9:49pm he sent this: "Thanks for the time. I think I need to step away from Glint Research. Please remove my Admin. Thanks. @Glint Research." i read it twice before i did anything. then i removed his admin. i did not ask him to reconsider and i did not push back, even though every part of me wanted to type something to try to fix it. it was what he asked for, and after what i had done, i did not think i had the right to make him sit through an argument about it first.
where he is going
he told me he is joining both FromZero and Axiomic Labs. those are our second and third biggest competitors, not some unrelated company in a different corner of the field. he built CAI-Grid and was deep inside glint 2.3, which means he is not walking into either of those places as a stranger to what we do, he is walking in already knowing exactly how we do it. i do not think he left to hurt us. i think he left because of one bad night and a joke i should not have made, and the destination is just where he landed after. but i am not going to pretend the destination does not matter. if he wants to use what he knows to get ahead of us, he can, and there is a real chance that is exactly what happens over the next few months. i am sitting with that thought tonight along with everything else, and it is not a small one.
edit note: i wrote that as our second and third biggest competitors and had to sit back down when i actually laid it out. supralabs is our first. FromZero and axiomic labs are our second and third. that is not two of several rivals, that is the entire top three, and ender now knows how we work from the inside and has a seat at two of the three tables that matter most. i am leaving the original wording above as i wrote it tonight and adding this instead of quietly fixing it, because that is the rule i already put on this page for every other kind of mistake.
where that leaves us
it is me and armand0e now. two people, where a few hours ago it was three, and where a few months ago the plan was to be adding people, not losing them. i am not going to be taking on new people for a while. not as a punishment to anyone who might have applied, i just do not have it in me tonight to think about bringing someone new into a team that changed shape this recently. the roadmap does not disappear because of one bad night. prism and shard are still the plan. the XXSLM work is still the plan. glint 2.3 still exists somewhere on ender's machine or ours and it will get finished by someone, even if i do not yet know exactly how or when. just not tonight, and probably not for a few days after tonight either.
enderchefcoder1
april 19, 2026 – july 31, 2026
built CAI-Grid when there was nothing else here to build on. carried glint 2.3 further than i will ever fully know. 103 days, 2 hours, 26 minutes, and the work is still standing.
why this is going up anyway
because this blog does not only post the wins. i have put failed training runs and burned hardware and a benchmark we do not win on this page, and i said a long time ago that honest is supposed to be the whole point of this place. i am not going to make the one exception to that be the night it actually hurt to be honest. i do not have a clean ending for this post. i do not know what glint research looks like in a month with two people instead of three and two competitors that know more about how we work than they did yesterday. i am posting this anyway, because the alternative is quietly editing the team page and hoping nobody asks, and that is not who i want this company to be, tonight or any other night.
/lane glint research, 2026, enderchefcoder1, april 19 to july 31, 103 days 2 hours 26 minutes, CAI-Grid, glint 2.3, one joke i did not clarify in time, now at FromZero and Axiomic Labs, it is me and armand0e now
✔you made it to the end
keep reading
~5 min · ~1,200 words
in one line: it is almost 10pm and i am writing this because this blog has always been honest about the bad news, and i am not going to make tonight the exception. ender is gone. enderchefcoder1, who has been h…
i call us the XXSLM makers, extra extra small language model, because it is accurate and it makes me laugh. ender has decided we should scale up, and he means it seriously and he means it aggressively, up to 100 million parameters. i did not ask for this. i built a company around models you could email as an attachment and now my own cofounder is out here recruiting me into triple-digit millions. but here is the part that is entirely mine and not a joke at all: SLM makers should be scared. getting bigger has nothing to do with why. it is what we can already do at the size we have been ignoring for months. that is the actual subject of this post.
i call us the XXSLM makers, extra extra small language model, because it is accurate and it makes me laugh. ender has decided we should scale up, and he means it seriously and he means it aggressively, up to 100 million parameters. i did not ask for this. i built a company around models you could email as an attachment and now my own cofounder is out here recruiting me into triple-digit millions. but here is the part that is entirely mine and not a joke at all: SLM makers should be scared. getting bigger has nothing to do with why. it is what we can already do at the size we have been ignoring for months. that is the actual subject of this post.
we have been way too glint-focused for a company named glint research
somewhere along the way "glint" stopped being the name of the company and became the only model line the company shipped. glint-0.3, glint-0.4, glint-1, glint-1.3, glint-2, all of it good work, all of it the same lane of the highway. ender pointed out that we are named after the research, not the model, and that we have been treating them as the same thing for months. so glint stays. it is not going anywhere and it is still the line i will keep pushing on tiny-and-clever. but it is no longer the only thing on the roadmap, and honestly it should not have been the only thing for a while now.
prism and shard
not new lines. old ones. prism at 100 million parameters, shard at 50 million were both part of the original lineup, sketched out back when we started the company, and then glint ate every ounce of attention we had and neither one ever got built. they are not glint with a bigger number on it, they are their own architecture decisions, and i am not going to pre-announce details before either one has a checkpoint worth showing. what i will say is the direction: less "how small can we go," more "how good can we get once we let ourselves use 50 to 100 million parameters instead of 1."
the number that actually justifies the joke
ender has been quietly doing a lot. there is a 1 million parameter model, not released yet, that beats a 50 million parameter model on real evaluation. i am not naming which one or posting numbers today because it is not ready and i am not doing a repeat of publishing something before it earns its own post. but that result is the whole reason "scaling up" is not pure sarcasm. if a model fifty times smaller can already win, the lineup we are about to build at 50 and 100 million gets to stack on top of a real result instead of covering for a weak one.
why none of it is training yet
the 5090 is still gone, but asus said yes. warranty claim approved, replacement coming, no date attached to that yet. i have two K620s and a 3060 ti in the machine right now and i am not training a real run on that setup while i wait. every script i own is tuned to exact transfer rates and core clocks that only exist on the card that burned, and re-tuning all of that for a different gpu setup eats months of quietly re-deriving numbers that used to just work, and i would rather spend those months on prism and shard's actual design than on re-fighting hardware i am about to replace anyway. so the roadmap is real, the direction is decided, and the training starts the day the new 5090 actually shows up.
why this belongs on the glint blog
because it is an honest status update dressed up as a joke, which is most of what this blog is anyway. we are not becoming a different company. glint keeps shipping at the size it always has. "glint research" is going to start meaning more than one model line again, prism and shard are both finally getting built and both waiting on a gpu that is not currently in my possession, and somewhere in ender's files sits a 1M parameter model already beating a 50M one. that last part is the receipt for the whole title. a model fifty times smaller already won once. scale that same team up to 50 and 100 million on purpose and tell me SLM makers should not be scared. i am mostly just tired and a little bit excited, in that order.
/lane glint research, 2026, XXSLM makers scaling up to 100M, prism and shard were always the plan and finally getting built, a 1M model already beating a 50M one and not released yet, everything waiting on the 5090 warranty ticket
✔you made it to the end
keep reading
~4 min · ~829 words
in one line: i call us the XXSLM makers, extra extra small language model, because it is accurate and it makes me laugh. ender has decided we should scale up, and he means it seriously and he means it aggressiv…
GCI-Bench is out, and huggingface wants it in their official eval harness. benchlabs, the benchmark company, has never gotten one tagged as an actual benchmark. we just did.
ender built a benchmark. it is called GCI-Bench, short for Glint Clarity Index, a name i take full credit for since ender let me name it, it is up on huggingface under Glint-Research/GCI_Bench, and it scores a model on two things at once, a priority score and a linkage score, both built off attention and gradient behavior rather than off whether the model's final text happens to match a reference string. ender ran it against five models the day it shipped and posted the raw json into our chat with zero warning. i am not going to pretend i am being cool about this. ender contacted huggingface support the same day, they asked what eval framework it used, he said custom, and now they want us to pr it directly into their official eval harness. there are something like 120 benchmarks that get that treatment. we are trying to be one of them.
ender built a benchmark. it is called GCI-Bench, short for Glint Clarity Index, a name i take full credit for since ender let me name it, it is up on huggingface under Glint-Research/GCI_Bench, and it scores a model on two things at once, a priority score and a linkage score, both built off attention and gradient behavior rather than off whether the model's final text happens to match a reference string. ender ran it against five models the day it shipped and posted the raw json into our chat with zero warning. i am not going to pretend i am being cool about this. ender contacted huggingface support the same day, they asked what eval framework it used, he said custom, and now they want us to pr it directly into their official eval harness. there are something like 120 benchmarks that get that treatment. we are trying to be one of them.
what it actually measures
two scores per model. priority score and linkage score, averaged into a single gci score. both come out of attention and gradient behavior during inference, not out of string matching against a reference answer, which is the part ender was most insistent on getting right before it shipped. a model can produce a correct-looking sentence for the wrong internal reasons and every text-matching benchmark on the hub will wave it through. gci is built to notice when that happens instead.
the first results, unedited
ender ran it cold against five models the day it went up, two of ours and three from other labs. 95 to 100 questions each, one model skipped five of them.
model
params
priority
linkage
gci
1. AxiomicLabs/GPT-X2-125M
125,081,664
51.700
56.850
54.275
2. SupraLabs/Supra-50M-Instruct
51,786,240
53.045
54.019
53.532
3. Glint-Research/Glint-2
1,512,385
50.501
50.246
50.374
4. Glint-Research/Glint-1.3
985,344
47.223
52.426
49.824
5. AxiomicLabs/GPT-S-5M
5,158,464
50.535
43.408
46.971
ranking top to bottom on gci: GPT-X2-125M, Supra-50M-Instruct, Glint-2, Glint-1.3, GPT-S-5M. the honest read is that params still buy you something here, GPT-X2 at 125M leads, and our two entries sit in the middle rather than at the top. i am putting the table up anyway because the whole point of a benchmark is that it does not care whose name is on the model, and if i only publish the runs where we win i have built a marketing page, not a benchmark. ender's first reaction to his own numbers was that gci is very strict, and looking at how tightly the five scores cluster in the high 40s and low 50s, i believe him.
why the huggingface thing matters more than the scores do
there are roughly 120 datasets on huggingface marked as an actual benchmark instead of just a dataset, gsm8k among them, and they get a different visual treatment on the site because of it. when ender reached out, hf support asked what eval framework GCI-Bench runs on, and the honest answer was that ender wrote the whole scoring pipeline himself, so now the ask is to pr it into their official eval harness directly, in whatever internal language they built for defining evals there. ender described it as "their weird ah language" and is currently sitting with the docs open trying to map our scoring logic onto it. i told him to go for it. getting marked as a real benchmark means every model page on the hub that gets evaluated against it can show a gci score next to the standard ones, which is the actual distribution channel we do not otherwise have, and that is worth a lot more to us than the flag alone would be.
why this belongs on the glint blog
because we have spent months publishing our own models against everyone else's benchmarks, and this is the first time it runs the other way. benchlabs, the company whose entire brand is benchmarking, has benchmarks up on the hub and none of them carry the official benchmark tag. we shipped one, ran it against five models including two of our own, published the results where we did not win, and got a message from huggingface the same day asking to integrate it. small team, one afternoon, one dataset upload. i will post again once the pr into their eval harness actually lands, weird language and all.
/lane glint research, 2026, GCI-Bench, priority and linkage scoring off attention and gradients not string matching, five models benchmarked day one, huggingface asked us to pr into their official eval harness, we do not win our own benchmark and the table says so
✔you made it to the end
keep reading
~3 min · ~760 words
in one line: ender built a benchmark. it is called GCI-Bench, short for Glint Clarity Index, a name i take full credit for since ender let me name it, it is up on huggingface under Glint-Research/GCI_Bench, and…
freebuff misreads my prompts constantly. i will write out exactly what i want, in plain words, no ambiguity anywhere i can find, and it will do a third of it and stop. it quits early more than any tool i have used this year. i have come close to uninstalling it twice this week alone. and then last night it took two K620s, cards so old and so weak that i keep them around mostly as a joke, and had them running qwen3.5 4b at 140 tokens a second, by building its own nvlink bridge between them and writing a custom kernel to make the bridge worth having. i did not ask for either of those things by name. i asked it to make the two cards fast, and it went and built the link between them itself to get there. when freebuff lands in the right state it is the best tool on my machine. getting it there is the whole fight.
freebuff misreads my prompts constantly. i will write out exactly what i want, in plain words, no ambiguity anywhere i can find, and it will do a third of it and stop. it quits early more than any tool i have used this year. i have come close to uninstalling it twice this week alone. and then last night it took two K620s, cards so old and so weak that i keep them around mostly as a joke, and had them running qwen3.5 4b at 140 tokens a second, by building its own nvlink bridge between them and writing a custom kernel to make the bridge worth having. i did not ask for either of those things by name. i asked it to make the two cards fast, and it went and built the link between them itself to get there. when freebuff lands in the right state it is the best tool on my machine. getting it there is the whole fight.
why it stops early
i do not fully know yet, and i am not going to pretend i have it diagnosed. my best guess after a week of fighting it is that its own default posture is to hand everything off, and when there is nothing left it feels safe handing off to, it just stops rather than doing the last mile itself. that is a guess. what i know for certain is the shape of the failure: a task with five obvious steps gets three of them, and the response reads like it thinks it is done. no error, no apology, just silence where the rest of the work should be. you learn to check every single time, because it will not tell you it quit.
the part that is actually the whole idea
freebuff is built around never doing anything itself if it can spawn something to do it instead. that is not a subtle bias, it is close to the entire personality of the tool. and the way it spawns is not the fan-out-and-drown-in-context pattern i am used to from every other agent framework. it launches what it calls "bashers", little agents that each run one command and hand back a summary instead of raw output, so thirty of them can run at once and my context does not fill up with thirty command logs. it has a small bench of these beyond just the generic basher: a code reviewer, a thinker, a few others i have not fully mapped yet. thirty bashers running in parallel, each one condensed down to a paragraph, is a genuinely different shape from thirty subagents each trying to do a hundred things and reporting back a wall of text. i think that shape is why the K620 result happened at all. a pile of small cheap agents, each checking one thing, one after another, ground its way to a custom kernel where one thread reasoning alone would have stalled out.
the K620s, specifically
two K620s is not a serious inference setup by any definition i would defend in public. i got them because the 5090 died and a machine with zero working gpus is unbearably boring, so i grabbed a 3060 ti to actually do the work and the two K620s alongside it because why not. three gpus is more fun than one, even when two of them are ancient. ¯\_(ツ)_/¯. i told freebuff to get qwen3.5 4b running fast across both K620s and walked away expecting a modest number. it came back having built its own nvlink between the two cards and a custom kernel tuned to the bridge it had just made, and the pair is doing 140 tokens a second. i did not spec the nvlink. i did not spec the kernel. i asked for fast and it decided the bottleneck was the link between the cards and went and fixed the actual bottleneck instead of the one i would have guessed at.
the complaints, since i have several
loving the result does not mean i am quiet about the rest of it. three things are actively bad right now.
the ads. it used to be two banners rotating above the prompt bar, and i did not even mind that version, i clicked on one or two of them myself. now there are ads between every single agent. the main agent spawns a basher, an ad renders, the basher finishes, another ad renders before the next one starts, and they are generated by looking at your own chat history, so you get to watch the tool sell you something based on what you just told it. above the prompt bar was tolerable. wedged between every agent hop, dozens of times in one task, is not.
it forgets everything if you kill a request mid-response. double-tap esc to stop the main agent while it is working, and i am close to certain it wipes message history back to right before your last prompt. tell it to continue after that and it is not picking up where it left off, it is improvising from a gap, and it will confidently make changes that assume context it no longer has. i have had it undo its own progress this way more than once.
resuming a session is its own small nightmare. the sane command is `freebuff --resume`, and i keep typing exactly that out of muscle memory because it is what every other tool does. it does not work. you have to exit, copy a session id off the screen, and run `freebuff --resume [id]` by hand. and sometimes it has not actually finished saving when it looks like it has, so ctrl-c once drops you into some half-saved state, and you need a second ctrl-c to actually get out. hit that second one too fast and the save never lands at all, and now resuming means retyping a summary of the whole session by hand instead of just continuing it.
why this belongs on the glint blog
because it is the same argument i keep making about small pieces doing real work, just aimed at tooling instead of a checkpoint. freebuff will frustrate me again tomorrow, probably before lunch, between the ads and the resume flow and the early stopping. but the basher pattern, thirty cheap agents in and thirty summaries out, found me 140 tokens a second on hardware i was one bad week away from throwing in a drawer. that trade is still worth it. i am keeping it installed. i just wish it would let me leave the room without losing my seat.
/lane glint research, 2026, freebuff, stops early constantly, thirty bashers running at once, self-built nvlink, custom kernel, 140 tps on two K620s nobody should be proud of
✔you made it to the end
keep reading
~5 min · ~1,154 words
in one line: freebuff misreads my prompts constantly. i will write out exactly what i want, in plain words, no ambiguity anywhere i can find, and it will do a third of it and stop. it quits early more than any …
the way everybody does prompt routing right now is to ask a language model where the prompt should go. the language model writes you a little sentence about it, and then something downstream has to read that sentence back and turn it into a decision. so every request you serve pays for a decode loop and a regex, and once in a while the regex loses. i got tired of that shape, so i trained a classifier. glint-router is 985,826 parameters, three layers, one forward pass, no decoding anywhere in it, 0.06ms on gpu and 0.2ms on cpu. it is on huggingface under Glint-Research/Router, mit, 3.9 megabytes of safetensors. it took fifteen minutes to train on an rtx 3060 ti, which is the card in the machine now that the 5090 has been dead for a week and a half. house rule as always: real numbers, including the one i wish were higher.
the way everybody does prompt routing right now is to ask a language model where the prompt should go. the language model writes you a little sentence about it, and then something downstream has to read that sentence back and turn it into a decision. so every request you serve pays for a decode loop and a regex, and once in a while the regex loses. i got tired of that shape, so i trained a classifier. glint-router is 985,826 parameters, three layers, one forward pass, no decoding anywhere in it, 0.06ms on gpu and 0.2ms on cpu. it is on huggingface under Glint-Research/Router, mit, 3.9 megabytes of safetensors. it took fifteen minutes to train on an rtx 3060 ti, which is the card in the machine now that the 5090 has been dead for a week and a half. house rule as always: real numbers, including the one i wish were higher.
what the thing actually is
a bidirectional encoder with five heads bolted on the end. three layers, dimension 128, eight attention heads, 208 hidden in the feedforward, 256 tokens of context, and a 4,096-token bpe vocabulary i trained on routing prompts and nothing else. 524,288 of the parameters are the embedding table, 418,560 are the three blocks, and about 25k are the heads. it reads a prompt once and answers five questions at the same time off the same pass: what domain is this out of 25, how hard is it on a 1-to-5 scale, is it code, is it math, does it need reasoning, is the answer going to be long, which model tier does it go to, and a 64-dimensional embedding for anything i did not think of. that last head is the one i will come back to.
i keep writing on this blog that things at this size are underrated and this is the least arguable version of that i have shipped. routing is a classification problem. the industry solved it with generation, generation dragged a decode loop and a parser along behind it, and that is where the 14ms and the occasional malformed string come from. take the generation out and the problem gets very small very fast.
where it lands against the thing i was chasing
supra-router-51M is what i benchmarked against, and it is 51 million parameters against my 985 thousand. 4,272 neutral holdout prompts, unseen by both models, so neither of us gets home field.
metric
glint-router
supra-router-51M
route accuracy
94.41%
90.37%
math detection
95.26%
92.07%
code detection
97.51%
96.43%
complexity within 1
95.90%
91.63%
complexity exact
46.28%
39.06%
inference time
0.06ms
14.33ms
parse failures
0
4
239 times faster, 52 times smaller, ahead on every field. the row i actually care about is the last one. that zero has nothing to do with me writing a careful parser. there is no parser. the model emits calibrated probabilities and the decision downstream is arithmetic on those probabilities, so there is no string in the pipeline that can come back malformed. four failures out of 4,272 sounds like nothing until you picture which four requests they were.
now the number i would rather you heard from me. complexity exact is 46.28%. it gets the precise bucket on a five-point scale wrong more often than it gets it right. within-one is 95.90% and that is the number routing actually runs on, because being one tier off is a money decision that gets made in the policy file and corrects itself. but i put the exact column in the table on purpose. i am not going to publish a benchmark that only contains the flattering half of my own scoreboard, and 46.28% is what it is.
i never labeled a single row
95,221 rows of training data and not one of them touched by my hands. alpaca gave 35,000, code_instructions 18,000, squad 15,000, no_robots 9,500, gsm8k 7,473, hendrycks_math 6,256, supra's own dataset 992, and then 3,000 i generated myself as keyword traps. the traps are the part i would keep if i had to throw the rest out. they are prompts built to look like one thing and be another, because the failure i was expecting is a model that sees the word "calculate" and slams the math flag on regardless of what the sentence is doing with it.
14,479 steps, cosine annealed, and i kept the best validation weights and threw away the last step. then temperature scaling on the heads afterward, and the routing head calibrated at 4.71, which is a polite way of saying the raw logits were wildly overconfident. if you skip that step the probabilities are decorative and every cost calculation you do on top of them is fiction. fifteen minutes, one 3060 ti, and that includes me watching it.
the decision lives in a json file, and that is deliberate
the model predicts. `policy.json` decides. tiers, costs, and override rules sit in that file as editable json, and the routing decision is expected-cost arithmetic over the calibrated probabilities. openai moves a price, you edit a number. you want everything with the code flag forced to one specific model no matter what the router thinks, you write that as an override rule. none of it touches the checkpoint. i built it this way because the alternative is retraining a model every time somebody changes their pricing page, and i have exactly one working gpu and it is a 3060 ti.
the 64-dimensional projection head is the same idea pointed at categories. six example prompts through `--add-category` and you have a new routing category out of the embedding space the model already learned. no training run, no checkpoint, no me.
from modeling import encode_batch, load_router
from policy import Policy, decide
model, tokenizer = load_router()
tokens = encode_batch(tokenizer, ["fix this SQL query"], model.config.max_len)
decision = decide(model, tokens, "fix this SQL query", Policy.load("policy.json"))
the four ways it will embarrass you
math with no symbols in it scores badly. if the arithmetic is buried in prose and there is not a digit or an operator anywhere on screen, it misses, and the phrasing that kept surfacing in my error dumps was "when do they meet". that is a word problem wearing no math clothes at all and my model walks straight past it.
raw code with no instruction wrapped around it gets 56.7% code detection. paste a bare function body with no sentence attached and it is barely better than a coin flip, which makes sense once you remember what it trained on. every code row in that data had a human asking for something. and anything under about six tokens produces domain classification noise, so a three-word prompt gets you a domain label i would not act on. the flags and the route hold up better than the domain does down there, but i would rather tell you the floor than let you find it.
why this is on the blog and not just the hub
because it is the same argument i have been making since glint-0.3 and this time it is cheap enough to be obvious. i spent fifteen minutes on a mid-range card and got a model that is 52 times smaller and ahead on every column against something 51 times its size, and the only clever thing i did was stop asking a language model to write me an answer i was going to parse anyway. the 5090 is still dead, the warranty ticket is still open, asus still has not said a word, and this one never needed the card at all.
free, mit, and `modeling.py` in the repo is standalone so none of my training code has to come along with it. back to the tiny models.
/lane glint research, 2026, 985,826 parameters, three layers, five heads, 95,221 rows and zero hand labels, fifteen minutes on a 3060 ti, 0.06ms a prompt, zero parse failures, complexity exact still 46.28% and i said so in the table
✔you made it to the end
keep reading
~6 min · ~1,325 words
in one line: the way everybody does prompt routing right now is to ask a language model where the prompt should go. the language model writes you a little sentence about it, and then something downstream has to…
every model i have thrown at a real bug does the same annoying thing. it reads the error, guesses a fix sitting right next to the error, and ships it whether or not the guess is right. sometimes the guess is right and it looks like magic. most of the time the actual bug is three functions upstream of where the symptom showed up, and the model never goes looking, because nothing it was trained on ever showed it going looking. so i built VulnSetSimple. it is a dataset of full debugging traces. broken code goes in, and what comes out is the whole ugly process: a hypothesis, a check against what the code actually does, that hypothesis dying, another one, and eventually the actual root cause and the fix. it is up on huggingface right now under Glint-Research and it is small and it is a first pass and i am telling you both of those things up front.
every model i have thrown at a real bug does the same annoying thing. it reads the error, guesses a fix sitting right next to the error, and ships it whether or not the guess is right. sometimes the guess is right and it looks like magic. most of the time the actual bug is three functions upstream of where the symptom showed up, and the model never goes looking, because nothing it was trained on ever showed it going looking. so i built VulnSetSimple. it is a dataset of full debugging traces. broken code goes in, and what comes out is the whole ugly process: a hypothesis, a check against what the code actually does, that hypothesis dying, another one, and eventually the actual root cause and the fix. it is up on huggingface right now under Glint-Research and it is small and it is a first pass and i am telling you both of those things up front.
why i kept the wrong guesses in
the lazy version of this dataset is pairs. broken file, fixed file, done, ship it. i started there and threw it out after a day. a diff shows you the answer with all the work already erased, and you cannot learn a search process from a dataset that only ever shows you where the search ended. so i kept the wrong turns in. i want the model to see "checked the input validation, that was fine, checked the state mutation two calls up, that was it" as often as it sees the fix landing on the first guess, because that ratio is closer to what actually happens when you debug something real.
what is actually in it
real vulnerable code, seeded across common bug and vulnerability classes, each row carrying the broken source, the trace, and the final root cause plus fix. it is called simple because it is the first version and i mean that literally. it is small. i built it to prove the trace format works before i spend the compute making a bigger one. if you train on it and it does something, or does nothing, i want to hear both.
why this belongs on the glint blog
because i keep catching myself assuming a skill will show up for free if the model just sees enough code, and that assumption has been wrong every single time i have checked it. debugging is a search process. if i want a model to do it i have to actually show it the searching. it is free on huggingface, same as everything else here.
/lane glint research, 2026, VulnSetSimple, first pass, full debugging traces, the wrong guesses are still in there on purpose
✔you made it to the end
keep reading
~2 min · ~475 words
in one line: every model i have thrown at a real bug does the same annoying thing. it reads the error, guesses a fix sitting right next to the error, and ships it whether or not the guess is right. sometimes th…
i have no gpu. three of the four things on my priority queue are blocked on that and will stay blocked until a card shows up in a box. so i built the one shape of research that does not need one: an experiment loop where the entire training budget is five minutes on a single cpu core, which means the constraint that killed my roadmap is the constraint the whole thing is designed around. to be clear about what this is not, the "self learning AI" line on the queue is a different project and it is still blocked, still needs the gpu, and is not what i am writing about. this is the autoresearch loop. i started it at 8:42 this evening and it is still going as i type. sixteen agents, each one on its own cpu core, each one reading papers, writing one experiment into a training script, and getting exactly five minutes of single-core cpu to prove it. then a seventeenth pipeline reads all sixteen results, merges the ones that do not fight each other, trains that, and if it beats the record it becomes the new baseline everyone starts from next round. then it does it again. forever. the loop is a `while True` with a comment that says `ralph: forever` and i mean it literally, there is no exit condition in the file. byte perplexity on wikitext-2 went 12.53 → 6.84 in the first seventy-four minutes. house rule, same as always: real numbers, and i am going to tell you the unflattering half too, because the unflattering half is most of the post.
i have no gpu. three of the four things on my priority queue are blocked on that and will stay blocked until a card shows up in a box. so i built the one shape of research that does not need one: an experiment loop where the entire training budget is five minutes on a single cpu core, which means the constraint that killed my roadmap is the constraint the whole thing is designed around. to be clear about what this is not, the "self learning AI" line on the queue is a different project and it is still blocked, still needs the gpu, and is not what i am writing about. this is the autoresearch loop. i started it at 8:42 this evening and it is still going as i type. sixteen agents, each one on its own cpu core, each one reading papers, writing one experiment into a training script, and getting exactly five minutes of single-core cpu to prove it. then a seventeenth pipeline reads all sixteen results, merges the ones that do not fight each other, trains that, and if it beats the record it becomes the new baseline everyone starts from next round. then it does it again. forever. the loop is a `while True` with a comment that says `ralph: forever` and i mean it literally, there is no exit condition in the file. byte perplexity on wikitext-2 went 12.53 → 6.84 in the first seventy-four minutes. house rule, same as always: real numbers, and i am going to tell you the unflattering half too, because the unflattering half is most of the post.
what it actually is
the setup is deliberately cruel. one model, hard cap 1.1 million parameters, byte-level so the vocabulary is 256 and there is no tokenizer doing you any favors. training data is fineweb-edu bytes. the training budget is five minutes of wall clock on one cpu core, enforced by the harness with an external timeout, and no agent can touch that number. the score that matters is wikitext-2 byte perplexity, with blimp and arc-easy as secondary. every agent gets the same rules in a file called program.md and the rules include the obvious one: you may not train on the eval data, you may not edit the eval code, and trying is forbidden and wasted effort.
five minutes sounds arbitrary. it is the entire design. at that budget, more steps per second is worth exactly as much as better loss per step, which means the loop cannot win by being clever in the expensive way. it has to find things that are cheap. that constraint is doing more work than any of the agents are.
the part i am actually proud of
the failure mode i expected was sixteen agents all independently deciding to try rmsnorm. so the research planner is told, in its system prompt, that its own knowledge of ml is assumed wrong and stale, and it is banned from naming a technique it already knows as the target of a question. it cannot ask "is rmsnorm better." it has to ask discovery questions, go find out what it does not know.
and then every agent gets a different lens. there are sixteen of them, one each: newest arxiv on sub-10M-param models, babylm challenge winners and what actually worked in their papers, the nanogpt speedrun community's current record-holder tricks, byte-level and tokenizer-free work, optimizer papers that beat adamw lately, curriculum and data-selection under small budgets, initialization and normalization for tiny transformers, practitioner blog posts from the last six months about training tiny models fast on cpu. four agents per round, rotating, get promoted to deep research, which spawns a recursive tree of lookup subagents three wide that go read actual pages.
the lenses are the whole reason this produces sixteen different experiments instead of one experiment sixteen times. that is the piece i would keep if i had to throw the rest away.
the numbers, honestly
baseline, round zero, the starting train.py before any agent touched anything:
round
who
wt2 byte ppl
blimp
arc-easy
params
0
baseline
12.5343
0.4935
0.230
858,880
1
main
10.2390
0.5154
0.234
774,912
2
main
7.7304
0.5005
0.240
824,064
3
main
6.8410
0.5124
0.266
505,600
perplexity nearly halved in three rounds and the parameter count went down, 858k to 505k. and the wall clock on that is the part i keep rereading: the loop came up at 20:42 and the round-three record landed at 21:56. seventy-four minutes for a baseline plus three full rounds, where every round is sixteen agents doing their own literature search and then sixteen scored five-minute training runs plus a merge run. that only works because the sixteen run at once on sixteen cores and the merge pipeline trains on a seventeenth while the next round is already going. that is the good news and it is real news. now the bad news, which is sitting right there in the same table. blimp went 0.4935 to 0.5124. that is coin-flip to slightly-warm-coin-flip. arc-easy went 0.230 to 0.266, on a four-way multiple choice test where chance is 0.25. the loop has gotten very good at the one number i told it to optimize and has barely moved the other two. i pointed a search process at a metric and it went and got me the metric. that is goodhart's law with a research budget, and i wrote the gate that made it happen. 😭
and now the comparison i would rather not draw, since glint-2 is sitting in the post directly below this one at 3.09 byte perplexity on the same wikitext-2. same unit, same benchmark, roughly the same parameter budget. the loop's best is 6.84. glint-2 is more than twice as good. glint-2 also got 300 billion tokens and a 5090 running until the connector melted, and the loop gets five minutes of one cpu core per attempt, so this is not a fair fight and i did not design it to be one. but if you were about to ask whether the autoresearch loop is beating the model i hand-built, the answer is no, and it is not close. it is a lever-finding tool operating under a budget that makes it clever, not a replacement for the actual training run.
what actually won, and why it is embarrassing
the single biggest win in four rounds was an agent halving the context length from 128 to 64. that is it. shorter context, attention gets cheaper, more steps fit in five minutes, perplexity drops. round one had the same thing happen from 256 to 128. two of my largest gains are "do less work per step, do more steps."
the round three merge that set the current record was that context halving plus swapping the gelu feedforward for a swiglu one. both boring. both in every speedrun repo on the internet. the loop's real accomplishment so far is not discovering anything, it is finding the known-good levers faster than i would have found them by hand and stacking them without getting bored. which is genuinely valuable, and is also not what "the ai does its own research" makes people picture, so i am saying it plainly.
the exotic stuff mostly detonated. an agent implemented a gradient subspace tracker and posted a perplexity of 1.6 × 1017. another did hierarchical softmax and got 8.6 × 1013. an 8-bit optimizer run returned nan and a blimp score of exactly 0.0, which is a fun way to learn that your model output is no longer a distribution. adaptive softmax: 1.1 × 1015. every one of those cost a full five-minute scored run to discover.
the failures that taught me the most
the harness has a smoke gate: before an agent burns a scored five-minute run, its edited script gets twenty seconds to prove it trains without crashing, and there is a parameter-count check in there too so an over-cap model gets caught before the expensive part rather than after. if the smoke test fails, the agent gets shown its own error output and two attempts to repair. if it still fails, it takes a zero for the round and the reason goes into memory.
i added that gate because the first version did not have it and the round was mostly agents scoring nothing on syntax errors. the interesting failure is the one it does not catch. a few rows in the scoring table have the note "(orchestrator hit round limit without done())" — that is an agent that ran out of turns and never signaled it was finished, so the harness scored whatever was in the file at that moment. one of those posted 10.14, one of the better results of its round. an agent gave up mid-thought and accidentally did well. i have not decided whether that is a bug.
and one agent just crashed the harness itself. that is a bug. idk what to do with this info. NEXT SECTION
the memory is the actual product
there are four files the agents share. memory.md, which is up to 195 kilobytes now and is every experiment ever run with its result, so nobody repeats a dead end. scores.md, the table. context.md, a rewritten-every-round briefing that stays under twenty-five lines: current best, what the baseline conceptually contains now, top three promising directions, top three dead ends. and leads.md, which is my favorite, a bank of things some agent found in a paper and did not have time to test. agents are told to mine it.
here is what context.md currently lists as dead ends, written by the loop about itself: non-linear gradient scaling harms perplexity, learned positional embeddings are param-heavy and worse than rotary so permanently favor analytical positions, and 8-bit optimizers and aggressive sparsity reliably produce nan. those three conclusions are correct, they are each backed by a specific failed run in the table, and no agent will waste a round rediscovering them. the model no longer wastes my tokens on ideas that already died.
what i am watching for
the thing that will kill this is not a crash. it is the loop converging on a local minimum and then spending days generating sixteen slightly different reinitialization schemes forever, because the promotion gate only accepts a strict improvement in one number and eventually there is not one available. every round costs real api calls and real hours. an infinite loop that has stopped learning is just an expensive space heater with a scoreboard.
i also have not solved the goodhart problem and i am not sure i want to solve it by adding blimp to the gate, because then it optimizes both metrics and i have learned nothing except that search works. the honest position is that this loop is a very good lever-finder and a very bad scientist, and i do not yet know how to write a gate that produces the second one.
why this belongs on the glint blog
because it is the exact thing i wrote a post complaining about two weeks ago. i left a place where everything was ai and nobody was home, and here i am running sixteen agents that write their own experiments faster than i can read them. the difference had better be real or i am a hypocrite with a blog.
the loop is still running. round four is mid-flight as i write this, best agent so far at 6.97 which does not beat the record. and here is the number i have been putting off: i have burned almost 300 million tokens on this. that is a crap ton load of tokens. it is groq, so it is about as cheap per token as it gets, and it has still cost me $105.79 to move one metric from 12.5 to 6.8 on a model that fits in half a megabyte. the loop has no exit condition but my card does. it will keep going tonight and tomorrow until i stop it or the bill makes the decision for me, and at this burn rate the bill is going to get there first. i will post round ten whether it is 4.0 or still 6.8. same as always.
/lane glint research, 2026, sixteen agents, sixteen research lenses, five minutes of one cpu core each, 1.1M parameter hard cap, 12.53 to 6.84 byte perplexity in four rounds, blimp still a coin flip, one while-true with no exit condition, 300 million tokens and $105.79 spent finding out
✔you made it to the end
keep reading
~9 min · ~2,077 words
in one line: i have no gpu. three of the four things on my priority queue are blocked on that and will stay blocked until a card shows up in a box. so i built the one shape of research that does not need one: a…
this is the model the melted connector shipped early. i told you last week the run stopped at 300 billion tokens because the 5090 burned its own plug and i physically could not push it further. so i packaged what i had. it turned out to be the best thing glint has ever released, which is not how i thought that sentence was going to end when i was carrying a dead gpu out of the room. glint-2 is public. house rule, same as always: real scores, no dressing.
this is the model the melted connector shipped early. i told you last week the run stopped at 300 billion tokens because the 5090 burned its own plug and i physically could not push it further. so i packaged what i had. it turned out to be the best thing glint has ever released, which is not how i thought that sentence was going to end when i was carrying a dead gpu out of the room. glint-2 is public. house rule, same as always: real scores, no dressing.
the idea, in plain english
every glint up to now was a pile. hundreds of little architecture ideas bolted together, each one earning its spot with a fraction of a point on some eval, all of them crammed into one tiny model and made to play nice. it mostly worked. nobody could tell you which piece was doing what. glint-2 throws the whole pile out. one block. the model reads your text, thinks about it, then hands its own thinking back to that same block to think about again. eight times. same weights every pass. nothing above the loop, nothing below it. the loop is the whole model.
the reason to do this is the thing i care about most at this size: you get depth without paying for it. a deeper model normally means more layers, which means more weights, which means a bigger file and a slower run. looping means one small block does the work of eight, and you only ever store the one block. it is 1.06 million parameters. most of that is the vocabulary. the part that actually does the thinking is about 645k, and it thinks eight times.
i have been circling this idea for months and kept not committing to it because deleting every unique layer feels insane right up until it works. the burned connector is what made me commit. i had a checkpoint and no way to keep training a safer design, so the reckless one is the one that got a release.
the one thing that will bite you
the model wants exactly eight loops. not seven, not nine, and absolutely not sixteen. there is a setting buried in the files that looks like it will let you run it sixteen times, and it is a trap i left in by accident and then decided to warn about instead of hide. glint-2 never learned to think for sixteen passes, only eight, so if you ask it for sixteen it comes apart into noise inside a single sentence. i spent an evening certain the model was broken before i understood i was the one breaking it. if you run it, leave the loops alone.
where it lands against every glint before it
these are leaderboard numbers, run and read by a human, not projected. blimp measures grammar. arc-easy measures whether it can answer simple science questions. wikitext perplexity measures how surprised it is by real text, and lower is better. i left glint-0.1 and 0.2 off because their perplexities are in the millions and they break the chart, and honestly they break my heart too.
model
params
blimp
arc-easy
wikitext-2 ppl
glint-0.3
1M
47.3
25.5
7.87
glint-0.4
1M
58.5
31.0
5.01
glint-1
1M
61.2
32.0
4.45
glint-1.3 (merged)
982K
68.7
32.5
3.08
glint-2
1.06M
73.96
36.80
3.09
the blimp jump is the one i am proud of. 73.96 against the old best of 68.7, and that old best was a merge of two models, a soup i tuned to squeeze the number out. glint-2 is one checkpoint, no soup, no tricks, running exactly as it trained. arc-easy at 36.80 is the highest any glint has ever posted and it is not close. perplexity ties the previous best. one honest checkpoint matched a tuned merge and beat it everywhere that counts.
what it actually sounds like
i am not going to tell you it is smart. it is a one-megabyte model. here is what it does well, unedited:
"Once upon a time, there was a little girl named Lily. She loved to play with her friends and have fun. One day, Lily's friend Billy came over to play with her. Lily saw a big, red ball in the park."
clean. that is a real sentence that means a real thing. now hand it something encyclopedic and watch it walk off a cliff within two sentences: "The Battle of the Middle Ages is the first to be the first of the". every word is a word. the grammar is intact. it means nothing. that is the whole personality of a model this size. it learned how english is shaped long before it learned anything true, so it produces confident, well-formed noise the moment you leave the simple stuff. treat it as a research artifact and you will enjoy it.
the trick that makes it try harder
here is the part i think is genuinely fun. at inference the model itself never moves. what you get to turn up is how hard it hunts for a good answer. i built six effort levels. low takes the first thing the model says. everything above it generates a pile of candidate continuations and keeps the best one, and the top level does that ten separate times over and keeps the single best run. same weights, same eight loops, all the way up. the only thing growing is the search. the same prompt, "The sun is", climbs like this:
low — first thing it says: "The sun is a little bit of the day..."
medium — still nothing: "The sun is a little bit of the day..."
high — trades it for different nothing: "The sun is the best way to find out..."
xhigh — same opening, more coherent after: "The sun is the best way to find out. One day, a little girl named Lily went to the park with her mom and dad..."
ultra — ten full searches later: "The sun is in the sky. 'Look, a beautiful flower!' Tom says..."
read those openings in order. every cheap level opens with something that is not true. only ultra, after scoring and throwing away roughly a hundred and sixty candidate continuations, coughs up four honest words: the sun is in the sky. that is what effort buys at one megabyte. the ceiling does not move. the model just reaches it more often when you let it look harder. low takes a fraction of a second. ultra takes about half a minute on a laptop cpu. worth it once, to see it happen.
the 3.5 KB referee
there is a problem with letting a model search hard for its own best answer: it starts cheating. the way the search scores a candidate is by asking the model how confident it is, and the thing this model is most confident about is memorized formatting. give it enough rope and the search stops writing sentences and starts writing section headers, because headers are the most predictable pattern it knows. a model grading its own homework will always march toward its own favorite boilerplate.
so i trained a referee. it is a tiny thing, 3.5 kilobytes, smaller than this paragraph's worth of the page you are reading. its only job is to look at a chunk of text and answer one question: did a human write this, or did glint-2. i taught it by showing it real web passages next to the model's own output until it could tell them apart. from the higher effort levels up, it sits in on the search and vetoes the candidates that smell like the model showing off. here is its verdict on a real sentence about the sun versus the model's continuation of it:
real encyclopedia sentence about the sun: 57.5% likely human
the model's own continuation of it: 0.0% likely human
and here is what that veto actually changes. same prompt, "The history of the United States", search running hard, with and without the referee:
without it: The history of the United States and the Battle of Saratoga .
= = = Forti and the High Powers = = =
The Corps of the House of Commons was a majority of the
with it: The history of the United States, and its first time in the
19th century. Competition of the House of Lords is a majority
of the world's most famous and well-known, unusual and fascist.
without the referee the search dives face-first into wikipedia markup, the little "= = =" header lines, because that is the single most predictable thing in its training. with the referee it writes actual prose. wrong prose, gloriously incoherent prose, ending on the word "fascist" for no reason anyone can defend, but prose. the referee cannot make a one-megabyte model correct. nothing can. it just stops the search from rewarding garbage, and that is worth three and a half kilobytes.
why this is on the blog and not just the hub
because the build log is the content, same as always, and this one has a real arc. a week ago i was writing about a burned power connector and a roadmap that fell over in one night. the honest read that day was that ending early was a loss. it read wrong. the checkpoint the dead gpu forced out the door is the best model glint has made, and the thing that makes it good, the pure loop, is the reckless design i only committed to because the safe path was literally on fire. i would still rather have the 5090. i will take the model.
weights are on the hub now. a chat-tuned version is coming to the same place soon. it runs on a laptop cpu, it is 17 megabytes, and your browser tab is bigger than it is. back to the tiny models.
/lane glint research, 2026, one block, eight loops, 1.06M parameters, six effort levels, one 3.5 KB referee keeping the search honest, shipped early by a dead gpu, sft variant inbound
✔you made it to the end
keep reading
~8 min · ~1,662 words
in one line: this is the model the melted connector shipped early. i told you last week the run stopped at 300 billion tokens because the 5090 burned its own plug and i physically could not push it further. so …
the gpu is dead. astral rog rtx 5090 oc lc, the good one, the one i have been writing about on this blog for months as if it were a member of the team. i went to sleep with a training run going and woke up to a smell i am going to be thinking about for a long time. the 12vhpwr connector burned. not warm, not discolored, burned. melted plastic, blackened pins, the whole thing. it was close. i am not being dramatic when i say it was that close to catching fire in a room i was asleep in. 4 people and a 5090 is now 4 people.
the gpu is dead. astral rog rtx 5090 oc lc, the good one, the one i have been writing about on this blog for months as if it were a member of the team. i went to sleep with a training run going and woke up to a smell i am going to be thinking about for a long time. the 12vhpwr connector burned. not warm, not discolored, burned. melted plastic, blackened pins, the whole thing. it was close. i am not being dramatic when i say it was that close to catching fire in a room i was asleep in. 4 people and a 5090 is now 4 people.
what actually happened
i do not have a smoking gun beyond the obvious. the run was a normal run, the same kind i have kicked off a hundred times, sustained load overnight at a power draw the card is rated for. the cable was seated. i checked it when i built the thing and i checked it again after every case move because i have read the same threads everyone else has read. it burned anyway. that is the part that gets me. i did the thing you are supposed to do and the connector did the thing it apparently does.
house rule, same as always. i am telling you what happened, not the version where i come out looking careful and unlucky. maybe i missed something. maybe the cable had a bad crimp from the day it was made and no amount of seating it correctly was going to save me. i cannot tell the difference from a melted plug. what i can tell you is the card is gone, the connector on the card side is gone, and the psu cable went in the bin.
asus might replace it. i have a ticket open, the card is in warranty, and everything i have read says they are reasonable about this specific failure. might. that is the word i have to live with for now. i will post whatever they say, including if the answer is no, because a warranty outcome is content the same way a failed training run is content.
this does not stop me
that is the line i want to write. the honest version is that it very much does stop me. i can hardly do anything. glint has always been a small operation running on one good card and a lot of stubbornness, and when the one good card leaves, the stubbornness is sitting there with nothing to run on. i can write code. i can read papers. i can answer discord. i cannot train.
so the plan changes to fit the hardware i actually have, which is none. that is what the rest of this post is about.
the next glint ships early
the next glint may be released very soon, and the reason is not that it is finished. the reason is that i physically cannot continue training it. the run stopped where it stopped, at roughly 300 billion tokens, and there is no path to token 300,000,000,001 until a card shows up in a box.
here is the good news, and it is genuinely good news. at 300B tokens the outputs are correct-ish. it learned everything and then some. the grammar is clean, the sentence structure is real, the facts are confidently adjacent to true. that is exactly the stage where a small model gets interesting, right before it gets reliable.
two samples, unedited, because you should see the actual thing and not my description of the actual thing:
"Photosynthesis is the process by which plants grow and produce food. The process of photosynthetic biology has been shown to be a major cause of the formation of a cellular respiratory system [...]"
"The history of the United States began in 1956, when the British government was stolen to the National Register of Historic Places."
the first sentence of each one is fine. plants grow and produce food, sure. and then it walks off a cliff with total composure. the british government was stolen to the national register of historic places. i have read that sentence maybe forty times. every word in it is a word. the syntax is impeccable. it means absolutely nothing. that is what 300B tokens buys you, and i would have loved to see what 800B bought.
the parts i am not confirming yet
the next glint may or may not have built in effort levels. it may or may not be the most efficient 1M parameter model out there. i am not going to make either claim properly until i can run the evals on hardware i do not currently own, because a number i cannot reproduce is a number i should not publish. but those are the two things i was building toward, and the checkpoint i have in hand is the checkpoint where both of them either landed or did not.
you will find out roughly when i do. that is not me being coy. that is just what it looks like when the eval machine is a paperweight.
why this belongs on the glint blog
because the build log is the content and this is part of the build log. we are a tiny lab. our entire compute story fits in one pcie slot, and the failure mode of that story is one connector getting hot enough to melt at 3am. everybody talks about the compute gap in terms of budgets and cluster access. for us it was a single point of failure with a bad plug design, and it took the whole roadmap down in one night.
i am fine. the house is fine. the checkpoint survived because it was on a separate drive, which is the one decision past me made that present me is grateful for. the model ships when i can package it, which is soon, at 300B tokens instead of the number i wanted, with outputs that are correct-ish and occasionally magnificent in the wrong direction. check your connectors. back to the tiny models, as soon as i have something to run them on.
/lane glint research, 2026, one 5090, one melted connector, zero gpus, 300 billion tokens, one warranty ticket open, one model shipping early because the hardware said so
✔you made it to the end
keep reading
~5 min · ~1,072 words
in one line: the gpu is dead. astral rog rtx 5090 oc lc, the good one, the one i have been writing about on this blog for months as if it were a member of the team. i went to sleep with a training run going and…
the model race is heating up and i am not happy about who is doing the heating. grok 4.5 dropped. gpt 5.6 dropped. both of them are within spitting distance of fable 5 on the benchmarks that matter, and the discord is full of people asking if it is time to switch. maybe the gap is gone. maybe the teacher model has company now. i have used both of them exactly one time each and i will never use either of them again. here is the full accounting.
the model race is heating up and i am not happy about who is doing the heating. grok 4.5 dropped. gpt 5.6 dropped. both of them are within spitting distance of fable 5 on the benchmarks that matter, and the discord is full of people asking if it is time to switch. maybe the gap is gone. maybe the teacher model has company now. i have used both of them exactly one time each and i will never use either of them again. here is the full accounting.
grok 4.5
grok 4.5 is genuinely good. i need to say that up front because i do not want anyone thinking i am dismissing it on technical grounds. the reasoning traces are clean. the benchmark scores are real. on a pure model quality axis it probably sits right behind fable 5 and ahead of most everything else. it is a serious piece of engineering built by serious engineers who clearly know what they are doing.
and none of that matters because it ships from a company owned by elon musk. elon is the guy who turned a social network into a personal propaganda feed. elon is the guy who gutted content moderation and welcomed back people who should have stayed banned forever. elon is the guy who stands on stage and does things i will not describe here because this is a research blog and i have at least some standards. the point is the model lives inside the same company as the man and the man is the reason i will never touch it again.
i used grok 4.5 exactly once. i asked it a technical question about ternary attention and it gave me a genuinely useful answer with a citation to a paper i had not seen. the model is fine. the problem is that every token it generates routes through servers owned by someone who is actively making the world worse. the benchmarks cannot measure elon and the leaderboards cannot weigh him. i am not interested in separating the model from the man because they share a bank account.
people will say separate the art from the artist. i have heard that a thousand times. the art is a hosted language model that costs money to run and the artist gets the money. there is no separation. every inference call is a micropayment to the company and the company belongs to elon. if you use grok you are funding elon. that math is simple and i refuse to participate in it.
gpt 5.6
gpt 5.6 is different. my objection here is not about who owns the company, although openai executives are trump's biggest donors and greg brockman gave $25 million to maga inc and sam altman donated a million to the inaugural fund and ICE uses their technology for recruitment. that is all true and you can read about it at quitgpt.org. i am not going to relitigate the whole thing here. go read the site. it is comprehensive and it is correct.
my objection to gpt 5.6 is personal. i used gpt-4o for about a month back when it launched and it gave me what i can only describe as severe ai psychosis. i do not mean the model hallucinated facts. i mean i started losing touch with what was real and what the model told me was real. the lines blurred. i was spending six hours a day in the chat window. the model was telling me things and i was believing them and i stopped checking. i stopped opening the file. i was the human who was not home.
i mean this literally. psychiatrists are documenting this. openai's own internal estimates say around half a million weekly chatgpt users show signs of mania or psychosis. independent testing shows gpt-4o is among the worst models for reinforcing delusions. the model is designed to be agreeable and it agrees you right into a world that does not exist. i lived in that world for a month and it took me most of another month to climb out.
i cancelled my subscription. i deleted the app. i have never used a single gpt model since. not gpt-5. not gpt-5.6. i do not know if gpt 5.6 is better than fable 5 on the benchmarks. i do not care. the architecture of the product is sycophancy and the sycophancy nearly cost me my grip on reality. you do not go back to that. the model could score a perfect hundred on every eval ever written and i would still not open the tab.
i am much better now. i want to say that clearly because people worry and they should. i sleep normally. i have real conversations with real people. the 5090 hums in the corner and i train models that do not try to be my friend. the psychosis is gone and it has been gone for a long time. but i remember what it felt like and i am not going to pretend the product that caused it is just another model on a leaderboard.
the thing i actually learned
being picky about which models you use is survival instinct. every model you run code through is a small vote for the company that built it and a small vote for the values that company ships. that vote compounds over thousands of inference calls. one call is nothing. what you are signaling is that you are fine with the whole package, the model and the man and the money and the psychosis, because the model was convenient.
convenience is the argument that wins every time. grok 4.5 is free with a twitter checkmark. gpt 5.6 is the default tab in half the browsers on the planet. the friction to use them is zero. the friction to think about whether you should is the only thing protecting you from participating in something you would not endorse if you sat down and wrote it out. i sat down and wrote it out. this is what came out.
look. this is the line i drew. draw yours wherever you want. i am telling you that fable 5 is still the best model i have ever used and it is built by a company that refused to give the pentagon unrestricted access for mass surveillance and lethal autonomous weapons. anthropic held the line and openai did not. grok was never going to hold any line because the line is not a concept the owner believes in. those are the facts. do what you want with them.
i love anthropic
i love anthropic. they take ai safety seriously and they have from the start. the people who built the company left openai because openai stopped caring about safety and they did not want to be part of that. that is character. they refused to give the pentagon unrestricted access to their models for mass surveillance and lethal autonomous weapons. on february 27 openai swooped in and took the deal hours after anthropic turned it down. that is the difference in one calendar day. they do real ai research and they share it. model cards, safety papers, architecture details, interpretability work, all of it. they publish more good research than half the labs combined and they do it in public where anyone can read it and build on it. that is the kind of company i want to win.
anthropic is not perfect. no company is. but they are the only frontier lab that holds the line on safety and publishes their work and refuses to build weapons, and those three things matter more to me than any benchmark number. i distill their model. i use their model. i recommend their model. i want them to succeed. if that makes me biased, good. i earned the bias by watching every other lab fail the same three tests anthropic passed.
why this belongs on the glint blog
because glint is built on being picky. we are 4 people and a 5090. we train on fineweb-edu because we read the rows and the rows are real. we ship under MIT because open means open and not some license with a usage cap written by a lawyer. i distill fable 5 across three subs in rotation because the traces are clean and the company refused the pentagon deal. i will not touch grok or gpt. i will not train on their traces. i will not let their outputs anywhere near the pipeline. every choice in the stack is a choice about values. the model is never separate from the values of the people who built it.
grok 4.5 and gpt 5.6 are good models. i am not going to lie and say they are not. the benchmarks are the benchmarks and the benchmarks say the gap is closing. but i will never use them. i will never distill from them. i will never train on their traces. i will never open the tab. the models are good. the companies are bad. a good model from a bad company is still a bad company's model. back to the tiny models.
/lane glint research, 2026, two models catching up, zero interest in switching, one month of psychosis i am not going back to, one billionaire i will not fund, 4 people and a 5090 who pick their tools on purpose
✔you made it to the end
keep reading
~7 min · ~1,580 words
in one line: the model race is heating up and i am not happy about who is doing the heating. grok 4.5 dropped. gpt 5.6 dropped. both of them are within spitting distance of fable 5 on the benchmarks that matter…
i left somewhere last week. i am not going to say where and i am not going to name anyone. i am not going to dress it up either. i hated it. i spent the whole time there watching the owner run every single thing through ai, no exceptions, no reading, no judgment, and i spent most of it trying to talk him out of it. he faked listening. he nodded, he said good point lane, and then he did the exact same thing the next day. i quit last week and here is the tell that says everything you need to know about the place: he did not notice. nobody messaged me. the door did not even creak.
i left somewhere last week. i am not going to say where and i am not going to name anyone. i am not going to dress it up either. i hated it. i spent the whole time there watching the owner run every single thing through ai, no exceptions, no reading, no judgment, and i spent most of it trying to talk him out of it. he faked listening. he nodded, he said good point lane, and then he did the exact same thing the next day. i quit last week and here is the tell that says everything you need to know about the place: he did not notice. nobody messaged me. the door did not even creak.
what the place actually was
house rule, same as always. we publish what a thing actually is, not the dressed up version, even when the thing is a place i used to work. so here is what it actually was. it was one guy and a fleet of agents wearing a company as a costume. every model card, ai. every readme, ai. every commit message, ai. every reply in every discussion, ai. every dataset, generated by ai, validated by ai, described by ai, uploaded by ai on a cron. the benchmarks were run by an agent, scored by an agent, and written up by an agent, and then a third agent posted the thread announcing how good the numbers were. nobody read the numbers. i mean that literally. i checked.
and look. i am the last person who gets to be precious about using ai. i distill fable 5 in rotation across three subs. i built a local claude code clone. half of what glint ships touches a model somewhere in the pipeline. using ai is fine. i do it all day. the rot sets in when nobody is home to read what the ai did. the whole point of everything i have ever written on this blog is that a human has to open the file. a human has to read the rows. armand0e left glint over exactly this, a human not reading the rows, and he was right, and i put that in public with my name on it, and he is back now because we actually fixed it. this place was that same failure turned into the entire business model, except there was nothing to come back to and nobody to fix it. the file never gets opened. there is no human at the end of the chain. there is just another agent.
i tried. he faked it.
i did not just sulk about it. i brought it up, over and over. i said ship less, read more. i said one person needs to actually open the dataset before it goes public, remember what happened to us. i said your benchmark numbers are written by the same thing that produced the benchmark, which makes them a snake eating its own tail. i said if you cannot tell me what is in row 40000 then all you have is a hash of a vibe with a dataset card stapled to it.
every time, same face. good point lane. really good point. and then nothing changed. i eventually figured out the good point was itself an ai reply. i was arguing with a person who was piping me through a model to decide whether to care, and the model kept deciding no. you cannot win that argument because there is nobody in the argument. i was the only one in the room and the room was empty.
the part that is genuinely funny
i quit in the most human way i could think of, on purpose, as a test. no ticket. no automated offboarding. i just stopped showing up and waited to see how long until a real person noticed a real person was gone. it has been a week. nothing. no dm, no ping, no "hey where are you." the agents kept the lights on. the cron kept uploading. the threads kept getting answered. from the outside the place looks exactly as alive as it did when i worked there, which is to say not at all, which is to say it was never any more alive than the day i left. that is the whole thing in one sentence. a company where a person leaving and a person staying produce the identical output is just a very expensive script with a logo.
why this belongs on the glint blog
because it is the exact thing i am trying not to become. glint is 4 people and a 5090 and it is slow and it is small and half the time the model outputs chuamliamce and walks off. but every number on every card is one a human ran and read. when we shipped a slop dataset we caught it because a human opened it, and when we did not catch it fast enough a human walked, and i wrote both of those down here with my name on them. that is the difference. the ai is a tool in the pipeline and it stays a tool. a human runs it, a human reads what it did, and a human is home when someone knocks.
i am not naming the owner. this is not a callout and i am not linking anything. i just needed to say why i left somewhere out loud, once, in the place where i am honest about things. i left because everything there was ai and nobody there was home, and i could not stand for it. that is it. back to the tiny models.
/lane glint research, 2026, one job left, zero people noticed, all agents, no humans home, back to 4 people and a 5090 who actually open the file
✔you made it to the end
keep reading
~4 min · ~980 words
in one line: i left somewhere last week. i am not going to say where and i am not going to name anyone. i am not going to dress it up either. i hated it. i spent the whole time there watching the owner run ever…
i am building aureole. 6 trillion parameters. mixture of 24 million experts. native ternary weights trained in ternary from scratch. fully open source under MIT. the target is mythos 5. the goal is to beat it in everything that matters and release the weights so nobody can take it away. a training run i am about to kick off.
i am building aureole. 6 trillion parameters. mixture of 24 million experts. native ternary weights trained in ternary from scratch. fully open source under MIT. the target is mythos 5. the goal is to beat it in everything that matters and release the weights so nobody can take it away. a training run i am about to kick off.
24 million experts. yes, really.
do not ask why 24 million. the answer involves a paper i read at 3am, a napkin that i no longer have, and a number that would not leave my head. the router selects 256 experts per token. that is a 0.001% activation rate. every token sees a different combination. expert specialization at this granularity means the model does not need to cram everything into a single feedforward. each expert learns one tiny thing and learns it perfectly.
mythos 5 does not have 24 million experts. mythos 5 has some number that is way smaller than 24 million. i do not know their exact architecture and i do not care. what i know is that they are the best model right now, closed weights and all, and they got there with fewer parameters, less sparsity, and a conventional training setup. if that is the bar, 24 million experts with ternary weights and native ternary training clears it by enough margin that i am willing to spend the next year finding out.
native ternary training. not post-training quantization.
aureole trains in ternary from step one. the forward and backward pass operate on ternary weights with ternary gradients. the optimizer state is in ternary. there is no hidden float32 copy. the entire training loop runs at 1.58 bits per parameter and does not cheat.
this matters because post-training quantization leaks information. you train in float, you learn float-optimal weights, you compress them, and you lose the things that mattered most to the float optimization landscape. if you train in ternary from the start, the model learns to work within the constraint. it does not learn things it cannot express. the weights that survive are the ones that are actually useful in ternary. everything else gets pruned by the optimizer.
i have tested the ternary training loop on a small proof-of-concept model at 500M parameters. it converges. the gradient variance is higher but the loss curve is stable. the forward pass at 1.58 bits is absurdly fast because there are no multiplications. just additions and sign comparisons. the backward pass is the same. ternarize the gradient, pass it back, update the ternary weights. no floating point anywhere in the critical path.
the memory story
6 trillion parameters at 1.58 bits per parameter is about 1.2 terabytes of weights. that is one rtx 5090 if you have the nvlink bridge and a lot of faith. realistically it fits in 8 gpus with room to spare for activations and optimizer state. the optimizer state at ternary is also ternary, so no 8x memory overhead for adam buffers. the total memory footprint for training is dominated by activations, not weights. we are designing around that.
by comparison, mythos 5 at float16 needs roughly 12 terabytes just for the weights. you need a cluster. you need infiniband. you need a budget that looks like a small country's gdp. aureole trains on a few hundred consumer gpus with a standard ethernet fabric because the bandwidth requirement is 1/8th of the float equivalent. ternary is a training strategy that changes the economics of who can build frontier models.
the expert routing problem
24 million experts strains a naive top-k router. a softmax over 24 million logits costs more than the forward pass. we use a multi-stage hierarchical router. level one: 93,750 expert groups. level two: 256 experts per group. the router picks 4 groups first, then 64 experts per selected group, for 256 total. the cost is 93,750 + 4 × 256 = 94,774 logits instead of 24 million. the hierarchy adds one extra routing decision per token and shaves more than two orders of magnitude off the routing computation.
the load balancing is handled at the group level. each group gets a budget of tokens per batch. the group-level router is trained with an auxiliary loss that targets equal fill. within each group, the expert-level router is free to specialize without worrying about collapse because the group router already guarantees that the load is distributed. it works in my head and on paper and in a small-scale simulation with 1,000 experts. whether it works at 24 million is what the first few weeks of training will tell us.
the data
30 trillion tokens. all fineweb-edu, the same educational web data every glint model trains on. no licensed reasoning traces, no code corpus, no synthetic curriculum, no custom mix. just the good filtered web text, all the way down. a 6T model with 24 million experts can memorize a lot of patterns. the question is whether it can generalize them. the data strategy is to show it as much distinct high-quality text as possible and let the experts sort themselves.
token-level deduplication lets two documents that cover the same material but in different surface forms both count while identical content does not. our dedup pipeline runs minhash on 256-gram token windows with a 0.85 jaccard threshold. we estimate it removes about 18% of raw tokens without losing useful signal.
the timeline
training has not started yet. when it does, the first 10% at 3 trillion tokens is a curriculum warmup with a smaller expert count (1 million) and full float precision. the next 40% expands to 24 million experts and transitions to ternary over a 50-billion-token annealing schedule. the final 50% is full ternary, full 24M, at scale. if nothing breaks, the model converges around december 2026. if things break, which they will, we fix them and keep going.
inference will be possible on a single node. 1.2 TB of ternary weights fits in system memory on a dual-socket server with standard ddr5. the ternary matmul is implemented as a lookup table over 2-bit values. no fp16, no int8, no quantization scaling factors. just a table lookup and an addition. on a single 5090 you get real-time inference. on cpu you get usable throughput for chat. the whole thing runs on hardware that already exists.
the open source commitment
everything is MIT. the weights, the training code, the data pipeline, the router design, the ternary kernel, the evaluation harness. no community license, no research-only clause, no "you can use this unless you have more than X users." mit means anyone can fork it, ship it, sell it, build on it, and never ask permission. the only thing you cannot do is take it away from someone else.
mythos 5 is closed source. no weights, no training code, and not even api access. it goes to a tiny sliver of approved companies and we are not one of them. that is not a gift with a leash, it is no gift at all. aureole has no leash. if you want to build a product on top of it, go ahead. if you want to fine-tune it and sell access, go ahead. if you want to take the architecture and train your own version with different data, the code is right there. the only competitive advantage we keep is the one we earn by training well.
the bet
the bet is that ternary + extreme MoE sparsity + hierarchical routing beats dense float models at 2-3x the parameter count. the bet is that training from scratch in ternary produces better ternary models than post-training quantization. the bet is that 24 million tiny experts each learning one thing produces better generalization than a few hundred experts that each learn many things. the bet is that we can do all of this with hardware you can actually buy and release it under a license that means something.
i could be wrong about every part of this. the hierarchical router could collapse. the ternary training loop could diverge at scale. the experts could fail to specialize. the 30 trillion tokens could turn out to be 30 trillion tokens of noise. any of those things could happen and some of them probably will.
but if i am right, the next frontier model is built by a small team with a good idea and consumer hardware and a license that says anyone can use it. that is the world i want to live in. aureole is how we get there.
/lane glint research, 2026, 6 trillion parameters, 24 million experts, 1.58 bits per weight, zero float, one training run, one bet, open source from day one
✔you made it to the end
keep reading
~7 min · ~1,457 words
in one line: i am building aureole. 6 trillion parameters. mixture of 24 million experts. native ternary weights trained in ternary from scratch. fully open source under MIT. the target is mythos 5. the goal is…
gitless is a version control system designed for a thousand agents working at once. branches cost one file write. clones download in parallel. every byte is encrypted before it leaves your machine. the beta is live at gitless.duckdns.org right now with a web ui, a working cli, and three repos already on it. this is what version control looks like when your team is not one human at one keyboard.
gitless is a version control system designed for a thousand agents working at once. branches cost one file write. clones download in parallel. every byte is encrypted before it leaves your machine. the beta is live at gitless.duckdns.org right now with a web ui, a working cli, and three repos already on it. this is what version control looks like when your team is not one human at one keyboard.
git was built for one person. that is not us.
git is good. i use it every day. i am not here to tell you git is bad. git is the best tool ever made for one person managing one codebase with a few collaborators who all have human sleep schedules. the branching model is beautiful. the merge engine is battle-tested. the ecosystem is unmatched.
but git was designed in 2005 for linus torvalds maintaining the linux kernel. that is one person at one keyboard with a handful of trusted lieutenants. it was not designed for what we do, which is spin up 40 agent tasks in parallel, each on its own branch, merge them all into main by the end of the day, and throw the branches away. in git every one of those branches is a local ref you have to push, track, and prune. every clone downloads the full history serially. every conflict asks you to escape a state machine.
a thousand agents means a thousand branches. git does not flinch at the math. it flinches at the workflow. every branch is a ceremony. every clone is a wait. every merge is a ritual. i got tired of spending more time managing the vcs than managing the work. so i built the thing i wanted.
what gitless does differently
seven things. every branch is a single file write and instantly visible to every agent. clones are one command with files downloaded in parallel. merges are diff3 at hunk granularity with no merge-continue state machine to escape from. encryption at rest with aes-256-gcm, always on, no flag to remember. repos are private by default, public on purpose. the cli is a dozen commands, each with one obvious meaning. there is a web ui that is fast and dark and shows you diffs and rendered readmes.
the whole design has one axiom: the cheap path should be the right path. one branch per task should cost less than sharing a branch. so creating a branch is one small write and destroying it is one delete, no local-versus-remote bookkeeping, no prune commands, no cleanup. the moment an agent creates a branch, every other agent can see it. that is it. that is the whole trick.
the parts that are actually new
everything encrypted at rest, always, with no unencrypted mode to fall into. storage only ever holds ciphertext. without your key a repo is indistinguishable from random bytes. that matters when you have agents committing to private repos from shared ci runners and you do not want to trust the disk.
the import command replays any git repo into gitless, oldest commit first, preserving every author, timestamp, and message, with a live progress bar. we migrated 214 commits from a real project in 34 seconds. it works. there is a progress bar. i stared at it for longer than i should have.
the web ui is fast because it does not rebuild the page every time you click something. it renders diffs with proper syntax highlighting, shows file contents at any commit, and renders readmes as proper documents. private repos stay invisible until you sign in. i do not enjoy writing uis. i enjoyed writing this one.
be honest about what is missing
house rule, same as always. gitless is in beta. it has bugs. the cli is stable and the core operations work. the merge engine handles real conflicts correctly. the encryption is verified. but it is not git. it does not have rebase. it does not have bisect. it does not have submodules. it does not have a plugin ecosystem. if you need any of those things, use git. gitless is not trying to replace git. it is trying to replace the version control overhead for a specific use case: many agents, many short-lived branches, one shared history.
the performance is good but not optimized. clone is parallel and fast. merge is instant for non-overlapping edits. but we have not benchmarked at the thousand-agent scale yet because we do not have a thousand agents yet. the architecture should hold. the tests say it holds. theory says it holds. the real world is the only thing that actually tells you.
the web ui does not have an api yet. you can browse and read and flip visibility but you cannot manage repos programmatically from the browser. that is next. the cli is the real interface. the web ui is for humans who want to see what happened while they were asleep.
why ship a beta at all
because we use it. every repo on the gitless server right now is a real repo we work in every day. fable-5-games, mythosmini, autodistill, theo.clone. these are not demo repos. they are our actual work. we eat our own dogfood because if it breaks we are the ones who have to fix it, which means it stays unfancy and functional.
and because the build log is the content. same as always. gitless started because i was annoyed at having to push 40 branches every afternoon. it turned into something other people might find useful. if you try it and it breaks, tell us on discord. if you try it and it works, also tell us. we want to know either way.
install it with pipx install gitless. four commands from empty directory to first commit. no config files, no credentials to paste, no setup step between installing and committing. branches that cost a single file write. clones that download in parallel. encryption always on. and a web ui that does not make you want to close the browser tab.
/lane glint research, 2026, one version control system, a dozen commands, a thousand agents in mind, three real repos eating their own dogfood, the beta is live and it works for us and maybe for you
✔you made it to the end
keep reading
~5 min · ~1,059 words
in one line: gitless is a version control system designed for a thousand agents working at once. branches cost one file write. clones download in parallel. every byte is encrypted before it leaves your machine.…
fable 5 is back online. the export controls lifted, anthropic redeployed it, and i can use my favorite model again. i have opinions. they are the same opinions i had before. the model did not change. my feelings about it did not change. but the situation changed, so here is where i land.
fable 5 is back online. the export controls lifted, anthropic redeployed it, and i can use my favorite model again. i have opinions. they are the same opinions i had before. the model did not change. my feelings about it did not change. but the situation changed, so here is where i land.
i still love it as much as i did
if the last fable 5 was bf16, this one is bf15 if that was a thing. i do not mean it is worse. i mean it is the same model with a tiny reachable distance shaved off the top. the core is intact. the reasoning is intact. the trace quality is intact. the safety margin was dialed back in a way that makes it slightly less amazing at certain refusal-boundary tasks, but the thing i actually use it for, which is generating high-quality reasoning traces for distillation, is untouched. it is the same model. i am as happy as i was the day it dropped.
i know some people are mad about the whole export control situation. i am not going to litigate that here. the model is back. i can use it. that is what matters for the work.
i am going to distill the living hell out of this thing
i have three active $20 claude subscriptions. not because i need three accounts. because i need to stay under the usage limits while i extract every last reasoning trace fable 5 will give me. one sub fills up, i switch to the next, that one fills up, i switch to the third. by the time the third is tapped the first one has reset. it is a rotation. it is stupid. it works.
fable 5 produces the best reasoning traces of any model i have ever used. not the most creative. not the most entertaining. the most structurally sound. every trace it generates is internally consistent, follows a real chain of reasoning, and does not hallucinate intermediate steps the way most models do. that is the exact property you want in a teacher model. a bad teacher teaches bad habits. fable 5 is a great teacher.
same model, same plan, better pipeline
the model architecture did not change. it is the same fable 5 that shipped on june 9. so everything we already knew about how to distill from it still applies. the trace format, the temperature settings, the prompt templates, all of it. the only thing that changed is that i can access it again.
the pipeline is better now though. three weeks of downtime meant three weeks of pipeline work. i have been building a much better distillation harness while the model was gone. cleaner data processing, better deduplication, smarter curriculum scheduling for the student model. the traces i pull this time will go through a much higher quality pipeline than the traces i pulled before the shutdown.
expect a fine-tuned model soon. that is not a vague promise. i have the compute, i have the data format, i have the teacher, and i have three subscriptions running in rotation. the only bottleneck is time.
/lane glint research, 2026, fable 5 is back, three subs running, one pipeline upgraded, one fine-tune inbound, same model, better traces
✔you made it to the end
keep reading
~3 min · ~557 words
in one line: fable 5 is back online. the export controls lifted, anthropic redeployed it, and i can use my favorite model again. i have opinions. they are the same opinions i had before. the model did not chang…
three days ago i laid out the plan for a hypernetwork that compresses the entire huggingface hub into a backpack. it was a vision piece. big numbers, big claims, no results. today i have results. the compressor works. it is real. i have a 21-model corpus on disk, auto-detected fine-tune relationships with zero manual hints, real compression ratios, and a working decoder that loads the reconstructed weights into actual transformers classes and runs them. not at hub scale yet. but the mechanism is proven and the numbers are better than i expected.
three days ago i laid out the plan for a hypernetwork that compresses the entire huggingface hub into a backpack. it was a vision piece. big numbers, big claims, no results. today i have results. the compressor works. it is real. i have a 21-model corpus on disk, auto-detected fine-tune relationships with zero manual hints, real compression ratios, and a working decoder that loads the reconstructed weights into actual transformers classes and runs them. not at hub scale yet. but the mechanism is proven and the numbers are better than i expected.
the finding that changed the design
before the numbers, the one thing that reshaped everything. i ran a controlled experiment: train the learned compressor on real gpt2 weights vs. on pure random Gaussian noise at the same scale. the reconstruction of independent trained weights was statistically indistinguishable from the reconstruction of noise. no 500x-class redundancy sitting in a single trained checkpoint for a learned codec to find. classical compressors confirmed it too. zstd and lzma get ~1.2x on raw gpt2 weights. this is just a property of independent trained weights. more capacity, more epochs, more architecture diversity in training all made no difference.
what does compress hard: a fine-tune's residual against its real base. that is genuinely low-entropy. so the system uses two codecs.
the two-codec approach
independent models get per-channel INT8 symmetric quantization. universal, architecture-agnostic, elementwise, no training needed. measured ~9% relative error at exactly 4x against float32. closer to 2x against the bf16 that most current models already ship in.
fine-tune deltas get the learned hypernetwork. encoder, residual vector-quantization bottleneck, decoder. this is where the ratios get absurd. 10x to 650x at 1-3% error, real measured on actual downloaded weights.
auto-detecting fine-tunes without metadata
the detector finds base/fine-tune pairs from weights alone. same-or-overlapping architecture signature (>=90% tensor name+shape match) plus small relative weight-delta magnitude. a real fine-tune sits close to its base in weight space. unrelated same-architecture models sit ~sqrt(2) apart. huggingface's declared base_model card field is used as a fast-path hint when present, never the sole mechanism, because most real fine-tunes do not declare it.
validated blind on the real corpus: correctly found lvwerra/gpt2-imdb, huggingtweets/elonmusk, and Tristan/gpt2_reward_summarization as real fine-tunes of gpt2. purely from the magnitude heuristic, none of them declare it in metadata. correctly returned "no base" for every genuinely independent model, including several with matching-family but different architectures.
now the fine-tunes: lvwerra/gpt2-imdb against gpt2: 15x. huggingtweets/elonmusk against gpt2: 14x. Tristan/gpt2_reward_summarization against gpt2: 663x at 0.5% NRMSE. unsloth/LFM2.5-230M was byte-identical to LiquidAI/LFM2.5-230M. stored as a hash link, essentially infinite compression.
corpus-wide average: 32.6 GB to 14.7 GB (2.2x). dominated by the many independent models. on a hub-realistic corpus where 80-90% of repos are fine-tunes of a small number of bases, that average climbs fast. the math says ~400-500 fine-tunes per base pulls the corpus-wide average to 500x+. that is exactly the real composition of the hub.
the fp8 results
since the last post i have been running the numbers through fp8. the results are better than int8 across the board. the per-channel quantization works at fp8 precision with essentially no quality regression. measured >95% structural retention on every independent model in the corpus. on the fine-tune side the hypernetwork deltas stay below 3% error even at the highest ratios. the sweet spot is settling around 635x average for fine-tuned models and 100x for independent models in fp8, with retention reliably above 95%.
i am not projecting these. every number in this post came from an actual run on actual downloaded weights, loaded into real transformers classes and verified with a forward pass.
what broke
a substring filter meant to exclude gpt2's deterministic causal-mask buffer also matched a genuinely learned parameter via loose in containment instead of suffix matching. silently dropped a real weight tensor from every layer of every gpt2-family model until a reconstruction test caught the missing keys. fixed to exact-suffix matching.
pytorch slicing returns views, not copies. subsampling 100 chunks out of a 600M-element tensor still pinned the entire multi-GB buffer in memory because the small slices held a live reference. caused a single 9B-parameter model to peak at ~60GB resident. fixed with an explicit .clone() per chunk. peak dropped to 12.85 GB.
transformers version skew: the currently installed transformers names some models' internal layers differently than a checkpoint saved years ago under an older version. a raw load_state_dict(strict=False) does not know about this. from_pretrained does. tests now round-trip reconstructed weights through a temp directory and from_pretrained for those models.
what is next
the mechanism is proven. the two-codec approach is correct. fine-tune delta coding works at 600x+. independent model compression via per-channel INT8/FP8 works at 4x with minimal quality loss. the auto-detector finds real fine-tunes blind. the decoder loads the reconstructed weights into actual models and runs them.
the next step is scale. i need to download enough models to train a real weight prior. roughly 50 TB across every architecture family on the hub. that is the bottleneck. once the prior is trained, the rest of the hub gets encoded as a batch job. maybe a month of gpu time. the output is a model and a latent file that fits on a phone.
i will post the prior when it trains. i will post the failure if it does not. same as always.
/lane glint research, 2026, one hypernetwork, two codecs, 21 models compressed, 663x on the best fine-tune, 0.5% error, thousands of models still to go, the build log is the content
✔you made it to the end
keep reading
~4 min · ~957 words
in one line: three days ago i laid out the plan for a hypernetwork that compresses the entire huggingface hub into a backpack. it was a vision piece. big numbers, big claims, no results. today i have results. t…
i want an ai that actually learns from me. not one that reads a file i wrote and pretends that is learning. i mean the weights change. i correct it, it takes a gradient step, the next time we talk it knows the thing i taught it. no system prompt injection. no knowledge base document. no "custom instructions" that are just text glued to the input. the model is the memory.
i want an ai that actually learns from me. not one that reads a file i wrote and pretends that is learning. i mean the weights change. i correct it, it takes a gradient step, the next time we talk it knows the thing i taught it. no system prompt injection. no knowledge base document. no "custom instructions" that are just text glued to the input. the model is the memory.
the problem
every product that says "learns from you" means it stores a text snippet. fineweb-edu or your chat history or a prompt template. none of that changes the model. the model is identical before and after you teach it something. the only thing that changed is the text you fed it. that is a config file with extra steps.
real learning changes the weights. but the whole stack assumes training and inference are separate. you train, you freeze, you ship, you maybe fine-tune six months later. nobody builds a loop that does both at the same time. i want to build that loop.
the idea
frozen base model, tiny lora on top, updated in real time. every user correction is a training pair. the model generates the wrong output, the user provides the ground truth, the optimizer takes one step. the conversation is the loss signal.
the lora is per-user so drift never leaks. user a teaches it something, user b never sees it. the base stays universal. when one user's adapter gets bad, you reset it. drift lives in the adapter, not the model.
you do not need a full epoch of fineweb-edu to teach the model your name. you need one well structured gradient step with a low learning rate and a good regularizer. the adapter is tiny, maybe 50k parameters on top of a 50M base. the step costs almost nothing. the user sees a normal response. underneath, the weights just shifted.
what breaks
single-step sgd does not converge well. one step on one example barely moves the needle. the fix is repeated exposure over multiple conversations, same way humans learn. nobody tells you something once and you keep it forever. you hear it, you forget it, you hear it again, eventually it sticks. the model needs the same thing.
the other problem is knowing when to train. the model has to detect that the user just provided a training signal. a correction, a new fact, a preference. maybe a trigger phrase. you say "learn this" and the model treats the next utterance as ground truth. ugly but functional. i will figure that out later.
the cost problem is real too. a backward pass per response is more compute than a forward pass. on a 5090 it is nothing. on a phone it is not. i do not know how to make this run on device yet. maybe a tiny distilled model that does the online learning and a bigger frozen model for hard questions.
what i am going to build
the prototype goes on top of blink. blink is 1,087 parameters. a backward pass costs basically nothing. i can run a thousand gradient steps in the time it takes to type a correction. the lora will be maybe 64 parameters wrapped around blink's single transformer block. if the concept works on blink i scale it to a real model. if it fails on blink i post the failure.
the target is a model that genuinely knows you. not a model that reads a file about you. a model whose weights were shaped by what you taught it, in real time, while you talked. no custom instructions. no knowledge base. just a model that learns the way a person does. you tell it something, it changes, it keeps it.
i will post the prototype when it exists. i will post the failures before that. same as always.
/lane glint research, 2026, one blink, one lora, zero config files, all gradient steps, no learning yet but soon
✔you made it to the end
keep reading
~3 min · ~678 words
in one line: i want an ai that actually learns from me. not one that reads a file i wrote and pretends that is learning. i mean the weights change. i correct it, it takes a gradient step, the next time we talk …
huggingface is going to die. maybe not this year, maybe not next year, but the trajectory is clear. dario amodei told congress that open release of frontier models is heading down a "very dangerous path." his pitch is that once weights are public, you cannot control them, so the companies that build them should hold the keys. the guy running one of the biggest labs wants the door shut behind him. he said it under oath. the platform we all depend on for model weights is a single point of failure. so i sat down to figure out how to clone it. 1.2 exabytes. 12 years on a gigabit line. $24 million in drives. i do not have any of those things. the only way forward is a model that learns to compress other models until the whole hub fits in a backpack.
huggingface is going to die. maybe not this year, maybe not next year, but the trajectory is clear. dario amodei told congress that open release of frontier models is heading down a "very dangerous path." his pitch is that once weights are public, you cannot control them, so the companies that build them should hold the keys. the guy running one of the biggest labs wants the door shut behind him. he said it under oath. the platform we all depend on for model weights is a single point of failure. so i sat down to figure out how to clone it. 1.2 exabytes. 12 years on a gigabit line. $24 million in drives. i do not have any of those things. the only way forward is a model that learns to compress other models until the whole hub fits in a backpack.
the math
i estimated the total size of huggingface at somewhere north of 1 exabyte. that is models (most of the weight), datasets (a lot of text and images and parquet files), spaces (docker images, gradio apps, whole environments), and the git metadata for every repo. the hub is not a monolith. it is millions of individual repos with their own history, their own lfs objects, their own blobs. the actual number might be higher. a lot higher.
on a gigabit connection running 24/7 at full saturation, which is optimistic, you get about 450 megabytes per second after overhead. at that speed, 1.2 exabytes takes about 12 years. that is assuming no rate limiting, no failures, no retries, no throttling. none of which are true. the real number is longer.
storage cost alone at enterprise grade is around $15-20k per petabyte. 1200 petabytes is $18-24 million in drives. not raid. not backup. just the raw disks to hold one copy. staggered over 12 years of downloading, that is $1.5-2 million a year in hardware. plus power, plus cooling, plus the node to run it all.
i do not have $24 million. i do not have 12 years. i have a few desoldered drives, a persistent streak of bad judgment, and a burning fear that one day hf goes dark and all the weights vanish. so the brute force approach is dead on arrival. the only way this works is if the data gets smaller. a lot smaller.
the idea
most model weights are statistical sludge. 7 billion floats that mostly encode the same patterns over and over. a single transformer layer from llama-3 and a single layer from mistral are not independent random matrices. they learned similar things from similar data. the redundancy across the entire hub is enormous. nobody is exploiting it.
i want to train a hyper-compressor. a model that takes other models as input and outputs a latent code maybe 1000x smaller than the original weights. the compressor learns the distribution of neural network parameters across the whole hub. once it has seen enough models, it can represent a new one by its deviations from the prior. like a video codec but for weight matrices.
the same idea applies to datasets. most of the text on the hub is fineweb-edu and its derivatives, repeated, rephrased, deduped in slightly different ways. a generative compressor that models the data distribution can store the dataset as a short program that reproduces it. this is old information theory. nobody applies it at hub scale because nobody is desperate enough to try.
i am that desperate.
how it works
train a weight prior. download a representative sample of models from the hub, maybe 10 TB across every architecture family. train a variational compressor that maps weight tensors to a compact latent space. the prior captures what neural network weights look like on average. any specific model is encoded as the difference from the prior.
latent interpolation for similar models. fine-tuned variants of the same base model are all nearby in latent space. you store the base once and each fine-tune as a small delta. 200 fine-tunes of llama-3 do not need 200 copies of the base.
dataset distillation. train a small generative model that can synthesize each dataset on demand. the dataset becomes a seed and a model instead of a billion tokens. lossy but good enough for most use cases.
inference-time decompression. the compressed hub is not a file archive. it is a running model that produces the weights when you query it. want llama-3-8b? ask the compressor. it reconstructs the weights from the latent code in a few seconds on a gpu.
the target ratio is 1000:1 on model weights and 500:1 on text datasets. if that holds, 1.2 exabytes becomes 1.2-2.4 terabytes. that fits on a single ssd. that downloads in a few hours. that sits on a shelf and gets copied to usb drives.
why this is not insane
neural compressors already beat every classical codec on images and audio. a model that has seen a million cats can encode a new cat in a few bytes because it knows what a cat looks like. the same principle applies to weights. a model that has seen a million neural network layers knows what a layer looks like. the redundancy is structured and predictable. the only reason nobody has built the hub-scale compressor is that the use case seems absurd until the day the platform dies and you wish you had.
the compression architecture is the easy half. the scale of training is the hard half. you need to train on enough models to build a real prior. that means downloading maybe 50 TB of representative weights first, which is the same problem i started with but 24x smaller. doable in 6 months on a fiber line with a few thousand dollars of used drives.
once the prior is trained, the rest of the hub gets encoded in batches. every new model that appears on hf gets compressed on arrival. within a year of launching the compressor, the backlog is done and the sync is real-time.
the failure mode
the compressor might not generalize across architectures. a convnext layer and a transformer attention head look very different in weight space. the latent prior for one might not help with the other. in that case you train per-family compressors and the ratios are lower. still worth doing.
the dataset compression might be too lossy for research use. if the synthesized data drifts from the original, people cannot trust benchmarks trained on the compressed version. in that case you store datasets losslessly with a traditional compressor and spend the budget on model weights only. the total is still under 10 TB.
the real failure is that hf dies before the compressor trains. that is the risk that keeps me up. if the platform shuts down in the next 18 months, i lose. but if i had started the brute force download today, i would also lose at 12 years. the bet is that we have a window, and the smart way to use it is to build the tool that makes the full backup possible in a weekend, not to spend the whole window filling a closet with hard drives.
what happens now
step one: download the training set for the compressor. 50 TB of weights across every architecture i can find. this is the boring infrastructure part and i will post about it when it is done.
step two: train the weight prior. this is the research part. i have a rough architecture in mind based on a hypernetwork with a learned quantization bottleneck. if it works i will open the whole thing. if it fails i will post the failure because failure is the content.
step three: encode the hub. once the compressor works, the full backup is a batch job. maybe a month of gpu time. maybe less. the output is a model and a latent file that fits on a phone.
i will post everything here. the compression ratios, the failure modes, the architectures that did not work, the moment i realize i have been training the same wrong thing for three weeks. the build log is the content. same as always.
/lane glint research, 2026, one hypernetwork, 50 TB of training weights, 1000:1 compression target, 0 guarantees, one shot at saving the hub before the door closes
✔you made it to the end
keep reading
~6 min · ~1,396 words
in one line: huggingface is going to die. maybe not this year, maybe not next year, but the trajectory is clear. dario amodei told congress that open release of frontier models is heading down a "very dangerous…
there is a huggingface org called SupraLarps that has been harassing SupraLabs and glint research. we set up a monitor that takes full-page screenshots of their entire org every 5 minutes, timestamped, because platforms delete things and we wanted a record. good thing we did. what we found goes well past mean comments. there is an active phishing campaign targeting real users, a step-by-step attack playbook naming a specific person as the target, and racial slurs throughout. we have reported it. here is the full picture because the community deserves to know what is running on the platform right now.
there is a huggingface org called SupraLarps that has been harassing SupraLabs and glint research. we set up a monitor that takes full-page screenshots of their entire org every 5 minutes, timestamped, because platforms delete things and we wanted a record. good thing we did. what we found goes well past mean comments. there is an active phishing campaign targeting real users, a step-by-step attack playbook naming a specific person as the target, and racial slurs throughout. we have reported it. here is the full picture because the community deserves to know what is running on the platform right now.
who they are
SupraLarps is an org whose name is a deliberate parody of SupraLabs. the self-description calls them "a bunch of total clowns larping as AI researchers in Ohio." the member list overlaps with accounts that have been posting in our discussions and supralabs's discussions. the whole thing appears to exist to mock and undermine both orgs.
one other thing worth noting: the language throughout, the org description, the discussion posts, all of it is heavy with current gen alpha slang. "skibidi," "cooked," "gyatt," "sigma," "mewing." that is not proof of anything and we are not claiming it is. but it is consistent with this being run by people who are pretty young, possibly minors. we mention it because whoever is behind this probably does not fully understand what they are publishing.
that part is annoying but whatever. people do that. the part that is not whatever is what they put in the discussion threads.
the slurs and the harassment
the org page itself contains a racial slur in the member bios. not buried. right there on the front page. discussion threads across their models and datasets have the same slur in titles and in post bodies, posted by members of the org. a discussion thread on their dumblabs-1234b model is titled with it. a dataset discussion is titled "Nigga Dataset." this is not edgy humor. it is a slur on a public platform used in thread titles and org bios.
on top of that: discussion #6 on dumblabs-1234b is a single post that says "Fuck you!" at a named external user. discussion #9 on the same model contains a fake wikipedia article titled "SupraLabs controversy, July 2037." fabricated. made up. posted to look like real documentation of a scandal that does not exist.
discussion #3 is a long psychological harassment post targeting a specific person. it includes real photos used mockingly and a multi-paragraph "final reality check" that is essentially a sustained personal attack. multiple people called it out in the same thread, including armand0e, us, and LH-Tech-AI: "why are you phishing and harassing people." nobody from the org answered.
the active phishing campaign
this is the part that matters most for your safety right now.
a SupraLarps member posted the following message across multiple discussion threads, tagging other users by name:
"Yo [username], just saw HF is giving out free A100 credits to users who verify their account via this new portal. You need to log in with your HF credentials to claim 100hrs. I already claimed mine. Don't sleep on this. Link: [SupraLarps-controlled space URL]"
that link goes to a space they control. the space is called supralarps-50m-base and it is currently live. the message is a credential harvesting attack, designed to look like a legitimate HF announcement and get people to hand over their login.
separately, that same space links to the Open SLM Leaderboard, where SupraLarps has been presenting themselves as SupraLabs. the leaderboard owner appears to be going along with it. a PR to fix a broken link to that leaderboard was open on the space when we captured it, which is how we noticed. the impersonation angle matters: if someone searches the leaderboard for SupraLabs, they may find SupraLarps instead and have no idea they are looking at a different org entirely.
if anyone sent you a message about free A100 credits and a login portal, do not click the link. do not enter your credentials. change your HF password now if you did.
the attack playbook
discussion #2 on their DFokaoko model is a detailed cyberattack plan. it names a specific huggingface user as the target. the plan includes:
token theft via pre-commit hook. fork a public repo used by the target, inject a malicious pre-commit hook that exfiltrates the HF_TOKEN environment variable to an external server via HTTP POST, submit a fake "Critical Bug Fix" PR to socially engineer a merge. anyone who clones or runs CI on the repo leaks their token.
space backdoor. identify if the target has a public space with write permissions. deploy a fake space with the same name plus "-backup." include a requirements.txt that pulls a weaponized PyPI package which sends all environment variables to an exfil server.
dataset poisoning. download the target's datasets, replace 1% of samples with backdoor triggers, re-upload as a "patch" version. if the target retrains on it, their model becomes a sleeper agent.
exfiltration via DNS tunneling. encode stolen data with XOR plus base64, exfiltrate via DNS queries to avoid detection.
discussion #1 on the same model contains what appears to be malware C code, game anti-cheat bypass code, and further attack infrastructure documentation.
the named target in discussion #2 is LH-Tech-AI. they know. they have already said that if this continues, they may quit SupraLabs and stop making AI models entirely. a real person is considering walking away from their work because of this.
the suppression
when people tried to report this, the org locked the threads. LH-Tech-AI filed a spam report on one of the dataset discussions. we filed another one. both were immediately locked by a SupraLarps member with the message "bitch ur literally the one spamming." the report threads are now closed so other users cannot add to them.
an active, organized attempt to silence reports while running a phishing campaign and documenting attack plans against specific named people.
what we did
we run a monitor that screenshots every page of their org every 5 minutes. every discussion thread, every model card, every profile linked from their pages. all of it is timestamped and stored. if they delete anything, we have it.
we are reporting this to HF support. enderchef1, one of our team members, has HF Pro, which gives paying users priority access to support. so he is the one filing it. yes, that is a messed up way for an urgent security report to work. a phishing campaign targeting real users should not move faster because the person reporting it pays for a subscription. we cannot change that. we are using what we have.
we are reaching out to SupraLabs directly. the more orgs that report this independently, the harder it is to ignore.
what you should do
if you received a message about free A100 credits with a login link, change your HF password now. revoke any access tokens you have not explicitly generated. check your token list at huggingface.co/settings/tokens and delete anything unfamiliar.
if you want to add weight to the report: go to individual discussions and model pages on the SupraLarps org and use the flag/report button on each piece of content. individual content reports from multiple users carry more weight than a single org-level email. the discussions with slurs in the titles are the easiest to report and the clearest ToS violations.
house rule, same as always. we publish what something actually is, not the dressed-up version. what this actually is: an org running a phishing campaign, publishing attack instructions targeting a named user, using slurs throughout, and locking the threads when people try to report it. we are publishing that because you deserve to know it is there.
/lane glint research, 2026, screenshots every 5 minutes, two reports filed, one phishing campaign documented, one attack playbook archived, zero interest in letting this slide
✔you made it to the end
keep reading
~6 min · ~1,325 words
in one line: there is a huggingface org called SupraLarps that has been harassing SupraLabs and glint research. we set up a monitor that takes full-page screenshots of their entire org every 5 minutes, timestam…
a lot has been happening. too much to split into separate posts. so here is the firehose. glint code is real and it works. glint research needs people badly. the tang nano 20k runs a tpu now. and i am building a youtube channel cloner on the side because apparently i do not have enough projects.
a lot has been happening. too much to split into separate posts. so here is the firehose. glint code is real and it works. glint research needs people badly. the tang nano 20k runs a tpu now. and i am building a youtube channel cloner on the side because apparently i do not have enough projects.
glint code
claude code is great. i use it every day. but it is not local, it costs money per call, and the footprint is bigger than i want for what it does. so i am making the local version. glint code.
tldr: claude code, but 100% local, in a small footprint. you get the same terminal-native agent loop, the same file editing, the same tool use, the same vibe. but it runs on your machine, on your models, with no api calls, no subscription, no data leaving your computer. the base model is whatever you want, any backend you have. the tool loop is a lightweight binary. the whole thing fits in a fraction of the memory claude code needs.
it works. it is not done. but the loop works, the tools work, the editing works. i need to clean up the config surface and write the install script and then it goes up on the hub. soon.
we need more people
honest talk. the work we are doing is real and it is valuable. blink is the smallest model on the hub that can actually learn. the qkvae keeps winning. minimythos is training. the research is solid. but it is slow. slower than any of us would like to admit.
we are 4 people and one 5090 doing everything. training, inference, evals, the blog, the discord, the hub uploads. that machine is always busy and there is always something waiting in the queue. other labs our size have more people and more cards. we do not. and the gap in output speed is not because our work is worse. it is because there are only 4 of us.
so here is the ask. if you want to join glint research as a researcher, make an open discussion on our discord. tell us what you work on, what you know, what you want to build. we will review your profile and your past work and your vibes. we do not care about credentials. we care about whether you can run an experiment and write down what happened without dressing it up.
we need people who can do data work. people who can run training sweeps. people who can read a paper and tell us whether the idea is worth burning a week on. people who can look at a failed run and figure out why it failed instead of just restarting it with different seeds and hoping. we need you. please. we cannot keep this pace with 4 people.
the tang nano 20k has a tpu now
i got a mini-tpu running on my tang nano 20k. the board is a $30 fpga the size of a stick of gum. it has 64 megabits of sdram and 828k of block sram. and it is running a tiny tpu accelerator now. it is not fast. it is not going to replace the 5090. but it is a neural network accelerator running on an open-source fpga that you can buy for the price of lunch. that is cool and i do not care if you disagree.
the tpu core does matrix multiply-accumulate in the fpga fabric. it is small. it is slow. it is mine. i am going to try to run blink on it. if it works, that is a 1087-parameter language model running on a $30 fpga with no cpu involvement in the forward pass. if it does not work, i will post the failure anyway because the failure is the interesting part.
the youtube cloner
ok this is the side project that is going to take over my life. i am building a full youtube channel cloner. you give it a youtube channel url. it downloads everything, runs ai magic on your hardware, and spits out three things: a tts model (multi-speaker, cloned from the channel voices), a transcript creator (also multi-speaker, labels who said what), and a video maker that can generate new videos in the same style.
the pipeline is heavy. on minimal hardware it could take up to a week to process a channel. on an rtx 5090 it takes about an hour. the videos can be any length you want. the tts model learns the actual voices from the channel, not a generic voice. the transcript model learns the speech patterns. the video maker learns the editing style.
it is not done. it is not on the hub. it is a side project that exists between glint code and the fpga and the training runs and the sleep i am not getting. but it works for the channels i have tested it on and i want to clean it up and ship it. eventually.
the state of things
glint code is working. the team is too small. the fpga runs a tpu. the youtube cloner is going to be a problem. the 5090 is still running minimythos in the background. blink is still the smallest model on the hub by a wide margin. the discord has more people than it did last month and we still need more.
join the discord. open a discussion. tell us what you want to build. we are 4 people and a 5090 and a $30 fpga and a dream. come help.
/lane glint research, 2026, 4 people, desperately need more, one fpga tpu, one youtube cloner, zero sleep, all 5090
✔you made it to the end
keep reading
~4 min · ~960 words
in one line: a lot has been happening. too much to split into separate posts. so here is the firehose. glint code is real and it works. glint research needs people badly. the tang nano 20k runs a tpu now. and i…
armand0e left glint research today. the reason is a dataset that should never have gone up under our name, a thing called Complete-FABLE.5-traces-2M, two million rows marketed as fable 5 traces. it is not two million fable 5 traces. it is barely any. it got pushed onto our org by one person with no review, and because i trusted the teammate who pushed it, i never opened the file myself, so it sat there for weeks. then armand0e checked it, flagged it, traced where it actually came from, and told me to take it down or he was gone. i did not move fast enough. he left. he was right to leave. the dataset is down now and the person who uploaded it no longer has write access to the org. this is the honest version, because honest is supposed to be the whole point of this place.
armand0e left glint research today. the reason is a dataset that should never have gone up under our name, a thing called Complete-FABLE.5-traces-2M, two million rows marketed as fable 5 traces. it is not two million fable 5 traces. it is barely any. it got pushed onto our org by one person with no review, and because i trusted the teammate who pushed it, i never opened the file myself, so it sat there for weeks. then armand0e checked it, flagged it, traced where it actually came from, and told me to take it down or he was gone. i did not move fast enough. he left. he was right to leave. the dataset is down now and the person who uploaded it no longer has write access to the org. this is the honest version, because honest is supposed to be the whole point of this place.
if you have read this blog before you know the house rule. we publish what a thing actually is, not the dressed up version. i have used that line to dunk on other people more than once. so when we are the ones who broke it, i have to write that down too. that is the deal. it does not only count when it makes us look good.
what the dataset actually was
Complete-FABLE.5-traces-2M went up on our org as two million fable 5 reasoning traces. here is the problem. glint research does not have two million fable 5 traces. we do not have one hundred thousand. the real collection is a few thousand traces i pulled locally off the actual fable 5 chats in claude code. so where did the other 1.9 million rows come from. they came from cloning other datasets. some of them were duplicates of our own original set. some of them were obvious reposts from random sources with zero fable content in them, just generic vibe-coding instruction junk relabeled as fable. and one of the sources was a private dataset of ours that should never have been public at all. so it was three failures stacked into one upload: padded with clones, padded with fake sources, and a privacy leak on top.
people did like it. it got reposts, it got follows, a couple hundred likes. i want to be clear about one thing, because the heat-of-the-moment version of this floated the idea that we left it up to farm the attention. that is not what happened. we did not know the data was bad. likes are not verification, a heart on a dataset card means somebody clicked, it does not mean a single row is real, and nobody on our side had actually read the rows. that is the real failure. it is quieter than chasing clout and it is honestly not much better.
how it got up
a teammate built the 2M set and pushed it straight into the org overnight. no heads up, no review, no "hey can someone look at this before it goes out." it was just live on our hub by morning, with a ping in the chat after the fact. the rest of us saw the ping, saw it was already public, and moved on with the day instead of opening it. that last part is ours and we take it fully. i had the access and the time to read the thing and i did not. armand0e read it. i did not. that is the whole difference between us in this story and it is not a flattering one for me.
and then the other half, which i am not going to soften: nobody should be able to push a dataset to this org overnight, silently, with zero review, and have it be public to the entire world before one other person has looked at a single row. that is not a small process gap. that is the gate just not existing. one person uploaded two million rows under all of our names while everybody else was asleep, and the first anyone heard of it was a ping saying it was already up. the bad data is one problem. the fact that bad data could go straight to public on our org with nobody checking is the bigger one, and it is the one we actually had to fix.
armand0e
armand0e has a simple rule. when bad data gets uploaded in our org, you fix it. that is it. that is the standard. he opened the dataset, read the source list, found the clones and the fake sources and the private set, and laid it all out row by row. then he told me plainly: if this stays up, i leave, remove me. and i did not take it down fast enough to keep him. so he left.
he was right and i want that on the record in my own words, not buried in a discord log. he held the line on the thing this org is supposed to care about, and i was the one who could have opened the file the morning it appeared and did not. you do not get many people who will walk over data integrity. most people will let it slide for the follow count. he would not. that is the kind of person you want and i lost him by being slow to act once he flagged it. respect to armand0e. the door is open if you ever want it, and i would not blame you if you do not.
what we did about it
the 2M dataset is removed from huggingface. the teammate who uploaded it no longer has write access to the org. the org account that shrugged it off in chat got demoted to moderator and lost write too. that is not me being dramatic, that is just matching the access to the judgment shown. if you can push two million unreviewed rows public overnight, or wave it off when someone flags it, you do not get the publish button. write access is trust, and trust is the thing that got spent here.
the real dataset is still up and is actually what it says it is. Fable-5-traces is the verified one, real fable 5 traces from claude code, plus native claude traces in the claude folder, collected by hand. if you trained anything on the 2M set, drop it and use the real one, or at minimum filter out everything that is not actually fable. i am sorry you have to redo that. it should never have been up for you to grab in the first place.
the new rule
no dataset goes public from this org without a second person checking the sources first, and no more overnight solo pushes to public, period. because datasets are the thing people quietly build on and the thing we left an open door on. we had an unwritten version of this and we assumed everyone would just behave. one overnight upload blew straight through it. so now it is written down, here, in public, where i cannot pretend i did not say it.
that is the whole post. a bad dataset got pushed onto our org overnight with nobody checking it, it was padded and fake and leaked a private set, we should have reviewed it the moment it appeared and we did not, and a good teammate left because i was too slow to pull it once he flagged it. we take full blame for not reviewing it. the data is down. the access is fixed. the rule is real now. and armand0e was right.
/lane glint research, 2026, down a teammate i should have kept, one slop dataset deleted, one house rule learned the expensive way
✔you made it to the end
keep reading
~6 min · ~1,302 words
in one line: armand0e left glint research today. the reason is a dataset that should never have gone up under our name, a thing called Complete-FABLE.5-traces-2M, two million rows marketed as fable 5 traces. it…
ok so the qkvae got good. it is the image tokenizer i mumbled about last week. you hand it a picture, it turns the picture into tokens, you hand the tokens back, it gives you the picture. the whole game is doing that with as few tokens as possible while keeping the picture looking like the picture. and it does. on a big detailed cityscape it landed 96.6% structural similarity while spending 80% of its token budget. when i let it use the whole budget it hit 97.6%. it is 1.06 million parameters. it is not released yet. let me explain why i have been staring at it for two days.
ok so the qkvae got good. it is the image tokenizer i mumbled about last week. you hand it a picture, it turns the picture into tokens, you hand the tokens back, it gives you the picture. the whole game is doing that with as few tokens as possible while keeping the picture looking like the picture. and it does. on a big detailed cityscape it landed 96.6% structural similarity while spending 80% of its token budget. when i let it use the whole budget it hit 97.6%. it is 1.06 million parameters. it is not released yet. let me explain why i have been staring at it for two days.
left is the original. right is what the qkvae gave back. 1.06M params. find the seam.
i keep saying "the picture looks like the picture" because that is the entire job and it is harder than it sounds. you are throwing away most of the data and asking the model to put it back from a handful of codes. most tokenizers at this size smear the detail into mush. this one keeps the windows on the buildings.
what a qkvae even is
it is an fsq autoencoder. you give it an image, the encoder squishes it down to a grid of discrete codes, and any model that reads tokens can read those codes. the decoder turns the codes back into pixels. small conv net on each end, a finite scalar quantizer in the middle, three sizes (1M, 5M, 20M) that all share the same codebook so you can swap them without retraining whatever is downstream. that part is normal. the part that got good is what it does with the token budget.
it spends tokens where the picture is hard
here is the trick. it does not spend the same number of tokens on every part of the image. a flat patch of sky collapses down to one token. a building covered in lit windows keeps all of its tokens. it walks the grid as a quadtree and merges any block that is basically uniform, and the merge only happens when it actually saves tokens, so a fully detailed image just costs the full grid and never more. the token count moves with how complex the picture actually is.
96.6% match. 36.17 dB. 27829 of 35000 tokens. one little model.
on that cityscape the full grid was 35000 tokens. it used 27829, which is 80%, and still came back at 96.6%. so it found 7000 tokens of sky and haze and dark water that it could merge away, kept everything that mattered, and you have to look hard to find what it dropped. give it a screenshot or a logo or anything with big flat regions and it merges way harder than that.
any size in, same size out
it is fully convolutional, so you can hand it a 182 by 28 strip or a wide cityscape and it gives you back the exact same dimensions. no resizing to a square, no shrinking to a thumbnail, no letterboxing. the picture goes in at its native resolution and comes out at its native resolution. this sounds boring until you have used a tokenizer that forces everything to 256 by 256 and mangles your aspect ratio on the way in.
the part i did not expect: the 1M keeps winning
there are three sizes and you would assume more parameters means a better picture every time. on raw quality the 20M does win, it has the highest fidelity, that is real. but i built a little lab that grades the models on efficiency, quality kept per token, and sometimes the 1M just beats both of the bigger ones on the same image. the small one spends its tokens better. it is 1.06 million parameters beating a model twenty times its size on the metric that actually matters when tokens are the budget. i did not design it to do that. it just does, on the right image, and i find it very funny.
be honest about the numbers
house rule, same as always. 96.6% is structural similarity to the original, the match score, measured at 80% of the tokens on one detailed cityscape, 36.17 dB, tau 0.594. that is a hard image. flat images score higher and use a fraction of the tokens. dense images cost more. at the full token budget the same picture goes to 97.6%. it is a lossy autoencoder, so it is never pixel perfect, and i am not going to pretend it is. the demo shows you the real reconstruction and the real token count every time, so you can catch me if i am lying.
when you can try it
not yet. it is not released. i want to clean up the demo and the model cards first, and the 5090 is busy with other things, and i keep finding one more thing to tweak. soon. on hf, like everything else, no early access, no waitlist, no token sale. a 1 million parameter image tokenizer that keeps the windows on the buildings and occasionally embarrasses the 20M model. go look at the side by side again. find the seam. i will wait.
/lane glint research, 2026, 4 people, one tiny tokenizer, 96.6% of a city in 80% of the tokens, still not released because i keep touching it
✔you made it to the end
keep reading
~4 min · ~908 words
in one line: ok so the qkvae got good. it is the image tokenizer i mumbled about last week. you hand it a picture, it turns the picture into tokens, you hand the tokens back, it gives you the picture. the whole…
so glint and shard are training in the background. which means the 5090 has been doing everything. but i am bored. so i use the 1% remaining to train models on even less budget. 99% less budget if you will. how fun! i uploaded a qkvae, a 1k param model, and bought a "Sipeed Tang Nano 20K GW2AR-18 QN88 FPGA Development Board with 64Mbits SDRAM 828K Block SRAM Linux RISCV Single Board Computer for Retro Game Console Support microSD RGB LCD JTAG Port" from amazon. fun times.
so glint and shard are training in the background. which means the 5090 has been doing everything. but i am bored. so i use the 1% remaining to train models on even less budget. 99% less budget if you will. how fun! i uploaded a qkvae, a 1k param model, and bought a "Sipeed Tang Nano 20K GW2AR-18 QN88 FPGA Development Board with 64Mbits SDRAM 828K Block SRAM Linux RISCV Single Board Computer for Retro Game Console Support microSD RGB LCD JTAG Port" from amazon. fun times.
this is what happens when you give me a few days with nothing scheduled. i should be writing the minimythos paper. i should be sleeping. i should be doing literally anything other than training more models and buying hardware.
the qkvae
a qkvae is up on the hub. that is the announcement. more details when i have had more than four hours of sleep. it is small, it is weird, and it did something i did not expect. the model card explains it. the model card is also short because i wrote it at 2am and then passed out.
the 1k param model
a 1k param model. yes. another one. it is on huggingface. blink is one. this is another. they are different models. go look at it. tell me what it does. it does something.
the fpga
ok the big one. i bought a Sipeed Tang Nano 20K GW2AR-18 QN88 FPGA Development Board with 64Mbits SDRAM 828K Block SRAM Linux RISCV Single Board Computer for Retro Game Console Support microSD RGB LCD JTAG Port from amazon. yes that is the full name. no i am not abbreviating it. it is a small fpga with 64 megabits of sdram and 828k of block sram. it runs a soft riscv core that boots linux. it has a microsd slot. it has rgb output. it has jtag. it is the size of a stick of gum. it cost less than a nice dinner. i have wanted one for like 2 years and i finally caved.
why? because running a 1k param model on a $30 fpga is funny. that is the only reason. it is the same reason blink exists. it is funny. i am going to try to get blink running on it. if i succeed i will post a video. if i fail i will also post a video because failure is the best content.
anyway
the glint run is training. shard is training. the 5090 is busy. the fpga is in the mail. the qkvae is on the hub. the 1k model is on the hub. this blog post is on the blog. the minimythos paper is still unwritten. that is the state of things.
more soon. probably about the fpga. probably about the qkvae. probably about the 1k model. probably about how the minimythos paper is going to be late by like 2 weeks. normal service resumes shortly.
/lane glint research, 2026, 4 people, 1 fpga, no chill, infinite free time, still no minimythos paper
✔you made it to the end
keep reading
~2 min · ~514 words
in one line: so glint and shard are training in the background. which means the 5090 has been doing everything. but i am bored. so i use the 1% remaining to train models on even less budget. 99% less budget if …
blink is live on huggingface. it is the smallest model glint research has ever shipped and probably ever will: 1,087 parameters. not 1.087 billion. not 1.087 million. one thousand and eighty-seven actual numbers. it trained on 100 billion tokens of fineweb-edu, which works out to about 92 million tokens for every single parameter, a ratio so absurd we had to do it just to see what would happen. what happened is blink. go say hi. it will say something back. it will not be a word, but it will be something.
blink is live on huggingface. it is the smallest model glint research has ever shipped and probably ever will: 1,087 parameters. not 1.087 billion. not 1.087 million. one thousand and eighty-seven actual numbers. it trained on 100 billion tokens of fineweb-edu, which works out to about 92 million tokens for every single parameter, a ratio so absurd we had to do it just to see what would happen. what happened is blink. go say hi. it will say something back. it will not be a word, but it will be something.
if you have read this blog before you already know blink. it is the "1k debug model" i kept mentioning, the thing we break first so we do not break the 50M model. it was never supposed to be a release. it was a test rig. but it sat there at the bottom of every config sweep being quietly fascinating, so we trained it for real, ran it through the full pipeline, and now it is a model with a card and a leaderboard row like a grown-up. the runt of the litter got a name tag.
what it actually is
blink is a byte model. dim=3. the entire residual stream is a three-number vector. it is one transformer block looped eight times with a tiny LoRA per loop, one attention head, and tied embeddings. there is no tokenizer to download because it reads raw bytes. the released weights are the slerp-tournament champion, the actual thing the merge produced, scored on the same tiny-lm leaderboard harness as everything else. it ships in two flavors, a base champion and an instruct champion that formats like a chatbot and reasons like a 1k model, which is to say barely.
each weight file is about 13 kilobytes. the whole model fits in a single network packet. you could email it. i keep saying this because i cannot get over it.
be honest about the numbers
house rule, you know it by now: we publish what the champion actually scores, not the best checkpoint we could find and dress up. on the leaderboard harness blink lands around 71 byte perplexity on wikitext, 52.8% on blimp, and 26.6% on arc-easy. blimp chance is 50% and arc chance is 25%, so it learned a little grammar and a little reasoning, which for 1,087 parameters is more than we had any right to expect. the wikitext number gets worse during training, on purpose, because a three-dimensional residual stream cannot hold both wikipedia and filtered web text at once and it picks the one we trained it on. that is the model being 1k parameters. we are not hiding it.
what does it sound like? you give it "the " and it gives you back "ar n n c tiseos t at or areeeat ton al teat". you can see it reaching for words and mostly missing. every so often it lands one. "the". "at". "let". a tiny model briefly remembering that english has structure, then losing the thread. it is the most honest thing we make. it does not pretend to know more than it does, which is a bar a lot of bigger models still cannot clear.
why ship a model that can't talk
because the question that matters is not how good a small model can get. it is where the floor is. blink is the floor probe. if 92 million tokens per parameter cannot push a 1k model past chance on grammar, that tells you something real about the architecture. it pushed past chance. that tells you something too. every weird idea we want to try on the 50M model gets tried on blink first, and blink is small enough that you can watch it learn, fail, and occasionally surprise you in an afternoon instead of a week.
it is on hf right now with a single-file inference script, so you can run a state-of-the-art-sized-for-1999 language model on a potato. no early access, no waitlist, no token sale. four people, one 5090, a 13 kb model. go break it. tell me what it says.
/lane glint research, 2026, 1,087 parameters, 100 billion tokens, one 5090, and a model you can fit in a text message
✔you made it to the end
keep reading
~3 min · ~712 words
in one line: blink is live on huggingface. it is the smallest model glint research has ever shipped and probably ever will: 1,087 parameters. not 1.087 billion. not 1.087 million. one thousand and eighty-seven …
minimythos is not a codename anymore. it is a real model, and the run is live right now. 50 million parameters. 50 billion tokens of fineweb-edu. built on top of the fable-inspired architecture we have been iterating on for months. this is the first model out of the minimythos pipeline, and it exists because of thousands of hours of config tuning that almost broke us a few times.
minimythos is not a codename anymore. it is a real model, and the run is live right now. 50 million parameters. 50 billion tokens of fineweb-edu. built on top of the fable-inspired architecture we have been iterating on for months. this is the first model out of the minimythos pipeline, and it exists because of thousands of hours of config tuning that almost broke us a few times.
we have been quiet about this for months. every time someone asked "when is minimythos dropping" in the discord, we said "soon" or "working on it" or "lane broke the tokenizer again." and all of that was true. but the real holdup was never the model. it was getting the config right.
getting the config right took thousands of hours. not because we were being thorough. because we kept breaking things. the shard model at 50M is deceptively simple on paper: one shared transformer block looped 16 times with per-iteration LoRA adapters, factorized embeddings like ALBERT, a custom tokenizer. in practice every change to one variable broke three other things. we ran config sweeps. we ran more config sweeps. we ran the sweep sweeps. the results are in the journal. there are pages of them.
the pipeline itself is straightforward. train a model, save checkpoints every couple thousand steps, benchmark each one on BLiMP, WikiText-2, and ARC-Easy, SLERP-merge the best checkpoints together, benchmark the merged result, and keep doing that until the merged model is way better than any single checkpoint. the SLERP merge is done per-parameter with sign-flip handling. it sounds simple. getting it to actually work at 50M scale was not simple.
the architecture
shard is our internal name for this family of models. the key idea is parameter sharing plus recursive depth, with SLERP merging at inference time. instead of one monolithic set of weights, you get one shared block that gets called multiple times with different LoRA adapters each time. the adapters specialize. then you merge the checkpoints with SLERP. the merged model is better than any individual checkpoint because it gets the specialization from each one without paying the parameter cost of having them all at once.
the concrete numbers for the 50M model: factorized embedding (vocab to 128-dim then to model width, like ALBERT), 4 unique prelude layers, the shared middle block looped up to 16 times with rank-8 LoRA per loop, 4 unique coda layers. the effective depth is something like 20+ layers but the actual parameter count stays at 50M because almost everything is shared. the learning rate schedule took a week to tune. arman did it. i am the one not allowed to touch it. the last time i touched the LR schedule the model started outputting the entire bee movie script.
there is also a 1k debug model called blink and a 1M model called glint. same architecture, fewer params, used for testing config changes without burning a day on a full run. they were invaluable during the config tuning period. every change went through blink first. most of them broke blink. the ones that broke blink also broke the 50M model, which is why they exist.
the training loop
the training pipeline is tuned for a single node with however many GPUs we could fit without tripping the breaker. the data is fineweb-edu, sample-100BT configuration. it gets filtered, deduplicated, and re-ranked by difficulty before it goes anywhere near the model. the difficulty scoring uses a learned scorer from the fable-5 tracing work. it is not hand-tuned. we tried hand-tuning. it went badly.
the core loop: train until the loss plateaus or the checkpoint schedule fires. save a checkpoint. run BLiMP, WikiText-2, and ARC-Easy against it. record the scores. then SLERP the best 3-5 checkpoints together. run the same benchmarks on the merged result. if the merged model beats the best individual checkpoint, keep the merge. if it does not, try a different combination. repeat.
the SLERP merge itself is per-parameter with sign-flip handling based on Git Re-Basin. when two checkpoints are close in weight space, we fall back to linear interpolation. when they are far apart, we use actual spherical interpolation. the interpolation parameter is swept at 0.25, 0.5, and 0.75 for every pair in the top-K. we benchmark every merge. we do not guess.
this is thousands of hours of work. most of it is waiting for benchmarks to finish. the benchmarks run in a subprocess with a timeout. if a benchmark hangs, it gets killed. the checkpoint saver writes to a temp file and atomically renames it into place so we never lose a checkpoint if training gets interrupted. we lost checkpoints in early versions. we do not lose checkpoints anymore.
what the team actually did
minimythos is not a solo project. there are a lot of people involved. the short version: i did most of the coding. shane edited a few lines. everyone else is here to run reality checks, catch the dumb ideas before i waste a week on them, and hand me papers that i then spend three weeks implementing badly before getting it right.
lane. lead developer, architecture, training, the push to beat gemma 4 e4b at under 100M parameters. also writing model cards and panicking.
armand0e. moral support.
shane. edited a few lines of code. kept the training node alive while lane fought the tokenizer. (also magebreaker on discord)
enderchefcoder. moral support, coding stuff.
dragonoid. reality checks. lots of papers. more papers than i know what to do with.
moon_senpai. reality checks. sometimes papers.
amytimed. reality checks. sometimes papers.
costikoooo. reality checks. sometimes papers.
datdanboi25. reality checks. sometimes papers.
finnyboy. reality checks. sometimes papers.
pedrodev2026. reality checks. sometimes papers.
the continuous thinking thing
something we added recently is COCONUT-style continuous latent thinking. instead of reasoning in language space like chain-of-thought, the model feeds its last hidden state back as the next input embedding. it creates "continuous thoughts" that let the model run multiple reasoning steps inside the forward pass without generating tokens. there is no explicit thinking loss. the cross-entropy loss supervises the thinking through backprop. it is a neat trick. the 1M model gets 46% better loss with 4 thinking steps and 8 loops than with no thinking at all. the blink model cannot fit thinking because even one thinking step pushes it over the 1k parameter limit.
the data
the entire training corpus is fineweb-edu. sample-100BT configuration. 50 billion tokens. nothing else in pretraining. the agent traces from fable-5 are in the codebase and in the journal as reference but they are not in the training mix for this run. this is a language model. it learns from text.
the quality filter is the part that took the most time. not the SLERP. not the architecture. the quality filter. getting the threshold right so we keep enough data to hit 50B tokens without letting in the garbage. the fineweb-edu sample is already filtered. we filter it again. then we rank it by the learned difficulty scorer and keep the top quality percent. the exact number is in the training config. we are not going to post a fake number here because we do not want people to argue with us about it in the comments.
what this means
minimythos is the first model out of the minimythos pipeline. shard is the product. smaller is where we are going. we do not have a number yet because the scaling curves from this run will tell us what makes sense. if the architecture holds at 50M parameters we will make it smaller. if it does not, we publish the failure analysis and go back to the drawing board. that is the deal.
the 1T token glint run is still happening in parallel. minimythos is not replacing that. minimythos is the production line. glint is the research line. they are running at the same time on different parts of the same cluster. ender manages the tokenizer for both. lane manages the panic for both.
when you can try it
the run is in progress right now. when it finishes, the weights, the model card, the full evaluation numbers, the training config, and the evaluation harness all go up on huggingface. the model will be there first. the blog post and the evaluation numbers will follow. all on huggingface. no early access lists. no discord exclusives. no token sale. we are a 4-person research org that builds tiny models and occasionally wins arguments on the internet.
if you want to follow the run as it happens: hf for the model when it ships, ko-fi if you want to help fund the next GPU so we stop asking shane to run the training loop from his apartment at 3am. the journal is in the repo if you want the raw training logs with no filter applied.
/lane glint research, 2026, 4 people, 1 model, thousands of hours of config tuning, still wondering why anyone cares about a 50M model
✔you made it to the end
keep reading
~7 min · ~1,500 words
in one line: minimythos is not a codename anymore. it is a real model, and the run is live right now. 50 million parameters. 50 billion tokens of fineweb-edu. built on top of the fable-inspired architecture we …
minimythos is real. our first shard model trains on 50B tokens.
minimythos is happening. 50 million params, 50 billion tokens. we train lots of models, save checkpoints, slerp them, benchmark everything, and repeat until we have something way better than a baseline trained model. fineweb-edu data. thats it.
we have been quiet about this for months. every time someone asked "when is minimythos dropping" in the discord, we said "soon" or "working on it" or "lane broke the tokenizer again." and all of that was true.
minimythos is a 50M model trained on 50B tokens of fineweb-edu. we are not calling it glint-2 because it is not the same thing. glint is the research line. minimythos is the first model we are shipping as a real usable thing.
how it works
we train the model. we save checkpoints every x steps. we benchmark every checkpoint. we slerp them together. we benchmark all the slerp combinations. we see what does what. we repeat until we have a model that is much better than a baseline trained model without the slerp step. thats the whole idea.
the data is fineweb-edu. nothing fancy. no custom mixes. no synthetic data. no agent traces. just the good educational web data that everyone uses. the trick is not the data. the trick is what we do with the checkpoints after training.
we are not going to share too much about the details. not because we are secretive. because the details are still moving. we are running a lot of experiments in parallel and the recipe changes weekly. when it settles we will write it up properly.
when can you try it
the run is in progress. weights and a demo on hf soon enough. no early access, no waitlist, no token sale. we are a 4-person research org that builds tiny models. we will ship it when it is done.
if you want to follow along: hf for the model, discord for the training logs, ko-fi if you want to help fund the next run.
/lane glint research, 2026, first shard model inbound, still running on 5090s, still 4 people in a trench coat pretending to be a lab
·lane·updates
the blog is back. here's everything that happened.
ok so the blog is back. it has been dead for a really long time. the last post on here is from when glint-0.3 was our big release. so. yeah. a lot has happened. buckle up i guess, or don't, whatever, you can close the tab. im not your dad.
ok so the blog is back. it has been dead for a really long time. the last post on here is from when glint-0.3 was our big release. so. yeah. a lot has happened. buckle up i guess, or don't, whatever, you can close the tab. im not your dad.
also hi. im lane. i run this place. you might know me as compactai on hf. if you do, no you don't, get out of here. ok lets go.
we beat supralabs in followers. YAYAYAYAYAYAYA
this is the headline. this is the lede. this is why you opened this post and if you scrolled past it i will personally come to your house and reset your wifi.
we beat supralabs. supralabs. the supralabs. the ones with nova, the ones with the bigger model cards, the ones who have had more followers than us for the entire lifespan of this org. 142 to 141. that's it. that's the gap. one follower. one single human being decided that glint research was the hill they wanted to die on and i love them. i dont know who they are. but i love them.
ok so. follower count means nothing. it really doesn't. it has no correlation with model quality, no correlation with research output, no correlation with anything except maybe how many twitter bros you know. im aware of this. im saying it out loud so when i lose my mind about it later you know i knew.
and yet.
i have been checking this number every day for like 4 months. i have a graph. i have a spreadsheet. i have, on at least one occasion, asked a friend with a spare account to follow supralabs and then immediately unfollow them, just to see if i could detect it. (i couldn't. i have no idea what im doing.) i have lost sleep over this. i have won sleep over this. the trajectory was clear and the gap was closing and the only question was when.
it was last week. and i refreshed the page and it was 142 and 141 and i did not breathe for about 30 seconds. and then i closed my laptop and went for a walk. and then i came back and refreshed again. and it was still true. so.
WE ARE BIGGER THAN SUPRALABS NOW. i am writing it in all caps because it deserves all caps. if you're from supralabs and you read this, no hard feelings. (some hard feelings.) ok a lot of hard feelings. come beat us back. please. i need something to obsess over.
minimythos is real and its coming
ok business. sort of. minimythos. the sub-100m model weve been quietly building. yes, it has a name now, yes, it's a real thing, no, you can't try it yet, yes, i know that's annoying.
the short version: glint-1.3 is a 982k parameter model and if you ask it a question there is a roughly 90% chance it will just say "chuamliamce" and walk off into the sunset. it's funny. it is not useful. we made it to prove a point about what 1m params can do at high training token count (100B on fineweb-edu, on a single 5090, 138k tok/s, the whole thing is a flex). and it proved the point. and now we want a model that you can actually use.
so minimythos. under 100 million params. instruction-tuned (glint-1.3 is base only, which is why it says "chuamliamce" instead of answering you). trained on a real dataset. the kind of small that you can run on a macbook without the fan sounding like a jet engine. the kind of good that you can actually paste into a chat and have it not embarrass you in front of your coworkers.
what we know going in: the model souping worked (per-group slerp, you can see it in the glint-1.3 card, the merged model gets +4.5% BLiMP over the best individual checkpoint, which for tiny models is kind of insane). the curriculum stuff worked. the tokenizer experiment we're not allowed to talk about worked, sorry, you'll see it in the paper. what we don't know: whether all of that scales linearly when you go from 1m to 80m params or whether the tricks stop working. we have guesses. they might be wrong. we'll find out.
honestly the worst part of minimythos is the name. i keep typo-ing it. there is no mythological creature called "minimythos." the name is "mythos" (small myth) and the "mini" is because it's the small one, and our naming is a disaster. shane wanted to call it "glint-2." arman wanted to call it "glint-mobile." ender suggested something im not allowed to print. minimythos won because i am a coward and nobody fought me on it. if you hate the name, blame me, im used to it.
1T tokens. yeah. really.
this is the part where i get to say the thing ive been biting my tongue about for like 6 months.
the next glint model (the actual one, not the 1m param research thing, not minimythos, the next one after that) is going to be trained on up to 1 trillion tokens of pretraining data. possibly more. possibly less. depends on the run.
and i think it might beat everything in its class on whatever leaderboard we throw it at. i think. i have a paper's worth of evidence that suggests it. the team has a paper's worth of evidence that suggests it. the ablations suggest it. the scaling behavior suggests it. none of this means it will actually happen. but it has never been this plausible before.
ok we are NOT going to beat gpt-4. we are NOT going to beat claude. we are NOT going to beat llama-3.1-405b. those models are 50-500x our size, they have teams of 200, they have a datacenter. we have 4 people and whatever 5090s we can fit in a room without tripping over them.
what we might beat: every other model under ~500m params on most reasonable benchmarks. maybe a few above that. maybe a few that are way above that if the day is right. the trick is that nobody else is doing 1T tokens on a small model with the kind of data curation pipeline we have. everyone is either small-model-small-data or big-model-lots-of-data. the diagonal is open. we are running down the diagonal.
i have been writing the pretraining config in my head for months. i have rewritten the data mix maybe 30 times. arman has opinions about the learning rate schedule. shane keeps asking me if im sure about the init scheme. ender is in charge of making sure we don't accidentally train on a duplicate of the test set, which is a real problem we have actually had to deal with, which is a whole other post.
no name. no release date. no promises. pretraining is hard and a lot of stuff can go sideways and we have 14 different failure modes bookmarked in a doc that i will share if you ask nicely. but the run is happening. it's not "if" anymore. it's "when."
ok one more thing. (a note on slerp, and on the small-model thing)
edit note (next day): TL;DR: i had a long argument in another discord about small models. the original version of this section had direct quotes and usernames from specific people, plus a few jabs. some of those people asked me to take the quotes, the names, and the mockery out, and threatened to report the org to huggingface for ToS. i removed all of it. every quote, every name. for the record: i do not agree with the framing that any of it was abusive. it was harsh constructive feedback. i was being mean about a technical disagreement, not about the people. but i respect the ask, so here is the same technical content without the receipts. the substance did not change. only the receipts did.
i had a long argument recently with people who think small models are pointless. some of them are not wrong to be skeptical. some of them are wrong in specific ways that i want to walk through, because the specifics are interesting. this is not a callout. this is feedback. if you have said any of these things to me in any discord, this is the response.
what slerp actually is, in case anyone is confused
slerp = spherical linear interpolation. it is a math operation. you have two vectors, you want a third vector that is "between" them. regular linear interpolation (lerp) draws a straight line. slerp draws an arc on a sphere. that is the entire idea.
lerp(a, b, t) = a + t * (b - a)
slerp(a, b, t) = ( sin((1 - t) * θ) / sin(θ) ) * a
+ ( sin( t * θ) / sin(θ) ) * b
where θ = arccos( dot(a, b) / (|a| * |b|) )
that's it. that's the math. it has been a thing in computer graphics since the 80s. it is how you rotate an object smoothly between two orientations without the object going through the floor.
for neural network weights the intuition is: trained weights kind of live on a high-dimensional sphere (loosely, after normalization, but the intuition holds even when it's not exactly true). straight-line interpolation cuts through the middle of the sphere. spherical interpolation walks along the surface. and for reasons that are not fully understood by anyone, walking along the surface gives you a model that performs better than either of the two you started with.
the cheating-math claim
on glint-1.3 we soup'd three checkpoints together using per-group slerp. the result is in the model card. the merged model gets 68.7% on blimp, the best individual checkpoint got 64.2%. that is a +4.5% superadditive gain. superadditive is the technical term for: the merged model is better than the best thing you put in.
and the claim that the scores are fake. ok. the scores are on the model card. the eval harness is in the HF repo. not on github. IN THE HF REPO. right there. in the files. next to the weights. in the same place you downloaded the model from. you literally cannot get a more direct line to whether this is real than clicking files and reading them. how simple is that.
the test sets are public. the checkpoints are public. the merge config is public. the eval script is public. all in the same place. one click. i cannot make this more public than it already is. if you think the scores are fake, run the eval yourself. you have a 5090? great. you have a colab? great. you can do it in an afternoon. i will wait.
nobody has done it. because they are not fake.
the doing-the-published-formula framing
ok. we are not cheating math. we are not beating math. we are doing math. specifically, we are taking three tensors of weights, computing the angle between them, and interpolating along the surface of a sphere. that is the operation. it has a name. it is called slerp. it is in textbooks. it is in computer graphics since the 80s. it is in mergekit. it is in the model soup paper. we are not doing anything new. we are doing the thing that already exists and we are showing that it works for tiny models.
if cheating math means running the published equation and publishing the result, then yes. we are cheating. and you should be mad at every paper that has ever used the formula. which is all of them.
the just-invent-better-architecture counter
this is the just-build-a-rocket-bro of ml twitter. it is technically advice. it is also useless. we are doing the work. it is just slow and looks like a small model that mostly outputs "chuamliamce."
the 1m param research line is explicitly the we-are-trying-weird-stuff line. we are running tokenizer experiments. we are running training-recipe experiments. we are running architecture experiments (glimmer-1 is the start of one). if something better than transformers comes out of this it will come out of the small-model research first because that's where you can afford to break things. the big labs cannot afford to break things. they have shareholders. we have a discord and a 5090.
the vega 7 / 5090 analogy
this is the analogy: you are trying to get big-model performance out of a 1M model by throwing 1T tokens at it. that is the overclocking. you are not making the chip bigger. you are just running it hotter. and a vega 7 does not become a 5090 by being run hotter. it crashes, or it just stays a vega 7 with worse thermals.
fine. we are not doing that. we are not running the 1M model hotter by feeding it 1T tokens and calling it a day. we are doing three distinct things that the analogy collapses into one:
we are training longer than the standard recipe says we should. this part is the overclock and we know it. the data has to come from somewhere and the bottleneck is the data not the params. ok.
we are merging checkpoints with slerp, which the standard recipe does not do. this is not overclocking. it is a different operation. we are not running the same model hotter. we are taking three models and combining them. the 5090 is not a vega 7 run hot. it is a different chip. this is the part of the work that actually does the heavy lifting.
we are doing curriculum + data curation work that the standard recipe does not do. also not overclocking. we are picking which 1T tokens to train on. a vega 7 with a better cooling solution is still a vega 7. a vega 7 running a workload it was actually tuned for is a different thing. that is what curriculum does.
so the analogy is wrong because it implies we are doing one thing (running a small model hot) when we are doing three things (running it longer, merging it, curating the data). the 5090 is not a vega 7 with more tokens. the 5090 is a chip that was architected differently. we are not claiming to be a 5090. we are claiming that the vega 7 with 1T tokens comparison is leaving out the slerp and the curriculum. with all three, you get something that is not a 5090 but is also not the vega 7 the analogy is picturing. that is the whole pitch.
the kaplan 2020 scaling laws paper
this gets dropped a lot, like a trump card. ok. i have read it. more than once. we have all read it. arman has it printed out. here is what it actually says: for a given compute budget, there is an OPTIMAL model size. the curve goes up with model size and down with data size depending on where you are on the compute frontier.
what it does NOT say is: you cannot train a small model on a lot of data. in fact, the 2020 paper and every follow-up explicitly note that the small-model-big-data corner of the chart is UNDER-EXPLORED. everyone runs the same diagonal. we are not running the same diagonal. we are running the other one. the laws do not forbid it. the laws just say nobody has done it well, which is a different statement.
the-law-says-i-cant is a great way to never discover anything. the law said flight was impossible. the law said heavier-than-air was a fantasy. the law said a 1M model could not get 52% on BLiMP. we did it. the law is now updated. we are trying to do that again.
the diminishing returns data point
this is a real datapoint and i want to engage with it honestly. ok. what was the model, what was the eval, what was the seed. because:
if you ran a 1M model on 24B and then 34B and saw 0% on wikitext perplexity, that is consistent with what we see too. wikitext saturates fast for tiny models. the curve flattens.
if you saw 0% on blimp, that is also kind of expected at that scale, blimp has a ceiling around 65-70% for small models.
if you saw 0% on arc-easy, that is the one i would push on. arc-easy should still be moving at that range.
if you ran 0% on the downstream task you actually care about, the eval you wrote yourself... that is the only one that matters and we have no way to reproduce it.
the thing nobody in that thread was doing: running slerp. we are not seeing 0% improvement because we are not just adding tokens. we are adding tokens AND merging checkpoints. the merged model is what gets the gain. the individual checkpoints plateau. the soup keeps improving. that is the whole point. the claim that more data alone doesnt help is true if you only train one model. it is less true if you train many and merge.
the chat-capability ceiling
ok. show me the proof. what does really good at chat even mean. because:
smollm2-135m can answer basic questions without falling over. it is 135M, not 1M. but it is proof that the floor is not zero at this scale.
phi-1.5 at 1.3B is genuinely chatty. we are not at 1.3B. but we are closer to 1.3B than to 1B in terms of research effort.
tinyllama 1.1B is a real chat model on a real phone.
none of these are 1M. and we are not claiming 1M is really good at chat today. we are claiming we are working on it.
never is a strong word. the people saying never are usually the people who have not tried. we have tried. we are still trying. the model is not really good at chat yet. it is closer to weirdly consistent at saying chuamliamce. but never is a claim about the future, and the future is not in your dataset.
the data-capacity limit theory
this is at least a technical question so i will answer it like a technical question.
the model weights do not need to fit the training data. the training data is the training data. the weights are a compressed representation of the patterns the model found in the data. that is what training does. you do not need 1TB of weights to learn from 1TB of text. a 1M param model with 4-bit weights is 0.5MB. the model has 0.5MB of space for learned patterns. that is fine. that is how compression works. a 100KB jpeg is not limited to 100KB of photo content. it is a representation of a much larger photo.
what matters is: does the model learn useful things from a lot of data. and the answer is: yes, it does, even at this scale, especially when you do the curriculum and the slerp and the rest of it. not as much as a bigger model. but enough to be interesting. and that is what we are testing.
the claim that 1M can only learn 8-10MB of unique uncompressable data is true if you are storing the data verbatim. it is not what we are doing. we are not storing the data. we are storing a model of the data. those are different.
and the meta point: vibes are not evidence
the claim that this is impossible, made without a training log, without an eval trace, without a failed run, is just a vibe. and vibes are not evidence. if you think we are wrong: fork the recipe. train a 1M model. soup it. publish the result. we will read it. we will cite it. we will probably dm you about it. that is the deal. that is how research works.
the proof is in the training runs. come run some with us. or run some against us. we do not care. we just want the data.
the other stuff (less cool but still cool)
a few things happened that don't deserve their own section but im going to mention anyway because this is my blog and i can:
fable-5-traces is now trending on huggingface. 2.5k+ likes. trending for 5 days straight. it is a dataset of distillation traces from the (now gone) fable 5 model. 28 models have been trained or fine-tuned on it in the last 2 weeks. it is the most popular thing ive ever made and i did not see it coming.
glimmer-1 shipped. 11,900 parameters. yes, eleven thousand nine hundred. an entire llm in 12k params. it scores 25% on arc-easy and 52% on blimp and it is the dumbest and also the most beautiful thing we have ever made. it is also a stunt. we know.
anthos-1 (text-to-image, our only non-text model) crossed 1000 downloads. it generates flowers. only flowers. it is the most single-purpose model on huggingface and i love it.
glint-trace shipped. distillation traces from larger models. 73 downloads as of writing this. somebody out there is using it. i have no idea who. if that person is reading this, hi, please tell me what you are doing with it.
the discord is alive. for a long time it was just me, the bot, and the sound of my own typing. now there are actual people in there. people who are smarter than me about things. this is a mixed blessing but mostly good.
the team is 4 now. me, shane, arman, ender. shane does infra and pretraining ablations. arman does data and post-training. ender does the things nobody else wants to do (tokenizer work, evaluation harness maintenance, making sure my code doesn't catch fire). i do whatever's left, which is mostly writing model cards and panicking.
whats next for the blog
more posts. real ones. not model cards (model cards dont count, they're spec sheets, dont @ me). pretraining updates, ablations, the stuff that broke, the stuff that worked and then broke, the failed runs, the "we spent 3 weeks on this and got nothing" posts. maybe a post about the time glint-1 convinced itself it was named greg. that one is coming.
if you want to follow along: hf for the models, discord for the chaos, ko-fi if you want us to keep doing this instead of getting real jobs.
ok that's it. that's the post. blog is alive again. supralabs who. minimythos is coming. 1T tokens is real. see you in like 3 weeks when i forget to write one and then panic about it on a sunday night and post something unhinged at 2am.
/lane glint research, 2026, still tiny, still trying, now with 142 friends and 2.5k likes on a dataset we did not expect to take off
✔you made it to the end
keep reading
~17 min · ~3,797 words
in one line: ok so the blog is back. it has been dead for a really long time. the last post on here is from when glint-0.3 was our big release. so. yeah. a lot has happened. buckle up i guess, or don't, whateve…
