I recently posted a Substack note about Claude’s new watermarking process and implementation. Since it’s such a popular topic and sparked such a lively discussion, I thought it might be interesting to go into a bit more detail when explaining how it works.
Instead of the usual text article, I recorded a little lecture on the topic (to change it up a bit from my usual articles). So, below is the video along with a transcript.
Originally, I planned to make 10 slides and record a short 10-min video. However, while putting it together, I added some crucial details here and there, resulting in >50 slides and a 48 min recording.
I hope that this now explains it well, though! Happy watching!
I also have a YouTube version if you prefer using the YouTube player
And here is a link to the slides
Video Transcript
Note: The transcript below is slightly edited and cleaned up for readability but preserves the overall order and flow of the video lecture above.
How Claude’s Text Watermarking Works
Hi everyone. So, a few days ago, Anthropic announced that they will watermark the text outputs of their Claude models. I then did a social media post briefly explaining how that works. And yeah, this was quite the popular post. So not the watermarking itself was popular, but I guess the explanation or the mechanism behind it. Then, it might be worthwhile expanding this a bit to explain it in more detail, because this post only had one figure, and there were a lot of questions and discussions.
So, I thought, well, let’s make a few more figures. I actually originally planned to do like 10 slides and walk you through it. It ended up being 50 slides, but I hope this really explains how this watermarking technique works well, how watermarking itself can fail or be removed, and so forth. So I think it might be an interesting topic because a lot of people use LLMs these days and also consume a lot of text on the Internet that might be generated by LLMs.
And now there’s going to be this watermarking, and there’s this, I guess, fear of watermarking making text worse, or what’s actually the benefit of this watermarking? And so what does it mean? And I think if we understand a bit better what watermarking is, that goes a long way, and then we can make up our own minds about whether that’s a good thing or not, and so forth, like the pros and cons.
So, my goal here is really to explain how the underlying mechanism works and how they are going to implement this type of watermarking, text watermarking.
It’s also a great example to illustrate why understanding things from scratch is actually quite useful. This watermarking technique is also a nice way to explain how conventional models or LLMs in general work under the hood. So yeah, you may know I like doing things from scratch. Like, I have my books: Build a Large Language Model From Scratch, Build a Reasoning Model From Scratch. I have some articles labeled from scratch.
So, for me, “ from scratch often includes coding. So this one will not be coding-related, but coding from scratch is actually a very, very useful technique because it really helps you understand how something is implemented. And then from that we can derive our understanding, figures, concepts, because if we don’t really implement things, if there’s no code, it’s really sometimes ambiguous. And of course, you know, as I realized, not everyone is coding from scratch anymore.
Like back in the day, coding something from scratch was all we had. I mean, there were only humans coding. Nowadays, coding can be done by LLMs. However, that doesn’t mean reading code is no longer useful, because it carries a lot of information. So in this case here with this watermarking, spending some time coding an LLM from scratch really makes you realize how this sampling inside is implemented. We still have some relevant code snippets.
And then that really, in turn, helps us understand, oh, the watermarking is applied at this position, and this has so-and-so consequences and so forth. So I think even though people may not be coding from scratch, at least not all the time anymore, it is still useful being able to, let’s say, build something from scratch for educational purposes to understand something deeply and then also for research purposes to manipulate this in a transparent way that is not hidden away in tons of layers of abstraction.
But that aside, I think it’s just a coincidental nice relationship here because, for this slide deck, I actually used a lot of figures from my from-scratch coding materials.
So a few days ago (this is August 14), there was this article, How Claude’s Text Watermark Works, and there was this article here; it’s just like a screen recording, so it can have everything in the slides, but there’s plenty of detail. They updated it actually a couple of times, so originally when I read this, it was a way shorter. Still, it is very, I guess, conceptual; there’s like this overview, and there’s, I mean, there’s not a single figure in there.
And so it’s kind of still hard to understand what they’re trying to do. So they explain a lot about why they’re going to do it, but they don’t explain how. They’re linking to one paper somewhere there, which is very technical also. So I do think it makes sense maybe to take a step back and start at the beginning to kind of understand what they’re trying to implement here with this watermarking technique.
And so the motivation, by the way, of watermarking is for them to identify if someone posts some text that they can say, oh, this text was generated by our Claude Opus 4.8 model, for example, so that they have a way to tell, OK, this text is AI-generated because it carries this watermark. And this watermark is invisible to users, so only they can decode it and find out whether the text has their watermark.
Why can only they do it? We will get to that later in this (hopefully not too long a video), but one thing at a time.
How LLM Text Generation Works
So I wanted to start with a brief prelude to explain how text generation works in LLMs, because based on that we can then more easily understand how the watermarking works and that this is actually not a huge, expensive thing on top of it. It’s really just like a minor, I guess, tweak inside the regular text generation process.
So when we are using something like ChatGPT, for example, let’s say I ask the question, the capital of Germany is, and yeah, ChatGPT or other LLMs, so this is just like an example would, for example, answer “Berlin”. So here, in this case, it’s generating two tokens, like “Berlin” and the period. But for simplicity, let’s assume it’s generating one token. So the next token is the “Berlin” token. How is this token generated internally?
What is happening under the hood when we type something here like the capital of Germany is and receive a token like “Berlin” back? What is actually going on there behind the scenes?
So in the next couple of slides, I want to briefly talk about what happens under the hood when this next token is generated.
So assume again that our prompt is the capital of Germany is. And the first step here is to convert this into token IDs. So tokenizing it and converting it into token IDs is one of the main steps at the beginning. This is outside. It’s not inside the LLM; it’s outside of the LLM. So we are simply converting the text into token IDs. It’s just a format that embedding layers can work with.
And then this passes through the LLM. And the LLM gives us a score distribution for the next token.
So again, this is just like a brief overview of how LLMs work internally. So I’m not covering the LLM machinery itself. I talked about it many times in my other From Scratch LLMs videos and books. The important part is that when we generate the next token (for example, “Berlin”), we have, at this point, a distribution of scores. So this is the output produced by the LLM.
Here in this case, we’re looking at logit values. So these are just scores from minus infinity to plus infinity, like a range of scores. Here’s an example, ranging from about -8 or -9 to 20. We could convert these into a probability distribution, but technically, it’s not strictly necessary depending on how we sample. But so you can think of the logit values as the raw scores.
And the raw scores go over the entire vocabulary.
That means every possible word that the LLM could generate. Now here, in the vocabulary index, a certain value (index position 19,846) receives the highest score. So I spread out the distribution. If you would run this prompt through an LLM, you would even see something more extreme: that everything is, like, very, very, very close to zero. And “Berlin” would probably be much, much higher even.
But just to show you a few, you know, like peaks here so it looks a bit more interesting, I kind of zoomed in; in and spread out the distribution a bit. Now here, “Berlin” is the highest score because you can think of it as the most, I guess, probable or plausible next token if I have a very specific prompt like this. So the other ones, I mean, it could be something like Hamburg or Munich that the LLM might guess incorrectly.
But nowadays an LLM should be fairly certain that “Berlin” is the correct answer here. You are also seeing here the vocabulary index. So that’s like over the whole vocabulary. Nowadays, LLMs have like 250,000 possible tokens as output. I’m truncating it here from 19,800 to 19,900 because there’s just so much space here on this slide. If I would have a very realistic vocabulary of 250,000 words, everything would be so narrow that we would barely even be able to tell or see anything on this distribution.
So this is just truncated for educational purposes. The important point is that in regular text generation, we get this score distribution. Now, what we do is look at the highest score.
I will get into more detail later on how this is selected. So it’s not necessarily precisely the highest one, but for simplicity, assume we are taking the highest score here. And in this case, it’s 19,846.
And this score is then detokenized, and we get “Berlin” back. So that is the process here on this slide: from an input prompt to conversion into token IDs and tokenization, passing it to the LLM, getting this score distribution, getting the next token, and converting it back into text.
And then this text is appended to the input. So if we have a question that requires multiple output tokens, we keep going in this loop until the answer is complete. That usually means that the LLM generates an end-of-text token, for example, here. For simplicity, I’m showing you only one iteration where it generates one token. But yeah, as I said, it would kind of continue like that, where we are feeding back the modified input to the LLM for the next round.
Now, how do we actually sample this next token here?
How Next-Token Sampling Works
I briefly said, well, we could just technically select the highest one, the one with the highest score. This is called greedy decoding. That’s one way to do it. But most LLMs, like if you use them, they don’t do greedy decoding where they always pick the highest one. Because if you ask it on some other prompt, it might not be what we want to always have the highest score, because then it would memorize the training data.
It would always kind of give the same response and so forth. So we actually often want some variation in the outputs, but not so much that it generates random stuff. So how it works is that, when we sample here from this distribution, we first typically convert it into probability scores.
So here I just have these scores shown in this plot. I’m just using NumPy for simplicity; whatever tool you use (e.g., PyTorch), the same concepts apply. But let’s assume we have the scores here in NumPy. So what I would do is I would compute the softmax. Technically, I would use a softmax function implemented in Torch or PyTorch, for example, that is numerically stable for both large and small values, including very high positive values, very low positive values, and very high negative values.
Here I’m just writing it out like that. That’s the canonical softmax, just to make it a bit more readable. But the details don’t matter here.
What matters is that after this conversion, the scores here, I mean, there’s only so much space on the slide, but the scores here, they would add up to one. So it’s essentially like a renormalization. So they would be normalized to sum up to one. That’s all that the probability conversion does: the softmax conversion. So then once we have these probabilities, we can use a random number or, like, a random sampling algorithm.
For example, here in NumPy, we could use the choice function or method. So this is with a specific random seed we are passing to the vocabulary indices. And then, and that’s the important part, we are passing the probabilities as the weights. So, these, essentially, yeah, are like: “How likely is a certain token to be selected?” So, for example, if “Berlin”, after this normalization step, the softmax step, has a 99% probability and the other ones together have a 1% probability, then if we would sample 100 times, 99 of the times, we would get “Berlin”.
In realistic LLMs, for example, that are well trained, “Berlin” might receive a probability of 99.999999 or something like that. So you’re almost certainly always sampling “Berlin” because it’s very confident that the answer is “Berlin” in this particular case. So yeah, that is how we would sample from this distribution. There are modifications like top-k sampling or top-p sampling where, let’s say, just for simplicity in top-k sampling, we would select the top 100 tokens and then apply this random choice only to the top 100, the 100 highest-scoring ones, so that we don’t get nonsense tokens in there.
For this example, it doesn’t really matter. I mean, it’s just like another thing to explain, so I’m skimming over this. So you can maybe assume that this is already the top 100 tokens using top-k or something like that.
And so, for example, here’s an example. If we sample 10,000 times with a probability of “Berlin” being very high, 99.9, we would sample “Berlin” 9,997 times, sample the word “Hal” twice, and one “Moh”. And these are basically nonsense tokens. It rarely happens that, in this case, the LLM might produce nonsense because, as I mentioned before, I spread out this distribution a bit to make it more interesting.
A real LLM would probably, 10,000 out of 10,000 times, sample “Berlin” because the probability of “Berlin” is so high. But this is for illustration purposes.
From Sampling to Watermarking
Now we briefly talked about how LLMs work under the hood, which I think is kind of an interesting concept in itself. But I’ve talked about this many times before, so I don’t want to bore you. I just wanted to set up some context for now, explaining how this watermarking works.
So, we mentioned that we select the highest-scoring token when sampling. Or we use this probability sampling, which will lead to one of the highest-scoring tokens being selected most of the time. Now here’s another example without watermarking. I changed the prompt slightly. Now the prompt is: today’s weather is “cold,” and a possible answer could be, for example, “gray” or “overcast”. So in contrast to the “Berlin” example, I would say “gray” and “overcast” kind of are interchangeable.
They are both reasonable next tokens for this prompt, given the goal of completing this text or writing the next token. So it’s almost like a coin flip which one we want to select. There is not really an objectively worse one of one or the other. So when we do the random sampling, because they also have relatively high scores and their scores are similarly high since they are both plausible tokens, we might get one or the other.
So almost half of the time we would get “overcast”, and almost half of the time we would get “gray” if we repeat the sampling multiple times. And that’s how LLMs often end up with different answers if you provide the same prompt. If you use the same prompt and you ask the LLM multiple times, you often get slightly different answers. And that’s because at certain positions, two possible tokens are almost equally likely, so it will choose one or the other.
And that token would then influence all subsequent tokens, and so forth.
Random Seeds and Deterministic Sampling
Now, I wanted to briefly talk about random number generation. So, for example, if we use a random number generator like this, it will generate a random sequence of numbers. If I run it again, the sequence of numbers is different here. So you can see every time we produce five numbers, they are different. If I set the random seed here, like one, two, three, and I run this multiple times, we still get random numbers, but they are now all the same, right?
So they are still random. If we use a random seed, we still get random numbers that are different from each other, but they are reproducible. So whether we use a random seed or not, we still get random numbers. But with a random seed, we get a reproducible sequence of numbers. So keep this in mind: this is just like a little primer, and we will use this concept in a few moments.
So, for example, I mentioned before that we might get either “gray” or “overcast” if we randomly sample. Now, if we use a specific random seed like 42, we would always, for example, select “overcast”. I mean, it’s still a random selection, but we make it deterministic. In this case, given this prompt, the model will always select “overcast”.
If we use a different random seed, the model might select “gray”. Every time we sample, it will always select “gray” as the next token. So it’s still random sampling, but we are making it deterministic based on the random seed.
How the Watermarking Key Works
So, in watermarking, Claude watermarking is kind of like the idea that it sets a random seed. But this random seed, instead of being like a number that is fixed based on, I don’t know, someone writing down a fixed number, they’re using a secret key that is essentially like an API key, a secret key, and from that key, together with the four previous words, they derive this random seed essentially.
But the idea is that if I go back one slide, it’s the same as here: there’s essentially a fixed random seed, and that random seed always selects the same next token. Okay, so instead of using random seed 99 here, for example, they have a secret key and also use information about the previous tokens to derive this random seed. But more on that later.
So the idea is that watermarking makes the text generation more deterministic in certain positions. So, for example, if we have these plausible texts on the left side. So if I have a text that says,
> The weather today is cold and
I may either pick “overcast” or “gray”. And the next sentence could be,
> and then “light” or “gentle”
They’re both interchangeable again.
> And then breeze is “moving” or “blowing” through the trees, and the streets seem “quiet” or “still”.
Which means basically I could say either “quiet” or “still”. So there are certain positions in the text where we have token choices where they are almost equally likely, like we have seen before. So that means if we are, this is without watermarking, if we are running the prompt, or given the prompt through the LLM, we might sometimes get this answer here, sometimes this answer, and so forth.
And based on the number of positions, we might have 128 possible answers here. And of course, the longer the text, the more positions we have where we can have terms interchangeably, the more combinations, or the more output texts, there are. So, for example, again, one possible output text could be
> The weather today is cold and overcast. A light breeze is moving through the trees, and the streets seem quiet. I think I’ll stay home and read a book with a cup of tea.
So that is one possible text. Another possible text is
> The weather today is cold and gray. A gentle breeze is blowing through the trees, and the streets seem still. I think I’ll stay inside and read a novel with a mug of tea. By the way, it’s also actually raining outside. I don’t know how good this microphone is, but it’s kind of a very fitting context here.
But yeah, the bottom line is that you can see there are two very reasonable texts here being generated, and there are more combinations. So they are all reasonable. There isn’t one that is necessarily better than the other. They’re just, you know, slight variations. And if we don’t use watermarking, we might get either one, or it’s just random, right? Because of the random sampling, we might get one or the other.
Now, if we fix the random seed, as I mentioned before, for example, if the random seed is 99, we might always get this text here. So, using a random seed, we can kind of fix which answer we get, because then the random sampling is still random, but it’s deterministic in the sense that it’s reproducible. It’s always going to be the same then. Okay, so that is still without watermarking, now with a random seed.
And the watermarking is essentially doing the same thing. Now, instead of just using a simple random seed, they have a so-called random key, where this random key is involved in selecting the text, essentially. But what we can already say is that, in the Claude blog post, they say the watermarking shouldn’t make the text worse. If we look at this mechanism, yeah, it makes sense why it would not make the text worse.
By the way, I’m not defending watermarks here. I’m just trying to explain. So please don’t kill the messenger here. But what I’m trying to say is that the watermarking is nothing else for the end user than fixing a random seed and making this sampling kind of deterministic, if that makes sense.
Where the Watermark Is Applied
Okay, so the summary so far is without watermarking. We often sample without a random seed because I know most people don’t even use one. I honestly don’t think you can necessarily do it with the Claude and OpenAI APIs. I know you can do it in Ollama, but I also always had some problems with that because I used Ollama in one of my books for the bonus material to generate some texts.
I was fixing the random seed, but it still wasn’t always deterministic, and so forth. So it’s tricky. Your mileage may also vary, depending on the software version and so forth. Anyways, so without watermarking, we have this random sampling. With watermarking on the right-hand side, we still have the random sampling. But in addition to just a random sampling being fully random, we have this watermarking key.
And this watermarking key is passed to the random seed generator to set a specific random seed, making this deterministic. But it’s essentially very similar, and like I mentioned, there’s a lot of benefit in terms of understanding things from scratch. And now we know essentially where this watermark is applied to. So this is essentially applied to the sampling. It’s not applied inside the LLM, which is actually cool knowledge.
So they don’t need to train a new LLM for that. They can just use an existing LLM, and they just apply it at this sampling stage. They don’t have to retrain anything or anything like that. So yeah, that is actually interesting, right?
How Watermark Detection Works
But we are not quite done yet. I would also like to talk about how we can understand or see whether text is watermarked. So detecting the watermark is only possible if we have access to the key. So, for example, if we have these different texts, and essentially, after the text was generated, you find some random text on the internet (for example, you find this text number four here on the internet somewhere), you want to know: is this watermarked?
Well, it’s impossible to know because, in order to know, you would need the watermarking key. You need this scoring function, and then you have to score basically the text with a scoring function. And then the idea is that if the score is above a certain threshold, then the text is watermarked. Otherwise, it’s not watermarked. But as the end user, we can’t do this because we don’t have this key. So the key is not available to us.
Only Anthropic will have the key. However, in this blog post, they mentioned that they are providing it, of course, or they’re going to develop an API for that that they will make available. I don’t know. Honestly, I’m not affiliated. I don’t know the details. I was just reading this in this blog post. That’s all I know. So that API might as well be private for some companies, like, let’s say, X or Substack Notes, when they want to label AI-generated posts.
They may make it public for end users to use. Who knows? We will have to wait on that. But yeah, so the bottom line here is that watermark detection is only possible if we have this watermarking key or, of course, the API that they are going to develop.
How to Remove a Watermark
Now, removing the watermark is interesting. So now that we know how the watermarking works, we also know the shortcomings. I mean, this is really highly dependent on specific tokens in certain positions. So, for example, in this given text, if these colored words or tokens are the watermarking positions, we know that we could remove the watermark by editing this, right? If we change all the words at these positions, we would be 100% able to defeat this watermark.
Now, the problem, though, is that we don’t know, right?
So we don’t know where these words are because we haven’t generated the watermark. So we don’t know which positions to look at. So the practical scenario here is that we could just randomly edit the text. So we would randomly change a few words and hope that we change enough positions to edit the watermark. So that would be one way to remove it. And since we also don’t know which are the highest-scoring ones, because that would require us to have access to the LLM and rerun the prompt through the LLM to find out which words are the highest-scoring, we can kind of only guess.
So for example, we might say, oh, we replace “overcast” with “cloudy” because we don’t know that “gray” was high-scoring, you know? So in this case, it might be intuitive to say “gray”, but there might be cases where it’s not so intuitive. So what I’m trying to illustrate here is just some general text editing where we are modifying positions, but we are still kind of guessing what a watermark position is.
So since we don’t know, we added just a few words here and there. And if we added enough words, that would also defeat the watermark.
How the Watermark Scoring Function Works
So yeah, that was the watermarking in a nutshell. I mentioned that there is a scoring function to find out whether something is watermarked. And I want to do it as a bonus here. It’s already a long video, but as a bonus here, I wanted to briefly also explain how this scoring function works because that is also interesting information. It’s a bit complicated. It’s not essential to understand how the scoring function works.
But the reason why they do it the certain way they do is to make the detection cheaper. Because otherwise, if I go back one slide or two slides, if you wanted to check if something is watermarked, if even they wanted to check, they would have to rerun the prompt to get these scores and then apply this watermarking random seed to get this text and then compare. And that would be very expensive because then essentially every text you want to compare, you would have to rerun the LLM.
You have to know which LLM, and that would be really unfeasible because you often also don’t even know the prompt, right? So yeah, so they have like a trick that they use to, yeah, I would say, modify the sampling so that you don’t use or don’t need the LLM later on for the scoring stage. And in the blog post, they mentioned that they derived this method from a paper.
It was a Nature paper, and this method is called SynthID-Text. So that was like a paper that came out maybe one or two years ago. It was by Google, and they use a similar technique they call Claude watermarking. I don’t know, sorry, I don’t know if they use exactly that technique, but that’s the one they mentioned.
SynthID Text and Tournament Sampling
So how does it work? So before we looked at the slides, we looked at the regular, let’s say, overview here, where we have some text. We put it through the LLM. We get this logit distribution and then we sample from the distribution and get the output token. And here, during the sampling, we use the watermarking key and the random seed generator. So this is still correct. This is still what’s going on, but there is a bit more, I guess, nuance to how this token is sampled.
So they’re not just using, let’s say, NumPy’s random choice. They’re using something a bit more sophisticated here.
So assume, again, our context is “the weather today is cold,” and we want to generate the next token. So, for example: “gray”, “overcast”, “gloomy”, “cloudy”. “Gray” is 50% probability, “overcast” is 30, “gloomy” is 15. Let’s say “cloudy” is 0.05 and the rest is, let’s say, 0. Here it looks, of course, a bit different. Let’s say that’s “gray” and “overcast”. I’m just reusing this figure. But now imagine these are the most likely ones, like “gray” and “overcast”, and everything else is just very small, except “gloomy” and “cloudy,” maybe.
So essentially, think about just a very small vocabulary for this example of four words instead of all these 50 words here, just to make it even simpler. Now, as I mentioned before, we could use ‘sNumPy’s random choice with these probabilities to sample the next token.
And we could use the watermarking key with this random seed generator to make it deterministic and get the certain watermark that we want. But as I mentioned before, this would be very expensive. Not the sampling itself. That doesn’t matter. This is pretty cheap. But the detection later on would be very expensive if we are trying to check random text on the internet.
So instead, what they use, they also use it during the sampling, during the generation, so that it can be reused later during detection. What they use is called tournament sampling. So this is instead of using something like random choice, they use a concept called tournament sampling. And so how does that work? It might look a bit complicated, but it looks really more complicated than it really is, to be honest.
So you might have to, I guess, stop the video at some point and just sit with the figure a bit. But I think it is actually simpler than it looks like. It’s like once you get the hang of it, it’s pretty straightforward. But let me try to explain here. So what we have is we have still this context, and then we have these probable or plausible next tokens with these different probabilities.
Now they have something they call random watermarking functions.
Random Watermarking Functions
Here we have three watermarking functions, G1, G2, and G3. In reality, they might have 30, 50, or even more. Here I’m just using three because that is simpler on this slide. It’s just smaller, you know, like it fits better on the slide. Now, if we look at this word “gray”, this might give us a signature 101. With that, I mean, if we use this watermarking key to generate this random seed, and we have three functions, G1, G2, G3.
If I put the word “gray”, what I’m skipping here is that usually you put the word “gray” together with the four or three previous words from the context. So it’s “cold” and “gray”. If I put that into G1 together with this watermarking key, I get the value one. Why? Well, that’s just how this function works. It’s like a random function. The random function either returns zero or one. In this case, with this random key and this token, it returns one.
With the same key, but a different function, you get the value zero. And then here you get a one again. So if we have more functions (of course, 30 functions), this will be a very long string of ones and zeros.
It’s basically like a bit string, like if you have bits of zeros and ones. Okay. So this is for “gray”. So we get the signature 101 through using these watermarking functions. Now we can do the same thing for all the other ones. So we can do it for “gray”. We can do it for “overcast”, “gloomy”, and “cloudy”. So each one has a different signature here. So, for example, “overcast” is zero, one, zero.
“Gloomy” has zero, zero, one. “Cloudy” has one, zero, zero. Okay. So we have these bits here now. The next step is a so-called tournament sampling where we just pair them.
Tournament Sampling Step by Step
Like, you know, like a soccer tournament, the knockout (KO) stages, or like the playoffs in American football, you always have two teams playing against each other. And that’s kind of like the same idea. We have a pair of tokens, and they’re playing against each other, essentially. And the scores, they come from these functions here. So we start with the first function in the first round. So we have “cloudy” and “gray”.
So we look up here: “gray” is a one and “cloudy” is a one. Okay. So one and one. “Overcast” and “gray”. So “overcast” is zero, “gray” is one. So we have zero, one. “Gloomy” and “overcast”. So here we have “gloomy” zero, “overcast” zero. So zero, zero. And then we have “gray” and “gray” again, because we are running out. So we don’t have enough of the others. So we have one duplicate.
So this is chosen randomly. And so you have one and one here. Now we look at the results. So this is a tie. In the case of a tie, we also select randomly using, you know, the random seed and the watermarking key. So here, “cloudy” survives. And from this one, G1 is, according to G1, “gray” is the winner because it has the one. So “gray” survives. And then here, “overcast” and “ gray “ are a tie, randomly selected, and “gray” also randomly selected.
So we have now “cloudy” and “gray” and “overcast” and “gray”. And we play the next round in this tournament. So in this next round, we use G2. So according to G2, “cloudy” has a zero here. “Gray” also has zero. “Overcast” has one. And “gray” also has zero, sorry. And so, the next stage of the tournament again.
So we have a tie. We randomly select “gray”. And here we have “overcast” as the winner. And so we have “gray” versus “overcast” in the final. And then we look again at the scores. So “gray” has a one. “Overcast” is a zero. So “gray” is the winner. And that’s how the token “gray” is sampled. What is the watermarking key doing here? So the watermarking key, if I go back a few slides, is selected for generating these scores using these random watermarking functions.
So the watermarking key determines essentially what values we get at these stages. So the watermarking key is still very important. Otherwise, these signatures would look different.
Detecting Watermarks Without Rerunning the LLM
So we now have sampled the next token. And that’s just how this modified sampling procedure works. We could have used NumPy’s `random.choice`. But the shortcoming of that is that if we want to score random text on the internet, we would have to rerun the LLM. With this technique, we don’t. I will show you in a moment. So this technique sounds like really weird and cumbersome, but it has the advantage that we can now score random text more easily without having to rerun the LLM.
So it’s essentially just to make the detection easier and cheaper.
So, for example, if we have a new text. So I’m just using the same text here, but let’s assume it’s new text. So this is after the sampling, when we are scoring. And let’s say we are discovering this text on the internet. And the text is the weather today is cold and “gray”, and we want to know if this is LLM-generated or not. So we would, or Claude/Anthropic would, have the watermarking key and these functions: G1, G2, and G3.
And it would put this text through these functions. For the one position here for “gray”, we would get 101, similar to what we got during the generation process. So this is the same as before. And this has, if we add up these bits, two bits, right? One and one here. So it has two bits of information, let’s say, for simplicity. This is just a really simple illustration.
But let’s assume we get a score of two here for the “gray” in this position. If we had a different word here, “overcast,” in this position, we would get one if we get “gloomy,” like we also have one, and “cloudy” one. So I’m just summing over each row here, right? So that’s just like a score we would get at each position. And here I’m only looking at the last position.
If I would do this at other positions, I would get a different score at different positions. So, for example, let’s assume at the first position I get a two. Here I get a two. For “today”, I get a three. For “is”, I get a two. “Cold”, two. And “gray”, three. So here I’m applying these watermarking functions as I’ve shown on the previous slide.
And I’m just adding up these numbers across the three functions. And the watermarking functions are very cheap. So you can just quickly run them on the whole text and get these scores. And then based on that, I can compute the average bits. So if I just average over all these values here, let’s say I get 2.23.
Now, if I have slightly different text, so here I swapped “today” with “now” and “gray” with “overcast”. These now get a score of one and one. And if I average over this whole string, then I get a 1.71. And so for that, I don’t need an LLM. All I need is the watermarking key, the random seed generator, and these functions, G1, G2, and G3. And that’s all I need.
I don’t need the LLM. And I can get this score here. And what they do is apply a threshold.
So, for example, I mean, they don’t use this exact threshold. But for example, we can say if the score is greater than two, then the text is watermarked. If the score is smaller than two, it’s not watermarked. So here, if the score is greater than two, it’s a yes. So yes, this is watermarked. In this case, 1.71 is not greater than two. So this text is not watermarked. Okay. So that’s just the way we can then detect whether random text on the internet is watermarked or not.
It’s essentially just applying these watermarking functions and then averaging over the scores and applying a threshold. Okay.
So again, the tournament sampling is mainly to make detection easier and cheaper. We could also use something like NumPy’s random choice with a random seed or to make the sampling deterministic. But then again, it would be hard to score any text on the internet.
Watermarking Summary and Limitations
So yeah, the summary is still the same, though. The thing that is different between no watermarking and watermarking is that we are controlling this sampling here with the watermarking key. And inside that, we have this tournament sampling. And yeah, as I mentioned before, detecting the watermarks requires the secret key and the watermarking functions G1 to Gn.
And again, removing the watermark, because I think that’s maybe interesting to some people, would ideally involve editing all the positions here. But since we don’t know which positions are watermarked and internally, they choose the positions so that they have equally likely tokens at those positions. And there might be positions where that’s not true. So here, for example, for “trees”, we might not even have an alternative word that is high scoring so they don’t watermark that position.
So they only do the watermarking at certain positions essentially. Since we don’t know which positions to kind of defeat or remove the watermark, we would...
...have to edit several places in the text. So what I think that means for the future of AI-generated text is that this actually...
What This Means for AI-Generated Text
…might result in worse AI-generated text. So I think if there’s a person who likes to use AI-generated text everywhere on the internet, let’s say there’s a news website that likes to use AI-generated text to write the news, I don’t think watermarking will necessarily stop them from doing that. They will probably still want to generate AI-generated text because that’s part of their workflow. So I think my guess is that they’ll use another model.
They’ll just use a second model to edit the text to get the so-called edited AI-generated text. So it’s complicating the pipeline. Instead of getting the text directly from Claude, it’s now using Claude to generate AI-generated text, passing it through a local model, and then having edited AI-generated text, which is likely not watermarked anymore. So why a local model? I just think a local model because I think all the providers- the proprietary LLMs, not only Claude, but also Google— I mean, Google wrote this paper, right?
So I’m thinking that they are also watermarking Gemini text. And I think OpenAI is probably already doing it or will do so as well. I mean, I’m just speculating, but I’m imagining everyone will probably do something like that because there’s like an EU regulation that requires that. And that’s, according to the blog post, apparently why Claude is doing it. Yeah, so I’m thinking local models may not, at least not yet, implement this watermarking.
So I think people will just use a local model and then generate edited AI-generated text. And my guess is it will be slightly worse than the original text because for the local model, you might now be using a smaller model. So, I mean, you could also technically just use the local model directly to generate text. But in my view, editing text is simpler than generating text. So for the generation of the text, you might use a very expensive high-end, I don’t know, like the highest, most expensive Claude model for complicated text.
And then you use a cheaper local model to make these surgical edits, essentially. That’s probably what’s going to happen. And why worse? So if we look back at this graphic where we just added random positions, you might be just changing words for the sake of changing them. And then it risks making the text worse. So you might still have generated text, but it’s kind of like it’s edited awkwardly.
Final Thoughts
But anyway, so my goal here was to explain how the watermarking works and not, let’s say, the worldwide ramifications of that. But I hope this kind of behind-the-scenes, under-the-hood look is useful. The watermarking is not as complicated as it might seem, but I think it was still 52 slides, so it was also not super trivial. So I hope you found this little lecture useful. And yeah, until next time, see you then.
PS: If you like more explainers in this style, I don’t post videos to YouTube regularly, but I have accumulated over 300 videos over the years, which you can find on my YouTube channel here.
So good. The walkthrough of token sampling and watermark detection makes a tricky topic really clear.
This feels very similar to that recent paper about using LLMs to pass secret messages, I don’t recall the title.