Great tutorial! I have a question about how ambiguous words like "duck" when they appear multiple times with different meanings would be treated in this case.
For example, in the sentence:
"He had to duck when the duck flew at him."
The first "duck" is a verb, and the second is a noun. Since we make use of cached key-value (KV) pairs for previously seen tokens, what happens in this case?
If we cache the first "duck" (the verb), do we simply reuse its KV pair when we encounter the second "duck"? Shouldn't their representations be different due to their distinct roles and contexts?
Great to see you back in action!! I hope your recovery is proceeding well.
Love this! Thank you for sharing. I hope your recovery is swift.
Great tutorial! I have a question about how ambiguous words like "duck" when they appear multiple times with different meanings would be treated in this case.
For example, in the sentence:
"He had to duck when the duck flew at him."
The first "duck" is a verb, and the second is a noun. Since we make use of cached key-value (KV) pairs for previously seen tokens, what happens in this case?
If we cache the first "duck" (the verb), do we simply reuse its KV pair when we encounter the second "duck"? Shouldn't their representations be different due to their distinct roles and contexts?
Could question, and I see what you mean. However, this would not be an issue since we retrieve the previous keys and values by position.
Please, I want to become a paid subscriber so I can see your closed content. Thank you