In this edition of the newsletter, we direct our attention to one of the most prominent highlights of the summer: the release of the Llama 2 base and chat models, as well as CodeLlama, the latest highlights in the open-source AI large language model (LLM) landscape.
It's a real pleasure to read the summary of such a knowledgeable man as you, kudos!
Now, I'm a little bit surprised that you didn't introduce multi-modality in LLMs as a main axis of research/differentiation. Pairing text to vision is already relatively straightforward and there is also sub-modality differentiation in the sound landscape with speech to text enlarged to text to sound (e.g. the Bark model), but this is only the beginning....
IMHO, generalized multi-modality it is a too neglected straightforward path towards AGI as it would solve a good part o the thorny issue of symbol grounding in those models (the other parts being feedback from retro-action from the world to the models, a direct pathway toward the synthesis of genuine evolved intentionality).
With models encompassing our 5 senses and proprioception, they should evolve inner world representations more aligned with human ones. One can even speculate if those LLMs would converge toward an universal underlying neural coding scheme like e.g. a kind of Grossberg' ART refinement, as described in this paper, https://www.mdpi.com/2078-2489/14/2/82
Awesome write-up. I tend to try to follow the news as they occur, but you do such a great job in distilling everything that I may just consume your newsletter. I wonder if you use any LLM to help you writing or organizing raw text.
I am not sure if I fully understand this sentence: "That's because new knowledge is usually ingested via pretraining, not finetuning; this is also true for open-source models." My impression was that finetuning was the way to inject new knowledge, or what exactly is meant here?
I have a question, are there any benchmarking analysis comparing finetuning model weights vs prompt (or prefix) tuning? My understanding is that the later is only preferred since it is a much easier training job, but performance-wise, finetuning the model weights yields better results. is that correct?
It's a real pleasure to read the summary of such a knowledgeable man as you, kudos!
Now, I'm a little bit surprised that you didn't introduce multi-modality in LLMs as a main axis of research/differentiation. Pairing text to vision is already relatively straightforward and there is also sub-modality differentiation in the sound landscape with speech to text enlarged to text to sound (e.g. the Bark model), but this is only the beginning....
IMHO, generalized multi-modality it is a too neglected straightforward path towards AGI as it would solve a good part o the thorny issue of symbol grounding in those models (the other parts being feedback from retro-action from the world to the models, a direct pathway toward the synthesis of genuine evolved intentionality).
With models encompassing our 5 senses and proprioception, they should evolve inner world representations more aligned with human ones. One can even speculate if those LLMs would converge toward an universal underlying neural coding scheme like e.g. a kind of Grossberg' ART refinement, as described in this paper, https://www.mdpi.com/2078-2489/14/2/82
Awesome write-up. I tend to try to follow the news as they occur, but you do such a great job in distilling everything that I may just consume your newsletter. I wonder if you use any LLM to help you writing or organizing raw text.
Thanks for the newlsetter!
I am not sure if I fully understand this sentence: "That's because new knowledge is usually ingested via pretraining, not finetuning; this is also true for open-source models." My impression was that finetuning was the way to inject new knowledge, or what exactly is meant here?
I was curious about the unnatural code Llama model. Why didn’t Meta release its weights? Its performance is the closest to GPT-4.
Thank you for the write up. enjoyed reading it.
I have a question, are there any benchmarking analysis comparing finetuning model weights vs prompt (or prefix) tuning? My understanding is that the later is only preferred since it is a much easier training job, but performance-wise, finetuning the model weights yields better results. is that correct?
ditto... Again WOW amazing coverage!
As always an amazing overview!
Wow amazing coverage!
I got overwhelmed following all the AI news last week. But reading your newsletter is a relief, I feel I am on top of the things now :)