In the last couple of months, we have seen a lot of people and companies sharing and open-sourcing various kinds of LLMs and datasets, which is awesome.
Delightful to read as always. I enjoy the concise summarizations of recent AI development, especially in this super fast pace era. Also thank you for sharing your thoughts on a lot of these problems. This opens up a lot of discussions and potential research areas.
Great insights on this AI issue. While the readings is a well fit-purpose for researchers and alike to allow replicating work, I was wondering in some sort of as an extended arm on these LLMs topics, if a pragmatic approach on how to implement all of these papers/research work via applications (streamlit, gradio, etc., just to name a few) can be covered as well with practical examples.
I like that suggestion. It could be something to do for selected LLMs. The only challenge is the resource requirements (e.g., when I used a int4-quantized Falcon-40B for inference, it still required > 30 Gb RAM; used an A100). But yeah, it's something to think about for sure!
Thanks for your reply. With your domain expertise on the field if you provide some How to Procedures to get these requirements and some sort of stepped guidance I believe we might attempt to get something along the way. I hope you don't take this as being myself/ourselves lazy, I just think since you have got a lot of ground walk on the field we may benefit from a point reference to start doing some cool stuff for the community.
Got it. Will get my hands on and keep reporting my findings. Thanks Sebastian, needless to say your work is tremendous and I am happy to be your supporter and subscriber. Keep up with the good work.
Delightful to read as always. I enjoy the concise summarizations of recent AI development, especially in this super fast pace era. Also thank you for sharing your thoughts on a lot of these problems. This opens up a lot of discussions and potential research areas.
Thanks for kind words! Glad to hear!
Great insights on this AI issue. While the readings is a well fit-purpose for researchers and alike to allow replicating work, I was wondering in some sort of as an extended arm on these LLMs topics, if a pragmatic approach on how to implement all of these papers/research work via applications (streamlit, gradio, etc., just to name a few) can be covered as well with practical examples.
I like that suggestion. It could be something to do for selected LLMs. The only challenge is the resource requirements (e.g., when I used a int4-quantized Falcon-40B for inference, it still required > 30 Gb RAM; used an A100). But yeah, it's something to think about for sure!
Thanks for your reply. With your domain expertise on the field if you provide some How to Procedures to get these requirements and some sort of stepped guidance I believe we might attempt to get something along the way. I hope you don't take this as being myself/ourselves lazy, I just think since you have got a lot of ground walk on the field we may benefit from a point reference to start doing some cool stuff for the community.
Sure! You could try Lit-Parrot for Falcon-40B for example. Here's an usage example for generating new texts: https://vimeo.com/833068632
And here are some setup instructions:
- https://github.com/Lightning-AI/lit-parrot/blob/main/howto/download_falcon.md
- https://github.com/Lightning-AI/lit-parrot/blob/main/howto/inference.md
Please let me know in case you have questions about this.
Got it. Will get my hands on and keep reporting my findings. Thanks Sebastian, needless to say your work is tremendous and I am happy to be your supporter and subscriber. Keep up with the good work.