Subscribe
Sign in
Home
Support & Books
Website
Archive
New
Top
Discussion
Research Papers in February 2024: A LoRA Successor, Small Finetuned LLMs Vs Generalist LLMs, and Transparent LLM Research
Once again, this has been an exciting month in AI research. This month, I'm covering two new openly available LLMs, insights into small finetuned LLMs…
Mar 3
•
Sebastian Raschka, PhD
72
Share this post
Research Papers in February 2024: A LoRA Successor, Small Finetuned LLMs Vs Generalist LLMs, and Transparent LLM Research
magazine.sebastianraschka.com
Copy link
Facebook
Email
Note
Other
13
February 2024
Improving LoRA: Implementing Weight-Decomposed Low-Rank Adaptation (DoRA) from Scratch
Low-rank adaptation (LoRA) is a machine learning technique that modifies a pretrained model (for example, an LLM or vision transformer) to better suit a…
Feb 18
•
Sebastian Raschka, PhD
148
Share this post
Improving LoRA: Implementing Weight-Decomposed Low-Rank Adaptation (DoRA) from Scratch
magazine.sebastianraschka.com
Copy link
Facebook
Email
Note
Other
32
Research Papers in Jan 2024: Model Merging, Mixtures of Experts, and Towards Smaller LLMs
Model Merging, Mixtures of Experts, and Towards Smaller LLMs
Feb 3
•
Sebastian Raschka, PhD
159
Share this post
Research Papers in Jan 2024: Model Merging, Mixtures of Experts, and Towards Smaller LLMs
magazine.sebastianraschka.com
Copy link
Facebook
Email
Note
Other
11
January 2024
Understanding and Coding Self-Attention, Multi-Head Attention, Cross-Attention, and Causal-Attention in LLMs
This article will teach you about self-attention mechanisms used in transformer architectures and large language models (LLMs) such as GPT-4 and Llama…
Jan 14
244
Share this post
Understanding and Coding Self-Attention, Multi-Head Attention, Cross-Attention, and Causal-Attention in LLMs
magazine.sebastianraschka.com
Copy link
Facebook
Email
Note
Other
23
December 2023
Ten Noteworthy AI Research Papers of 2023
This year has felt distinctly different. I've been working in, on, and with machine learning and AI for over a decade, yet I can't recall a time when…
Dec 30, 2023
•
Sebastian Raschka, PhD
332
Share this post
Ten Noteworthy AI Research Papers of 2023
magazine.sebastianraschka.com
Copy link
Facebook
Email
Note
Other
31
Research Papers in Nov 2023: Tackling Hallucinations, Boosting Reasoning Abilities, and New Insights into the Transformer Architecture
This month, I want to focus on three papers that address three distinct problem categories of Large Language Models (LLMs): Reducing hallucinations…
Dec 9, 2023
•
Sebastian Raschka, PhD
109
Share this post
Research Papers in Nov 2023: Tackling Hallucinations, Boosting Reasoning Abilities, and New Insights into the Transformer Architecture
magazine.sebastianraschka.com
Copy link
Facebook
Email
Note
Other
13
November 2023
Practical Tips for Finetuning LLMs Using LoRA (Low-Rank Adaptation)
Things I Learned From Hundreds of Experiments
Nov 19, 2023
•
Sebastian Raschka, PhD
186
Share this post
Practical Tips for Finetuning LLMs Using LoRA (Low-Rank Adaptation)
magazine.sebastianraschka.com
Copy link
Facebook
Email
Note
Other
38
Research Papers in Oct 2023: A Potential Successor to RLHF for Efficient LLM Alignment and the Resurgence of CNNs
From Vision Transformers to innovative large language model finetuning techniques, the AI community has been very active with lots of interesting…
Nov 4, 2023
•
Sebastian Raschka, PhD
70
Share this post
Research Papers in Oct 2023: A Potential Successor to RLHF for Efficient LLM Alignment and the Resurgence of CNNs
magazine.sebastianraschka.com
Copy link
Facebook
Email
Note
Other
October 2023
AI and Open Source in 2023
The Highs and Lows: A Year in Review
Oct 23, 2023
•
Sebastian Raschka, PhD
104
Share this post
AI and Open Source in 2023
magazine.sebastianraschka.com
Copy link
Facebook
Email
Note
Other
11
LLM Business and Busyness: Recent Company Investments and AI Adoption, New Small Openly Available LLMs, and LoRA Research
Discussing Recent Company Investments and AI Adoption, New Small Openly Available LLMs, and LoRA Research
Oct 8, 2023
•
Sebastian Raschka, PhD
73
Share this post
LLM Business and Busyness: Recent Company Investments and AI Adoption, New Small Openly Available LLMs, and LoRA Research
magazine.sebastianraschka.com
Copy link
Facebook
Email
Note
Other
17
September 2023
Research Papers Aug-Sep 2023: From Self-Alignment to LongLoRA
Another month, another round of interesting research papers ranging from large language modeling to computer vision. One recent focus is on refining…
Sep 23, 2023
•
Sebastian Raschka, PhD
56
Share this post
Research Papers Aug-Sep 2023: From Self-Alignment to LongLoRA
magazine.sebastianraschka.com
Copy link
Facebook
Email
Note
Other
4
LLM Training: RLHF and Its Alternatives
I frequently reference a process called Reinforcement Learning with Human Feedback (RLHF) when discussing LLMs, whether in the research news or…
Sep 10, 2023
•
Sebastian Raschka, PhD
135
Share this post
LLM Training: RLHF and Its Alternatives
magazine.sebastianraschka.com
Copy link
Facebook
Email
Note
Other
5
This site requires JavaScript to run correctly. Please
turn on JavaScript
or unblock scripts