Subscribe
Sign in
Home
Support & Books
Website
Upcoming Talks
Archive
Latest
Top
Discussions
How Good Are the Latest Open LLMs? And Is DPO Better Than PPO?
Discussing the Latest Model Releases and AI Research in April 2024
May 12
•
Sebastian Raschka, PhD
89
Share this post
How Good Are the Latest Open LLMs? And Is DPO Better Than PPO?
magazine.sebastianraschka.com
Copy link
Facebook
Email
Note
Other
4
April 2024
Using and Finetuning Pretrained Transformers
What are the different ways to use and finetune pretrained large language models (LLMs)? The most common ways to use and finetune pretrained LLMs…
Apr 20
•
Sebastian Raschka, PhD
106
Share this post
Using and Finetuning Pretrained Transformers
magazine.sebastianraschka.com
Copy link
Facebook
Email
Note
Other
13
March 2024
Tips for LLM Pretraining and Evaluating Reward Models
Discussing AI Research Papers in March 2024
Mar 31
•
Sebastian Raschka, PhD
127
Share this post
Tips for LLM Pretraining and Evaluating Reward Models
magazine.sebastianraschka.com
Copy link
Facebook
Email
Note
Other
16
Research Papers in February 2024: A LoRA Successor, Small Finetuned LLMs Vs Generalist LLMs, and Transparent LLM Research
Once again, this has been an exciting month in AI research. This month, I'm covering two new openly available LLMs, insights into small finetuned LLMs…
Mar 3
•
Sebastian Raschka, PhD
77
Share this post
Research Papers in February 2024: A LoRA Successor, Small Finetuned LLMs Vs Generalist LLMs, and Transparent LLM Research
magazine.sebastianraschka.com
Copy link
Facebook
Email
Note
Other
13
February 2024
Improving LoRA: Implementing Weight-Decomposed Low-Rank Adaptation (DoRA) from Scratch
Low-rank adaptation (LoRA) is a machine learning technique that modifies a pretrained model (for example, an LLM or vision transformer) to better suit a…
Feb 18
•
Sebastian Raschka, PhD
158
Share this post
Improving LoRA: Implementing Weight-Decomposed Low-Rank Adaptation (DoRA) from Scratch
magazine.sebastianraschka.com
Copy link
Facebook
Email
Note
Other
37
Research Papers in Jan 2024: Model Merging, Mixtures of Experts, and Towards Smaller LLMs
Model Merging, Mixtures of Experts, and Towards Smaller LLMs
Feb 3
•
Sebastian Raschka, PhD
168
Share this post
Research Papers in Jan 2024: Model Merging, Mixtures of Experts, and Towards Smaller LLMs
magazine.sebastianraschka.com
Copy link
Facebook
Email
Note
Other
15
January 2024
Understanding and Coding Self-Attention, Multi-Head Attention, Cross-Attention, and Causal-Attention in LLMs
This article will teach you about self-attention mechanisms used in transformer architectures and large language models (LLMs) such as GPT-4 and Llama…
Jan 14
260
Share this post
Understanding and Coding Self-Attention, Multi-Head Attention, Cross-Attention, and Causal-Attention in LLMs
magazine.sebastianraschka.com
Copy link
Facebook
Email
Note
Other
27
December 2023
Ten Noteworthy AI Research Papers of 2023
This year has felt distinctly different. I've been working in, on, and with machine learning and AI for over a decade, yet I can't recall a time when…
Dec 30, 2023
•
Sebastian Raschka, PhD
341
Share this post
Ten Noteworthy AI Research Papers of 2023
magazine.sebastianraschka.com
Copy link
Facebook
Email
Note
Other
32
Research Papers in Nov 2023: Tackling Hallucinations, Boosting Reasoning Abilities, and New Insights into the Transformer Architecture
This month, I want to focus on three papers that address three distinct problem categories of Large Language Models (LLMs): Reducing hallucinations…
Dec 9, 2023
•
Sebastian Raschka, PhD
111
Share this post
Research Papers in Nov 2023: Tackling Hallucinations, Boosting Reasoning Abilities, and New Insights into the Transformer Architecture
magazine.sebastianraschka.com
Copy link
Facebook
Email
Note
Other
13
November 2023
Practical Tips for Finetuning LLMs Using LoRA (Low-Rank Adaptation)
Things I Learned From Hundreds of Experiments
Nov 19, 2023
•
Sebastian Raschka, PhD
201
Share this post
Practical Tips for Finetuning LLMs Using LoRA (Low-Rank Adaptation)
magazine.sebastianraschka.com
Copy link
Facebook
Email
Note
Other
42
Research Papers in Oct 2023: A Potential Successor to RLHF for Efficient LLM Alignment and the Resurgence of CNNs
From Vision Transformers to innovative large language model finetuning techniques, the AI community has been very active with lots of interesting…
Nov 4, 2023
•
Sebastian Raschka, PhD
70
Share this post
Research Papers in Oct 2023: A Potential Successor to RLHF for Efficient LLM Alignment and the Resurgence of CNNs
magazine.sebastianraschka.com
Copy link
Facebook
Email
Note
Other
October 2023
AI and Open Source in 2023
The Highs and Lows: A Year in Review
Oct 23, 2023
•
Sebastian Raschka, PhD
106
Share this post
AI and Open Source in 2023
magazine.sebastianraschka.com
Copy link
Facebook
Email
Note
Other
11
Share
Copy link
Facebook
Email
Note
Other
This site requires JavaScript to run correctly. Please
turn on JavaScript
or unblock scripts