Sharing a minimal GPT implementation in Rust using Burn

[izhangshuo]

Hi Burn community,

I wanted to share a minimal GPT-like model implemented in Rust using Burn.
The goal of this project is to help anyone interested in learning how to implement and train a transformer/GPT model with Burn. It includes examples for both English and Chinese datasets.

You can check out the repository here: https://github.com/izhangshuo/mygpt

📘 mygpt — GPT Implementation in Rust using Burn

A minimal GPT‑style model implemented in Rust using the Burn machine learning framework.
This project demonstrates how to build and train a simple transformer language model from scratch. It includes both English and Chinese example models and provides training and inference code.

🎥 Video Demonstrations

You can also find full walkthrough videos:

🌐 YouTube (Chinese): https://www.youtube.com/playlist?list=PL3lRZ4mOrPobeASemC7hvDfGc93a530gB

📺 Bilibili (Chinese): https://www.bilibili.com/video/BV1A2tnzSEsv

Feel free to watch them for setup, training, and usage guidance.

🚀 How to Use

1. Choose a Dataset

Copy one of the model folders under the root (artifact_en, artifact_cn, or artifact_4in1) to a folder named artifact.

Example:

cp -r artifact_en artifact

2. Modify Source File

Open the source code in your editor, and replace the model file name inside the code:

let text = include_str!("4in1.txt");

Replace "4in1.txt" with the filename you want to train on.

3. Train or Inference

Use cargo to train or generate text — just like any Rust project:

cargo run --release

The training script will train a simplified GPT model on the chosen dataset.
After training, inference will generate text continuations.

📌 Notes

• This implementation is designed for education and experimentation, not state‑of‑the‑art performance.

• The model and code are kept minimal and easy to understand.

• Burn allows us to build and train neural networks fully in Rust.

🧠 Why This Project

This repository aims to:

• Show how to structure transformer models in Rust using Burn

• Provide training and inference examples for different language datasets

• Help Rust/Burn users understand GPT architectures

🗂 Additional Notes

The note folder contains the original files of the diagrams used in the video. These can be opened with Microsoft OneNote.

For users who do not have OneNote installed, you can view the diagrams in the note.mht file, which is a single web page.

📝 License

This project is licensed under MIT License.

[laggui]

Thanks for sharing!

If you're not already on discord, come join us. We have a #showcase channel where community members share burn-related projects 🙂

[izhangshuo]

Thanks for the invitation! I'd be happy to share the project there.

Could you share the Discord invite link? I couldn't find it in the message.

And if there are any suggestions to improve the project before sharing it in the showcase channel, I'd be glad to update it.

[laggui]

Weird, I could have sworn I had linked it in my previous message but apparently I only put [join us] without the link 😅 I updated my message, but here is the direct invite link: https://discord.gg/uPEBbYYDB6

And if there are any suggestions to improve the project before sharing it in the showcase channel, I'd be glad to update it.

I haven't looked at the implementation more carefully, but looks like you're still on burn 0.17. Would be nice to update it to a more recent release.

[izhangshuo]

Thanks for the suggestion! I’ve updated the project to use the latest Burn release (0.20.1). I’ll also try to keep it updated as new versions come out. Thanks again for taking a look at the project and for sharing the Discord invite!

…

________________________________ From: Guillaume Lagrange ***@***.***> Sent: Wednesday, March 11, 2026 7:58 PM To: tracel-ai/burn ***@***.***> Cc: izhangshuo ***@***.***>; Author ***@***.***> Subject: Re: [tracel-ai/burn] Sharing a minimal GPT implementation in Rust using Burn (Discussion #4605) Weird, I could have sworn I had linked it in my previous message but apparently I only put [join us] without the link 😅 I updated my message, but here is the direct invite link: https://discord.gg/uPEBbYYDB6 And if there are any suggestions to improve the project before sharing it in the showcase channel, I'd be glad to update it. I haven't looked at the implementation more carefully, but looks like you're still on burn 0.17. Would be nice to update it to a more recent release. — Reply to this email directly, view it on GitHub<#4605 (reply in thread)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ABRWXVZBPIMUPHMNCS5V5734QFIGTAVCNFSM6AAAAACWJ6R3FKVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTMMBYGI3DKNA>. You are receiving this because you authored the thread.Message ID: ***@***.***>

[laggui]

My pleasure! Don't hesitate to ask questions on discord if you have any.

[CytITDchen]

Wait I saw you on bilibili teaching rust burn, I had followed you before to learn burn.