- Notifications
You must be signed in to change notification settings - Fork11.9k
-
Hi, I'm interested in using Build a Large Language Model (From Scratch) for a school project, but I’d like to make sure my PC specs are sufficient before purchasing the book. Here are my specs: CPU: 12th Gen Intel Core i7-12650H @ 2.30 GHz RAM: 32 GB GPU: NVIDIA GeForce RTX 3050 Laptop GPU (6GB VRAM) Available storage: ~150 GB Would this setup be enough to follow the practical parts of the book, at least for training small-scale models or running examples locally? Thank you for your help! |
BetaWas this translation helpful?Give feedback.
All reactions
❤️ 1
Hi,
Imo, on CPU training with your ram, you'll have no problem, you'll be able to run all chapters smoothly without having to worry about memory. Training on CPU is a bit slow though.
If instead you want to use your Cuda GPU, some parts might lead you into OOM territory in f32, depending on the chapters, not sure.
@rasbt can give a better answer on that, I don't remember exactly. But It'll be feasible in bf16model.to(torch.bfloat16).
You can also reduce some hyperparameters likebatch_size orcontext_length to fit everything in VRAM. Even use Google Colab too, if needed.
Sebastian made the book with these limits and a broader audience in mind, I don't think you'll run into trouble tbh.
Replies: 2 comments 3 replies
-
Hi, Imo, on CPU training with your ram, you'll have no problem, you'll be able to run all chapters smoothly without having to worry about memory. Training on CPU is a bit slow though. If instead you want to use your Cuda GPU, some parts might lead you into OOM territory in f32, depending on the chapters, not sure. Sebastian made the book with these limits and a broader audience in mind, I don't think you'll run into trouble tbh. |
BetaWas this translation helpful?Give feedback.
All reactions
❤️ 1
-
Thank you so much for your kind and detailed response — I really appreciate it! Thanks again for your help! |
BetaWas this translation helpful?Give feedback.
All reactions
👍 1
-
Hi there, Additionally, I think you will probably be able to use the GPU for most if not all chapters. 6GB RAM on the GPU might be tight but I think it is enough (the code will automatically use a CUDA-supported GPU if available). However, in case the 6 GB VRAM are too tight, you can always run the code on the CPU; that's what I've done on the 2020 MacBook Air mentioned above. Alternatively, you can use bf16 as recommended by another reader. (There are some more tips here:https://github.com/rasbt/LLMs-from-scratch/tree/main/ch05/10_llm-training-speed but I would not worry and read that until you have finished the book to be honest) Long story short, your setup should be sufficient. |
BetaWas this translation helpful?Give feedback.
All reactions
❤️ 1
-
Thank you so much for your kind reply and confirmation — I’m very happy to hear that my setup should be sufficient! |
BetaWas this translation helpful?Give feedback.
All reactions
-
I have a similar question, I find this tool very helpful on monitoring single gpu machinehttps://github.com/ulixcode-labs/GPU-pro |
BetaWas this translation helpful?Give feedback.