Movatterモバイル変換

[0]ホーム
URL:

Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

 Sign up
Appearance settings

Commita4ad4f3

Browse files
committed
readme
1 parent44ddee1 commita4ad4f3

File tree

2 files changed

+15
-3
lines changed

2 files changed

+15
-3
lines changed

‎mftcoder_accelerate/README.md

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -193,7 +193,13 @@ accelerate launch --config_file accelerate_fsdp_config.yaml pefts/mft_accelerate
193193
or
194194
FSDP config in command line arguments
195195
```bash
196-
sh ds_single_launch.sh
196+
sh fsdp_single_launch.sh
197+
```
198+
199+
####MultiNode Launch
200+
Refer to the deepspeed multi-node launch script below.
201+
```bash
202+
sh ds_multinode_launch.sh
197203
```
198204

199205
####Traing Arguments

‎mftcoder_accelerate/README_cn.md

Lines changed: 8 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -146,7 +146,7 @@ QLoRA通过4-bit的nf4量化，且加入更多adapter，在大幅减少显存消
146146
QLoRA论文指出，该方法可以在一张V100上对33B的模型进行微调，并且性能逼近全量参数微调。
147147

148148
执行如下命令即可进行 Lora/QLora/全量 微调：
149-
####Launch viaDeepspeed
149+
####Deepspeed 单机启动
150150
DeepSpeed配置在accelerate_ds_config.yaml中。
151151
```bash
152152
accelerate launch --config_file accelerate_ds_config.yaml pefts/mft_accelerate.py --train_config configs/xxx_train_config.json --distributed_type"DeepSpeed"
@@ -163,7 +163,7 @@ DeepSpeed Zero3 配置在脚本中通过命令行输入
163163
sh ds_zero3_single_launch.sh
164164
```
165165

166-
####Launch viaFSDP
166+
####FSDP 单机启动
167167
FSDP配置在accelerate_fsdp_config.yaml中。
168168
```bash
169169
accelerate launch --config_file accelerate_fsdp_config.yaml pefts/mft_accelerate.py --train_config configs/xxx_train_config.json --distributed_type"FSDP"
@@ -175,6 +175,12 @@ FSDP配置在脚本中通过命令行输入。
175175
sh fsdp_single_launch.sh
176176
```
177177

178+
####多机启动
179+
多机启动请参考如下deepspeed多机启动脚本
180+
```bash
181+
sh ds_multinode_launch.sh
182+
```
183+
178184
####训练参数
179185
_**训练需要的参数配置在```configs/*_train_config```中，主要参数说明如下：**_
180186

0 commit comments

Comments
 (0)
[8]ページ先頭
©2009-2025 Movatter.jp