-**coba_update_interval**: The frequency at which CoBa updates weights. It is commonly set to 1, meaning weights are updated at every step.

-**coba_sample_valid_num**: The number of validation batches to be sampled by CoBa at each step. Theoretically, when this value equals the total number of validation batches, the fitted convergence slope most closely approximates the actual situation. However, considering computational requirements, it is recommended to set it to 1.

-**xxpo**: preference optimization type, "dpo" or "orpo".

-**beta**: DPO beta, smaller beta allows larger distance between dpo model and ref model.

-**rpo_alpha**: The coefficient of the```chosen``` NLL loss added to dpo loss.

-**coba_update_interval**: CoBa更新权重的频率。一般设置为1，即每一步都对权重做更新。

-**coba_sample_valid_num**: CoBa每一步要取的valid batch数。理论上当该值等于valid batch总数量时，拟合出的收敛斜率最逼近真实情况，但考虑到计算需求，建议设置为1。

-**xxpo**: 偏好对齐方法, "dpo" 或者 "orpo".

-**beta**: DPO beta, beta 越小，允许对齐后的dpo模型与ref模型的距离越远

-**rpo_alpha**: 加到dop损失的```chosen``` NLL损失的系数，0的话就是原始DPO。