***[2023.10.30]** Add the AIOps Leaderboard.

***[2023.10.25]** Add the AIOps samples, including log parsing, time series anomaly detection, time series classification and root cause analysis.

***[2023.10.18]** Update the initial Leaderboard...

<br>

Below are zero-shot and five-shot accuracies from the models that we evaluate in the initial release. We note that five-shot performance is better than zero-shot for many instruction-tuned models.

|**ModelName**| plan| code| build| test| release| deploy| operate| monitor|**AVG**|

| Baichuan2-7B-Chat| 60.61| 64.95| 81.19| 75.88| 71.23| 75.69| 78.36| 79.17| 70.49|

| Internlm-7B-Base| 62.12| 65.25| 77.52| 80.7| 74.06| 78.82| 79.85| 75.46| 69.17|

|**ModelName**| LogParsing| RootCauseAnalysis| TimeSeriesAnomalyDetection| TimeSeriesClassification|**AVG**|

|:-------------------:|:------------:|:------------------:|:---------------------------:|:-------------------------:|:-------:|

***[2023.10.30]** 增加针对AIOps场景的评测排行榜

***[2023.10.25]** 增加AIOps样本，包含日志解析、时序异常检测、时序分类和根因分析

***[2023.10.18]** DevOps-Eval发布大模型评测排行版

<br>

以下是我们获得的初版评测结果，包括多个开源模型的zero-shot和five-shot准确率。我们注意到，对于大多数指令模型来说，five-shot的准确率要优于zero-shot。

|**模型**| plan| code| build| test| release| deploy| operate| monitor|**平均分**|

| Internlm-7B-Base| 62.12| 65.25| 77.52| 80.7| 74.06| 78.82| 79.85| 75.46| 69.17|

|**模型**| 日志解析| 根因分析| 时序异常检测| 时序分类|**平均分**|

|:-------------------:|:-----:|:----:|:------:|:----:|:-------:|