Movatterモバイル変換


[0]ホーム

URL:


Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

Commit3fb511d

Browse files
author
jimmy.xj
committed
Update README.md
1 parent3e784a4 commit3fb511d

File tree

2 files changed

+53
-5
lines changed

2 files changed

+53
-5
lines changed

‎README.md‎

Lines changed: 26 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -11,9 +11,9 @@ DevOps-Eval is a comprehensive evaluation suite specifically designed for founda
1111

1212
📚 This repo contains questions and exercises related to DevOps, including the AIOps.
1313

14-
🔥️ There are currently**4850** multiple-choice questions spanning 8 diverse general categories, as shown[below](images/data_info.png).
14+
💥️ There are currently**4850** multiple-choice questions spanning 8 diverse general categories, as shown[below](images/data_info.png).
1515

16-
💥 There are a total of**2200** samples in the AIOps subcategory, covering scenarios such as**log parsing**,**time series anomaly detection**,**time series classification**,**and root cause analysis**.
16+
🔥 There are a total of**2200** samples in the AIOps subcategory, covering scenarios such as**log parsing**,**time series anomaly detection**,**time series classification**,**and root cause analysis**.
1717

1818
<palign="center"> <ahref="resources/devops_diagram_zh.jpg"> <imgsrc="images/data_info.png"style="width:100%;"id="data_info"></a></p>
1919

@@ -28,6 +28,7 @@ DevOps-Eval is a comprehensive evaluation suite specifically designed for founda
2828
-[🏆 Leaderboard](#-leaderboard)
2929
-[⏬ Data](#-data)
3030
-[👀 Notes](#-notes)
31+
-[🔥 AIOps Sample Example](#-aiops-sample-example)
3132
-[🚀 How to Evaluate](#-how-to-evaluate)
3233
-[🧭 TODO](#-todo)
3334
-[🏁 Licenses](#-licenses)
@@ -133,6 +134,29 @@ explanation:
133134
分析原因:
134135
git diff--name-onlySHA命令会显示与SHA参数对应的提交中已修改的文件列表。参数--name-only让命令只输出文件名,而忽略其他信息。其它选项中的命令并不能实现此功能。
135136
```
137+
#### 🔥 AIOps Sample Example
138+
👀 👀 Taking log parsingas an example, hereis a brief showcase of the AIOps samples:
139+
```
140+
id:0
141+
question:
142+
Here are some running logs
143+
004:21:15,429WARN Cannotopen channel to2 at election address/10.10.34.12:3888
144+
119:18:56,377WARN*******GOODBYE/10.10.34.11:52703********
145+
219:13:46,128WARN*******GOODBYE/10.10.34.11:52308********
146+
319:16:26,268WARN*******GOODBYE/10.10.34.11:52502********
147+
409:11:16,012WARN Cannotopen channel to3 at election address/10.10.34.13:3888
148+
516:37:13,837WARN Cannotopen channel to2 at election address/10.10.34.12:3888
149+
609:09:16,008WARN Cannotopen channel to3 at election address/10.10.34.13:3888
150+
715:27:03,681WARN Cannotopen channel to3 at election address/10.10.34.13:3888
151+
The first three parts of the log are index, timestamp,and log level. Without considering these three parts, Here we assume that the variablesin the logs are representedas'<*>', separated by spaces between tokens. Whatis the specific log templatefor the above logs?
152+
A: Notification time out:<*> 和 Connection brokenforid<*>, myid =<*>, error =
153+
B: Send worker leaving thread 和 Connection brokenforid<*>, myid =<*>, error =
154+
C: Received connection request/<*>:<*> 和 Interrupting SendWorker
155+
D: Cannotopen channel to<*> at election address/<*>:<*>*******GOODBYE/<*>:<*>********
156+
answer: D
157+
explanation: The log includes the fixed template fragments"Cannot open channel to <> at election address /<>:<>"and"****** GOODBYE /<>:<> ********," both of which appearin option D. Meanwhile, the template fragmentsin the other options donot match the contentin the log. Therefore, option Dis the most consistentwith the log template.
158+
```
159+
136160

137161
## 🚀 How to Evaluate
138162
If you need to test your own huggingface-formatted model, the overall steps areas follows:

‎README_zh.md‎

Lines changed: 27 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -10,9 +10,9 @@ DevOps-Eval是一个专门为DevOps领域大模型设计的综合评估数据集
1010

1111
📚 该仓库包含与DevOps和AIOps相关的问题和练习。
1212

13-
🔥️ 目前有 4850 个多项选择题,根据DevOps的通用流程将其归纳未8个模块,如[下图](images/data_info.png)所示。
13+
💥 目前有 4850 个多项选择题,根据DevOps的通用流程将其归纳未8个模块,如[下图](images/data_info.png)所示。
1414

15-
💥 AIOps子类当前已有中英文样本2200例,覆盖的场景包括**日志解析****时序异常检测****时序分类****根因分析**
15+
🔥 AIOps子类当前已有中英文样本2200例,覆盖的场景包括**日志解析****时序异常检测****时序分类****根因分析**
1616

1717

1818
<palign="center"> <ahref="resources/devops_diagram_zh.jpg"> <imgsrc="images/data_info.png"style="width:100%;"id="data_info"></a></p>
@@ -27,7 +27,8 @@ DevOps-Eval是一个专门为DevOps领域大模型设计的综合评估数据集
2727

2828
-[🏆 排行榜](#-排行榜)
2929
-[⏬ 数据](#-数据)
30-
-[👀 Notes](#-notes)
30+
-[👀 说明](#-说明)
31+
-[🔥 AIOps样本示例](#-AIOps样本示例)
3132
-[🚀 如何进行测试](#-如何进行测试)
3233
-[🧭 TODO](#-todo)
3334
-[🏁 Licenses](#-licenses)
@@ -132,6 +133,29 @@ explanation:
132133
分析原因:
133134
git diff --name-only SHA命令会显示与SHA参数对应的提交中已修改的文件列表。参数--name-only让命令只输出文件名,而忽略其他信息。其它选项中的命令并不能实现此功能。
134135
```
136+
#### 🔥 AIOps样本示例
137+
👀 🔥 此处以日志解析为例,对AIOps样本做一些简要的展示:
138+
```
139+
id: 0
140+
question:
141+
下面是一些运行日志
142+
0 04:21:15,429 WARN Cannot open channel to 2 at election address /10.10.34.12:3888
143+
1 19:18:56,377 WARN******* GOODBYE /10.10.34.11:52703********
144+
2 19:13:46,128 WARN******* GOODBYE /10.10.34.11:52308********
145+
3 19:16:26,268 WARN******* GOODBYE /10.10.34.11:52502********
146+
4 09:11:16,012 WARN Cannot open channel to 3 at election address /10.10.34.13:3888
147+
5 16:37:13,837 WARN Cannot open channel to 2 at election address /10.10.34.12:3888
148+
6 09:09:16,008 WARN Cannot open channel to 3 at election address /10.10.34.13:3888
149+
7 15:27:03,681 WARN Cannot open channel to 3 at election address /10.10.34.13:3888
150+
日志最前面三部分别为序号、时间戳和日志Level,在不考虑这三部分内容的情况下,此处我们设定日志的变量用'<*>'代替,token与token之间用空格分隔,那么请问上述日志的日志模版具体是什么?
151+
A: Notification time out: <*> 和 Connection broken for id <*>, my id = <*>, error =
152+
B: Send worker leaving thread 和 Connection broken for id <*>, my id = <*>, error =
153+
C: Received connection request /<*>:<*> 和 Interrupting SendWorker
154+
D: Cannot open channel to <*> at election address /<*>:<*> 和******* GOODBYE /<*>:<*>********
155+
answer: D
156+
explanation: 根据日志中的内容,选项D是最符合日志模板的。日志中包含了"Cannot open channel to&lt;*&gt; at election address /&lt;*&gt;:&lt;*&gt;"和"******* GOODBYE /&lt;*&gt;:&lt;*&gt;********"这两个固定的模板片段,它们都在选项D中出现了。同时,其他选项中的模板片段与日志中的内容不匹配。因此,选项D是最符合日志模板的。
157+
```
158+
135159
136160
## 🚀 如何进行测试
137161
如果需要在自己的 HuggingFace 格式的模型上进行测试的话,总的步骤分为如下几步:

0 commit comments

Comments
 (0)

[8]ページ先頭

©2009-2025 Movatter.jp