- Notifications
You must be signed in to change notification settings - Fork80
Open
Description
Hi, could someone explain what problems in this category are?
'domain': 'Long In-context Learning''sub_domain': 'Many-shot learning'
For example, take a look at problem340
:
importdatasetsds=datasets.load_dataset("THUDM/LongBench-v2")['train']print(ds[340]['_id'])print(ds[340]['question'])forchoicein ('A','B','C','D'):print(ds[340]['choice_'+choice])
Output:
6725dabfbb02136c067d8450Document: I was there in 89-91. Some of the best years of my life.\n\nQuestion: Only considering the given document, what are the emotions of the document?aaa,aaraap,aajaaw,aayaat,aai[context_below]
I do not understand the question or the candidate answers.
The context is here:https://gist.github.com/Naqu6/8b7628d1308e53199f98e7a53498f2f2
As far as I can tell, the context is an LLM conversation, where the model answers with some 3 character string.
I do not understand this task. Maybe I'm missing something, but I didn't see a description of it in the paper (it is described as something else in the paper)
Further, there are lots of swearwords and other inappropriate content in the context - is this intentional?
Thanks for your help!
Metadata
Metadata
Assignees
Labels
No labels