Repository files navigation

This is the official repository for the GPT-2 models described in the EMNLP2020 paper[STORIUM: A Dataset and Evaluation Platform for Machine-in-the-LoopStory Generation]. It has all the code necessary to reproduce the models andanalysis from the paper.

A high-level outline of our dataset and platform. In this example from a realSTORIUM game, thecharacter ADIRA MAKAROVA uses the strength card DEADLY AIM to DISRUPT THEGERMANS, a challenge card. Our model conditions on the natural languageannotations in the scene intro, challenge card, strength card, and character,along with the text of the previous scene entry (not shown) to generate asuggested story continuation. Players may then edit the model output, by addingor deleting text, before publishing the entry. We collect these edits, usingthe matched text as the basis of our USER metric. New models can be added tothe platform by simply implementing four methods: startup, shutdown,preprocess, and generate.

This repository contains the code that makes our GPT-2 story generation modelsdeployable on ourevaluation platform, so itserves as a great template for how to structure your code. Please see the filefigmentate.py for the simple API required for making yourmodel deployable on our platform. You will also need to provide a json filewith any properties needed to pass to your startup method. See for example theproperties below:

{"scene_entry":  {"properties": {"checkpoint_path":"/var/lib/figmentator/checkpoint","sample": {"top_p":0.9,"temperature":0.9,"repetition_penalty":1.2      }    },"requires": ["torch==1.3.0","transformers==2.2.0","kiwisolver==1.1.0"],"cls":"model=figmentate:GPT2Figmentator"  }}

The keyscene_entry defines the type of model being created. Currently, weonly support models that generate the text of a scene entry, though we mightsupport other types of prediction models in the future, like suggesting cardsor narrator actions.

Theproperties object will be passed to your startup method. It allows fordefining any parameters needed for sampling from your model.

Therequires list, is simply a list of python packages that need to beinstalled for your model to run. These will be automatically installed whenyour model is deployed. If you notice, we specify the deep learning packagetorch as a requirement. That's because our code isagnostic to the underlying deep learning framework being used by your model.That means it should support models using other frameworks liketensorflow orjax.

Finally, thecls string is the class that wraps your model. It is specifiedusing Python'sentrypointssyntax.

@inproceedings{akoury2020storium,Author ={Nader Akoury, Shufan Wang, Josh Whiting, Stephen Hood, Nanyun Peng and Mohit Iyyer},Booktitle ={Empirical Methods for Natural Language Processing},Year ="2020",Title ={{STORIUM}: {A} {D}ataset and {E}valuation {P}latform for {S}tory {G}eneration}}