Repository files navigation

ScienceWorld is a text-based virtual environment centered around accomplishing tasks from the standardized elementary science curriculum. This code accompanies the paperScienceWorld: Is your Textual Agent Smarter than a 5th grader?.

You can try ScienceWorld yourself via ourHuggingFace Space or read some of theplaythrough transcripts.

@inproceedings{wang-etal-2022-scienceworld,    title = "{S}cience{W}orld: Is your Agent Smarter than a 5th Grader?",    author = "Wang, Ruoyao  and      Jansen, Peter  and      C{\^o}t{\'e}, Marc-Alexandre  and      Ammanabrolu, Prithviraj",    editor = "Goldberg, Yoav  and      Kozareva, Zornitsa  and      Zhang, Yue",    booktitle = "Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing",    month = dec,    year = "2022",    address = "Abu Dhabi, United Arab Emirates",    publisher = "Association for Computational Linguistics",    url = "https://aclanthology.org/2022.emnlp-main.775/",    doi = "10.18653/v1/2022.emnlp-main.775",    pages = "11279--11298",}

Before running: You will have to haveJava 1.8+ installed on your system (shipped with most linux distributions) andPython 3.8+. We recommend creating a conda environment like this:

conda create --name scienceworld python=3.8conda activate scienceworld

Then, install ScienceWorld either from PyPi:

pip install scienceworld

or from source in development mode:

git clone https://github.com/allenai/ScienceWorld.gitcd ScienceWorldpip install .

Run an example random agent, on task 13 (classification: place a non-living thing in a box), for 5 episodes:

python examples/random_agent.py --task-num=13 --num-episodes=5 --simplifications-preset easy

Run a user console where you can interact with the environment, on task 3 (change of state: melting):

python examples/human.py --task-num=3 --num-episodes=5

A web server demo is also available, that allows running a ScienceWorld user console that can be interacted with in a web browser.

To run the web server demo:

conda create --name scienceworld python=3.8conda activate scienceworldpip install scienceworld[webserver]

Run the web server:

python examples/scienceworld-web-server-example.py

Point your web browser to:localhost:8080

ScienceWorld is written in Scala (2.12.9), and compiles usingsbt into a JAR file that is run with Java. For convenience, a Python API is provided (Python >= 3.8), which interfaces using thepy4j package.

If you modified the Scala code, you can recompile the JAR file by running:

./simulator/package.shpip install -e.

The tasks are listed in the table below along with their number of variations. Either the task ID or its name can be used to a task withenv.load().

ScienceWorld supports a number of simplifications that can be applied to the environment to make it easier for agents to learn. These simplifications can be applied by passing the--simplifications-preset argument to the command line interface, or by passing thesimplifications argument to the Python API.

The available simplifications are:

• teleportAction: Allows agents to instantly move to any location in the environment.
• openDoors: All doors in the environment are open by default.
• selfWateringFlowerPots: Automatically waters all flower pots in the environment.
• noElectricalAction: Disables electrical actions, making it easier for agents to learn tasks that do not require electrical actions.
• openContainers: All containers in the environment are open by default.

The--simplifications-preset argument can be set toeasy to apply the following simplifications:

• teleportAction
• openDoors
• selfWateringFlowerPots
• noElectricalAction (for non-connectivity tasks)

DRRN:https://github.com/cognitiveailab/drrn-scienceworld

KG-A2C:https://github.com/cognitiveailab/kga2c-scienceworld

CALM:https://github.com/cognitiveailab/calm-scienceworld

Behavior Cloning and Decision Transformer:https://github.com/cognitiveailab/t5-scienceworld

To compile the ScienceWorld JAR file, follow these steps:

You will need to haveJava 1.8+ SDK installed on your system (shipped with most linux distributions). E.g. on Ubuntu, you can install it with:

sudo apt-get install openjdk-21-jdk

Then, install sbt (Scala Build Tool) by running:

echo"deb https://repo.scala-sbt.org/scalasbt/debian all main"| sudo tee /etc/apt/sources.list.d/sbt.listecho"deb https://repo.scala-sbt.org/scalasbt/debian /"| sudo tee /etc/apt/sources.list.d/sbt_old.list    curl -sL"https://keyserver.ubuntu.com/pks/lookup?op=get&search=0x2EE0EA64E40A89B84B2DF73499E82A75642AC823"| sudo tee /etc/apt/trusted.gpg.d/sbt.asc    sudo apt-get update    sudo apt-get install sbt

Once you havesbt installed, you can compile the ScienceWorld JAR file by running:

./simulator/package.sh