TextWorld Commonsense (TWC) is a new text-based environment for RL agents that requires the use of commonsense knowledge from external knowledge sources to solve challenging problems. This repository provides the code for the work described below.

TextWorld Commonsense (TWC) dataset/environment and code for the sample RL agents reported in the paper Text-based RL Agents with Commonsense Knowledge: New Challenges, Environments and Baselines.

Install the following requirements:

# Dependencies
conda create -n twc python=3.7 numpy scipy ipython matplotlib
conda activate twc
conda install pytorch=1.3.0 torchvision cudatoolkit=9.2 -c pytorch
pip install textworld==1.2.0
conda install  nltk gensim networkx unidecode
pip install -U spacy
python -m spacy download en_core_web_sm
python -m nltk.downloader 'punkt'

Create the following directories in the main folder for the storing the model, result and log files.

mkdir results
mkdir logs

Download the Numberbatch embedding for the knowledge-aware agents:

cd embeddings
Download https://conceptnet.s3.amazonaws.com/downloads/2019/numberbatch/numberbatch-en-19.08.txt.gz
gzip -d numberbatch-en-19.08.txt.gz

This directory contains a set of 45 games created based on the dataset available in the folder twc_dataset (under game_generation directory). The games are grouped into three difficulty levels as follows.

• Easy level: games in the easy directory have only 1 room and 1 object that need to be placed in the appropriate location.
• Medium level: games with a medium difficulty level have 1 room and 2 or 3 objects shuffled across the room. Given the high number of objects, in this case we expect that the agent will need to rely more on the commonsense knowledge graph.
• Hard level: games with a hard difficulty level have 1 or 2 rooms with 6 or 7 objects shuffled across the rooms. In this case, we expect that the agent needs to leverage other knowedge sources such as historical state description etc efficiently in addition to the commonsense knowledge.

You can use the benchmark games provided in the games/twc folder for your custom agents. You may use the game_generation folder to generate customized games generated with commonsense knowledge for your agents. See the instruction for more details.

To run Text-only agent, use the following command:

The above command uses state description/observation (with GloVe embedding for the word representation by default) to select the next action.

To run Text + Commonsense agent, use the following command:

The above command uses state description/observation (with GloVe embedding for the word representation by default) and the commonsense subgraph extracted from ConceptNet using Contextual Direct Connection (CDC) method (with Numberbatch embedding for the graph representation) to select the next action.

Use test_agent.py with --split and --pretrained_model options to test the above agents.

We give some sample results for Text and Text + Commonsense RL agents against the Human and the Optimal performances. The Text-based RL agent uses only the state description/observation only for selecting the next action, where as, Text + Commonsense-based RL agent uses both the state description and commonsense knowledge graph to select next action (See our paper for more details).

The above results confirm that there is still much progress to be made in retrieving and encoding the commonsense knowledge effectively to solve Text + Commonsense problems; and we hope that TWC can spur further research in this direction for the text-based RL.

If you use our TWC environment and/or the code, please cite us by including the following articles in your work:

@inproceedings{murugesan2021textbased,
    title={{Text-based RL Agents with Commonsense Knowledge: New Challenges, Environments and Baselines}},
    author={Keerthiram Murugesan, Mattia Atzeni, Pavan Kapanipathi, Pushkar Shukla, Sadhana Kumaravel, Gerald Tesauro, Kartik Talamadupula, Mrinmaya Sachan and Murray Campbell},
    year={2021},
    booktitle={Thirty Fifth AAAI Conference on Artificial Intelligence}
}

@article{murugesan2020textbased,
      title={Text-based RL Agents with Commonsense Knowledge: New Challenges, Environments and Baselines}, 
      author={Keerthiram Murugesan, Mattia Atzeni, Pavan Kapanipathi, Pushkar Shukla, Sadhana Kumaravel, Gerald Tesauro, Kartik Talamadupula, Mrinmaya Sachan and Murray Campbell},
      year={2020},
      eprint={2010.03790},
      archivePrefix={arXiv},
      journal={CoRR},
      volume={abs/2010.03790}
}

Please share commonsense-rl using http://ibm.biz/commonsense-rl

• File an issue on GitHub.
• Ask a question on Stack Overflow.

• ConceptNet http://conceptnet.io/
• Microsoft TextWorld https://www.microsoft.com/en-us/research/project/textworld/