Repository files navigation

• WoZ4U - What is this?
• Installation
  • Docker
  • Windows Guide
  • Linux and Mac Guide
    • Download NAOqi 2.5
    • Setting up the environment
    • Running WoZ4U
• Configuring WoZ4U
  • Using a different configuation file
  • A word on YAML syntax
  • Connecting WoZ4U to your Pepper
  • Configuring Pepper's default state
  • Adding tablet items
  • Adding text message
  • Adding audio files
  • Adding LED colors
  • Adding gesture animations
• Citation
• Troubleshooting

In this repository, we host WoZ4U, a fully configurable interface forSoftbank's Pepper robot.

WoZ4U was developed by theIntelligent Robotics Group, Umeå University, Sweden, as a collaborative project with theKnowledge Technology Group from Hamburg University, Germany.

Our main objective is to provide an easy to use yet flexible and powerful tool for the conduction of Wizard-of-Oz experiment with Pepper,which are an essential method for general HRI research.Further, we hope to make Pepper as a research platform more accessible fornon-technical groups (social sciences), by providing means to conduct Wizard-of-Oz experiments on Pepper without needingexpert programming knowledge.

WoZ4U is implemented as a Flask HTTP server and is accessed via browser, nevertheless, a few steps are required to run WoZ4U on your machine. Firstly, WoZ4U requires theNAOqi API v2.5, which only supportsPython 2.7. Thus, you need some environment where these requirements are satisfied. You can either run WoZ4U from our Docker image (where the entire installation process has been taken care of, seeDocker) or install the system manually, as described inWindows Guide orLinux and Mac Guide.

Hence, we recommend to set up a dedicated Python 2.7 virtual environment and install all requirements there in that virtual environment. If you follow the steps as described below, everything will be installed in a virtual environment and you won't have to worry about conflicting package versions.

The easiest way run the WoZ4U interface is through our docker image, which won't require any manual installation steps beyond the installation of docker itself. Thus, if you don't have it, install docker fromhere.Then, you can immediately start the interface from our image hosted on dockerhub (however, the docker call needs to be parameterized differently depending on the OS, which is what is happening in the script). Simply run the script like this:

bash start_docker.sh

Now, the interface can be access via URL in the browser of your choice onhttp://0.0.0.0:5000

While this is great and probably easiest to get started right away, if you want to make changes to the configuration of the interface (as described underConfiguring WoZ4U), you will have to either rebuild the docker image from the providedDockerfile locally after making your changes, or install the system without Docker, directly on your machine, as described below.

To make changes if you've opted for docker, first clone the repo, then make your changesAfter having made changes to the either theconfig.yaml configuration file, rebuild the docker image with the following command:

# git clone https://github.com/frietz58/WoZ4U.git# make your changes# then:cd WoZ4U&& sudo docker build -t woz4u.

This will build a local version of the image, which uses the files you've changed locally. The command to start the local version is the same as we used to run remote image, except the name of the images changes:

# before we were running frietz58/woz4u, now its just woz4u, based on how we tagged the image during the buildsudo docker run -it --network host --device /dev/snd woz4u# this is the linux command.

Consider changing the calls in thestart_docker.sh script so that the locally buildwoz4u container is started instead of the remotly hostedfrietz58/woz4u...

WoZ4U can be installed on all major platforms, namely Debian Based Linux (Ubuntu etc), MacOS, and Windows. However, the installation process on Windows is different, because Bash is not default shell on Windows. Thus, if I wish to install and run WoZ4U on a Windows machine, please follow the steps described in ourwindows_readme.md,available here.

If you wish to install and run WoZ4U on Linux or MacOS, just follow the steps below.

For now, just clone this Github Repo to your local machine:git clone https://github.com/frietz58/WoZ4U.git

Change working directory inside the repo folder:cd WoZ4U
This is important for the following commands!

1. Download the NAOqi APIfrom Softbank's website:From the download page, select the SDK (not Choregraphe), and download the archive. Extract the archive to an arbitrarylocation. We propose extracting the API inside theWoZ4U directory, so that it is not deleted by accident.
2. To send requests from WoZ4U's browser interface to the NAOqi API (and ultimately onto the physical Pepper),WoZ4U needs to know where to find the API on your machine. We do this via the scriptset_paths.sh:
   • Edit line 10 inset_paths.sh by replacing everything after the colon with the path to thesite-packagesfolder inside the extracted NAOqi API folder, that we extracted in the previous step:

   # export PYTHONPATH=${PYTHONPATH}:/ABSOLUTE/PATH/TO/SITE-PACKAGES/FOLDER/IN/NAOQI-API-FOLDERexport PYTHONPATH=${PYTHONPATH}:/Users/finn/Desktop/WTM/pepper_scripts/pynaoqi-python2.7-2.5.7.1-mac64/lib/python2.7/site-packages

   • Edit line 11 inset_paths.sh by replacing everything after the colon with the path to thelib folder inside the extracted NAOqi folder:

   # export PYTHONPATH=${PYTHONPATH}:/ABSOLUTE/PATH/TO/LIB/FOLDER/IN/NAOQI-API-FOLDERexport DYLD_LIBRARY_PATH=${DYLD_LIBRARY_PATH}:/Users/finn/Desktop/WTM/pepper_scripts/pynaoqi-python2.7-2.5.7.1-mac64/lib

In order to be able to execute the script, make it executable with the following command:chmod +x set_paths.sh
Note, that we assume that your terminal's working directory is locatedinsider thw WoZ4U folder, otherwise adjust the command to point to the location of theset_paths.sh script.

Now, every time you want to use WoZ4U (in a new terminal session), firstsource theset_paths.sh script in that terminal session:source set_paths.sh
Executing that command adds the the NAOqi API to yourPATH, so that they can be found by Python later. See sectionRunning WoZ4U for a concrete example. Obviously, you can adjust your.bashrc accordingly, the familiar won't need additional instructions ;)

1. Install Python 2.7, if you don't have it, fromhere. Verify installation by runningpython2.7 --version, which should output something similar toPython 2.7.16, if you have Python 2.7 installed.
2. Verify that Pythons package manager, PIP, is available, by typing:pip into your terminal. This sould output somethig likepip 19.3.1 from .... If the command is not availabel, install PIP fromhere.
3. Install Python's virtualenv package:pip install virtualenv --user
4. Make a fresh Python 2.7 virtual environment:virtualenv -p /usr/bin/python2.7 woz4u_venv.
   Pay attention to providing the correct Python (2.7) interpreter by setting the correct path to the-p /path/to/interpreter argument.
5. Activate the environment:source woz4u_venv/bin/activate. In your terminal, the prompt should now be prefixed with(woz4u_venv), indicating that the environment is active and that requirements will be installed in that environment.
6. Install WoZ4U's requirements from therequirements.txt file in the virtual environment:pip install -r requirements.txt.
   Here, I assume that your current working directory is still set to the main repository folder:WoZ4U. If you changed working directories in the meantime, provide the path to therequirements.txt file to the-r argument of thepip install call.

As a reminder, to install and run WoZ4U on Windows, please see thewindows_readme.mdhere. If you have installed WoZ4U on Linux or MacOS, follow the steps below to start the interface.

Assuming that you followed the installation instructions to this point, the following sequence of commands starts WoZ4U (even in a new terminal session or after having restarted your computer):

1. Change the working directory into theWoZ4U directory (where you cloned this repository, during the installation process):cd PATH/TO/YOUR/WoZ4U
2. Activate the virtualenv containg all the required packages:source woz4u_venv/bin/activate
3. Tell the interface where to find the NAOqi API:source set_paths.sh
4. Run the main command:python server.py

You can also chain the commands together with the&& operator (assuming your working directory is set toWoZ4U):

source woz4u_venv/bin/activate&&source set_paths.sh&& python server.py

While the server is running, you can assess the interface via your browser of choice (developed with Firefox), via theURLhttp://0.0.0.0:5000. The server will run until you either close the terminal session or kill the process by pressingCTRL + C.

If you are running WoZ4U on a Mac and the operating system does not allow the execution of binaries from unoffical sources, see sectionTroubleshooting.

In order to use WoZ4U with you own Pepper robot and configuring the context and experiment specific items, you only have to edit one central YAML file, that holds the concrete values for all the UI elements in the frontend. As a general rule for working with WoZ4U, only edit, add, or deletevalues, but never editkeys! When theconfig.yaml file is parsed by the server, the keys in the file (eg names of fields, list, or object) are the only thing that bridgesthe backend with the frontend. For example, buttons for text messages are generated because the list namedanimated_speech in theconfig.yaml file is sent to the frontend (hence changing the name of that list will break the system).As a guideline in laymen's terms: You are fine to edit whatever is to the right of a colon, but now that is to the left! The general idea behind the configuration file isillustrated here:For most sections of the interface, you populate a YAML list with the concrete items (text messages, images, gestures, etc)that are relevant for your use-case. For each element in the list corresponding to each section, UI elements will becreated in the WoZ4U frontend. Hence, via this configuration file, the interface can be fully configured to any specificuse-case.

The WoZ4U server can be started with the command line argument "-c", which expects a string value. The string value has to correspond to a path of a valid configuration file for WoZ4U. For example, when the server is started without explicitly appending the "-c" argument, the default value will be inserted, which is equivalent to callingserver.py -c "config.yaml"Thus, to use a different configuration file as the defaultconfig.yaml, simply pass the name of the file configuration file via the "-c". Using again the full argument fromabove, we get:

source woz4u_venv/bin/activate&&source set_paths.sh&& python server.py -c"new_config_file.yaml"

YAML is a vastly popular markup language. A good guide is availablehere.The things you should know: The character "-" (followed by a whitespace) indicates a list item, like so:

-list_item_a-list_item_b-list_item_X

Lists can also be provided in a more condensed syntax:[list_item_a, list_item_b, list_item_X]. You will find both notations in our exemplaryconfig.yaml file. Apart from lists, it's really onlykey: value pairs, where the values can recursively bekey: value pairs or lists, like so:

list_name:# list_name is a key, value is a list    -# denotes first list itemchild_key_a:child_val_a# list item is object with two key: value pairschild_key_b:child_val_b    -# second list itemchild_key_a:different_child_val_achild_key_b:different_child_val_b

In order to connect the interface, you need to provide Pepper's IP address in theconfig.yaml file. You can provide multiple IP addresses, but the listpepper_ips should at least contains one value. The list items will populate adropdown menu in the frontend. Example:

pepper_ips:# Entry in the dropdown menu will be created for every list item  -192.168.10.1# pepper ip address, should be reachable from the network where the WoZ4Uinterface is hosted.  -192.168.10.2  -192.168.10.3

Naturally, Pepper must actually be reachable via TCP/IP (aka be in the same network as the machine that hosts WoZ4U).

Here, we refer to Pepper's state as a combination of autonomous life settings. These control how Pepper responds to stimuli in the environment, whether Pepper emits lifelike idle animations, whether Pepper actively looks for interaction partners, etc. The dictionaryautonomous_life_config inconfig.yaml has a key for each of those settings. The concrete values you put there depend on the setting (documentation),if you are not sure about those, you can simply put an empty string("") as values for the keys, in which case the system will ignore the setting.For all keys that have values supplied in theconfig.yaml file, these settings will be applied to Pepper as soon asyou connect the interface to the Pepper robot.

In case you want to use Pepper's tablet to display something to your experiment participants, you need to store these itemsin the configuration file beforehand. Items can be

• Images (hosted locally or remote)
• Videos (hosted locally or remote)
• Websites (hosted locally or remote)

You can find examples for all cases in the exemplaryconfig.yaml file

For images or video hosted locally, they should be stored in the folder specified by the fieldtablet_root_location inconfig.yamlAs you can see,tablet_items inconfig.yaml is a list, containing multiple items. Each list item has the followingfields:

• title: A short text that will be displayed next to the button in the frontend.
• file_name: The name of the file (for images or videos hosted locally, in thetablet_root_location folder). For remotely hosted images or videos, this is the full URL to the resource. For websites, also the URL.
• key_comb: A list (using the compressed notation), containing validkeycodesto make up a shortcut, that will be assigned to the button in the frontend.

You likely want access to some predefined text message for Pepper to say in the frontend interface. The gist of this process isequivalent to how you'd add tablet items: You populate the listanimated_speech inconfig.yaml, adjusting thefields with the desired values:

• title: A short text that will be displayed next to the button in the frontend.
• string: The text message you want Pepper to say. Can contain tags valid under theNAOqi animated speech module.
• tooltip: A tooltip that will be displayed in the frontend when the button for each message is hovered with the cursor.
• key_comb: A list (using the compressed notation), containing validkeycodesto make up a shortcut, that will be assigned to the button in the frontend.

Adding audio files for playback via Pepper speakers requires a bit more setup effort than just editing theconfig.yamlfile. That is because the NAOqi API is limited to playing audio files that are storeddirectly on Pepper. Hence,if you desire to play audio tunes during your experiment the steps are as follows:Either

1. ssh onto your Pepper robot:ssh nao@PEPPER_IP_ADDRESS
2. Download the audio files on your Pepper via, for example,wget:wget https://www2.cs.uic.edu/~i101/SoundFiles/ImperialMarch60.wav --no-check-certificate

Alternatively, you usescp to copy a sound file onto your Pepper.

Either way, once the audio files are stored on the Pepper robot, you can provide a list item for each file in the listaudio_files. The field values are similar to before:

• title: A short text that will be displayed next to the button in the frontend.
• location: The complete, absolute path to the sound file stored on your Pepper robot
• key_comb: A list (using the compressed notation), containing validkeycodesto make up a shortcut, that will be assigned to the button in the frontend.

In case you want to manipulate Pepper's eye LED colors, you must predefine the colors you wish to access via the interface. As with all previous sections, UI elements will be generated for each item in the list namedcolors inconfig.yaml.

The fields of the list items must be set a follows:

• title: A short text that will be displayed next to the button in the frontend.
• red: The red color channel in range [0 - 1].
• green: The green color channel in range [0 - 1].
• blue: The blue color channel in range [0 - 1].
• key_comb: A list (using the compressed notation), containing validkeycodesto make up a shortcut, that will be assigned to the button in the frontend.

There are two ways to execute gestures vie the WoZ4U interface on Pepper: Either, you can embed them into text messagesvie the animated speech module in sectionAdding text messages. Alternatively, and this is whatwe will address here, you can provide gestures via the listgestures inconfig.yaml. Set the fields of the list itemsas follows:

• title: A short text that will be displayed next to the button in the frontend.
• gesture: An animation for Pepper to execute, fromthis list.NAOqi animated speech module.
• tooltip: A tooltip that will be displayed in the frontend when the button for each gesture is hovered with the cursor.
• key_comb: A list (using the compressed notation), containing validkeycodesto make up a shortcut, that will be assigned to the button in the frontend.

If you use WoZ4U, please cite our work:

@ARTICLE{10.3389/frobt.2021.668057,    AUTHOR={Rietz, Finn and Sutherland, Alexander and Bensch, Suna and Wermter, Stefan and Hellström, Thomas},       TITLE={WoZ4U: An Open-Source Wizard-of-Oz Interface for Easy, Efficient and Robust HRI Experiments},          JOURNAL={Frontiers in Robotics and AI},          VOLUME={8},          PAGES={213},         YEAR={2021},          URL={https://www.frontiersin.org/article/10.3389/frobt.2021.668057},           DOI={10.3389/frobt.2021.668057},          ISSN={2296-9144},   }

• With newer versions of the Mac operating systems, binaries from unoffcial source are not executable. Some information on this can be foundhere. To be able to run these binaries, for example from the NAOqi API, execute the following command:
  sudo spctl --master-disable.

• With Mac OS El Capitan, Apple introduced additional security measures, that introduced some issue with the NAOqi API. If you get anSystemError: dynamic module not initialized properly when importing the naoqi library, run the following command to disable the new security features:csrutil disable followed byreboot. More on thishere.

• Python errorOSError: PortAudio library not found: On Linux, this can be fixed by running:sudo apt-get install libportaudio2 libasound-dev.