An Approach to Deploy Interactive Robotic Simulators on the Web for HRI Experiments: Results in Social Robot Navigation
(pdf)
Evaluation of social robot navigation inherently requires human input due to its qualitative nature. Motivated by the need to scale human evaluation, we propose a general method for deploying interactive, rich-client robotic simulations on the web. Prior approaches implement specific web-compatible simulators or provide tools to build a simulator for a specific study. Instead, our approach builds on standard Linux tools to share a graphical desktop with remote users. We leverage these tools to deploy simulators on the web that would typically be constrained to desktop computing environments. As an example implementation of our approach, we introduce the SEAN Experimental Platform (SEAN-EP). With SEAN-EP, remote users can virtually interact with a mobile robot in the Social Environment for Autonomous Navigation, without installing any software on their computer or needing specialized hardware. We validated that SEAN-EP could quickly scale the collection of human feedback and its usability through an online survey. In addition, we compared human feedback from participants that interacted with a robot using SEAN-EP with feedback obtained through a more traditional video survey. Our results suggest that human perceptions of robots may differ based on whether they interact with the robots in simulation or observe them in videos. Also, they suggest that people perceive the surveys with interactive simulations as less mentally demanding than video surveys.
If you use this work in your research, please consider citing
(pdf):
@inproceedings{Tsoi_2021_Sean_EP,
author = {Tsoi, Nathan and Hussein, Mohamed and Fugikawa, Olivia and Zhao, J. D. and V\'{a}zquez, Marynel},
title = {An Approach to Deploy Interactive Robotic Simulators on the Web for HRI Experiments: Results in Social Robot Navigation},
booktitle = {2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)},
year = {2021},
organization={IEEE}
}System
Using modern web technologies, the SEAN-EP can be embedded in a web-based survey form to allow quick collection data from online participants.
SEAN-EP consists of SEAN, our Unity and ROS-based simulator. It is described in detail in our SEAN
setup instructions. In addition, SEAN-EP allows multiple instances of the simulation environment to be run concurrently via a web-based orchestration tool. Each simulator session is managed by the Process Manager, allowing many concurrent SEAN Sessions. NGINX, the Processes Manager, and a local Datastore are used to serve many SEAN Sessions to users via a standard web browser. Supporting web-scale data collection, SEAN-EP is able to scale horizontally using a session-aware load balancer.
Deploying SEAN-EP
Survey Integration
Our simulation is integrated with the Qualtrics Survey Platform, which makes it easier for researchers to conduct large-scale social robot navigation experiments on the web.
Human Avatar Control
Recording Experimental Data
SEAN-EP reports data via ROS topics. These include: avatar, robot, and player positions. Data recording can be done via
rosbag.
SEAN: Social Environment for Autonomous Navigation (pdf)
If you use this work in your research, please consider citing
(pdf):
@inproceedings{Tsoi_2020_HAI,
author = {Tsoi, Nathan and Hussein, Mohamed and Espinoza, Jeacy and Ruiz, Xavier and V\'{a}zquez, Marynel},
title = {SEAN: Social Environment for Autonomous Navigation},
booktitle = {Proceedings of the 8th International Conference on Human-Agent Interaction (HAI)},
month = {November},
year = {2020},
}Features
For systematic evaluation, we provide the
trial runner which allows a user to run their navigation algorithms against a set of common tasks in both indoor and outdoor environments.
Modifying the Environment
For systematic evaluation, we provide the
trial runner which allows a user to run their navigation algorithms against a set of common tasks in both indoor and outdoor environments.