Policy Training

Assuming you have extracted bags of data that you want to train a controller with, the following steps will be necessary to transform the data into a consumable format:

• In a Python script, use a library like rospy to extract topics of interest as listed in Communication. For instance, one might want to predict the robot's velocity based on what the robot is currently seeing—you could extract the /robot_firstperson_rgb/camera_info and /social_sim/cmd_vel topics.
• You will need to write a script/notebook that trains a model based on your extracted data. For instance, one could use PyTorch on a GPU-enabled cloud machine, writing a custom PyTorch dataloader that sends data on disk to the GPU to train the model. For predicting velocity, you would want to set up your model optimization to minimize the difference between the ground-truth velocity and the predicted velocity.
• For your model to be deployable, you will need to embed it in a Python script that consumes from the ROS topics in Communication that you used for your dataset, and converts the data in those topics to samples of the form that your model expects. For instance, the Python script could subscribe to the /robot_firstperson_rgb/camera_info and /social_sim/cmd_vel topics as they are populated by Unity in real-time. It would then extract and transform the data, feed it into your trained model, and publish the predicted/desired velocity to /mobile_base_controller/cmd_vel to move the robot in Unity according to your model.

In Policy Rollout, we will cover how to embed this last script into your tmuxinator config, so that when you start tmuxinator and Unity this script will push velocities to Unity for the robot to move.