Deep Learning with Domain Randomization Python

Learn how to train any robot to recognize an object and pinpoint its 3D location with only an RGB camera and a lot of training with Keras.

Course Overview


Welcome to this micro-course! This course is intended for the people that want to learn about deep learning using Keras.

In this case, we use a very interesting approach to learning, which is Environment Randomization. This method exploits the versatility of environment generation in simulations to train a robot in a way that the resulting model is very robust, no matter the lighting conditions. It also makes the transition from simulated learning to reality much smoother and faster. Learn through hands-on experience how to train a robot for 3D object recognition using random environments.

Keras will be the cornerstone of this system and you will learn all the necessary skills to generate training data, convert it to a database, train a MobileNetV2 model, retrain it, and make predictions with it.

The final project is the training of a garbage picking robot, from training data generation to the final garbage detection and picking program. Dive into the fantastic world of Deep Learning with Keras right now!

Learning Objectives

  • Use Keras in a basic way
  • Train a deep neural network using a Gazebo Simulation
  • Work with ROS + Gazebo + Keras in tandem.
  • Understand how the Random Environment generation works in Gazebo

Simulation robots used in this course

Fetch Robot, GarbageCollector Robot.







This course is part of this learning path:

What projects will you be doing?

[ROS Q&A] 168 - What are the differences between global and local costmap

Create a Simple Random Environment

Start with a simple environment training

ROS Mini Challenge #2 - RViz

Exercises For XY motion Spam

Train the simulation to find the Spam anywhere on the table


Exercises for a Distractor & Random Environment

Train the model with a random environment 

Microproject Garbage Collector

Use the two-wheeled Magician robot to find an object amidst other distractions

What you will learn

Course Syllabus

Unit 1: Introduction

Random Environment: Quick Demo.

1 hr.

Unit 2: Step By Step Simple Guide

We will start with a simple environment training. We will talk about all the steps from training image generation to validation.

1 hr.

Unit 3: Exercises For XY Motion Spam

Here we indicate how to retrain a previously trained model and work on improving our detection model.

1 hr.

Unit 4: Exercises for Spam and a Distractor

Here we are trying to distract the model with another object in the scene that moves around and could be mistaken by the Spam object.

1 hr. 

Unit 5: Exercises with Distractor and Random Env

You are going to train the model with a random environment to make it more robust in any lighting condition. You will also train with a huge image database to emulate the real deal.

1 hr. 

Unit 6: Microproject Garbage Collector

Ready to get started?

Start learning ROS & Robotics online quickly and easily

What’s next

People interested in this course also viewed


ROS Deep Learning with TensorFlow 101


Using OpenAI with ROS

Top universities choose The Construct for Campus to teach ROS & Robotics.