Deep reinforcement learning control of hand-eye coordination with a software retina

Research output: Chapter in Book/Report/Conference proceedingConference contribution book

1 Downloads (Pure)

Abstract

Deep Reinforcement Learning (DRL) has gained much attention for solving robotic hand-eye coordination tasks from raw pixel values. Despite promising results, training agents using images is hardware intensive often requiring millions of training steps to converge incurring long training times and increased risk of wear and tear on the robot. To speed up training, images are often cropped and downscaled resulting in a smaller field of view and loss of valuable high-frequency data. In this paper, we propose training the vision system using supervised learning prior to training robotic actuation using Deep Deterministic Policy Gradient (DDPG). The vision system uses a software retina, based on the mammalian retino-cortical transform, to preprocess full-size images to compress image data while preserving the full field of view and high-frequency visual information around the fixation point prior to processing by a Deep Convolutional Neural Network (DCNN) to extract visual state information. Using the vision system to preprocess the environment improves the agent's sample complexity and network update speed leading to significantly faster training with reduced image data loss. Our method is used to train a DRL system to control a real Baxter robot's arm, processing full-size images captured by an in-wrist camera to locate an object on a table and centre the camera over it by actuating the robot arm.
Original languageEnglish
Title of host publication2020 International Joint Conference on Neural Networks (IJCNN)
Place of PublicationPiscataway, NJ
PublisherIEEE
Number of pages8
ISBN (Electronic)9781728169262
DOIs
Publication statusPublished - 28 Sep 2020
Event2020 International Joint Conference on Neural Networks (IJCNN)- IEEE World congress on computational intelligence(WCCI) 2020: IJCNN - glasgow, Glasgow, United Kingdom
Duration: 19 Jul 202024 Jul 2020
Conference number: 48605X
https://wcci2020.org/ijcnn-sessions/
https://wcci2020.org/

Publication series

NameProceedings of the International Joint Conference on Neural Networks
PublisherIEEE
ISSN (Print)2161-4393
ISSN (Electronic)2161-4407

Conference

Conference2020 International Joint Conference on Neural Networks (IJCNN)- IEEE World congress on computational intelligence(WCCI) 2020
Abbreviated titleIJCNN
Country/TerritoryUnited Kingdom
CityGlasgow
Period19/07/2024/07/20
Internet address

Keywords

  • software retina reprocessor
  • reinforcement learning
  • robotic vision
  • CNN

Fingerprint

Dive into the research topics of 'Deep reinforcement learning control of hand-eye coordination with a software retina'. Together they form a unique fingerprint.

Cite this