State Entropy Maximization with Random Encoders for Efficient Exploration

Cited 0 time in webofscience Cited 0 time in scopus
  • Hit : 46
  • Download : 0
DC FieldValueLanguage
dc.contributor.authorSeo, Younggyoko
dc.contributor.authorChen, Liliko
dc.contributor.authorShin, Jinwooko
dc.contributor.authorLee, Honglakko
dc.contributor.authorAbbeel, Pieterko
dc.contributor.authorLee, Kiminko
dc.date.accessioned2022-01-14T06:56:15Z-
dc.date.available2022-01-14T06:56:15Z-
dc.date.created2021-12-02-
dc.date.issued2021-07-
dc.identifier.citation38th International Conference on Machine Learning, ICML 2021-
dc.identifier.issn2640-3498-
dc.identifier.urihttp://hdl.handle.net/10203/291830-
dc.description.abstractRecent exploration methods have proven to be a recipe for improving sample-efficiency in deep reinforcement learning (RL). However, efficient exploration in high-dimensional observation spaces still remains a challenge. This paper presents Random Encoders for Efficient Exploration (RE3), an exploration method that utilizes state entropy as an intrinsic reward. In order to estimate state entropy in environments with high-dimensional observations, we utilize a k-nearest neighbor entropy estimator in the low-dimensional representation space of a convolutional encoder. In particular, we find that the state entropy can be estimated in a stable and compute-efficient manner by utilizing a randomly initialized encoder, which is fixed throughout training. Our experiments show that RE3 significantly improves the sample-efficiency of both model-free and model-based RL methods on locomotion and navigation tasks from DeepMind Control Suite and MiniGrid benchmarks. We also show that RE3 allows learning diverse behaviors without extrinsic rewards, effectively improving sample-efficiency in downstream tasks.-
dc.languageEnglish-
dc.publisherJMLR-JOURNAL MACHINE LEARNING RESEARCH-
dc.titleState Entropy Maximization with Random Encoders for Efficient Exploration-
dc.typeConference-
dc.type.rimsCONF-
dc.citation.publicationname38th International Conference on Machine Learning, ICML 2021-
dc.identifier.conferencecountryUS-
dc.identifier.conferencelocationVirtual-
dc.contributor.localauthorShin, Jinwoo-
dc.contributor.nonIdAuthorChen, Lili-
dc.contributor.nonIdAuthorLee, Honglak-
dc.contributor.nonIdAuthorAbbeel, Pieter-
dc.contributor.nonIdAuthorLee, Kimin-
Appears in Collection
RIMS Conference Papers
Files in This Item
There are no files associated with this item.

qr_code

  • mendeley

    citeulike


rss_1.0 rss_2.0 atom_1.0