Multimodal estimation and communication of latent semantic knowledge for robust execution of robot instructions

Cited 21 time in webofscience Cited 9 time in scopus
  • Hit : 303
  • Download : 0
DC FieldValueLanguage
dc.contributor.authorArkin, Jacobko
dc.contributor.authorPark, Daehyungko
dc.contributor.authorRoy, Subhroko
dc.contributor.authorWalter, Matthew R.ko
dc.contributor.authorRoy, Nicholasko
dc.contributor.authorHoward, Thomas M.ko
dc.contributor.authorPaul, Rohanko
dc.date.accessioned2020-11-17T00:55:04Z-
dc.date.available2020-11-17T00:55:04Z-
dc.date.created2020-11-17-
dc.date.created2020-11-17-
dc.date.created2020-11-17-
dc.date.issued2020-09-
dc.identifier.citationINTERNATIONAL JOURNAL OF ROBOTICS RESEARCH, v.39, no.10-11, pp.1279 - 1304-
dc.identifier.issn0278-3649-
dc.identifier.urihttp://hdl.handle.net/10203/277314-
dc.description.abstractThe goal of this article is to enable robots to perform robust task execution following human instructions in partially observable environments. A robot's ability to interpret and execute commands is fundamentally tied to its semantic world knowledge. Commonly, robots use exteroceptive sensors, such as cameras or LiDAR, to detect entities in the workspace and infer their visual properties and spatial relationships. However, semantic world properties are often visually imperceptible. We posit the use of non-exteroceptive modalities including physical proprioception, factual descriptions, and domain knowledge as mechanisms for inferring semantic properties of objects. We introduce a probabilistic model that fuses linguistic knowledge with visual and haptic observations into a cumulative belief over latent world attributes to infer the meaning of instructions and execute the instructed tasks in a manner robust to erroneous, noisy, or contradictory evidence. In addition, we provide a method that allows the robot to communicate knowledge dissonance back to the human as a means of correcting errors in the operator's world model. Finally, we propose an efficient framework that anticipates possible linguistic interactions and infers the associated groundings for the current world state, thereby bootstrapping both language understanding and generation. We present experiments on manipulators for tasks that require inference over partially observed semantic properties, and evaluate our framework's ability to exploit expressed information and knowledge bases to facilitate convergence, and generate statements to correct declared facts that were observed to be inconsistent with the robot's estimate of object properties.-
dc.languageEnglish-
dc.publisherSAGE PUBLICATIONS LTD-
dc.titleMultimodal estimation and communication of latent semantic knowledge for robust execution of robot instructions-
dc.typeArticle-
dc.identifier.wosid000538262400001-
dc.identifier.scopusid2-s2.0-85085926678-
dc.type.rimsART-
dc.citation.volume39-
dc.citation.issue10-11-
dc.citation.beginningpage1279-
dc.citation.endingpage1304-
dc.citation.publicationnameINTERNATIONAL JOURNAL OF ROBOTICS RESEARCH-
dc.identifier.doi10.1177/0278364920917755-
dc.contributor.localauthorPark, Daehyung-
dc.contributor.nonIdAuthorArkin, Jacob-
dc.contributor.nonIdAuthorRoy, Subhro-
dc.contributor.nonIdAuthorWalter, Matthew R.-
dc.contributor.nonIdAuthorRoy, Nicholas-
dc.contributor.nonIdAuthorHoward, Thomas M.-
dc.contributor.nonIdAuthorPaul, Rohan-
dc.description.isOpenAccessN-
dc.type.journalArticleArticle-
dc.subject.keywordAuthorHuman-robot collaboration-
dc.subject.keywordAuthorsemantic state estimation-
dc.subject.keywordAuthorBayesian modeling-
dc.subject.keywordAuthormultimodal interaction-
dc.subject.keywordAuthornatural language understanding-
Appears in Collection
CS-Journal Papers(저널논문)
Files in This Item
There are no files associated with this item.
This item is cited by other documents in WoS
⊙ Detail Information in WoSⓡ Click to see webofscience_button
⊙ Cited 21 items in WoS Click to see citing articles in records_button

qr_code

  • mendeley

    citeulike


rss_1.0 rss_2.0 atom_1.0