Relation extraction is a work that inferring semantic relations of two entities identified in natural language text. The extracted relation is saved in the knowledge base in triple form. The knowledge base is widely used in the field of Natural Language Processing, such as Question-and-Answering systems and Information Retrieval. Therefore, research to augment Knowledge Base through various methods of relation extraction is essential. Distant Supervision data, used to train machine learning based relation extraction models, can be obtained by annotating a predefined Knowledge Base to the target corpus. With this method, we can easily obtain data, but it has the disadvantage of having many errors. In this paper, we propose an improved method in the methodology of handling error data through existing reinforcement learning based relation extractor. The existing reinforcement learning based relation extraction could not overcome the limitations of the performance of the relation extractor itself since it reflected the reward for its agent dependent on relation extractor. In this study, we propose to supplement the limits of performance of the relation extractor by adding reward independent to relation extractor to the agent. In addition, because of the characteristic of a defined knowledge base, there are cases where multiple relations are classified in distant supervision data. In this case, the agent cannot get the optimal reward at a specific state. To improve this, this study presents a way to obtain the optimal reward for each state by separating the state by relation.