A Reinforcement Learning Memristive Circuit Based on Q-Learning and Operant Conditioning

In most memristive neural network circuits based on operant conditioning, the agent’s tendency towards certain behaviors is simply reflected through changes in synaptic weight. No specific analysis has been conducted on the changes in the tendency of intelligent agent behavior. Therefore, an operant conditioning circuit based on memristors and Q-learning is proposed to analyze the behavior and decision-making of intelligent agents in complex environments. The designed network uses Q-learning to update the decision voltage in the circuit based on the optimal Bellman equation, allowing agents to make different strategies according to the constantly changing environment to achieve optimal results. In addition, this study also uses the Double Q-learning to effectively reduce the overestimation bias of Q-learning and improve the stability and strategy performance of agent learning by introducing Q value estimation. This design provides more References for the development of automobile obstacle avoidance systems.