|
In this paper, a queue batch sampling algorithm is proposed to address the problem of deep reinforcement learning in continuous action control tasks that affects the efficiency of the algorithm due to high sampling correlation. The proposed sampling algorithm reduces the sampling correlation while using the forward value of sampling as a measure to ensure the value of sampling. First, the starting sample is randomly selected as the sampling starting point in the memory bank generated by the robot's interaction with the environment. Second, to save computational resources, the samples in the repository are sampled for the first time, the correlation between the sampling starting point and other samples and the forward multi-step reward of all samples except the sampling starting point is calculated, and the score of each sample based on the ranking position is obtained based on the weighted cumulative results. Finally, the samples with the previous minimum sampling batch size are selected based on the scores to train the deep reinforcement learning model. The proposed queueing batch sampling algorithm is fused with the deep proximal policy optimization algorithm and applied to the forward motion task of a bipedal robot. The proposed improved deep proximal optimization algorithm is compared with the original deep proximal optimization algorithm of the actor-critic framework in simulation, and the learning efficiency of the deep neural network is improved due to the queueing than sampling algorithm which reduces the correlation of sampling and ensures the quality of sampling, and the learning effect of the deep proximal policy optimization algorithm based on queueing batch sampling is significantly improved, that is, the proposed queueing batch sampling algorithm can effectively improve the learning efficiency of bipedal robot learning forward.
|
