In order to make the unmanned aerial vehicle quickly find an accident-free, shorter and safer flight path from the starting point to the end point in the complex three-dimensional urban environment, An advanced swarm optimization algorithm based on proximal policy optimization (PPO) for unmanned aerial vehicle aided distribution system in dynamic environment is designed, and a PPO-PSO algorithm is proposed. Based on the characteristics of the standard PPO algorithm and the particle swarm optimization (PSO) algorithm, the PPO algorithm is improved; It integrates long short time memory (LSTM), convolutional neural network (CNN), and uses particle optimization to modify the iteration mode of agents, which solves the problem of poor local search ability of neural network; The convergence of the algorithm is demonstrated, and its effectiveness is verified by simulation in Python environment. The simulation results show that PPO-PSO is better in convergence speed and solution speed, and has better robustness.