In order to enable UAVs to quickly find an accident-free, shorter and safer flight path between the take-off point and the end point in a complex 3-dimensional urban environment, an advanced population optimization algorithm based on proximal policy gradient optimization (PPO) for unmanned aerial vehicle (UAV)-assisted distribution system in dynamic environments is designed, and the proposed PPO-PSO algorithm is proposed. Based on the characteristics of standard PPO algorithm and particle swarm optimization (PSO), new improvements were made to the PPO algorithm; long short term memory (LSTM), convolutional neural network (CNN) were incorporated, and the PPO-PSO algorithm was utilized, and modified the iterative method of intelligences using particle optimization to solve the problem of poor local search ability of neural networks; the convergence of the algorithm is demonstrated, and simulations are carried out in Python environment to verify its effectiveness. The simulation results show that PPO-PSO is better in convergence speed and solution speed, and has better robustness.