Aiming at the problem that the strong coupling and nonlinearity of quadrotor UAV system make it difficult to track the desired trajectory accurately, a controller combining improved particle swarm optimization (PSO) algorithm and fractional order integral sliding mode is designed. The sliding mode control method is used to design the inner attitude loop and outer position loop controllers, and the function of controller output and tracking error is selected as the optimization objective function; To improve the high-frequency chattering problem in the system, a fractional order exponential reaching law is introduced to improve the control effect, and sufficient conditions for the system to be stable in finite time are given; In response to the problem of difficult tuning of sliding mode parameters and error coefficients, an improved PSO algorithm was adopted to optimize the sliding mode parameters and accelerate the convergence speed. Matlab simulation results show that the controller can meet the requirements of quadrotor trajectory tracking control.