Aiming at the speed limit of detection and tracking of the flying target in TV tracking system, a real-time TV tracking system, based on the core-processor of Altera Company’s EP2C70F672C6 FPGA and cooperated with TMS320C6416 DSP is designed. In this system, FPGA is used to realize video image collection, preprocessing, moving target detection and wave-gate overlap for its powerful parallel-pipeline processing. DSP is used to implement template-matching to ensure the tracking target. The result shows that the system can automatically execute the detection and tracking for moving target timely.