In order to improve the intrusion detection performance of communication system, the K-means algorithm is optimized. Aiming at the problem that the number of clusters of network data features can not be estimated in advance, the validity index of K value is proposed to determine the number of clusters and evaluate the quality of clustering. At the same time, the influence of various cluster features on clustering is considered, and the feature weighted distance is used to consider the closeness within the cluster and the separation between the clusters, which is used as the clustering center. The experimental results show that the improved K-means intrusion detection algorithm has better detection rate and false alarm rate, and can effectively improve the quality of system security defense.