To improve the reliability and transmission rate of chaotic communication method using chaos segment shift correlation delay shift keying (SSCDSK) system, a new segment shift M-ary differential chaos shift keying aided by discrete sine transform (DST-SS-MDCSK) is proposed. At the transmitter, the chaotic base signals are used as the reference signals. After an improved segment shift, two approximately orthogonal signals are generated and then modulated through M-ary phase-shift keying (MPSK) modulator to obtain the data bearing signals. This method combines index modulation(IM) technology to effectively increase the system's transmission rate. To further improve the transmission rate, the reference signals generated by different chaos generators are summed up with the data bearing signals by discrete sine transform (DST) codes. At the receiver, the recovered reference signals are correlated with the data bearing signals. The theoretical bit error rate (BER) is derived for additive white Gaussian noise (AWGN) channel and multipath Rayleigh fading channel (RFC), and the accuracy of the BER is verified through Monte Carlo simulations. The experimental results indicate that, under similar conditions, the DST-SS-MDCSK system achieved a (4N-1)x100%\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$(4N-1)\times 100 \%$$\end{document} increase in transmission rate compared with the SSCDSK system , where N represents the dimension of the DST matrix. With equal BER, a performance gain of approximately 1 dB can be achieved.