Quantum secret sharing is an important technology in quantum cryptography, and it is used in the fields of secure multiparty computation, quantum auction and fiber network configuration. In this paper, we construct two quantum secret sharing schemes with adversary structure by using three-qudit GHZ state and three qubits. In the proposed scheme, the dealer first generates the private keys based on the adversary structure, then the participants in authorized subset use their private keys to perform the unitary operations on the quantum state and recover the original secret. The security analysis shows the attacks of intercept-and-resend, entangle-and-measure and participant are impossible in our schemes. Compared with the existing quantum secret sharing schemes, we realize the sharing of classical information and quantum state, and need less computational cost. Moreover, our schemes are more efficient when the adversary structure can be obtained directly.