The rapid development of communication technologies, the network, advanced computing methods and wireless medical sensors gives rise to a modern medical system. In this system, large-scale electronic health records (EHRs) are often outsourced to be stored at the third parties, such as cloud service providers (CSPs). However, CSPs are not trustworthy, that is, serious security and privacy concerns about cloud service exist because it may expose the user's sensitive data to CSPs or unauthorized users in transmission, storage and sharing. To prevent the privacy disclosure of patients better and realize information sharing more effectively, this paper proposes an anonymous EHRs sharing scheme based on decentralized hierarchical attribute-based encryption (ABE). In the proposed scheme, (1) Multiple attribute authority (AA) ABE is leveraged to achieve fine-grained and scalable data access control and avoid bottleneck. Meanwhile, hierarchical access tree is used to encrypt multiple files in one operation, thereby saving calculation and storage load greatly. Moreover, the hidden access policy enhances user privacy protection. (2) The global identifier (GID) of a user is introduced to resist the collusion attack of users. Subsequently, an anonymous key generation mechanism is equipped to prevent multiple AAs from building a full profile using the user's GID. (3) To ensure the correctness and integrity of EHRs, users can conduct double verification based on the verification tag and convergent key. Finally, the efficiency analysis and experiments show that the scheme meets the security requirements of key management and privacy preservation in cloud and is proven secure and efficient in practice under the decisional bilinear Diffie-Hellman (DBDH) assumption.