Medical records are one of the crucial documents and a significant asset for anyone seeking treatment. Electronic health records (EHRs) have made a dynamic shift by making them easier to manage, facilitate and share among various stakeholders such as doctors, lab technicians, and insurance agents. EHRs are vulnerable to hacker, cybercriminal attacks, and data breaches. Once compromised, health records cannot be retrieved. As a result, patients must have control over who gets their EHRs, when they get them, and where they get them. To address the aforementioned issue, this paper proposes a blockchain-based secure record-keeping and trustworthy sharing system. In order to do this, a distributed off-chain storage architecture for large-scale medical data storage is developed, which overcomes the drawbacks of on-chain data storage and enhances scalability. The distributed storage, i.e., InterPlanetary File System, is a content-addressable storage that ensures the integrity of the content such that a slight modification in the stored EHR records results in a change in the obtained hash value. Furthermore, a Ciphertext Policy Attribute-Based Encryption (CP-ABE) algorithm integrated with blockchain technology is designed for fine-grained access control, allowing only authorized users to access specific EHR data based on their attributes. The combination of CP-ABE with blockchain technology provides a tamper-proof and verifiable audit trail of all data access and updations made to EHRs. This enhances accountability and ensures that the patients or owners can track and verify all actions taken on the data. To implement the proposed system, the Remix-Ethereum IDE is used. Smart contracts (SCs) are designed with access permissions so patients have complete control over their records. The scalability and immutability of the system is ensured by storing the hash of the encrypted EHRs on the blockchain and the actual encrypted records on IPFS. The security analysis of the proposed system is carried out by evaluating its resistance to various attacks. Additionally, potential security flaws in the proposed SCs are investigated using the Oyente tool. Different test cases are presented to demonstrate the functionality and cost analysis of the proposed system.