Bromodomain-containing protein 4 (BRD4), as the reader of epigenetics, could regulate gene transcription by recognizing acetylated lysine of histones. In recent years, researchers have found that dysregulation of BRD4 leads to the occurrence and development of various cancers, making BRD4 a promising target for cancer therapy. To identify novel BRD4 inhibitors from natural products, a hierarchical virtual screening method including pharmacophore modeling, molecular docking, and molecular dynamic simulation was performed to obtain five hit compounds with potential BRD4 inhibitory activity. Subsequently, structural optimization of the hit compound (ZINC2648030) with chromone structure was conducted to afford a series of derivatives (8a–13e), and their BRD4 inhibitory activities were evaluated. Among them, 13b showed remarkable BRD4 inhibitory activity (IC50 = 0.60 μM). Moreover, 13b displayed a potent inhibitory effect on A549 cells with an IC50 value of 0.79 μM, and further investigations demonstrated that it has the potential to induce apoptosis, inhibit colony formation, and suppress cell invasion. These findings indicated that 13b might be a candidate for cancer treatment.