Research has shown that adverse childhood experiences (ACEs) lead to substance use and this relationship may differ by gender. There is limited knowledge about how ACEs relate to the risk of substance use in South Korea. The current study aims to identify the latent patterns of co-occurring ACEs and examines their relationships with substance use among college students. This study used data collected in 2019 from a national sample of Korean college students (N = 1,037). This study conducted a Latent Class Analysis (LCA) with distal outcomes, using the Bolck, Croons, and Hagenaars method. The independent variable was the patterns of ACEs identified using 14 indicators of childhood adversity. Two dependent variables measured self-reported substance use in smoking and alcohol use. The LCA identified four heterogeneous ACEs classes: High Adversity, Family Violence, Economic Hardship, and Low Adversity. The analysis showed different associations between the ACE patterns and substance use by gender: For women, the Family Violence classes (OR = 1.13, p < .05) were significantly associated with the odds of daily smoking; for men, the High Adversity class was significantly associated with the odds of problem drinking (OR = 1.24, p < .01). This study discusses the importance of understanding the roles of ACEs on substance use and gender differences. These findings highlight the need for gender-specific understanding and interventions efforts to reduce the impact of ACEs on substance use.