Safety and mission-critical systems span across an extensive array of research areas, including transportation (aviation, aerospace, automotive, rail), cybersecurity, robotics, and medicine. Significant advancements in computational performance continue to drive an increased desire to automate systems, including safety and mission-critical systems. As autonomous systems gain popularity on the ground, in the air, and throughout space, the use of artificial intelligence (AI) technologies to manage onboard decision-making has become an increasingly appealing approach. The growing demand to improve the effectiveness of automation has pressed contemporary systems to increasingly deploy machine learning strategies that further enhance the speed and accuracy of system responses. Safety-critical systems require critical decision-making to be performed at specific decision points based on timely and reliable data. Conventional designs are based on deterministic methodologies that are thoroughly verified and validated against known expected behaviors. However, the unpredictability of AI systems introduces considerable risks that are challenging to mitigate. The joining of AI with autonomous systems that require safe operations poses a significant challenge-guaranteeing the safety requirements in the context of the unpredictable nature of AI. This paper presents an analysis of the current state of research on the use of artificial intelligence (AI) approaches for safety and mission-critical systems. The goal of this paper is to understand how researchers are approaching these problems and identify and characterize distinct areas of research that show a potential to advance the use of AI for these types of systems.