Artificial intelligence (AI) can be used to solve complex problems in a short amount of time or give machines the ability to make decisions based on previous data. In recent years, AI has been used in many fields such as medicine, robotics, and aerospace. However, in many fields of the construction industry, such as structural engineering, AI has not been widely used in practice. One potential application of AI that has been the focus of several research efforts over the years is to predict the compressive strength of concrete in existing reinforced concrete (RC) structures. Accurate estimation of concrete strength of existing RC structures is an important challenge for engineers. The most reliable method to obtain the compressive strength of concrete is to perform the core test (destructive test) which causes damage to the structure and is very costly and time-consuming. Moreover, due to safety or project conditions, it is not always possible to extract cores. Two common non-destructive methods that correlate with the compressive strength of concrete are the ultrasonic pulse velocity (UPV) and the rebound hammer test. Several equations and regression models have been proposed to estimate the strength of concrete based on these two non-destructive tests. Each of these models is limited by specific boundary conditions, and the equations are not universally accurate or reliable. In recent years, studies have been conducted to develop AI models based on non-destructive testing of concrete. This paper first reviews the studies conducted in this field and then discusses the remaining challenges and explains why; despite several years of studies around the world, there is still no accurate and usable model for the industry. Finally, advantages for future AI models are described.