This paper presents a material optimization framework for identifying optimal material typologies to improve structural performance under the presence of uncertainties. Specifically, the focus in this work is on carbon nanotube (CNT)-reinforced concrete with the optimization problem consisting in finding the optimal CNT orientation in the host material so as to minimize the total deformation of structures made up from the composite. Regarding the material modeling, a two-level approach is considered to characterize the mechanical properties of the reinforced concrete. Specifically, cement mortar enhanced with carbon nanotubes is studied at a microscale level where a Drucker-Prager plasticity model is assumed to describe its inelastic behavior. Subsequently, the reinforced mortar along with the concrete's larger aggregates is studied at a mesoscale level using continuum micromechanics. For the analysis of structural systems comprised of this composite material, an extension of the FE2 technique, termed FE3, is employed. To overcome the immense computational demands associated with FE3, efficient neural network-based surrogates are developed to approximate the nonlinear constitutive law of the composite. In this setting, the stochastic optimization problem equates to finding the optimal orientation of CNTs in the cement mortar, so as to achieve small structural deformations with low variability, under the presence of uncertainty in the loading conditions. To solve this problem, the Covariance Matrix Adaptation Evolution Strategy is chosen herein, and even though this approach requires a massive number of model runs, it is performed at a reasonable computational cost by virtue of the elaborated surrogate modeling scheme.