Observational prospects of self-interacting scalar superradiance with next-generation gravitational-wave detectors

Current- and next-generation gravitational-wave observatories may reveal new, ultralight bosons. Through the superradiance process, these theoretical particle candidates can form clouds around astrophysical black holes and result in detectable gravitational-wave radiation. In the absence of detections, constraints-contingent on astrophysical assumptions-have been derived using LIGO-Virgo-KAGRA data on boson masses. However, the searches for ultralight scalars to date have not adequately considered self-interactions between particles. Self-interactions that significantly alter superradiance dynamics are generically present for many scalar models, including axion-like dark matter candidates and string axions. We implement the most complete treatment of particle self-interactions available to determine the gravitational-wave signatures expected from superradiant scalar clouds and revisit the constraints obtained in a past gravitational-wave search targeting the black hole in Cygnus X-1. We also project the reach of next-generation gravitational-wave observatories to scalar particle parameter space in the mass-coupling plane. We find that while proposed observatories have insufficient reach to self-interactions that can halt black hole spin-down, next-generation observatories are essential for expanding the search beyond gravitational parameter space and can reach a mass and interaction scale of similar to 10-13-10-12 eV /c2 and greater than or similar to 1017 GeV, respectively.