A compactness theorem for SO(3) anti-self-dual equation with translation symmetry

Motivated by the Atiyah-Floer conjecture, we consider SO(3) anti-self-dual instantons on the product of the real line and a three-manifold with cylindrical end. We prove a Gromov-Uhlenbeck type compactness theorem, namely, any sequence of such instantons with uniform energy bound has a subsequence converging to a type of singular objects which may have both instanton and holomorphic curve components. In particular, we prove that holomorphic curves that appear in the compactification must satisfy the Lagrangian boundary condition, a claim which has been long believed in the literature. This result is the first step towards constructing a natural bounding cochain proposed by Fukaya for the SO(3) Atiyah-Floer conjecture.(C) 2022 Elsevier Inc. All rights reserved.