Transformer for long-term effect estimation

Determining the long-term causal effects as well as making decisions in a timely fashion are the most challenging and significant tasks in A/B tests. The key challenge is that long-term and short-term effects may be different, especially in an online controlled experiment that a low-qualified ads strategy with high short-term revenue may hurt users' service experience in the future. We propose a 'Transformer for Long-Term Effect (TLTE)' method to encode the short-term latent features and predict the outcome of the long-term to estimate the long-term effects. TLTE utilises the transformer structure to capture the reasoning feature, considering the global information by the self-attention mechanism. Extensive experiments are conducted on public day-level data sets and our collected large-scale hourly data set, providing the flexible and sensitive analysis. Both quantitative and qualitative experiments demonstrate that our model has the ability to meet requirements of real-world scenarios and improves accuracy of long-term effect estimation.