The main objective of this paper is to analyse whether the Transformer neural network, which has become one of the most influential algorithms in Artificial Intelligence over the last few years, exhibits predictive capabilities for high-frequency Forex data. The prediction task is to classify short-term Forex movements for six currency pairs and five different time intervals from 60 to 720 min. We find that the Transformer exhibits high predictive power in the context of intraday Forex trading. This performance is slightly better than for the carefully selected benchmark - ResNet-LSTM, which currently is a state-of-the-art algorithm. Since intraday Forex trading based on deep learning models is largely unexplored, we offer insight on which currency pair and time interval are amenable to devising a profitable trading strategy. We also show that high predictive accuracy can be misleading in real world trading for short time intervals, as models trained on OHLC data tend to report the highest accuracy when the spread cost is the highest. This renders assessment based on typical machine learning metrics overly optimistic. Therefore, it is critical to backtest frequent intraday Forex trading strategies with realistic cost assumptions, which is rarely the case in empirical literature. Lastly, sensitivity analysis shows that the length of the time interval used for training does not play a critical role in the Transformer's predictive capabilities, whereas features derived from technical analysis are essential.
