Živko Sokolović

The forecast of electricity consumption is a key aspect of planning, operation, maintenance and management of the electric power system, as it enables the optimal use of production resources, increasing the reliability of the system and reducing operating costs. In the last few years, the development of artificial intelligence and machine learning methods have found wide application in this area, providing advanced tools for the analysis and modeling of complex time series. Deep learning models, especially recurrent neural networks (RNNs) and convolutional neural networks (CNNs), have dominated time series prediction tasks due to their ability to model nonlinear dependencies in data. However, these models have certain limitations, especially in capturing long-term dependencies and efficient processing of long data sequences. In natural language processing (NLP), a new architecture based on deep learning has emerged – Transformer, which was initially introduced to overcome the limitations of RNN models arising from their sequential nature. Thanks to the self-attention mechanism, Transformer enables direct modeling of dependencies between all elements in the sequence, regardless of their mutual distance. This feature makes Transformer particularly suitable for application in time series prediction problems, where the observation of long-term and complex patterns is of key importance. In this work, the Transformer model was applied to forecast electricity consumption. The model enables simultaneous processing of the entire time series, where each input data can affect any position in the output, which significantly improves the modeling of long-term dependencies and complex patterns in the data. An additional advantage of this approach is reflected in the possibility of parallelizing computations, which leads to faster training and prediction performance compared to traditional sequential models. The experiments were carried out on a data set comprising the electricity consumption of 370 customers in Portugal, with an hourly time resolution, where the aggregated consumption was considered. Model evaluation was performed using standard accuracy metrics, such as MAE, RMSE, and MAPE. The influence of varying the length of the input sequence on the performance of the model was examined. In addition, special attention is paid to the assessment of model performance at longer forecast horizons, as well as the impact of additional weather attributes, such as hour of the day, day of the week, and month, on forecast accuracy. Also, the influence of different hyperparameters of the model on the final performance was analyzed. The obtained results indicate that the Transformer model achieves high forecast accuracy, with stable performance in all considered scenarios.

Keywords: Load forecasting, Time series, Transformer, Deep learning