Document Type : Research Paper
Authors
1 Department of Computer Engineering, Faculty of Engineering, Shahid Bahonar University of Kerman, Kerman, Iran.
2 Department of Computer Engineering, Faculty of Engineering, Shahid Bahonar University of Kerman, Kerman, Iran
3 Department of Computer Engineering, Shahid Bahonar University of Kerman, Kerman, Iran
Abstract
Long-term multivariate time series forecasting is one of the most challenging problems in machine learning. Among the proposed solutions, deep learning networks—particularly transformer-based models—have demonstrated superior performance. However, these models are vulnerable to noise, uncertainty, and abrupt changes, and often lack interpretability. To address these limitations, this study introduces a novel hybrid architecture called FuzzyPE-KAN, which integrates fuzzy logic into the transformer framework. The proposed architecture incorporates: (1) a learnable Gaussian noise-based fuzzy attention mechanism that enhances robustness against noise; (2) a learnable fuzzy positional encoding relying on Gaussian membership functions and multilayer perceptrons to effectively model the inherently vague and graded nature of time; and (3) complete replacement of feed-forward layers with Kolmogorov–Arnold Networks to dramatically reduce the number of parameters and improve interpretability. The proposed architecture was applied to five state-of-the-art baseline models (Transformer, Informer, PatchTST, Crossformer, and iTransformer) and evaluated on eight standard benchmark datasets (ETTh1/2, ETTm1/2, Weather, Electricity, Traffic, and Exchange Rate). Results indicate that the proposed variants achieve an average improvement of 26–49% in Mean Squared Error and 17–29% in Mean Absolute Error across most scenarios compared to the baseline models. The most substantial gains were observed on the Exchange Rate dataset (78% improvement), Weather (71.28%), ETTh2 (76.41%), and ETTm2 (78.66%). This research demonstrates that the simultaneous integration of fuzzy logic and Kolmogorov–Arnold Networks within a transformer architecture not only enhances accuracy and robustness but also elevates model interpretability to a significant level, paving the way for real-world applications in finance, energy, and healthcare domains.
Keywords
- Multivariate Time series forecasting
- Fuzzy logic
- Fuzzy positional encoding
- Transformer
- Kolmogorov–Arnold networks
Main Subjects
Analysis and Applications, 13(1) (2022), 499-511. https://doi.org/10.22075/ijnaa.2022.5521
for high-dimensional data, International Conference on Artificial Intelligence and Soft Computing, 15948 (2026),
231-242. https://doi.org/10.1007/978-3-032-03705-3_20
Advanced Computer Science and Applications, 14(1) (2023), 313-320. https://doi.org/10.14569/IJACSA.2023.
0140133
[4] J. Alcal´a-Fdez, et al., KEEL: A software tool to assess evolutionary algorithms for data mining problems, Soft
Computing, 13(3) (2009), 307-318. https://doi.org/10.1007/s00500-008-0323-y
forecasting, Energy Reports, 8(9) (2022), 1084-1091. https://doi.org/10.1016/j.egyr.2022.07.139
(2018). https://doi.org/10.48550/arXiv.1803.01271
(2025). https://doi.org/10.48550/arXiv.2504.00070
and temporal convolution, International Journal of Approximate Reasoning, 186 (2025). https://doi.
org/10.1016/j.ijar.2025.109521
10.1093/oso/9780198714835.001.0001
2403.18361
Machines, 12(6) (2024), 380. https://doi.org/10.3390/machines12060380
Healthcare, 11(18) (2023). https://doi.org/10.3390/healthcare11182530
48550/arXiv.2408.11306
Computing Surveys, 58(7) (2026), 1-35. https://doi.org/10.1145/3786319
Cybernetics, 23(3) (1993), 665-685. https://doi.org/10.1109/21.256541
(2025). https://doi.org/10.48550/arXiv.2508.01575
30(1) (2023), 727-747. https://doi.org/10.1007/s11831-022-09815-7
driven networks, Future Internet, 15(8) (2023), 255. https://doi.org/10.3390/fi15080255
https://doi.org/10.48550/arXiv.1907.00235
arXiv.2405.06721
//doi.org/10.48550/arXiv.2310.06625
arXiv.2408.10205
2404.19756
https://doi.org/10.48550/arXiv.2503.17970
Electrical and Computer Engineering Innovations, (2021). https://doi.org/10.22061/jecei.2020.7426.391
learning systems, arXiv, (2025). https://doi.org/10.48550/arXiv.2504.03749
2022 IEEE International Conference on Distributed Computing and Electrical Circuits and Electronics (ICDCECE), (2022), 1-6. https://doi.org/10.1109/ICDCECE53908.2022.9793213
exponential smoothing (ES) based on time series model for forecasting road accidents, Arabian Journal for Science
and Engineering, 46(11) (2021), 11113-11138. https://doi.org/10.1007/s13369-021-05650-3
water level prediction, Journal of Hydrology, 649 (2025). https://doi.org/10.1016/j.jhydrol.2024.132430
series prediction, IEEE Transactions on Fuzzy Systems, 33(7) (2025). https://doi.org/10.1109/TFUZZ.2025.
3549920
forecast wind power, Applied Energy, 353 (2024). https://doi.org/10.1016/j.apenergy.2023.122015
Machine Learning Research, 1(11) (2025). https://doi.org/10.20944/preprints202506.1173.v1
multiple seasonal time series: An overview, Fluctuation and Noise Letters, 20(04) (2021). https://doi.org/10.
1142/S0219477521300032
Information Processing Systems, (2017), 6000-6010. https://doi.org/10.48550/arXiv.1706.03762
Reports, 13(1) (2023). https://doi.org/10.1038/s41598-023-47812-3
with AutoML method for time series classification, Scientific Reports, (2024). https://dx.doi.org/10.2139/
ssrn.4884435
MethodsX, 13 (2024). https://doi.org/10.1016/j.mex.2024.102946
series forecasting, NIPS’21: Proceedings of the 35th International Conference on Neural Information Processing
Systems, (2021), 22419-22430. https://doi.org/10.48550/arXiv.2106.13008
in Engineering, 2021(1) (2021). https://doi.org/10.1155/2021/6655102
10594
Sciences, 15(23) (2025), 12698. https://doi.org/10.3390/app152312698
//doi.org/10.48550/arXiv.2211.14730
S0019-9958(65)90241-X
forecasting, International Conference on Learning Representations, (2023). https://api.semanticscholar.org/
CorpusID:259298223
of the AAAI Conference on Artificial Intelligence, 35 (2021), 11106-11115. https://doi.org/10.48550/arXiv.
2012.07436