Enhanced High-Dimensional Data Classification by Combining Fuzzy Learning Integration and Graph Transformers

Bastami, Sajad; Pirmohamadiani, Rojiar; Dowlatshahi, Mohammad Bagher; Abdollahpouri, Alireza

doi:10.22111/ijfs.2025.50945.8999

Document Type : Research Paper

Authors

¹ Department of Computer Engineering, University of Kurdistan, Sanandej, Iran

² Department of Computer Engineering, University of Lorestan, Khorramabad, Iran

https://doi.org/10.22111/ijfs.2025.50945.8999

Abstract

Graph neural networks and fuzzy models offer effective and practical methods for solving various tasks at the large-scale graph level. Large-scale graph embedding based on deep methods and fuzzy models is categorized into fusion and integration. Feature extraction and graph structure at the local and global levels are based on augmented graph fusion. In fusion-based graph embedding, the fuzzy model is used as an activation function based on an aggregated process. In some cases, the fusion of graph neural network methods with fuzzy systems has been successful. However, no effective methods have been developed for integrating fuzzy models with deep methods. Two main issues are associated with this integration: (1) computational complexity due to the exponential increase in fuzzy rules with the number of features, and (2) the complexity of the solution space due to the combination of fuzzy regression rules between inputs and outputs. Additionally, modeling at the large-scale graph level using linear regression and graph neural networks is not sufficient. Therefore, this paper proposes a feature and structure combination method at the local and global levels using a combination of fuzzy modeling and graph transformers, an integrated deep learning technique called Fuzzy Graph Transformer (FuzzyGT). We conducted experiments on deep learning graph datasets to compare with the proposed model. Our method achieved the best results compared to other advanced models

Keywords

Main Subjects

Social Sciences

References

[1] M. Akram, A. Khan, A. Borumand Saeid, Complex Pythagorean Dombi fuzzy operators using aggregation operators
and their decision making, Expert Systems, 38(2) (2021), e12626. https://doi.org/10.1111/exsy.12626

[2] M. Alateeq, W. Pedrycz, Logic-oriented fuzzy neural networks: A survey, Expert Systems with Applications, 257
(2024), 125120. https://doi.org/10.1016/j.eswa.2024.125120

[3] S. Bastami, A. Abdollahpouri, R. Pir Mohammadiani, FARW: A feature-aware random walk for node classification,
Journal of Innovations in Computer Science and Engineering JICSE, no. Online First, 1(2) (2025). https://doi.
org/10.48308/jicse.2025.237378.1039

[4] E. Bastami, et al., A gravitation-based link prediction approach in social networks, Swarm and Evolutionary Computation, 44 (2019), 176-186. https://doi.org/10.1016/j.swevo.2018.03.001

[5] K. D. Bollacker, S. Lawrence, C. L. Giles, CiteSeer: An autonous web agent for automatic retrieval and identification
of interesting publications, in Proceedings of the Second International Conference on Autonomous Agents -
AGENTS ’98, Minneapolis, Minnesota, United States: ACM Press, (1998), 116-123. https://doi.org/10.1145/
280765.280786

[6] X. Chang, J. Wang, M. Wen, Y. Wang, Y. Huang, M-Graphormer: Multi-channel graph transformer for node
representation learning, IEEE Transactions on Big Data, (2024), 1-13. https://doi.org/10.1109/TBDATA.2024.
3489418

[7] J. Chen, G. Li, J. E. Hopcroft, K. He, SignGT: Signed attention-based graph transformer for graph representation
learning, arXiv, (2023). https://doi.org/10.48550/ARXIV.2310.11025

[8] F. Dernoncourt, J. Y. Lee, PubMed 200k RCT: A dataset for sequential sentence classification in medical abstracts,
arXiv, (2017). https://doi.org/10.48550/ARXIV.1710.06071

[9] M. B. Dowlatshahi, V. Derhami, H. Nezamabadipour, Fuzzy particle swarm optimization with nearest-better neighborhood for multimodal optimization, Iranian Journal of Fuzzy Systems, 17(4) (2020), 7-24. https://doi.org/10.
22111/ijfs.2020.5403

[10] M. B. Dowlatshahi, A. Hashemi, Unsupervised feature selection: A fuzzy multi-criteria decision-making approach,
Iranian Journal of Fuzzy Systems, 20(7) (2023), 55-70. https://doi.org/10.22111/ijfs.2023.7630

[11] F. Fanian, M. Kuchaki Rafsanjani, CFMCRS: Calibration fuzzy-metaheuristic clustering routing scheme simultaneous
in on-demand WRSNs for sustainable smart city, Expert Systems with Applications, 211 (2023), 118619.
https://doi.org/10.1016/j.eswa.2022.118619

[12] F. Fanian, M. Kuchaki Rafsanjani, M. Shokouhifar, Combined fuzzy-metaheuristic framework for bridge health
monitoring using UAV-enabled rechargeable wireless sensor networks, Applied Soft Computing, 167 (2024), 112429.
https://doi.org/10.1016/j.asoc.2024.112429

[13] B. R. HamaKarim, R. P. Mohammadiani, A. Sheikhahmadi, B. R. Hamakarim, M. Bahrami, A method based on
k-shell decomposition to identify influential nodes in complex networks, The Journal of Supercomputing, 79(14)
(2023), 15597-15622. https://doi.org/10.1007/s11227-023-05296-y

[14] M. Ishaque, M. G. M. Johar, A. Khatibi, M. Yamin, A novel hybrid technique using fuzzy logic, neural networks
and genetic algorithm for intrusion detection system, Measurement: Sensors, 30 (2023), 100933. https://doi.
org/10.1016/j.measen.2023.100933

[15] W. Jia, M. Sun, J. Lian, S. Hou, Feature dimensionality reduction: A review, Complex and Intelligent Systems,
8(3) (2022), 2663-2693. https://doi.org/10.1007/s40747-021-00637-x

[16] M. Joodaki, M. B. Dowlatshahi, N. Z. Joodaki, An ensemble feature selection algorithm based on PageRank centrality
and fuzzy logic, Knowledge-Based Systems, 233 (2021), 107538. https://doi.org/10.1016/j.knosys.2021.
107538

[17] T. R. Khalifa, A. M. El-Nagar, M. A. El-Brawany, E. A. G. El-Araby, M. El-Bardini, A novel fuzzy Wiener-based
nonlinear modelling for engineering applications, ISA Transactions, 97 (2020), 130-142. https://doi.org/10.
1016/j.isatra.2019.07.017

[18] D. Kreuzer, D. Beaini, W. L. Hamilton, V. L´etourneau, P. Tossou, Rethinking graph transformers with spectral
attention, arXiv, (2021). https://doi.org/10.48550/ARXIV.2106.03893

[19] Krishankumar R, R. Ks, A. B. Saeid, A new extension to PROMETHEE under intuitionistic fuzzy environment
for solving supplier selection problem with linguistic preferences, Applied Soft Computing, 60 (2017), 564-576.
https://doi.org/10.1016/j.asoc.2017.07.028

[20] D. Krleza, K. Fertalj, Graph matching using hierarchical fuzzy graph neural networks, IEEE Transactions on Fuzzy
Systems, 25(4) (2017), 892-904. https://doi.org/10.1109/TFUZZ.2016.2586962

[21] W. Li, H. Fujita, C. Zhang, S. F. Su, Editorial: Fuzzy big data-driven computational intelligence models and
applications, International Journal of Fuzzy Systems, (2024), s40815-024-01821–0. https://doi.org/10.1007/
s40815-024-01821-0

[22] F. Li, T. Zheng, Q. Cao, Modeling and identification for practical nonlinear process using neural fuzzy networkbased
Hammerstein system, Transactions of the Institute of Measurement and Control, 45(11) (2023), 2091-2102.
https://doi.org/10.1177/01423312221143777

[23] T. Lin, Y. Wang, X. Liu, X. Qiu, A survey of transformers, AI Open, 3 (2022), 111-132. https://doi.org/10.
1016/j.aiopen.2022.10.001

[24] R. Liu, S. Duan, L. Xu, L. Liu, J. Li, Y. Zou, A fuzzy transformer fusion network (FuzzyTransNet) for medical
image segmentation: The case of rectal polyps and skin lesions, Applied Sciences, 13(16) (2023), 9121. https:
//doi.org/10.3390/app13169121

[25] Y. Liu, M. Liu, X. Xu, Adaptive control design for fixed-time synchronization of fuzzy stochastic cellular neural
networks with discrete and distributed delay, Iranian Journal of Fuzzy Systems, 18(6) (2021), 13-28. https://doi.
org/10.22111/ijfs.2021.6330

[26] J. Lu, G. Ma, G. Zhang, Fuzzy machine learning: A comprehensive framework and systematic review, IEEE
Transactions on Fuzzy Systems, 32(7) (2024), 3861-3878. https://doi.org/10.1109/TFUZZ.2024.3387429

[27] G. Mali, S. Misra, CORA: Cooperative communication and optimal resource allocation in multihop wireless multimedia
sensor networks, IEEE Internet of Things Journal, 11(3) (2024), 4076-4084. https://doi.org/10.1109/
JIOT.2023.3300774

[28] V. Ojha, A. Abraham, V. Sn´aˇsel, Heuristic design of fuzzy inference systems: A review of three decades of research,
Engineering Applications of Artificial Intelligence, 85 (2019), 845-864. https://doi.org/10.1016/j.engappai.
2019.08.010

[29] K. Oono, T. Suzuki, Graph neural networks exponentially lose expressive power for node classification, Published
as a Conference Paper at ICLR, (2020). https://doi.org/10.48550/ARXIV.1905.10947
[30] P. Pham, An integrated simplicial neural network with neuro-fuzzy network for graph embedding, International Journal of Machine Learning and Cybernetics, 16 (2025), 233-251. https://doi.org/10.1007/s13042-024-02201-8

[31] M. Rehyani Hamedani, S. W. Kim, AdaSim: A recursive similarity measure in graphs, in Proceedings of the 30th
ACM International Conference on Information and Knowledge Management, Virtual Event Queensland Australia:
ACM, (2021), 1528-1537. https://doi.org/10.1145/3459637.3482316

[32] M. Riyahi, A. Borumand Saeid, M. Kuchaki Rafsanjani, Improved q-rung orthopair and T-spherical fuzzy sets,
Iranian Journal of Fuzzy Systems, 19(3) (2022), 155-170. https://doi.org/10.22111/ijfs.2022.6949

[33] A. Saeedi, M. Kuchaki Rafsanjani, S. Yazdani, Energy efficient clustering in IoT-based wireless sensor networks
using binary whale optimization algorithm and fuzzy inference system, The Journal of Supercomputing, 81(1)
(2025), 209. https://doi.org/10.1007/s11227-024-06556-1

[34] S. Salmani Pour Avval, N. D. Eskue, R. M. Groves, V. Yaghoubi, Systematic review on neural architecture search,
Artificial Intelligence Review, 58(3) (2025), 73. https://doi.org/10.1007/s10462-024-11058-w

[35] W. Samek, G. Montavon, S. Lapuschkin, C. J. Anders, K. R. Muller, Explaining deep neural networks and beyond:
A review of methods and applications, Proceedings of the IEEE, 109(3) (2021), 247-278. https://doi.org/10.
1109/JPROC.2021.3060483

[36] P. Shaw, J. Uszkoreit, A. Vaswani, Self-attention with relative position representations, arXiv, (2018). https:
//doi.org/10.48550/ARXIV.1803.02155

[37] V. Sn´aˇsel, M. ˇStˇepniˇcka, V. Ojha, P. N. Suganthan, R. Gao, L. Kong, Large-scale data classification based on
the integrated fusion of fuzzy learning and graph neural network, Information Fusion, 102 (2024), 102067. https:
//doi.org/10.1016/j.inffus.2023.102067

[38] L. Sun, J. Hou, C. Xing, Z. Fang, A robust Hammerstein-Wiener model identification method for highly nonlinear
systems, Processes, 10(2) (2022), 2664. https://doi.org/10.3390/pr10122664

[39] T. Takagi, M. Sugeno, Fuzzy identification of systems and its applications to modeling and control, in Readings in
Fuzzy Sets for Intelligent Systems, Elsevier, (1993), 387-403. https://doi.org/10.1016/B978-1-4832-1450-4.
50045-6

[40] N. Talpur, S. J. Abdulkadir, H. Alhussian, M. H. Hasan, N. Aziz, A. Bamhdi, Deep neuro-fuzzy system application
trends, challenges, and future perspectives: A systematic survey, Artificial Intelligence Review, 56(2) (2023), 865-
913. https://doi.org/10.1007/s10462-022-10188-3

[41] H. H. Tang, N. S. Ahmad, Fuzzy logic approach for controlling uncertain and nonlinear systems: A comprehensive
review of applications and advances, Systems Science and Control Engineering, 12(1) (2024), 2394429.https:
//doi.org/10.1080/21642583.2024.2394429

[42] J. Tavoosi, A. Mohammadzadeh, K. Jermsittiparsert, A review on type-2 fuzzy neural networks for system identification, Soft Computing, 25(10) (2021), 7197-7212. https://doi.org/10.1007/s00500-021-05686-5

[43] P. Veliˇckovi´c, G. Cucurull, A. Casanova, A. Romero, P. Li´o, Y. Bengio, Graph attention networks, arXiv, (2017).
https://doi.org/10.48550/ARXIV.1710.10903

[44] T. Vo, An integrated fuzzy neural supervision and attention-based graph neural network for improving network
clustering, Neural Computing and Applications, 35(33) (2023), 24015-24035. https://doi.org/10.1007/
s00521-023-08974-3

[45] X. Wei, G. Wang, B. Schmalz, D. F. T. Hagan, Z. Duan, Evaluation of transformer model and self-attention
mechanism in the Yangtze river basin runoff prediction, Journal of Hydrology: Regional Studies, 47 (2023), 101438.
https://doi.org/10.1016/j.ejrh.2023.101438

[46] Z. Wu, S. Pan, F. Chen, G. Long, C. Zhang, P. S. Yu, A comprehensive survey on graph neural networks, IEEE
Transactions on Neural Networks and Learning Systems, 32(1) (2021), 4-24. https://doi.org/10.1109/TNNLS.
2020.2978386

[47] L. Xiao, X. Wu, G. Wang, Social network analysis based on graph SAGE, in 2019 12th International Symposium
on Computational Intelligence and Design (ISCID), Hangzhou, China: IEEE, (2019), 196-199. https://doi.org/
10.1109/ISCID.2019.10128

[48] S. Xu, N. Feng, K. Liu, Y. Liang, X. Liu, A weighted fuzzy process neural network model and its application in
mixed-process signal classification, Expert Systems with Applications, 172 (2021), 114642. https://doi.org/10.
1016/j.eswa.2021.114642

[49] Y. Xue, et al., Neural architecture search with progressive evaluation and sub-population preservation, IEEE Transactions on Evolutionary Computation, (2024), 1-1. https://doi.org/10.1109/TEVC.2024.3393304

[50] Y. Xue, X. Han, F. Neri, J. Qin, D. Pelusi, A gradient-guided evolutionary neural architecture search, IEEE
Transactions on Neural Networks and Learning Systems, 36(3) (2025), 4345-4357. https://doi.org/10.1109/
TNNLS.2024.3371432

[51] G. Yuan, B. Xue, M. Zhang, An evolutionary neural architecture search method based on performance prediction and
weight inheritance, Information Sciences, 667 (2024), 120466. https://doi.org/10.1016/j.ins.2024.120466

[52] S. Yun, et al., Graph transformer networks: Learning meta-path graphs to improve GNNs, Neural Network, 153
(2022), 104-119. https://doi.org/10.1016/j.neunet.2022.05.026

[53] S. Yun, M. Jeong, R. Kim, J. Kang, H. J. Kim, Graph transformer networks, arXiv, (2019). https://doi.org/
10.48550/ARXIV.1911.06455

[54] Y. Zhang, et al., Takagi-Sugeno-Kang fuzzy system fusion: A survey at hierarchical, wide and stacked levels,
Information Fusion, 101 (2024), 101977. https://doi.org/10.1016/j.inffus.2023.101977

[55] S. Zhang, H. Tong, J. Xu, R. Maciejewski, Graph convolutional networks: A comprehensive review, Computational
Social Networks, 6(1) (2019), 11. https://doi.org/10.1186/s40649-019-0069-y

[56] J. Zhang, H. Zhang, C. Xia, L. Sun, Graph-Bert: Only attention is needed for learning graph representations,
arXiv, (2020). https://doi.org/10.48550/ARXIV.2001.05140

Enhanced High-Dimensional Data Classification by Combining Fuzzy Learning Integration and Graph Transformers

References

References

Volume 22, Issue 2March and April 2025Pages 129-146

Volume 22, Issue 2
March and April 2025
Pages 129-146