IDENTIFYING ARITHMETIC OPERATION IN MATH WORD PROBLEM BASED ON RECURSIVE NEURAL NETWORK AND SUPPORT VECTOR MACHINE

Agung Prasetya


DOI: https://doi.org/10.29100/joeict.v8i2.7421

Abstract


Math word problems act as a test bed to design an intelligent system. An approach is needed to identify arithmetic operations including addition, substraction, multiplication and division. Template-based approaches have addressed this problem. However, the template-based approach is less efficient because it requires the process of building a template repository that have to cover a wide variety of story implied by math words. The template-based approach is potentially sub-optimal when solving story problems that have not been covered yet by templates. The proposed approach resolves this by using Recursive Neural Network and Support Vector Machine. Recursive neural network is used as an encoder that can generate semantic vectors of math word problems. Then, this vector becomes as an input for a Support Vector Machine-based classifier. Tests were conducted on a dataset collected manually from Kemdikbud’ electronic school books. The results showed that the proposed approach does not require the formation of templates, thereby reducing human involvement. In addition, the use of Recursive Neural Network reduces feature engineering making it more efficient. Experimental results by applying k-fold cross validation show that the proposed approach has an accuracy of 81% and a precision of 66%

Full Text:

PDF

Article Metrics :

References


S. R. Raiyan, M. N. Faiyaz, S. M. J. Kabir, M. Kabir, H. Mahmud, and M. K. Hasan, ‘Math Word Problem Solving by Generating Linguistic Variants of Problem Statements’, Jun. 24, 2023, arXiv: arXiv:2306.13899. doi: 10.48550/arXiv.2306.13899.

A. M. Kareem and A. Obied, ‘Testbed for intelligent agent: A survey’, J. Al-Qadisiyah Comput. Sci. Math., vol. 13, no. 2, p. Page-23, 2021.

A. Mukherjee and U. Garain, ‘A review of methods for automatic understanding of natural language mathematical problems’, Artif. Intell. Rev., vol. 29, no. 2, pp. 93–122, Apr. 2008, doi: 10.1007/s10462-009-9110-0.

L. Wang et al., ‘Template-based math word problem solvers with recursive neural networks’, in Proceedings of the AAAI Conference on Artificial Intelligence, 2019, pp. 7144–7151. Accessed: Dec. 24, 2024. [Online]. Available: https://ojs.aaai.org/index.php/AAAI/article/view/4697

J. Qin, Z. Yang, J. Chen, X. Liang, and L. Lin, ‘Template-Based Contrastive Distillation Pretraining for Math Word Problem Solv-ing’, IEEE Trans. Neural Netw. Learn. Syst., 2023, Accessed: Dec. 24, 2024. [Online]. Available: https://ieeexplore.ieee.org/abstract/document/10113691/

L. Wang, Y. Wang, D. Cai, D. Zhang, and X. Liu, ‘Translating a Math Word Problem to an Expression Tree’, Nov. 15, 2018, arXiv: arXiv:1811.05632. doi: 10.48550/arXiv.1811.05632.

Y. Wang, X. Liu, and S. Shi, ‘Deep neural solver for math word problems’, in Proceedings of the 2017 conference on empirical methods in natural language processing, 2017, pp. 845–854. Accessed: Dec. 24, 2024. [Online]. Available: https://aclanthology.org/D17-1088/

W. Zhao, M. Shang, Y. Liu, L. Wang, and J. Liu, ‘Ape210K: A Large-Scale and Template-Rich Dataset of Math Word Problems’, Oct. 09, 2020, arXiv: arXiv:2009.11506. doi: 10.48550/arXiv.2009.11506.

R. Socher, C. C. Lin, C. Manning, and A. Y. Ng, ‘Parsing natural scenes and natural language with recursive neural networks’, in Proceedings of the 28th international conference on machine learning (ICML-11), 2011, pp. 129–136. Accessed: Dec. 24, 2024. [Online]. Available: https://nlp.stanford.edu/pubs/SocherLinNgManning_ICML2011.pdf

Q. Guo, Z. Yu, Y. Wu, D. Liang, H. Qin, and J. Yan, ‘Dynamic recursive neural network’, in Proceedings of the IEEE/CVF Confer-ence on Computer Vision and Pattern Recognition, 2019, pp. 5147–5156. Accessed: Dec. 24, 2024. [Online]. Available: http://openaccess.thecvf.com/content_CVPR_2019/html/Guo_Dynamic_Recursive_Neural_Network_CVPR_2019_paper.html

O. Irsoy and C. Cardie, ‘Deep recursive neural networks for compositionality in language’, Adv. Neural Inf. Process. Syst., vol. 27, 2014, Accessed: Dec. 24, 2024. [Online]. Available: https://proceedings.neurips.cc/paper/5551-deep-recursive-neural-networks-for-compositionality-in-language.

D. A. Pisner and D. M. Schnyer, ‘Support vector machine’, in Machine learning, Elsevier, 2020, pp. 101–121. Accessed: Dec. 24, 2024. [Online]. Available: https://www.sciencedirect.com/science/article/pii/B9780128157398000067

S. Suthaharan, ‘Support Vector Machine’, in Machine Learning Models and Algorithms for Big Data Classification, vol. 36, in Inte-grated Series in Information Systems, vol. 36. , Boston, MA: Springer US, 2016, pp. 207–235. doi: 10.1007/978-1-4899-7641-3_9.

M. A. Hearst, S. T. Dumais, E. Osuna, J. Platt, and B. Scholkopf, ‘Support vector machines’, IEEE Intell. Syst. Their Appl., vol. 13, no. 4, pp. 18–28, 1998.

A. Prasetya and T. A. Cahyono, ‘PARSING STRUKTUR PARAGRAF BERBASIS NEURAL NETWORK’, JIPI J. Ilm. Penelit. Dan Pembelajaran Inform., vol. 7, no. 2, pp. 608–614, 2022.

A. Prasetya, C. Fatichah, and U. L. Yuhana, ‘Parsing struktur semantik soal cerita matematika berbahasa indonesia menggunakan recursive neural network’, Register, vol. 5, no. 2, pp. 106–115, 2019.

A. Prasetya, ‘Deteksi Operasi Hitung Pada Soal Cerita Campuran Menggunakan Recursive Neural Network’, PhD Thesis, Institut Teknologi Sepuluh Nopember, 2019. Accessed: Dec. 24, 2024. [Online]. Available: https://repository.its.ac.id/61173/

F. J. Pelletier, ‘The Principle of Semantic Compositionality’, Topoi, vol. 13, no. 1, pp. 11–24, Mar. 1994, doi: 10.1007/BF00763644.

Y. Goldberg and O. Levy, ‘word2vec Explained: deriving Mikolov et al.’s negative-sampling word-embedding method’, Feb. 15, 2014, arXiv: arXiv:1402.3722. doi: 10.48550/arXiv.1402.3722.

Q. Le and T. Mikolov, ‘Distributed representations of sentences and documents’, in International conference on machine learning, PMLR, 2014, pp. 1188–1196. Accessed: Dec. 24, 2024. [Online]. Available: http://proceedings.mlr.press/v32/le14.html?ref=https://githubhelp.com

T. Mikolov, I. Sutskever, K. Chen, G. S. Corrado, and J. Dean, ‘Distributed representations of words and phrases and their composi-tionality’, Adv. Neural Inf. Process. Syst., vol. 26, 2013, Accessed: Dec. 24, 2024. [Online]. Available: https://proceedings.neurips.cc/paper/2013/hash/9aa42b31882ec039965f3c4923ce901b-Abstract.html

B. Galitsky, ‘Learning parse structure of paragraphs and its applications in search’, Eng. Appl. Artif. Intell., vol. 32, pp. 160–184, 2014.

E. M. Sibarani, M. Nadial, E. Panggabean, and S. Meryana, ‘A study of parsing process on natural language processing in bahasa Indonesia’, in 2013 IEEE 16th International Conference on Computational Science and Engineering, IEEE, 2013, pp. 309–316. Ac-cessed: Dec. 24, 2024. [Online]. Available: https://ieeexplore.ieee.org/abstract/document/6755234/

B. Taskar, D. Klein, M. Collins, D. Koller, and C. D. Manning, ‘Max-margin parsing’, in Proceedings of the 2004 Conference on Empirical Methods in Natural Language Processing, 2004, pp. 1–8. Accessed: Dec. 24, 2024. [Online]. Available: https://aclanthology.org/W04-3201.pdf

A. Bauer, S. Nakajima, N. Görnitz, and K.-R. Müller, ‘Optimizing for measure of performance in max-margin parsing’, IEEE Trans. Neural Netw. Learn. Syst., vol. 31, no. 7, pp. 2680–2684, 2019.