NeSy is alive and well: A LLM-driven symbolic approach for better code comment data generation and classification

Hanna Abi Akl (1, 2)

1 DSTI Labs - Data ScienceTech Institute
2 WIMMICS - Web-Instrumented Man-Machine Interactions, Communities and Semantics

We present a neuro-symbolic (NeSy) workflow combining a symbolic-based learning technique with a large language model (LLM) agent to generate synthetic data for code comment classification in the C programming language. We also show how generating controlled synthetic data using this workflow fixes some of the notable weaknesses of LLM-based generation and increases the performance of classical machine learning models on the code comment classification task. Our best model, a Neural Network, achieves a Macro-F1 score of 91.412% with an increase of 1.033% after data augmentation.

Mots clés: Neuro-symbolic AI; Natural Language Processing; Machine Learning; Large Language Models; Code Generation; Synthetic Data

Domaines: Génie logiciel [cs.SE]

Hanna Abi Akl. NeSy is alive and well: A LLM-driven symbolic approach for better code comment data generation and classification. Springer Nature. Generative Artificial Intelligence for Code - Impact of Large Language Models on Code Generation and Summarization, Transactions on Computer Systems and Networks, , 2024, Transactions on Computer Systems and Networks. ⟨hal-04481420⟩