Although showing remarkable zero-shot and few-shot capabilities across a wide variety of tasks, Large Language Models (LLMs) are still not mature enough for off-the-shelf use in engineering design tasks. Organizations implementing model-based systems engineering practices into their product development processes can leverage on ontologies, models, and procedures to enhance LLMs applied to engineering design tasks. We present a methodology to integrate an Object-Process Methodology model of a space system into an LLMbased spacecraft design assistant and show a performance improvement, as compared to a conventional LLM. The benchmark is evaluated through subjective expert-assessed and an objective cosine-similarity-based criteria. The results motivate additional efforts in integrating Model-Based Systems Engineering practice into LLMs as means to improve their performance and reduce shortcomings such as hallucinations and black-box, untraceable behavior.
Bringing Systems Engineering Models to Large Language Models: An Integration of OPM with an LLM for Design Assistants / Alarcia, R. M. G.; Russo, P.; Renga, A.; Golkar, A.. - 1:(2024), pp. 334-345. (Intervento presentato al convegno 12th International Conference on Model-Based Software and Systems Engineering, MODELSWARD 2024 tenutosi a ita nel 2024) [10.5220/0012621900003645].
Bringing Systems Engineering Models to Large Language Models: An Integration of OPM with an LLM for Design Assistants
Russo P.;Renga A.;
2024
Abstract
Although showing remarkable zero-shot and few-shot capabilities across a wide variety of tasks, Large Language Models (LLMs) are still not mature enough for off-the-shelf use in engineering design tasks. Organizations implementing model-based systems engineering practices into their product development processes can leverage on ontologies, models, and procedures to enhance LLMs applied to engineering design tasks. We present a methodology to integrate an Object-Process Methodology model of a space system into an LLMbased spacecraft design assistant and show a performance improvement, as compared to a conventional LLM. The benchmark is evaluated through subjective expert-assessed and an objective cosine-similarity-based criteria. The results motivate additional efforts in integrating Model-Based Systems Engineering practice into LLMs as means to improve their performance and reduce shortcomings such as hallucinations and black-box, untraceable behavior.I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.