Although showing remarkable zero-shot and few-shot capabilities across a wide variety of tasks, Large Language Models (LLMs) are still not mature enough for off-the-shelf use in engineering design tasks. Organizations implementing model-based systems engineering practices into their product development processes can leverage on ontologies, models, and procedures to enhance LLMs applied to engineering design tasks. We present a methodology to integrate an Object-Process Methodology model of a space system into an LLMbased spacecraft design assistant and show a performance improvement, as compared to a conventional LLM. The benchmark is evaluated through subjective expert-assessed and an objective cosine-similarity-based criteria. The results motivate additional efforts in integrating Model-Based Systems Engineering practice into LLMs as means to improve their performance and reduce shortcomings such as hallucinations and black-box, untraceable behavior.

Bringing Systems Engineering Models to Large Language Models: An Integration of OPM with an LLM for Design Assistants / Alarcia, R. M. G.; Russo, P.; Renga, A.; Golkar, A.. - 1:(2024), pp. 334-345. (Intervento presentato al convegno 12th International Conference on Model-Based Software and Systems Engineering, MODELSWARD 2024 tenutosi a ita nel 2024) [10.5220/0012621900003645].

Bringing Systems Engineering Models to Large Language Models: An Integration of OPM with an LLM for Design Assistants

Russo P.;Renga A.;
2024

Abstract

Although showing remarkable zero-shot and few-shot capabilities across a wide variety of tasks, Large Language Models (LLMs) are still not mature enough for off-the-shelf use in engineering design tasks. Organizations implementing model-based systems engineering practices into their product development processes can leverage on ontologies, models, and procedures to enhance LLMs applied to engineering design tasks. We present a methodology to integrate an Object-Process Methodology model of a space system into an LLMbased spacecraft design assistant and show a performance improvement, as compared to a conventional LLM. The benchmark is evaluated through subjective expert-assessed and an objective cosine-similarity-based criteria. The results motivate additional efforts in integrating Model-Based Systems Engineering practice into LLMs as means to improve their performance and reduce shortcomings such as hallucinations and black-box, untraceable behavior.
2024
Bringing Systems Engineering Models to Large Language Models: An Integration of OPM with an LLM for Design Assistants / Alarcia, R. M. G.; Russo, P.; Renga, A.; Golkar, A.. - 1:(2024), pp. 334-345. (Intervento presentato al convegno 12th International Conference on Model-Based Software and Systems Engineering, MODELSWARD 2024 tenutosi a ita nel 2024) [10.5220/0012621900003645].
File in questo prodotto:
Non ci sono file associati a questo prodotto.

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11588/968953
Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus 1
  • ???jsp.display-item.citation.isi??? ND
social impact