Going shopping with a tanker? Language models and their purpose

Nürnberg, 22 July 2025

AI-based text generators, so-called language models, have long permeated our everyday lives and the economy. Some models are run locally on a user´s personal device, while others rely on cloud-based servers. Which models are suitable for which purposes, what can they achieve and where are their limits? Experts and participants discussed these topics at the Forschungsdialog which was jointly hosted by acatech and Fraunhofer-Gesellschaft at Josephs – Das offene Innovationslabor (Open innovation laboratory) in Nuremberg on 15 July 2025.

Which language model for what application?

Language models also differ according to their areas of application, because who would do their weekly shopping with a tanker or move house using a bicycle? With this analogy to mobility behaviour, Jan Plogsties (Head of Generative AI at Fraunhofer IIS) opened his keynote speech.

From everyday life, we all know that the purpose defines the means used to achieve something, and that they should be coordinated in the best possible way. The same goes for language models, that is, AI systems. Large models with several hundred billion parameters are usually cloud-hosted and can simultaneously write poems, generate images, and produce computer code. However, using a model that is able to do so many things at the same time does not always make sense, as this requires enormous computational power, thereby consuming a lot of energy and requiring longer response times. Smaller language models, on the other hand, are trained for specific tasks and can be deployed in embedded systems – for example, in medical devices or machine controls. Local operation allows for handling sensitive data and is simultaneously optimized for speed and efficiency. Using the example of a Fraunhofer project on AI-supported voice isolation, Jan Plogsties illustrated these specific application areas of smaller language models.

In his conversation with moderator Christina Merkel, Jan Plogsties explained the framework conditions of his research in further detail. The starting situation in the research landscape is very good, with numerous bright minds publishing a lot and holding several patents. However, the European competitive disadvantage lies in a lack of infrastructure and hardware dependencies. Current initiatives such as the AI Giga-Factories could contribute to more sovereignty.

From ELIZA to bot-assisted customer service

With Joseph Weizenbaum’s ELIZA, the first voice assistant that reacted to keywords started to operate in 1966. Today, current models are increasingly being used in companies. Thomas Geiger (Project Manager at DATEV eG) provided an insight into the development and deployment of voice assistants in customer service. Long waiting loops, recurring questions, and documentation requirements prompted DATEV as early as 2020 to develop its own language assistant. In cooperation with the service staff, they were to relieve human employees so that they had more time to deal with more complex inquiries.

Teamwork of humans and machines

Asking for customer authentication and a summary of the specific problem, the voice assistant initiates the conversation. Simple queries, such as “I forgot my password, what can I do?” or “How can I extract specific information from the system?” can be answered directly: Here, the system is very confident when it comes to the response. However, if the query is more challenging, the voice assistance makes the necessary preparations so that the service representatives can engage directly in problem-solving. Questions that cannot be answered directly or that turn out to be incorrectly answered are continuously fed into the training database, so the system is continually optimized and improved. “A crucial prerequisite for developing a voice assistant that provides valid answers is the use of a proprietary knowledge data base hosted at DATEV. “And of course, we have also implemented our very high data protection and security requirements for the voice assistant!” said Thomas Geiger.

Quality Assurance in Voice Assistants in Customer Service

In their conversation with moderator Christina Merkel, the two panellists elaborated on their approach to quality assurance. Thomas Geiger described in detail how interruptions are handled: transcripts are evaluated, feedback from customers and service employees is collected, and all this is taken into account in further development. Quality assurance is also crucial in research projects. Jan Plogsties described their approach as a combination of objective measures such as benchmark tests (testing abilities) and subjective assessments (assessing the quality of human responses).

After a lively discussion with other attendees in small groups, the audience was particularly interested in finding out how fabricated responses or hallucinations can be prevented. The experts agreed that more extensive training data would improve the responses. With limited training data and specialized applications, mechanisms are used that trigger an interruption for certain questions and limit the scope of the response.