Prof. Dr. Matthias Rottmann conducts research at the Osnabrück University on machine learning and, in particular, on how reliable artificial intelligence actually is in safety-critical applications such as autonomous driving. In the interview series “UOS wants to know,” he explains why it’s not enough for AI to be right most of the time, what problems AI has with a unicorn on the road, and whether he would get into a robotaxi himself.
Prof. Rottmann, reports on artificial intelligence have often had negative connotations lately: There’s been talk of AI models acting on their own and seeming to spiral out of control. How much do you trust artificial intelligence?
Prof. Rottmann: I ask myself this question again and again, and it’s difficult to give a good answer. For me, trustworthiness in prompt-based language models has at least two dimensions: First, one should verify an AI’s outputs afterward, especially when important decisions are made based on them. Second, it’s crucial to consider what information you’re willing to entrust to a language model in the first place. I would certainly provide sensitive data to an AI that I host privately with appropriate data protection measures in place. The AI doesn’t directly store what it receives; rather, it’s essentially a statistical model that only actively learns something new when placed in the appropriate training phase. So it’s important to read the fine print. That said, we’re talking about language models here; in my academic work, however, I primarily focus on image-processing AI.
And how reliable is artificial intelligence in this context?
With image-processing AI, it was long the case that the models had specific “categories” into which they sorted objects. Everything that appeared in front of the camera had to fit into one of these categories. As a result, an unknown object could easily be misclassified, for example, without the AI flagging this uncertainty. In safety-critical applications, such as road traffic, this can be dangerous.
What happens when the AI sees something it has never seen before?
That’s exactly the big problem. Take an autonomous car: It doesn’t necessarily need to know that what’s standing on the road is a unicorn. It just needs to recognize that something is there and react accordingly.
We distinguish between different types of uncertainty: On the one hand, there is aleatory or intrinsic uncertainty. It is based on mathematically determined unpredictability and cannot be further resolved using the available image data. Then there’s epistemic uncertainty, which arises from insufficient knowledge—for example, when a model is confronted with a completely new situation. This brings us back to the unicorn on the road. The distinction is important: Only epistemic uncertainty can be reduced through more data or training; aleatory uncertainty persists, no matter how good the model becomes.
To address this, we’re developing methods designed to recognize: “Here’s something the AI may never have seen before. We shouldn’t trust it, therefore.”
Would you get into a self-driving car yourself?
Well, for a family vacation that crosses state lines and involves a long drive—that would definitely be exciting. That’s because it involves what’s called a “domain shift”: the environment changes, and that’s a problem for AI models. I’d be rather cautious in that situation.
It would be different, for example, with a robotaxi in San Francisco that operates in a relatively clearly defined and familiar environment. I’d definitely get in one of those. They don’t drive insanely fast, either. But on the highway? There are already highway assistants, for example from Mercedes-Benz, that travel at around 95 km/h. It would be important to me that I could still intervene myself, that there’s a steering wheel, and that I’m not completely at the mercy of the AI. Then I’d feel confident enough to do it.
Where do you currently see the limits of AI?
Even modern models with open vocabularies have their limits. Certain combinations of objects and contexts may not have been encountered yet, so the accuracy of these models suffers. That’s why we have to constantly develop new methods to evaluate them. That’s what I’m working on with my team. Our problem is that language models, in particular, are evolving rapidly—much faster than image models. It’s hard to keep up without the necessary resources.
As a scientist, do you feel like things are moving too fast? Do you feel like you’re playing catch-up with technology?
Even Anthropic employees are saying this now: We really need a break. We need to pause for a moment and ensure that the programs become reliable and trustworthy. We need to understand where the limits lie.
Will AI development ever reach a limit at all?
Whenever I think a certain limit has been reached, it shifts again in the very next step. That’s why it’s hard for me to say what AI will be capable of in a year. Currently, for example, there are still situations where language models lose track of the big picture in large projects. In some cases, humans are still better at that. But even that can change quickly, so we should constantly question our own role: What part of the work is my responsibility, and where do I make sensible use of AI? Furthermore, AI’s insatiable appetite for data is pushing us up against certain limits: The freely available internet has now been largely drained of high-quality text data.
For me, however, what ultimately matters is that we not only develop increasingly powerful models, but also understand when we can trust them and when we cannot.
This brings us back to the initial question and the dystopian scenario in which we are slowly losing control over AI. What’s currently keeping you up at night?
I hope that the major tech companies act thoughtfully and cautiously, that they don’t open Pandora’s box and endow AI with the wrong capabilities. The recent cyberattacks, in which an AI simply breaks free, acts on its own, and steals data from another company—that’s certainly cause for concern. In my view, this makes it all the more important for science and society to critically monitor this development, rather than merely marveling at it or fearing it.
About the Author: Prof. Dr. Matthias Rottmann has held the Chair of “Practical Computer Science/Digitalization” at the Osnabrück University since August 2025. His research focuses on machine learning, particularly on uncertainty quantification, the detection of atypical objects, and data-centric AI—that is, the systematic improvement of training data for AI systems.
About the Series: In the interview series “UOS wants to know,” experts from the Osnabrück University speak with the press office about their research and share their views on current and everyday topics. From politics to education, from art to AI—UOS asks.
Further information for the media:
Prof. Dr. Matthias Rottmann, University of Osnabrück
Department of Mathematics/Computer Science/Physics
