Interdisciplinary Institute in Artificial Intelligence

DESCRIPTION

AI methods produce results that are expected to directly impact decisions or predictions about people in multiple sensitive areas such as justice, healthcare, finance, autonomous driving, and defense. Two important research directions—which can be combined—emerge: on the one hand, ensuring that AI is reliable (whether through a priori methods or runtime monitoring), and, on the other hand, making its workings explainable. With regard to reliability, the goal is to develop solutions that incorporate rigorous approaches—based on formal methods as well as data engineering and model engineering techniques—in order to provide guarantees regarding reliability (safety and transparency), efficiency (data volume, data quality, energy consumption during model inference, etc.), and robustness (with respect to realistic perturbations).

In addition, the explainability, transparency, and generalizability of AI models are significant factors in the trust-based acceptability of AI systems, whether they are based on machine learning algorithms or algorithms for diagnosis, recommendation, voting, assignment, or collective decision-making. The goal here is to make the elements considered by the AI system when it produces results understandable. In this context, the goal of this chair is to develop reliable and/or explainable AI methods related (but not limited) to the following emerging research areas:

• Methods for Runtime Monitoring
• Methods for scalable automated verification,
• Learning interpretable features,
• Approximation (whether local or global) of black-box models by interpretable models,
• Safe/Inverse Reinforcement Learning
• Abstraction and generalization,
• Calculation of the importance of attributes for decision-making or prediction
• Causal models,
• Argument-based explanations,
• Assessment and validation of explanations,
• Convergence of learning-based and model-based approaches,

The chair is interested in all types of relevant applications, with an initial focus on AI for cyber-physical systems (e.g., in robotics, autonomous driving, and medical devices).

ACTIVITIES

During the first few months of the chair's tenure and as a continuation of previous collaborations, we have been working on several topics involving cyber-physical systems with machine learning components (CPS-ML), including:

• Runtime Monitoring and Learning of Signal Temporal Logics (STL): STL is a formal language for rigorously expressing properties involving temporal evolution. It is therefore well-suited for monitoring the behavior of CPS-ML. STL properties that are shown to be satisfied by a system can be manipulated in various ways for systematic testing, verification, and explanation.
• Uniform input generation with temporal constraints: When characterizing the behavior of an AI, a crucial aspect is its operational domain—that is, the actual domain of inputs within which we would expect it to behave properly. Defining such a domain is not trivial, especially when time is involved; in such cases, STL or timed automata can be used. The ability to sample the operational domain uniformly makes it possible to test AI systems in a more efficient and rigorous manner.
• Imitation learning for control systems: For complex control system tasks, it may be the case that a controller exists but only for limited cases, or at a high computational cost. Imitation learning aims to design a controller by imitating its behavior at a lower cost and/or achieving better generalization. In this work, we also use STL to define and evaluate control objectives more rigorously, and uniform sampling to guide the generation of data used incrementally in the learning process.
• Safe Reinforcement Learning with Complex Goals: Reinforcement learning is a type of control synthesis in which control laws are learned from experience. Controllers obtained through RL can perform complex tasks, but their convergence is difficult to predict and explain. By using monitoring and our set of formal methods, we aim to improve our understanding of RL agents and verify that they satisfy certain requirements or ensure that they do so.

In 2025, a postdoctoral research position and a Ph.D. opportunity will be available starting in September or earlier, to work on these topics or others related to the chair's thematic focus.