Multidisciplinary Institute in Artificial intelligence

DESCRIPTION

AI methods produce results that are expected to directly impact decisions or predictions about people in multiple sensitive areas such as justice, healthcare, finance, autonomous driving, and defense. Two important research directions, which can be combined, emerge: on one hand, ensuring that AI is reliable (whether by a priori methods or runtime monitoring), and, on the other hand, making its workings explainable. With respect to reliability, the goal is to develop solutions incorporating rigorous approaches, based on formal, as well as data engineering and model engineering methods, so as to obtain guarantees regarding reliability (safety and transparency), efficiency (volume of data, data quality, energy consumption of model inference...) and robustness (with respect to realistic perturbations).

Also, explainability, transparency, and ability to generalize AI models are significant issues for the trust‐based acceptability of AI systems, be they based on machine learning algorithms or algorithms for diagnosis, recommendation, voting, assignment, or collective decision. The goal here is to render understandable the elements taken into account by the AI system when it produces results. In this context, this chair goal is to develop reliable and/or explainable AI methods related (but not limited) to the following emerging research areas:

• Methods for runtime monitoring
• Methods for scalable automated verification,
• Learning of interpretable characteristics,
• Approximation (whether local or global) of black‐box models by interpretable models,
• Safe/Inverse Reinforcement Learning
• Abstraction and generalization,
• Computation of the importance of attributes for decision‐making or prediction
• Causal models,
• Argument‐based explanations,
• Assessment and validation of explanations,
• Convergence of learning‐based and model‐based approaches,

The chair is interested in all types of relevant applications, with an initial focus on AI for Cyber-Physical Systems (e.g. in robotics, autonomous driving, medical devices).

ACTIVITIES

During the first months of the chair and in continuation of previous collaborations, we have been working on several topics involving cyber-physical systems with machine learning components (CPS-ML) including:

• Runtime Monitoring with and learning of Signal Temporal Logics (STL): STL is a formal language to express rigorously properties involving temporal evolution. It is thus suitable for monitoring the behaviors of CPS-ML. STL properties that are shown to be satisfied by a system can be manipulated in different ways for systematic testing, verification and explanation.
• Uniform inputs generation with temporal constraints: when characterizing the behavior of an AI, a crucial aspect is its operation domain, i.e., the actual domain of inputs in which we would require it to behave properly. Defining such a domain is not trivial, especially when time is involved, in which case STL or Timed Automata can be employed. Being able to sample the OD uniformly makes it possible to test AI systems in a more efficient and rigorous manner.
• Imitation learning for control systems: for complex control systems tasks, it may be the case that a controller exists but only for limited cases, or at a high computation cost. Imitation learning aims at designing a controller, imitating its behavior at a lower cost and/or benefitting from better generalization. In this work, we also use STL to define and evaluate more rigorously control objectives, and uniform sampling to guide the generation of data used incrementally in the learning process.
• Safe Reinforcement Learning with complex goals: Reinforcement Learning is a type of control synthesis where control law is learned from experience. Controllers obtained by RL can achieve complex tasks, but their convergence are difficult to predict and explain. Using monitoring and our tool sets of formal methods, we aim at improving our understanding of RL agents and verify that they satisfy certain requirements or enforce them to do so.

In 2025, a post-doctoral researcher position and a PhD opportunity will be available starting in September or earlier, to work on these topics or others related to the thematic focus of the chair.