Jekyll2025-11-13T21:01:08+00:00https://ai4ai-lab.github.io/ai4ai.github.io/feed.xmlAI4AI LabAI for Actionable Impact Lab at Imperial College London. Led by Dr. Sonali Parbhoo. Developing trustworthy AI systems for healthcare through causal inference, reinforcement learning, and interpretability. The Alignment Auditor: A Bayesian Framework for Verifying and Refining LLM Objectives2025-10-09T06:41:00+00:002025-10-09T06:41:00+00:00https://ai4ai-lab.github.io/ai4ai.github.io/blog/2025/alignment-auditorWe’re thrilled to share our new paper, “The Alignment Auditor: A Bayesian Framework for Verifying and Refining LLM Objectives”! We introduce a novel framework for auditing the alignment of Large Language Models.

Our Three-Stage Audit Framework

Our framework provides a comprehensive approach to LLM alignment through three key stages:

  1. Recovering a posterior distribution over reward functions to quantify ambiguity
  2. Auditing trustworthiness via uncertainty diagnostics
  3. Policy-level validation of the inferred reward, reframing reward inference as a complete audit rather than mere estimation
Our three-stage framework: (1) Posterior recovery using variational inference, (2) Uncertainty diagnostics through iterative posterior contraction, (3) Policy-level reward validation via RLHF

Key Finding 1: Systematic Ambiguity Reduction

We demonstrate that sequential updates contract the reward posterior, yielding steady AUROC and accuracy gains on Llama-1B over 5 rounds. This provides clear evidence that we’re converging on the true objective function.

The iterative refinement process shows:

  • Consistent improvement in reward prediction accuracy
  • Reduced uncertainty in posterior estimates
  • Better alignment between inferred and true objectives

Key Finding 2: Trustworthiness Diagnostics

Our uncertainty diagnostics successfully flag untrustworthy inputs. We find that reward uncertainty correlates strongly with:

  • Out-of-distribution examples
  • Ambiguous preference comparisons
  • Cases where the model’s predictions are unreliable

This correlation allows us to identify when the auditor’s judgments should be treated with caution, providing crucial safety guarantees for deployment.

Implications for AI Safety

This work represents a significant step toward making LLM alignment more:

  • Transparent: By explicitly modeling uncertainty in the reward function
  • Auditable: Through systematic validation at multiple levels
  • Reliable: By flagging cases where alignment may be uncertain

By treating alignment as an auditing problem rather than just an estimation task, we can build more trustworthy AI systems.

Read the full paper: [arXiv link coming soon]

Tags: #AI #LLM #AISafety #Alignment #BayesianMethods

]]>Improving ARDS Diagnosis Through Context-Aware Concept Bottleneck Models2025-08-15T10:00:00+00:002025-08-15T10:00:00+00:00https://ai4ai-lab.github.io/ai4ai.github.io/blog/2025/ards-diagnosis-cbmHappy to share our latest work presented at #MLHC2025!

Context-Aware Concept Bottleneck Models for ARDS

🧠 Improving ARDS Diagnosis Through Context-Aware Concept Bottleneck Models

Large, publicly available clinical datasets have emerged as a novel resource for understanding disease heterogeneity and exploring personalization of therapy. Our work leverages these datasets to improve diagnosis of Acute Respiratory Distress Syndrome (ARDS).

Key Innovation

✨ Combines multiple data modalities:

  • Structured ICU data (vital signs, lab values)
  • LLM-extracted concepts from clinical notes
  • Information from medical imaging

Results

📈 Achieves 8-10% accuracy improvement without compromising interpretability

By using concept bottleneck models, we maintain the ability to understand why the model makes its predictions, which is crucial for clinical adoption and trust.

Why This Matters

ARDS is a critical condition that requires timely and accurate diagnosis. Our approach demonstrates that:

  • Combining structured and unstructured clinical data improves predictions
  • Interpretable AI models can achieve high performance
  • LLMs can extract meaningful concepts from clinical text

📄 Read the full paper: arXiv:2508.09719

Conference: Machine Learning for Healthcare (MLHC) 2025

]]>Concept-Driven Off-Policy Evaluation2025-08-04T09:00:00+00:002025-08-04T09:00:00+00:00https://ai4ai-lab.github.io/ai4ai.github.io/blog/2025/concept-driven-opeExcited to share that our paper “Concept-Driven Off-Policy Evaluation” is being presented at #RLC2025 this week!

Rethinking Off-Policy Evaluation

We introduce a novel approach to off-policy evaluation (OPE) that uses interpretable concepts rather than raw features.

Why Concept-Driven OPE?

Traditional OPE methods often suffer from:

  • High variance in value estimates
  • Black-box evaluation processes
  • Difficulty understanding what drives policy performance

Our concept-driven approach addresses these challenges by:

Key Contributions

🎯 Low-variance estimation: By operating at the concept level, we reduce the dimensionality and variance of OPE estimates

🔍 Transparency: Concepts provide interpretable explanations for why one policy outperforms another

⚡ Actionable insights: Understanding performance at the concept level enables targeted policy improvements

Applications

This work has important implications for:

  • Healthcare: Evaluating treatment policies with interpretable feedback
  • Robotics: Understanding which perceptual concepts matter for task success
  • Autonomous systems: Auditing policies before deployment

Technical Approach

We leverage causal inference to:

  1. Identify relevant concepts for policy evaluation
  2. Construct low-variance OPE estimators using concept-level importance sampling
  3. Provide concept-level explanations for policy differences

📄 Read the full paper: arXiv:2411.19395

Conference: Reinforcement Learning Conference (RLC) 2025

Tags: #RL #CausalAI #Interpretability #OffPolicyEvaluation

]]>