Concept-Driven Off-Policy Evaluation

Excited to share that our paper “Concept-Driven Off-Policy Evaluation” is being presented at #RLC2025 this week!

Rethinking Off-Policy Evaluation

We introduce a novel approach to off-policy evaluation (OPE) that uses interpretable concepts rather than raw features.

Why Concept-Driven OPE?

Traditional OPE methods often suffer from:

• High variance in value estimates
• Black-box evaluation processes
• Difficulty understanding what drives policy performance

Our concept-driven approach addresses these challenges by:

Key Contributions

🎯 Low-variance estimation: By operating at the concept level, we reduce the dimensionality and variance of OPE estimates

🔍 Transparency: Concepts provide interpretable explanations for why one policy outperforms another

⚡ Actionable insights: Understanding performance at the concept level enables targeted policy improvements

Applications

This work has important implications for:

• Healthcare: Evaluating treatment policies with interpretable feedback
• Robotics: Understanding which perceptual concepts matter for task success
• Autonomous systems: Auditing policies before deployment

Technical Approach

We leverage causal inference to:

1. Identify relevant concepts for policy evaluation
2. Construct low-variance OPE estimators using concept-level importance sampling
3. Provide concept-level explanations for policy differences

---

📄 Read the full paper: arXiv:2411.19395

Conference: Reinforcement Learning Conference (RLC) 2025

Tags: #RL #CausalAI #Interpretability #OffPolicyEvaluation