Umer Siddique

PhD Candidate, Department of Electrical and Computer Engineering, The University of Texas at San Antonio

I am a final-year PhD candidate in the Unmanned Systems Lab at UT San Antonio, advised by Prof. Yongcan Cao. My research is on AI alignment through reinforcement learning: building decision-making systems that are fair, safe, and aligned with the people they affect. I work on multi-objective and preference-based RL, multi-agent systems, and inference-time alignment, often using social welfare functions such as the Generalized Gini Function to balance efficiency and equity. My work is supported by the Office of Naval Research and the Army Research Laboratory.

Before UTSA, I received my MS from the UM–SJTU Joint Institute in Shanghai, where I worked with Prof. Paul Weng on fairness in reinforcement learning.

News

Oct 2026	Paper on re-grounding frozen VLAs under actuation shift accepted at the CoRL 2026 Workshop on The Science of Physical AI Safety.
Sep 2026	Interviewed by Texas Standard for The New Wild West: Texans on the AI Frontier.
Aug 2026	Paper on rubric aggregation for multi-criterion LLM alignment accepted at the COLM 2026 Workshop on Agent Behavior.
2026	Serving as a reviewer for ICML, NeurIPS, COLM, CoRL, AAAI, AAMAS, and TMLR in 2026.
May 2026	Received the Gold Reviewer Award at ICML 2026.
May 2026	Inference-Time Policy Alignment for Fair RL accepted at RLC 2026.
Apr 2026	Multi-view media profiling suite accepted at ACL 2026.
Mar 2026	Received the Jamshidi Fellowship from UT San Antonio.

All news →

Selected Publications

RLC
Inference-Time Policy Alignment for Fair Reinforcement Learning

Umer Siddique, Peilang Li, Conor Wallace and Yongcan Cao

Reinforcement Learning Conference (RLC), 2026

arXiv OpenReview

Deep reinforcement learning (RL) agents achieve strong performance by optimizing scalar reward functions. However, once deployed, the policies of these RL agents are often rigid and costly to adapt to new performance criteria. For instance, an agent trained to maximize expected cumulative reward may not accommodate previously unknown stakeholder preferences. Existing approaches to achieve fairness, a type of preference, in RL typically assume that such preferences are known a priori and require complete retraining of the policy under a fairness-oriented metric. Inspired by inference-time alignment in large language models, we investigate the problem of steering a pretrained RL policy toward welfare-based fairness objectives at inference time without updating the base policy's parameters. We formalize inference-time fairness alignment as a policy shaping problem and propose a multiplicative policy shaping framework that adjusts action probabilities using action-dependent welfare scores, thus requiring no modification to the base policy. Our framework is general and compatible with any deep RL agent. Through extensive experiments across multiple domains, we demonstrate that inference-time policy shaping substantially improves welfare-based fairness objectives while preserving core task performance.
```
@inproceedings{siddique2026inference,
  title     = {Inference-Time Policy Alignment for Fair Reinforcement Learning},
  author    = {Umer Siddique and Peilang Li and Conor Wallace and Yongcan Cao},
  booktitle = {Reinforcement Learning Conference (RLC)},
  year      = {2026}
}
```
ACC
Adaptive Event-Triggered Policy Gradient for Multi-Agent Reinforcement Learning

Umer Siddique, Abhinav Sinha and Yongcan Cao

American Control Conference (ACC), 2026

arXiv

Conventional multi-agent reinforcement learning (MARL) methods rely on time-triggered execution, where agents sample and communicate actions at fixed intervals. This approach is often computationally expensive and communication-intensive. To address this limitation, we propose ET-MAPG (Event-Triggered Multi-Agent Policy Gradient reinforcement learning), a framework that jointly learns an agent's control policy and its event-triggering policy. Unlike prior work that decouples these mechanisms, ET-MAPG integrates them into a unified learning process, enabling agents to learn not only what action to take but also when to execute it. For scenarios with inter-agent communication, we introduce AET-MAPG, an attention-based variant that leverages a self-attention mechanism to learn selective communication patterns. AET-MAPG empowers agents to determine not only when to trigger an action but also with whom to communicate and what information to exchange, thereby optimizing coordination. Both methods can be integrated with any policy gradient MARL algorithm. Extensive experiments across diverse MARL benchmarks demonstrate that our approaches achieve performance comparable to state-of-the-art, time-triggered baselines while significantly reducing both computational load and communication overhead.
```
@inproceedings{siddique2026eventmarl,
  title     = {Adaptive Event-Triggered Policy Gradient for Multi-Agent Reinforcement Learning},
  author    = {Umer Siddique and Abhinav Sinha and Yongcan Cao},
  booktitle = {American Control Conference (ACC)},
  year      = {2026}
}
```

MLJ

Learning Fair Policies in Multi-Objective Preference-based Reinforcement Learning

Umer Siddique, Abhinav Sinha and Yongcan Cao

Machine Learning (Springer), 2025

Paper

@article{siddique2025fpbrl,
  title     = {Learning Fair Policies in Multi-Objective Preference-based Reinforcement Learning},
  author    = {Umer Siddique and Abhinav Sinha and Yongcan Cao},
  journal   = {Machine Learning (Springer)},
  year      = {2025}
}

RLC
Learning Fair Pareto-Optimal Policies in Multi-Objective Reinforcement Learning

Umer Siddique, Peilang Li and Yongcan Cao

Reinforcement Learning Conference (RLC), 2025

PDF arXiv

Fairness is an important aspect of decision-making in multi-objective reinforcement learning (MORL), where policies must ensure both optimality and equity across multiple, potentially conflicting objectives. While single-policy MORL methods can learn fair policies for fixed user preferences using welfare functions such as the generalized Gini welfare function (GGF), they fail to provide the diverse set of policies necessary for dynamic or unknown user preferences. To address this limitation, we formalize the fair optimization problem in multi-policy MORL, where the goal is to learn a set of Pareto-optimal policies that ensure fairness across all possible user preferences. Our key technical contributions are threefold: (1) We show that for concave, piecewise-linear welfare functions (e.g., GGF), fair policies remain in the convex coverage set (CCS), which is an approximated Pareto front for linear scalarization. (2) We demonstrate that non-stationary policies, augmented with accrued reward histories, and stochastic policies improve fairness by dynamically adapting to historical inequities. (3) We propose three novel algorithms, which include integrating GGF with multi-policy multi-objective Q-Learning (MOQL), state-augmented multi-policy MOQL for learning non-statoinary policies, and its novel extension for learning stochastic policies. We evaluate our algorithms across various domains and compare our methods against the state-of-the-art MORL baselines. The empirical results show that our methods learn a set of fair policies that accommodate different user preferences.
```
@inproceedings{siddique2025pareto,
  title     = {Learning Fair Pareto-Optimal Policies in Multi-Objective Reinforcement Learning},
  author    = {Umer Siddique and Peilang Li and Yongcan Cao},
  booktitle = {Reinforcement Learning Conference (RLC)},
  year      = {2025}
}
```
ICML
Learning Fair Policies in Decentralized Cooperative Multi-Agent Reinforcement Learning

Matthieu Zimmer, Claire Glanois, Umer Siddique and Paul Weng

International Conference on Machine Learning (ICML), 2021

PMLR arXiv

We consider the problem of learning fair policies in (deep) cooperative multi-agent reinforcement learning (MARL). We formalize it in a principled way as the problem of optimizing a welfare function that explicitly encodes two important aspects of fairness: efficiency and equity. As a solution method, we propose a novel neural network architecture, which is composed of two sub-networks specifically designed for taking into account the two aspects of fairness. In experiments, we demonstrate the importance of the two sub-networks for fair optimization. Our overall approach is general as it can accommodate any (sub)differentiable welfare function. Therefore, it is compatible with various notions of fairness that have been proposed in the literature (e.g., lexicographic maximin, generalized Gini social welfare function, proportional fairness). Our solution method is generic and can be implemented in various MARL settings: centralized training and decentralized execution, or fully decentralized. Finally, we experimentally validate our approach in various domains and show that it can perform much better than previous methods.
```
@inproceedings{zimmer2021soto,
  title     = {Learning Fair Policies in Decentralized Cooperative Multi-Agent Reinforcement Learning},
  author    = {Matthieu Zimmer and Claire Glanois and Umer Siddique and Paul Weng},
  booktitle = {International Conference on Machine Learning (ICML)},
  year      = {2021}
}
```
ICML
Learning Fair Policies in Multi-Objective (Deep) Reinforcement Learning with Average and Discounted Rewards

Umer Siddique, Paul Weng and Matthieu Zimmer

International Conference on Machine Learning (ICML), 2020

PMLR arXiv Talk

As the operations of autonomous systems generally affect simultaneously several users, it is crucial that their designs account for fairness considerations. In contrast to standard (deep) reinforcement learning (RL), we investigate the problem of learning a policy that treats its users equitably. In this paper, we formulate this novel RL problem, in which an objective function, which encodes a notion of fairness that we formally define, is optimized. For this problem, we provide a theoretical discussion where we examine the case of discounted rewards and that of average rewards. During this analysis, we notably derive a new result in the standard RL setting, which is of independent interest: it states a novel bound on the approximation error with respect to the optimal average reward of that of a policy optimal for the discounted reward. Since learning with discounted rewards is generally easier, this discussion further justifies finding a fair policy for the average reward by learning a fair policy for the discounted reward. Thus, we describe how several classic deep RL algorithms can be adapted to our fair optimization problem, and we validate our approach with extensive experiments in three different domains.
```
@inproceedings{siddique2020fair,
  title     = {Learning Fair Policies in Multi-Objective (Deep) Reinforcement Learning with Average and Discounted Rewards},
  author    = {Umer Siddique and Paul Weng and Matthieu Zimmer},
  booktitle = {International Conference on Machine Learning (ICML)},
  year      = {2020}
}
```

All publications →