Nitin Kamra
Research Scientist
Reality Labs at Meta


I am a Research Scientist currently working at Reality Labs, Meta since June 2021. Here, I work on developing efficient planning and reinforcement learning algorithms to enable household assistive agents in AR/VR to provide guidance to users for day-to-day tasks.

Earlier, I graduated with an MS in Intelligent Robotics and a Ph.D. in Computer Science from the University of Southern California (USC) in May 2021. I was advised by Yan Liu in the Melady Lab. My research was primarily focused on prediction and control in multi-agent settings with dense interactions amongst the various agents. I also worked on many projects involving reinforcement learning, continual learning, game theory, robotics, natural language understanding and graph-based relational learning. Thanks to my wonderful collaborators Milind Tambe (Harvard), Fei Fang (CMU), Nora Ayanian (USC) and TK Satish for their guidance over these years.

Before this, I attended Indian Institute of Technology, Delhi where I graduated with an undergraduate degree in Electrical Engineering in May 2014. My primary focus was on Control Theory and Signal Processing and I was advised by Shouribrata Chatterjee. I was also the Technical Secretary of the Electrical Engineering Society and served as the General Secretary of the Electronics Club during my final year at IIT Delhi.

Research Interests

My broad interest lies in understanding "understanding" itself. As a broad and ambitious goal, I want to figure out how the human mind works and potentially develop architectures and algorithms which would allow artificial agents to achieve at least the same level of understanding as humans one day. Consequently, I work on reinforcement learning and deep learning to design agents capable of autonomous planning and learning in multi-agent settings. My research interests broadly span deep reinforcement learning, continual learning, multi-agent learning, language understanding and robotics.


Treatment Recommendation with Preference-based Reinforcement Learning
We present an open simulation platform to model the evolution of two diseases, namely Cancer and Sepsis. Secondly, we systematically examine preference-based reinforcement learning approaches for treatment recommendation in the simulated environments, where the reward function is itself learned based on treatment goals, without requiring either expert demonstrations in advance or human involvement during policy learning.
Nan Xu, Nitin Kamra and Yan Liu
IEEE International Conference on Big Knowledge (ICBK), 2021
Gradient-based Optimization for Multi-resource Spatial Coverage Problems
We propose the coverage gradient theorem, which provides a gradient estimator for a broad class of spatial coverage objectives using a combination of Newton-Leibniz theorem and implicit boundary differentiation. We also propose a tractable framework to approximate the coverage objectives and their gradients using spatial discretization and empirically demonstrate the efficacy of our framework on multi-resource spatial coverage problems.
Nitin Kamra and Yan Liu
Conference on Uncertainty in Artificial Intelligence (UAI), Jul 2021
PolSIRD: Modeling Epidemic Spread under Intervention Policies
We present the PolSIRD, a compartmentalized mathematical model to model epidemic spread. Our model accounts for under-reporting and also captures the effects of intervention policies on the disease spread parameters. We apply our model to predict the spread of the recent global outbreak of COVID-19 in the United States and provide counterfactual simulations from our model to analyze the effect of lifting the intervention policies prematurely. Our model correctly predicts the second wave of the epidemic.
Nitin Kamra, Yizhou Zhang, Sirisha Rambhatla, Chuizheng Meng and Yan Liu
Journal of Healthcare Informatics Research (J-HIR), Jun 2021
An Examination of Preference-based Reinforcement Learning for Treatment Recommendation
We present an open simulation platform to model the progression of Cancer and Sepsis. Next, using this platform we investigate important practical problems in adopting preference-based RL approaches for treatment recommendation.
Nan Xu, Nitin Kamra and Yan Liu
NeurIPS workshop on Deep Reinforcement Learning, 2020
Gradient-based Optimization for Multi-resource Spatial Coverage
Resource allocation for coverage of geographical spaces is a challenging problem in robotics, sensor networks and security domains. We propose a tractable framework to approximate a general class of spatial coverage objectives and their gradients using a combination of Newton-Leibniz theorem, spatial discretization and implicit boundary differentiation.
Nitin Kamra and Yan Liu
NeurIPS workshop on Interpretable Inductive Biases and Physically Structured Learning, 2020
Multi-agent Trajectory Prediction with Fuzzy Query Attention
We present a general architecture to address multi-agent trajectory prediction which models the crucial inductive biases of motion, namely, inertia, relative motion, intents and interactions. Specifically, we propose a relational model to flexibly model interactions between agents in diverse environments. At the core of our model lies a novel attention mechanism which models interactions by making continuous-valued (fuzzy) decisions and learning the corresponding responses. Our architecture demonstrates significant performance gains over existing state-of-the-art predictive models in diverse domains such as human crowd trajectories, US freeway traffic, NBA sports data and physics datasets.
Nitin Kamra, Hao Zhu, Dweep Trivedi, Ming Zhang and Yan Liu
Advances in Neural Information Processing Systems (NeurIPS), 2020
Correction of Speech Recognition on Repetitive Queries
Detected speech recognition errors by considering consecutive repetitive queries by the user as an unsupervised hint. Created small query-specific language models on-the-fly by clustering the repeated queries and improved the speech recognition by jointly inferring the best possible text output for the queries in the cluster.
Pinar Donmez Ediz, Ranjitha Kulkarni, Shawn Chang and Nitin Kamra
US patent 10,650,811 -- Issued May 12, 2020
Microsoft AI and Research, Sunnyvale CA, Summer 2017
Where is the World Headed? Trajectory Prediction for Interacting Agents
We present a novel relational neural network model to address multi-agent trajectory prediction, which flexibly models interaction between agents by making fuzzy decisions and combining the corresponding responses with a fuzzy operator. Our approach shows significant performance gains over many existing state-of-the-art predictive models in diverse domains such as human crowd trajectories, US freeway traffic and physics datasets.
Nitin Kamra, Hao Zhu, Dweep Trivedi, Ming Zhang and Yan Liu
Southern California Machine Learning Symposium (SCMLS), 2020
DeepFP for Finding Nash Equilibrium in Continuous Action Spaces
We present DeepFP, an approximate extension of fictitious play in continuous action spaces. DeepFP represents players’ approximate best responses via highly expressive implicit density approximators and trains them with a model-based learning regime. We demonstrate stable convergence to Nash equilibrium on several classic games and in a forest security domain. DeepFP learns strategies robust to adversarial exploitation and scales well with players’ resources.
Nitin Kamra, Umang Gupta, Kai Wang, Fei Fang, Yan Liu and Milind Tambe
Conference on Decision and Game Theory for Security (GameSec), 2019
Deep Fictitious Play for Games with Continuous Action Spaces
We develop an approximate extension of fictitious play to two-player games with high-dimensional continuous action spaces.
Nitin Kamra, Umang Gupta, Kai Wang, Fei Fang, Yan Liu and Milind Tambe
18th International Conference on Autonomous Agents and Multiagent Systems (AAMAS), 2019
Policy Learning for Continuous Space Security Games using Neural Networks
We present OptGradFP, a novel and general algorithm that searches for optimal defender strategies, and can also be used to learn policies over multiple game states simultaneously in stackelberg security games with continuous action spaces. We demonstrate the potential to predict good defender strategies via experiments and analysis on discrete and continuous game settings.
Nitin Kamra, Umang Gupta, Fei Fang, Yan Liu and Milind Tambe
Thirty-Second AAAI Conference on Artificial Intelligence, February 2018
Handling Continuous Space Security Games with Neural Networks
We propose OptGradFP, a novel and general algorithm that searches for optimal defender strategies in stackelberg security games with continuous action spaces.
Nitin Kamra, Fei Fang, Debarun Kar, Yan Liu and Milind Tambe
IJCAI International Workshop on A.I. in Security (IWAISe), August 2017
DynGEM: Deep Embedding Method for Dynamic Graphs
We present DynGEM, an efficient algorithm based on a deep autoencoder which produces embeddings for graphs evolving over time, to perform graph visualization, link prediction and node classification etc. DynGEM: (a) produces stable embeddings over time, (b) can handle growing dynamic graphs, and (c) has better running time than using static embedding methods on each snapshot of a dynamic graph.
Nitin Kamra*, Palash Goyal*, Xinran He and Yan Liu
IJCAI International Workshop on Representation Learning for Graphs (ReLiG), August 2017
Combinatorial Problems in Multi-Robot Battery Exchange Systems
We present solutions to combinatorial problems in multi-robot systems characterized by task robots, which provide services at requested locations and delivery robots, which deliver batteries to task robots. Multiple resource scheduling and path planning problems, at least as hard as the m-TSP problem, are solved using heuristic algorithms inspired by techniques from approximation algorithms.
Nitin Kamra, T. K. Satish Kumar and Nora Ayanian
IEEE Transactions on Automation Science and Engineering (T-ASE), 2018
A mixed integer programming model for timed deliveries in multirobot systems
We address the scheduling problem arising when several deployed task robots perform long-duration missions and can request resources (e.g. batteries), which are to be delivered by delivery robots. We incorporate multiple incoming time-bound delivery requests, while permitting relaxed deliveries when available resources are scant and allowing dynamic re-routing of delivery robots. The problem is posed as a variant of the Vehicle Routing Problem with Time Windows, and solved as a Mixed Integer QP with a branch and bound based solver.
Nitin Kamra and Nora Ayanian
IEEE International Conference on Automation Science and Engineering (CASE), August 2015


Machine Learning in Interacting Multi-agent Systems
In this thesis, I study and propose methods to advance the state-of-the-art for several multi-agent learning problems.
Nitin Kamra
PhD Thesis, University of Southern California. July 2021
Towards Zero-shot Dialog Act Classification
We propose and train novel dialog act classifiers in single-domain, multi-domain and unseen-multi-domain, zero-shot-generalization settings.
Nitin Kamra, Daniel Elkind and Angeliki Metallinou
Alexa Natural Understanding, Amazon. Summer 2020
Deep Generative Dual Memory Network for Continual Learning
We derive inspiration from human complementary learning systems (hippocampus and neocortex) to develop a dual memory architecture capable of learning continuously from sequentially incoming tasks, while averting catastrophic forgetting. We perform memory consolidation via generative replay of past experiences and demonstrate improved retention on challenging tasks.
Nitin Kamra, Umang Gupta and Yan Liu
ArXiv, May 2018
Parallel Gradient Descent for Multilayer Feedforward Neural Networks
Implemented and analyzed gradient descent with parallelization over: (a) training examples in gradient computation, and (b) matrix multiplication for gradient computation over a single training example. A serial variant, a parallel multithreaded version (with C++ Pthreads), a BLAS parallelized version and a CUDA implementation were compared for the speedup obtained on MNIST digit classification.
Nitin Kamra, Palash Goyal, Sungyong Seo and Vasilis Zois
Project, Spring 2016
RF-Based Relative Localization for Robot Swarms
Used the radio signal strength indicator (RSSI) data from a CrazyFlie robot swarm to fit a novel power vs. log-distance model, which in turn provided the foundation for a centralized anchor-free localization algorithm. We presented a computationally simple gradient descent based localization approach which is easily extendable to distributed swarms and scales efficiently to large number of robots.
Wolfgang Hoenig and Nitin Kamra
Project, Spring 2015
Predicting Rainfall with Polarimetric Radar Data
Explored a set of polarimetric radar data and rain gauge readings collected in the Midwestern US over several months, to improve upon the existing Marshall-Palmer baseline for rainfall prediction using various supervised learning algorithms. This competition was sponsored by the Artificial Intelligence Committee of the American Meteorological Society and hosted by Kaggle here.
Nitin Kamra and James Preiss
Kaggle Competition, Fall 2015
Output Power Maximization in Energy Harvesting Applications
Project focused on increasing the efficiency of an Energy Harvesting Integrated Circuit (EHIC) being developed by the ICE group at IIT Delhi.
  1. Explored the usage of a Discrete Time Parametric Amplifier (DTPA) to boost the charging speed of a DC-DC converter.
  2. Implemented maximum power point tracking for the Energy Harvesting IC to track the frequency of charging cycles and harness maximum output power from multiple energy sources.
Nitin Kamra and Shouribrata Chatterjee
Undergraduate Thesis, IIT Delhi. 2014
ROSHNI: Indoor Navigation System for Visually Impaired
This project involves the design of an indoor navigation system for visually impaired people. This is a long-term ongoing project at Indian Institute of Technology, Delhi supervised by Prof. M. Balakrishnan. I joined the project in 2012 to improve the circuit design and sensing of infrared modules.
I implemented a localization and step-detection algorithm with infra-red checkpointing for precise localization, along with a complete circuit design of wall-mounted and waist-worn modules for the project.
Nitin Kamra, Devesh Singh, Dhruv Jain and M. Balakrishnan
Project, Spring 2012
Elementary Iterative Methods and the Conjugate Gradient Algorithm
Presented several iterative methods and the Conjugate Gradient algorithm for non-linear optimization.
Nitin Kamra
High Performance Computing, Indo-German Winter Academy, December 2012


  • Teaching Assistant for CSCI-567: Machine Learning, USC (Spring 2020, Fall 2016)

  • Tutorial for Reinforcement Learning, CS-699: Advanced topics in Deep Learning, USC (Spring 2019)

  • Hosting the Artificial General Intelligence Reading Group at USC (Fall 2018)

  • Teaching Assistant for EEL301: Control Engg - I, IIT Delhi (Spring 2014)

  • Teaching Assistant for EEL201: Digital Electronics, IIT Delhi (Fall 2013)


  • Deep Learning Best Theory Project Award, CSCI-599: Deep Learning, University of Southern California (2017)

  • Viterbi Graduate Ph.D. Fellowship, University of Southern California (2014-18)

  • Best Mentor Award, Awarded by Mentorship Review Committee, Indian Institute of Technology, Delhi (2013)

  • SOF 3rd International Mathematics Olympiad, International Rank 16, School Topper and Gold Medalist (2010)

  • SOF 12th National Science Olympiad, National Rank 45, School Topper and Gold Medalist (2010)

  • FIITJEE Talent Reward Exam, Zonal Topper and Gold Medalist (2009)