Nitin Kamra
Ph.D. Student in Machine Learning
University of Southern California

Bio


I am a Ph.D. Student in the Computer Science Department at University of Southern California (USC). I work with Prof. Yan Liu in the Melady Lab . My work spans both: (a) learning to predict, and (b) learnable control in multi-agent settings featuring interactions amongst the various agents. Learning to predict effectively in multi-agent settings requires incorporating inductive biases from physics and graph-based models, while learning to control often utilizes tools from game theory and Nash equilibria in order to design deep architectures which can be trained to achieve optimal prediction performance or control policies in multi-agent settings. Working in these diverse domains has provided me with a strong fundamental background in deep learning, reinforcement learning, game theory and graph-based learning. I also work in collaboration with Prof. Milind Tambe (Teamcore group, Harvard University) and Prof. Fei Fang (CMU) on finding optimal strategies for Stackelberg Security Games with Deep Reinforcement Learning. Previously, I have also worked with Prof. Nora Ayanian in the ACT Lab at USC. Here, my work was on multirobot coordination and on developing planning algorithms for efficient resource delivery.

Before this, I attended Indian Institute of Technology, Delhi where I got my undergraduate degree in Electrical Engineering. My primary focus was on Control Theory and Signal Processing and I was advised by Prof. Shouribrata Chatterjee . I was also the Technical Secretary of Electrical Engineering Society at IIT Delhi (Aug 2012 - Aug 2013) and served as the General Secretary of the Electronics Club (August 2013 - May 2014) during my final year.

I am originally from New Delhi, India. I started my Ph.D. in Spring 2015 and currently live in Los Angeles, California.

Research Interests


My broad interest lies in understanding "understanding" itself. As a broad and ambitious goal, I want to figure out how the human mind works and potentially develop architectures and algorithms which would allow artificial agents to achieve at least the same level of understanding as humans one day. Consequently, I work on reinforcement learning and deep learning to design agents capable of autonomous planning and learning in multi-agent settings. My research interests broadly span deep reinforcement learning, continual learning, multi-agent learning, language understanding and robotics.

Publications


An Examination of Preference-based Reinforcement Learning for Treatment Recommendation
We present an open simulation platform to model the progression of Cancer and Sepsis. Next, using this platform we investigate important practical problems in adopting preference-based RL approaches for treatment recommendation.
Nan Xu, Nitin Kamra and Yan Liu
NeurIPS workshop on Deep Reinforcement Learning, 2020
Gradient-based Optimization for Multi-resource Spatial Coverage
Resource allocation for coverage of geographical spaces is a challenging problem in robotics, sensor networks and security domains. We propose a tractable framework to approximate a general class of spatial coverage objectives and their gradients using a combination of Newton-Leibniz theorem, spatial discretization and implicit boundary differentiation.
Nitin Kamra and Yan Liu
NeurIPS workshop on Interpretable Inductive Biases and Physically Structured Learning, 2020
Multi-agent Trajectory Prediction with Fuzzy Query Attention
We present a general architecture to address multi-agent trajectory prediction which models the crucial inductive biases of motion, namely, inertia, relative motion, intents and interactions. Specifically, we propose a relational model to flexibly model interactions between agents in diverse environments. At the core of our model lies a novel attention mechanism which models interactions by making continuous-valued (fuzzy) decisions and learning the corresponding responses. Our architecture demonstrates significant performance gains over existing state-of-the-art predictive models in diverse domains such as human crowd trajectories, US freeway traffic, NBA sports data and physics datasets.
Nitin Kamra, Hao Zhu, Dweep Trivedi, Ming Zhang and Yan Liu
Advances in Neural Information Processing Systems (NeurIPS), 2020
Correction of Speech Recognition on Repetitive Queries
Detected speech recognition errors by considering consecutive repetitive queries by the user as an unsupervised hint. Created small query-specific language models on-the-fly by clustering the repeated queries and improved the speech recognition by jointly inferring the best possible text output for the queries in the cluster.
Pinar Donmez Ediz, Ranjitha Kulkarni, Shawn Chang and Nitin Kamra
US patent 10,650,811 -- Issued May 12, 2020
Microsoft AI and Research, Sunnyvale CA, Summer 2017
Where is the World Headed? Trajectory Prediction for Interacting Agents
We present a novel relational neural network model to address multi-agent trajectory prediction, which flexibly models interaction between agents by making fuzzy decisions and combining the corresponding responses with a fuzzy operator. Our approach shows significant performance gains over many existing state-of-the-art predictive models in diverse domains such as human crowd trajectories, US freeway traffic and physics datasets.
Nitin Kamra, Hao Zhu, Dweep Trivedi, Ming Zhang and Yan Liu
Southern California Machine Learning Symposium (SCMLS), 2020
DeepFP for Finding Nash Equilibrium in Continuous Action Spaces
We present DeepFP, an approximate extension of fictitious play in continuous action spaces. DeepFP represents players’ approximate best responses via highly expressive implicit density approximators and trains them with a model-based learning regime. We demonstrate stable convergence to Nash equilibrium on several classic games and in a forest security domain. DeepFP learns strategies robust to adversarial exploitation and scales well with players’ resources.
Nitin Kamra, Umang Gupta, Kai Wang, Fei Fang, Yan Liu and Milind Tambe
Conference on Decision and Game Theory for Security (GameSec), 2019
Deep Fictitious Play for Games with Continuous Action Spaces
We develop an approximate extension of fictitious play to two-player games with high-dimensional continuous action spaces.
Nitin Kamra, Umang Gupta, Kai Wang, Fei Fang, Yan Liu and Milind Tambe
18th International Conference on Autonomous Agents and Multiagent Systems (AAMAS), 2019
Policy Learning for Continuous Space Security Games using Neural Networks
We present OptGradFP, a novel and general algorithm that searches for optimal defender strategies, and can also be used to learn policies over multiple game states simultaneously in stackelberg security games with continuous action spaces. We demonstrate the potential to predict good defender strategies via experiments and analysis on discrete and continuous game settings.
Nitin Kamra, Umang Gupta, Fei Fang, Yan Liu and Milind Tambe
Thirty-Second AAAI Conference on Artificial Intelligence, February 2018
Handling Continuous Space Security Games with Neural Networks
We propose OptGradFP, a novel and general algorithm that searches for optimal defender strategies in stackelberg security games with continuous action spaces.
Nitin Kamra, Fei Fang, Debarun Kar, Yan Liu and Milind Tambe
IJCAI International Workshop on A.I. in Security (IWAISe), August 2017
DynGEM: Deep Embedding Method for Dynamic Graphs
We present DynGEM, an efficient algorithm based on a deep autoencoder which produces embeddings for graphs evolving over time, to perform graph visualization, link prediction and node classification etc. DynGEM: (a) produces stable embeddings over time, (b) can handle growing dynamic graphs, and (c) has better running time than using static embedding methods on each snapshot of a dynamic graph.
Nitin Kamra*, Palash Goyal*, Xinran He and Yan Liu
IJCAI International Workshop on Representation Learning for Graphs (ReLiG), August 2017
Combinatorial Problems in Multi-Robot Battery Exchange Systems
We present solutions to combinatorial problems in multi-robot systems characterized by task robots, which provide services at requested locations and delivery robots, which deliver batteries to task robots. Multiple resource scheduling and path planning problems, at least as hard as the m-TSP problem, are solved using heuristic algorithms inspired by techniques from approximation algorithms.
Nitin Kamra, T. K. Satish Kumar and Nora Ayanian
IEEE Transactions on Automation Science and Engineering (T-ASE), 2018
A mixed integer programming model for timed deliveries in multirobot systems
We address the scheduling problem arising when several deployed task robots perform long-duration missions and can request resources (e.g. batteries), which are to be delivered by delivery robots. We incorporate multiple incoming time-bound delivery requests, while permitting relaxed deliveries when available resources are scant and allowing dynamic re-routing of delivery robots. The problem is posed as a variant of the Vehicle Routing Problem with Time Windows, and solved as a Mixed Integer QP with a branch and bound based solver.
Nitin Kamra and Nora Ayanian
IEEE International Conference on Automation Science and Engineering (CASE), August 2015

Projects


Towards Zero-shot Dialog Act Classification
We propose and train novel dialog act classifiers in single-domain, multi-domain and unseen-multi-domain, zero-shot-generalization settings.
Nitin Kamra, Daniel Elkind and Angeliki Metallinou
Alexa Natural Understanding, Amazon. Summer 2020
PolSIRD: Modeling Epidemic Spread under Intervention Policies and an Application to the Spread of COVID-19
We present the PolSIRD, a compartmentalized mathematical model to model epidemic spread. Our model accounts for under-reporting and also captures the effects of intervention policies on the disease spread parameters. We apply our model to predict the spread of the recent global outbreak of COVID-19 and provide actionable guidance on the lifting of intervention policies via counterfactual simulations from our model.
Nitin Kamra, Yizhou Zhang, Sirisha Rambhatla, Chuizheng Meng and Yan Liu
ArXiv, September 2020
Deep Generative Dual Memory Network for Continual Learning
We derive inspiration from human complementary learning systems (hippocampus and neocortex) to develop a dual memory architecture capable of learning continuously from sequentially incoming tasks, while averting catastrophic forgetting. We perform memory consolidation via generative replay of past experiences and demonstrate improved retention on challenging tasks.
Nitin Kamra, Umang Gupta and Yan Liu
ArXiv, May 2018
Parallel Gradient Descent for Multilayer Feedforward Neural Networks
Implemented and analyzed gradient descent with parallelization over: (a) training examples in gradient computation, and (b) matrix multiplication for gradient computation over a single training example. A serial variant, a parallel multithreaded version (with C++ Pthreads), a BLAS parallelized version and a CUDA implementation were compared for the speedup obtained on MNIST digit classification.
Nitin Kamra, Palash Goyal, Sungyong Seo and Vasilis Zois
Project, Spring 2016
RF-Based Relative Localization for Robot Swarms
Used the radio signal strength indicator (RSSI) data from a CrazyFlie robot swarm to fit a novel power vs. log-distance model, which in turn provided the foundation for a centralized anchor-free localization algorithm. We presented a computationally simple gradient descent based localization approach which is easily extendable to distributed swarms and scales efficiently to large number of robots.
Wolfgang Hoenig and Nitin Kamra
Project, Spring 2015
Predicting Rainfall with Polarimetric Radar Data
Explored a set of polarimetric radar data and rain gauge readings collected in the Midwestern US over several months, to improve upon the existing Marshall-Palmer baseline for rainfall prediction using various supervised learning algorithms. This competition was sponsored by the Artificial Intelligence Committee of the American Meteorological Society and hosted by Kaggle here.
Nitin Kamra and James Preiss
Kaggle Competition, Fall 2015
Output Power Maximization in Energy Harvesting Applications
Project focused on increasing the efficiency of an Energy Harvesting Integrated Circuit (EHIC) being developed by the ICE group at IIT Delhi.
  1. Explored the usage of a Discrete Time Parametric Amplifier (DTPA) to boost the charging speed of a DC-DC converter.
  2. Implemented maximum power point tracking for the Energy Harvesting IC to track the frequency of charging cycles and harness maximum output power from multiple energy sources.
Nitin Kamra and Shouribrata Chatterjee
Undergraduate Thesis, 2014
ROSHNI: Indoor Navigation System for Visually Impaired
This project involves the design of an indoor navigation system for visually impaired people. This is a long-term ongoing project at Indian Institute of Technology, Delhi supervised by Prof. M. Balakrishnan. I joined the project in 2012 to improve the circuit design and sensing of infrared modules.
I implemented a localization and step-detection algorithm with infra-red checkpointing for precise localization, along with a complete circuit design of wall-mounted and waist-worn modules for the project.
Nitin Kamra, Devesh Singh, Dhruv Jain and M. Balakrishnan
Project, Spring 2012
Elementary Iterative Methods and the Conjugate Gradient Algorithm
Presented several iterative methods and the Conjugate Gradient algorithm for non-linear optimization.
Nitin Kamra
High Performance Computing, Indo-German Winter Academy, December 2012

Teaching


  • Teaching Assistant for CSCI-567: Machine Learning, USC (Spring 2020, Fall 2016)

  • Tutorial for Reinforcement Learning, CS-699: Advanced topics in Deep Learning, USC (Spring 2019)

  • Hosting the Artificial General Intelligence Reading Group at USC (Fall 2018)

  • Teaching Assistant for EEL301: Control Engg - I, IIT Delhi (Spring 2014)

  • Teaching Assistant for EEL201: Digital Electronics, IIT Delhi (Fall 2013)

Awards


  • Deep Learning Best Theory Project Award, CSCI-599: Deep Learning, University of Southern California (2017)

  • Viterbi Graduate Ph.D. Fellowship, University of Southern California (2014-18)

  • Best Mentor Award, Awarded by Mentorship Review Committee, Indian Institute of Technology, Delhi (2013)

  • SOF 3rd International Mathematics Olympiad, International Rank 16, School Topper and Gold Medalist (2010)

  • SOF 12th National Science Olympiad, National Rank 45, School Topper and Gold Medalist (2010)

  • FIITJEE Talent Reward Exam, Zonal Topper and Gold Medalist (2009)