Building Robots That Can Perform Complex Tasks With and Around People

Designing robots and developing their complex algorithms to solve real world issues

There are over six million people in the US alone—ranging from people with disabilities to older adults— who need assistive care. Current technology has great potential to improve their quality of life, and to ease the burden of their healthcare costs. Dr. Siddhartha Srinivasa, Finmeccanica Associate Professor at Carnegie Mellon University is addressing this issue with his research in robotics. His work involves building robots that can physically and forcefully interact with the world (i.e. pick up objects, feed someone, and defuse bombs), with and around people. Through his technological breakthroughs in robotics, Dr. Srinivasa aims to help people in need of care.

He focuses on both developing algorithms for motion planning, manipulation, control, perception, and machine learning, and also on designing, building, and deploying physical systems that perform complex manipulation tasks in the real world. Robots work very well on factory floors which are structured for robots. But a home looks nothing like a factory floor; it is filled with clutter and uncertainty. By endowing robots with simple models of physics, Dr. Srinivasa’s work enables robots to understand the clutter and uncertainty present in our everyday environment, enabling them to work seamlessly and effortlessly with and around humans.

Dr. Srinivasa works with a team of members from over six countries who specialize in computer science, mechanical engineering, electrical engineering, mathematics, and physics, calling themselves the Personal Robotics Lab. They also collaborate extensively with researchers around the world, as well as with clinicians at the Rehabilitation Institute of Chicago (RIC) for patient trials. 

His team not only focuses on the theory and algorithms of robotic manipulation, but they take the building blocks of autonomy and apply them on a physical system to address real world tasks, with the aim to accomplish this within the next five years. He and his team have made huge progress, both in the algorithms they’ve developed in autonomy, as well as with the hardware they have designed. They currently work with the two robots they developed: “HERB” (Home Exploring Robot Butler)—a two armed robot with a segway base—and “ADA” (Assistive Dexterous Arm), a wheelchair-mounted robot arm. ADA is used in collaboration with the RIC to help people with high spinal cord injuries. The hardware platforms Dr. Srinivasa and his team have iterated on for the past 10 years are now approaching the stages of applied capability. 

Current Research Includes: 

  • The Mathematics of Human-Robot Interaction — Instead of a robot cooking a meal by itself inside an individual’s kitchen, Dr. Srinivasa and his team want the individual to be able to work side-by-side with the robot when preparing a meal. They want a robot to work as an equal partner with the individual they care for. However, this involves both great dexterity and seamless coordination. Dr. Srinivasa’s research focuses on developing the building blocks of autonomy, combining perception, motion planning, control, and machine learning algorithms to enable robots to perform complex tasks. However, autonomy is necessary but not sufficient for collaboration. To be an effective collaborator, a robot has to understand and predict its human partner, and also act expressively and predictably such that the human partner understands its intentions, leading to seamless coordination. Dr. Srinivasa is formalizing this coordination via Bayesian models of prediction combined with techniques from optimal control for feedback-driven action.
  • Physics-based Manipulation — Traditionally, industrial robots follow the “pick-and-place” paradigm: they pick something up, and place it somewhere else. However, our daily lives are richer than mere pick-and-place. We push, pull, and slide objects around, exploiting the fact that we can use physics and even other surrounding objects to help accomplish our task. Dr. Srinivasa  and his team are developing algorithms that use simple yet expressive models of real-world physics to enable robots to understand the consequences of their interaction with the world. This allows them to perform the kinds of tasks that humans perform, in human environments. This is a crucial step for getting robots out of the factory floor and into people’s homes.
  • Building Robots for Assistive Care — Because Dr. Srinivasa believes we have a societal and moral obligation to produce technology that can help people in need, his lab significantly focuses on assistive care. He is collaborating with the RIC to help people with high spinal cord injuries use wheelchair-mounted robot arms to perform complex manipulation tasks, such as opening a refrigerator, pulling out a meal, microwaving it, and eating it. Controlling a complex robot arm is incredibly challenging—even for an able-bodied person—but much harder for people with disabilities, who have to use high latency and low bandwidth interfaces, like head-mounted joysticks or Sip-N-Puff. He and his team are developing a framework for shared autonomy, in which the robot predicts the user's intentions and goals, and automatically slides the autonomy, giving the user more or less control depending on the confidence of its predictions. There is often a social stigma around the requirement of constant assistance, so a robotic system that helps an individual perform daily living tasks on their own allows them a sense of independence and confidence. He is extremely passionate about this research project, as well as the societal impact of this thread of research. 


Born in an university campus—Indian Institute of Technology Madras—Dr. Siddhartha Srinivasa spent his first 19 years in India, including his undergrad. He was surrounded by academics: his father was a professor, and so were all of his friends' fathers. Being a professor seemed like the obvious and logical thing to do. Considering himself an “accidental roboticist,” his undergraduate research was in theoretical fluid dynamics and he was set to choose that as a career. He was excited about its complication and intricity. However, Raj Reddy, a famous professor from CMU's Robotics Institute, visited his university and gave a talk about robots and how cool they were. He spontaneously decided to apply and, after much deliberation, chose to pursue robotics instead of fluid dynamics. He hasn't regretted that choice one moment since. 

Chasing his passions, he felt (and admits this feeling is even stronger today) robotics have great potential for making an impact and improving people's’ lives. Dr. Srinivasa wants to make that potential a reality. With the realization that there is so much to do in the field of robotics, he knows his work could fundamentally change the way our world works. He remembers his passion for robotics he first had as a 19-year-old, fascinated by what he saw and excited to “play with cool toys,” and doesn’t see a difference with his current self. He grew up watching and reading science fiction, and has always been fascinated at by the people that make that fiction a reality. He’s excited to now be part of that process himself.

He wants to build robots that can actually help people. Thinking of of his 80-year-old grandma in India, Dr. Srinivasa often worries about how she is able to interact with the world. He wants to build technology that can help her perform everyday activities, like making breakfast. As the aging population increases, a shortage of qualified caregivers grows, increasingly making current care-giving services more expensive.

He believes we build a lot of technology that can help able bodied people be lazier, but often forget about the real people who need care. He was inspired when he first visited a collaborator at RIC and met patients, clinicians, and occupational therapists face-to-face, and really getting a glimpse into their lives. He calls them real heroes, working tirelessly to make lives better.

In his free time, Dr. Srinivasa enjoys playing squash and captaining the Carnegie University squash team. He’s currently training for an ultra-marathon—a 50 mile race— and enjoys cooking, playing with his kids (he jokingly considers them one of the best examples of how you can start with very poor manipulation skills and greatly improve them), and recently began taking piano lessons. 


Deliberate delays during robot-to-human handovers improve compliance with gaze communication


Exploiting Domain Knowledge for Object Discovery


Generating Legible Motion


Towards seamless human-robot handovers


The MOPED framework: Object recognition and pose estimation for manipulation


Task Space Regions: A Framework for Pose-Constrained Manipulation Planning



RSS Best Systems Paper Award Finalist, 2015

Paper: CHISEL: Real Time Large Scale 3D Reconstruction Onboard a Mobile Device

IEEE ICRA Best Paper Award Finalist, 2015

Paper: Movement Primitives via Optimization

IEEE ICRA Best Video Award Finalist, 2014

Paper: Extrinsic Dexterity: In-Hand Manipulation with External Forces

Finmeccanica Chair in Computer Science, 2013-16

RSS Best Paper Award Finalist, 2013

Paper: Generating Legible Motion

RSS Early Career Spotlight, 2013

IEEE ICRA Best Manipulation Paper Award Finalist, 2013

Paper: Efficient Touch Based Localization through Submodularity