Kathrine

To start, would you mind introducing yourself, where you are currently in your education, and the kinds of robotics problems that you are most interested in solving?

Duo

Okay, cool. So my name is Duo, and I'm currently doing my CS PhD program in Rutgers University, and I'm mainly doing the robotics side of the science department. And in robotics, I'm mostly doing task and motion planning and motion planning.

So to expand a little bit, basically, task planning is what you tell your robot to do. For example, you want your robot to go to your kitchen to grab you a mug, and which mug should it grasp, right? It's called task planning.

It's more abstract on the object level. Also, another thing is called motion planning. It's like when I tell the robot to grasp something or to do something, how can the robot actually do that?

Because you need to give instructions to every joint, every part of the robot, some kind of command to make the robot actually execute that move to get to, for example, grasp a mug for you. So I guess it's a very high level explanation of what I'm doing right now, and hopefully we can get a little bit deeper into this side later.

Kathrine

Yeah, sounds super exciting. So to touch a little bit on your career path, so you began college studying energy and power engineering before moving into computer science. So how did you make that decision to change studies?

Duo

Actually, I have tons of reasons, but I think I'll conclude it as maybe two reasons. So first of all, I don't have much choices at the beginning to pick a major when I get to college, because I don't know how much you know about Chinese high school education. We are just focusing on all the coursework, like all kinds of math, physics, chemistry, English, everything.

But we can't really put ourselves into one project to discover our interests. And I was born in a power station. All my family are working in a power station.

So it was so natural for me to just go for this direction. But after the first two years of my college life, I tried more things. I started to learn programming myself, and I realized, okay, maybe this is now where my interest is in.

And so I was seeking out for another direction. And this is my second reason. And at that time, AI started to be huge.

Everybody was talking about AI, AI, AI, this, AI, that. AI will be our future. So at that time, in China, a major for AI just got approved, and it's held under the computer science department.

And they are recruiting some new students to just try out. So I was lucky. I got picked into that major.

So I made my decision in one night, and I just applied for it, and I got it. So that's basically the whole story. And I'm still glad I made this decision.

Because before, I know like power, like energy, power engineering, they are also doing cutting-edge research. They're super good. They're doing like nuclear energy, like fluid dynamics.

But in my first two years, all I learned is how to boil water to generate electricity, right? It's just, I mean, it could get very far, but at that moment, I'm just not interested in that anymore.

Kathrine

Yeah. That was a brave decision to switch majors. So next, your recent work often focuses and emphasizes like mathematical guarantees, such as ensuring that a robot's path is collision-free or approaches the best possible solution.

So how did you become interested in this more theoretical side of robotics?

Duo

Yes, actually, I tried a lot of stuff to do when I was doing my master's. But the reason I choose this, okay, so like guarantee is a good thing to hear. Like every time you hear guarantee, you feel safe, right?

If you're doing something else without a guarantee, sometimes you don't know what will happen. You don't know how good it is. You don't know the worst case.

So once you've got this guarantee, it's safer for you to know, I have this guarantee and it's a good... So, okay, so if I can say, I guarantee this robot won't have any collisions, right? It's a fantastic property.

But if I say, okay, so it's going to be collision-free 95% of the time. If you are in production, in a factory, you have to have someone there to monitor everything. So that's kind of the difference here.

I guess it's more like a mental thing. I can predict my results. I can know what's going to happen.

And yeah, most importantly, it feels safe. That's the main reason, I guess.

Kathrine

Yeah. So next, could you walk us through a recent project that you did regarding a dual arm robot? So what kind of work is involved behind the scenes in coordinating two arms to interact with an object?

Duo

Yeah. So basically, our lab has a long history working with robot arms and doing tabletop rearrangements, this kind of tasks. And we're trying to explore the optimal way to do the task planning part.

Like how should we arrange the cues, the objects on the table that we can have, for example, minimum number of movements and things like this. And it's very natural to think, okay, so maybe one arm is doing great. How about we have two arms?

Maybe they can do things together so we can have a better performance, a more efficient system. But in the turnout, it's not that straightforward because if you have two arms, the system is a lot more complicated. There's a lot of entanglement between the two arms.

One arm could be an obstacle for the other arm. And so it's blocking the movements of the other arm. So how do we actually handle this kind of interaction?

How do we maximum the parallelism between the two arms so they can do things together without bothering each other? Yeah. So that's the initial question we had when we started this project.

I think that's the problem we tried to solve. And then we actually kind of solved it at a very preliminary stage because I don't want to make this thing too deep. But to plan a trajectory for a robot arm, it requires you to give every time step, each joint on the robot arm has to have an angle together.

And for just one arm, if it has six joints, you have to search for a trajectory in a six-dimensional space. And when you have two arms, if they are planned all together, you will have 12 dimensions. So the search space is growing exponentially, which makes things a lot harder.

Yeah. So we really tried tons of ways to make this thing easier because not a lot of motion planners can currently do this. Okay.

Sorry, but I don't know how deep should I get from this. I don't want to do a whole presentation on this, but I also... Do you think it's still at the right level or you're still following?

Kathrine

Yeah. Could you talk us a little bit more through the kind of work that you guys are doing? Were you guys doing a lot of coding?

Were you guys tinkering a lot with the actual mechanical parts?

Duo

Yeah. That's a great question. So we do tons of coding.

That's for sure. Everything is done by code. I cannot manually do anything.

And you have to make your... Once you have an algorithm, you have to implement it so it can be efficient and secure. And so you have some guarantees.

So we do tons of coding, but we don't really build our robots. We don't really do hardware, but we actually do the control part of the robot. So it's more like we give the intelligence to robots instead of building one.

So we take the robot for granted. Whatever we buy, we have it. And then we need to do some real robot experiments.

So we have to give the command to the robots from our code and we have to make sure they are safe. And we need to make sure the actual robot can follow our trajectory and things like that. So that's a little bit the hardware side.

Kathrine

That's pretty cool. I mean, I never really realized that... I kind of always thought that people who worked in robotics would always be making their robots from scratch.

So I guess it's interesting that you guys were buying them and I guess just teaching them how to act.

Duo

Yeah. So actually, that's the major two directions of robotics. One side, they are building robots from scratch, making all the properties to the best.

And on the other side, people are trying to... Okay, that's actually building the body for the robot. And the other side is more like the mental stuff.

You are trying to develop the brain for the robot so it can actually handle something autonomously.

Kathrine

Yeah. So next I want to ask, so after you earned a master's degree, you actually worked for a while at a company called Tencent America. So after that, how did you decide that pursuing a PhD was still important to you?

Duo

Yeah. So actually, I never thought not to get a PhD because when I was working at Tencent America, which of course, it's a great company. I'm working in one of their game studios, Lightspeed and Quantum Studios.

Basically, what I do there is also research. So we don't have products in our team. We just do research.

So it's actually more like a different approach of testing different styles of research at school or in industry. So I think it's a great experience for me to have a different style of research. And then I go back to school to get my PhD.

So I guess it's more like a bridge connecting my master's to my PhD at that moment. And also, everyone around me, I'm the only master's student here. So that's another peer pressure.

So I got pushed to get a master's student there. So that's a different angle here.

Kathrine

Very understandable. So in school, many students, including myself, see clips of these very polished robot arms, smoothly grabbing things and moving them around in these robotics research presentations. So what part of being a robotics PhD student are most different from those images of robotics research that people would see in those polished videos?

Duo

I guess one of the big things is, yes, they are polished. So behind that, it's days, months of work of failure just to get that thing to work. Yeah.

So people always watch those videos and say, okay, we are so cooked. Robots are replacing human beings. And they can do this.

They can do that. They can fold enclosures, picking males for 24 hours nonstop. But in reality, everything, it's specifically designed for that task to happen.

It's generally not robots doing things in the wild. Most of the stuff is controlled and very well calibrated cameras environment. I mean, people are trying to make those things more and more general, and robots should be more generalized in all kinds of stuff.

But I guess that's the difference. It's really not happening now. Especially when you see that in the videos, it's actually still far from how one knows that robot in your home doing all the jobs like that.

Kathrine

Of course. So my next question is, as AI systems are becoming more capable, what skills do you think young robotics researchers or people who are generally interested in CS still need to develop for themselves rather than relying on like Chachi, BT or Quad?

Duo

I'll say it's taste. So I'll make an analogy here. Basically, all kinds of AI agents, they are fantastic cooks.

They can cook in different styles. They are masters. Basically, they are better than you in most of the stuff, except maybe your own expertise there.

But they are better in almost everything. So they can tell you everything, and you need to choose whether to believe it or not. So while you're doing research, so taste is an important thing to have.

You cannot believe everything AI said to you. You cannot just do everything I said to you. It's like you're hiring your own cook.

You have your own style. They're all equally good, but you want to go for this direction. So you have to hire a cook that works for you, or you have to work with a cook to make the best dishes for you.

If you just ask AI, what should I do as my next research project? It will give you tons of stuff, and it will say, okay, everything is good, and maybe you need to pick one. And you have to have that thing in you to tell you, okay, which one's good, which one's bad, whether the AI is doing a great job, whether AI is doing a bad job.

You need to learn tons of stuff. You need to work really hard. So I know in the current undergrad education system, especially in computer science, you probably don't have to do all the homework yourself.

You don't have to do all the coding assignments yourself. AI can handle everything perfectly. But if you get the chance, if you don't do that yourself, you cannot absorb everything into your own brain.

So at the time when you need to pick something from, or to judge how the AI is doing, you kind of lack of the taste to say, this is a good taste, this is a bad taste, I don't want this, I want that. You are the supervisor. You have a great worker right here, but you need to guide it to infinity and beyond.

Kathrine

So I'm curious, earlier you mentioned that most of your job involved coding. So now with AI, how much of that code is actually written by humans, and how much of it is AI-assisted?

Duo

Okay, two months ago, I'll say I'm still doing coding myself, but now I'll say most of the code is done by AI, because you have to test it out. After I test it out, they're really good. They're really good at not extreme hard problems.

So I'll say probably 99% of my code are AI in the last month. But I still have to do some of the coding myself, because I have a project which has tons of corner cases. It's tricky for AI to debug, and I tried it multiple times, always do a bad job.

So I had to do it myself. That's probably the only part that I need to code myself. Also, for example, I code myself for the hard parts, but for the other stuff, mostly it's not that hard.

Sometimes, for example, if I need to make a figure, if I need to just implement some easy algorithms, I can do it a thousand times in one minute. I have to code myself probably for days. For those kind of jobs, it's 100% there.

There's no chance I'm doing it myself.

Kathrine

Very great decisions. So how are you thinking about the decision between academia and returning to industry after you achieve your PhD?

Duo

Yeah, so my goal is to go to industry. Because right now, you see all the large models, all the fast robots are from major companies that are doing it. They have the ability to buy the equipment and to spend a huge amount of money on computer resources.

So that's something that schools, universities currently don't have. It's not that far from academia. I still want to do research, but I want to do it in industry.

That's probably the path I will follow later.

Kathrine

Yeah, of course. So finally, what is a realistic project that you would recommend for high school students who are interested in robotics?

Duo

Actually, I have one. Our lab has one. Can I share my screen?

Yeah, sure. Yeah, okay. So basically, it's super fun to have this in your home.

And I'll say... Here. So can you see it?

Kathrine

Yes.

Duo

Nice. So basically, it's a little cart, very small. And this is developed way back in our lab by Professor Yu and all the senior students at that time.

It's a little cart, and it has a marker on top. And basically, once you use your camera, your camera can detect these kind of markers, right? It can distinguish which cart this is and where it is.

So basically, with one camera and with one of this little cart, you can control them on the ground and do all kinds of stuff. And it's really easy to build. Basically, it's very cheap.

You can just buy everything from Amazon, and you can build this little cart yourself. So here are some little demos here. So basically, on the left side, it's the camera view, and on the right, it's the simulator, basically the real-time recognition of all the robots.

And you can actually drag with your mouse on one robot and tell it to go there, and the robot can actually go there. And this is some other demos. So it includes building the robot from scratch.

Also, it includes the robot intelligence part if you're interested, right? So basically, this is a circle. You want the robot to move themselves onto the circle, and they can do a circular movement here.

This is just another demo. So they are doing multi-robot path planning on this kind of nest-like graph, right? They can only go from vertices to vertices, and they have to go from one start configuration to a goal configuration.

And we are also developing some new versions of this. Let me see. I think I have this somewhere.

Yeah, I guess I can show this to... Yeah, so now it's a brand new interface, and it includes my algorithms into motion planning algorithms into the system. Okay, so that algorithm basically gives you a very optimal path to go across, like for example, short narrow passages.

Yeah, things like this.

Kathrine

Oh, wow. That was really cool.

Duo

Yeah, so this is a new demo, like a real demo of the circular movement. So they're just testing things out. And now, it's all wireless, and the interface is much, much faster.

Oh, this is another kind of movement. And they are just changing patterns in real time.

Kathrine

This is incredible.

Duo

Yeah, so it's pretty fun. It's like Lego, right? Oh, this is another thing.

So this is actually a research project done by our lab mates. So basically, this robot has to push the block to some target positions. So this is just the robot doing some pushing.

Oh, this one's too heavy, so it can't really push it. But this one, this wooden bar is lighter, so it can push it at this stage. So you just print out your own markers and stick it to your objects, and it can recognize and do something you want.

Yeah, so this is pretty cool. And other than this, I think... Okay, I guess I'll stop sharing now.

So this robot is called MicroMVP. So you can Google it, and you'll find it on our website. But other than this, there's tons of things you can do, not just to build a robot.

I guess you can try different approaches to do coding, solve your own maze. The robot can go from place to place, and you can build your own simulation system. Okay, maybe it's a little bit harder because it requires some basic knowledge in computer graphics, robotics.

But it's fun to build all the stuff from scratch, and you'll learn. It's definitely rewarding. You'll learn tons of stuff from those kinds of experiences.

And also, because you're actually designing your own system, like I said, it builds your taste over time. So you have to do those kinds of stuff yourself, and you interact with AI, and you'll know, okay, so that is my design. That is AI's design.

Which one's better? You'll learn over time to be a better designer, a programmer, and most importantly, a researcher.

Kathrine

Yeah. Thank you. That was a wonderful note to end on.

I love those tiny robots because they were so cute, and they look like a super fun and accessible project for students. So yeah, thank you for sharing your work and your advice with me. I really appreciate your time and everything that you had to share, and I really hope students will learn a lot from hearing your perspective.

Duo

Cool. Thank you so much for interviewing me.