Kathrine

Joining me today to start, would you mind introducing yourself, your field and the main questions that your research focuses on?

Professor Latecki

Okay, I am Longin Jan Latecki and I got my PhD in 1992 at the University of Hamburg in Germany and computer science, but I got a master's in Poland in mathematics. Yeah, and I am doing computer science for like 36 years. So this is a long time and I, from the beginning, I was doing artificial intelligence and it changed a lot.

So this is, I am surprised how many things happened during my life. So this is really, I am blessed to some extent to witness this transformation. The things which I thought never would be possible, just become possible.

So this is really extremely surprising. And yeah, what was very important and kind of a main question of artificial intelligence was how to really make computers and robots have human-like abilities. And we witnessing this kind of, and it's getting better and better really every day.

Kathrine

Absolutely. You definitely had a lot of foresight when you picked your major, but you originally, as you mentioned, studied computer science. So how did you decide to transition into studying computer vision?

Professor Latecki

Yeah, first I started to do mathematics because I kind of didn't know really computer science. I was fascinated, to be honest. First, let me tell you why I switched to computer science.

Because I was, during my math, I had some programming classes. My first programming language was Algol, but it was on big mainframe machines. And the machine has a lot of colorful blinking lights.

So this is like cool disco lights. So this was looking really cool that you could program it and whatever you write the instructions and did, if you wrote them correctly. So this was really cool.

Yeah. And then, yeah, artificial intelligence basically was kind of, for me, always the most exciting to do. And I didn't start doing computer vision.

I started doing natural language processing. So understanding, text understanding. Basically, you read a story and we know what it is about and we wanted computers to do it.

But unfortunately, I started doing it in symbolic logic. So there were two main streams of AI. One was symbolic, which was dominant when I was a student.

But what is winning this neural computation. At that time, it was basically, I didn't understand this neural computation. But my professors said, many of my professors, it's not good.

So I followed the symbolic. And time showed that the symbolic is useless, but the neural processing is what was needed. And then I did natural language processing.

I tried to do it in a symbolic way, was very hard. And I thought, yeah, computer vision seems easier. So I started doing computer vision.

Now I do both actually, computer vision and natural language processing.

Kathrine

That's really cool. So in some of your recent research, you guided a model's attention towards relevant visual information. So how exactly do you teach an AI where it should look?

And how do you determine if it has really learned the correct relationship between an image and a word?

Professor Latecki

Yes, this is actually first, maybe the last question you said, it's very easy. And because now everything, neural networks, and as you may be aware, mechanistically, we know what they are doing. But just on the surface, but how everything is thinking, whatever we call it, this thinking, we don't know really how it works.

So now computer scientists kind of become psychologists. So now kind of, it's very interesting, or psychoanalytics, that we try to understand what we program, what does it actually do in the inner working. And here I work at the intersection of natural language and computer vision.

So you give a, say, vision transformer, we call them vision language models, but this is like ChatGPT, which also understands images. You can ask a question, what color is the cat in this image? And it will reply, oh, the cat is red, or the cat is white.

But also we know that we cannot 100% trust that these neural networks give us correct answers. So to check very easily is, we look at the attention, and attention is actually, where is actually this transformer or neural network looking at the image when answering your question. And there are several ways to utilize this attention, kind of go inside inner working and see where at the image it is looking when answering your question.

And if it is looking at the right part of the image where you as a human would be looking, then you have higher confidence that it's really correctly answering your question. So this is basically how it is with attention. This was, by the way, started by psychologists for many, many years with extra devices around the year 2000 only started to basically tracking human eye movements to see when humans analyze images where they look.

So this is kind of similar, but mathematically very different way.

Kathrine

Yeah. So could you walk us through one computer vision or medical imaging project, including the research problem, training data, and evaluation of the results?

Professor Latecki

Oh, so it's now everything is changing because nowadays we have tools like this advanced neural networks, which we call transformers. And this started with 2017 paper by Google, attention is all you need. It's basically very recent paper.

So this is kind of, by the way, side remark, the history is made in front of our eyes and it's changing very fast because not even yet 10 years since this paper was published. And this everything started with paper. So this is the tool which most people are using.

Some trying to develop better versions of transformers. Some try to utilize them in different applications to do different tasks. But big part now of this, since these models are not programmed, like hard-coded with human-made rules, they find out the rules themselves.

But how you train them? Usually you train them by providing input data and what would be expected output. So you basically prepare a training data set showing for this input, I would like this output.

For this input, I want the other output. So basically, a lot of now work is done in collecting certain data sets for the output you would really like to have. So this is the big part of work.

And then another thing is how you measure the correctness of your prediction. So you prepare the data sets based on this data set. I mean, my projects now currently are too complicated, but I can describe a very simple project which was important at the beginning.

If you want the computer to distinguish cats and dogs, you will basically have a set of images containing either a cat or a dog, a natural environment in the background, and a label, one called cat, the other dog. But then you still need to measure whether what the computer output is cat and dog. So you need so-called loss function.

And then you backpropagate this loss with partial derivative to adjust the weights in this neural network. So this is kind of on the high level what is done. And there are many different loss functions because you may not only want to classify whether the image contains cat and dog, but you also want to know exact location where is the cat.

So for example, highlight the pixels containing a cat or highlight the pixels containing a dog and so on.

Kathrine

Yeah. So next, could you give us an overview of how computer vision research has changed during your career?

Professor Latecki

Changed a lot. Particularly that things which we thought never possible to be done can be achieved now. So this is, as I said, in the 90s and also early 2000s, we still try to do basically logical inference.

So which means look how we humans do it and try to learn from this and write a set of rules. So this, say, we have a distinction of cat and dog, how you would describe, for example, how their face is different, how their ears are different, and you need some mathematics. And so basically just describe certain shape features and so on.

So you try to generate set of features and handcraft them basically by coding them. And then describe rule-based systems. And then maybe a little bit classifier like logistic regression, very simple classifiers on top.

So this was basically, at that time, didn't work well. So I tried to make around year 2000 a system for image retrieval and recognizing objects based on their shape. It worked on certain images, but it couldn't scale to real-life applications.

But since neural networks started, this was first 2012 really, paper showed that image classification can be done much better with neural networks. Then came transformers. So all these tasks are now based on learning.

So you just provide examples and the neural network will learn from examples. You don't write any rules. You just collect the data set and come up with a loss function.

So it's big transformation. But the main transformation is where before we only have toy systems which you couldn't deploy anywhere. Nowadays, you have systems which really can make a difference in the real life.

And the best example in the US, I think, in addition to digital system, like ChakGPT can answer your question and analyze images, the real-life system is like full-serve driving of Tesla. So I don't know whether you had a chance to be driven by Tesla full-serve driving, but it's really fun to see that it can drive you from one location to the other and you do not need to do anything. So the things change in the way we program and then the outcomes change because before we had only demo systems which couldn't really work.

Now suddenly this thing works and can be deployed in reality.

Kathrine

Yeah, of course. So also, what does a typical day in your life look like and how is your time divided amongst developing ideas, teaching students, analyzing results, perhaps writing papers and other activities?

Professor Latecki

This, yeah, it's hard to say during the day, but I can describe. I have few meetings usually with my students or collaborators, mostly on Zoom. So Zoom is a really great tool.

And yeah, teaching is kind of regular teaching except that now grading became a very old-fashioned grading, just pen and paper in a classroom because you cannot grade, you cannot give assignments. I even stopped doing programming assignments because before if you gave student programming assignments, they would program themselves. Nowadays programming assignments make no sense because like codecs can program everything for the students.

Then I have a lot of discussion with my PhD students. So we have usually Zoom meeting to discuss mostly things which do not work because doing research is 99% of your ideas will not work. And it's basically understanding why things do not work the way you want them to work which basically drives the progress.

So yeah, kind of we humans learn from mistakes and this basically, in my view, also drives a lot of research. Basically, if you need to do something wrong to really understand why it does not work and then you can learn and improve on it.

Kathrine

Absolutely.

Professor Latecki

And then writing also, yes, some part is also writing the ideas, writing research publications.

Kathrine

So next, what misconceptions in your experience do people often have about computer vision or artificial intelligence?

Professor Latecki

Depending who are the people, there are many misconceptions. Because also they are only recent. So maybe I need to go back in time.

Already in the 90s, not the first time, actually, with the beginning of computers, people thought about artificial intelligence. But particularly in the 80s, 90s, many people, also famous professors, particularly in the US, because in the US you need to get research grants. In order to get research grants, you need to promise a lot.

And before, many people promise a lot that they will do in artificial intelligence, but they didn't deliver. What happens, they didn't understand how hard the problems are. And they try to basically use simple rule-based systems, so basically logic-based system, symbolic computation, and this didn't work.

And so this happened. So the first misconception was that artificial intelligence is the research with broken promises. So when I got my PhD, it was impossible to get any position in artificial intelligence.

So I have to say I do computer vision, I couldn't say I do artificial intelligence because nobody would hire me, because everybody who knew this term knew that this does not work. Things only change after year 2012. So suddenly everybody's doing artificial intelligence.

Nowadays also people still do not understand artificial intelligence. So many people use full self-driving in Tesla and they enjoy it, but they don't really know that artificial intelligence is behind, which makes it possible. Meanwhile, yeah, chatGPT most people understand and like to use it, but still many people actually prefer to use Google search, or Google search is more or less the same as chatGPT meanwhile.

But because they were used to it, they do not, they know it got more clever and better, but they do not understand why.

Kathrine

Mm-hmm.

Professor Latecki

So yeah, this is kind of, misconceptions come, in my view, mostly because people do not understand how they already benefit from it, without knowing what it is. But before it was different misconceptions, this was basically, it's funny how it changed over time, because those people who knew in the 90s and years 2000, early 2000, artificial intelligence, they knew it is basically the field of broken promises. So basically this is something that does not work.

Nowadays we think about it very differently.

Kathrine

Yeah, absolutely. I think it's really incredible that you like, you had the foresight to go study artificial intelligence, like at a time where people didn't really like it, and now it's like paying off definitely.

Professor Latecki

Yes, but still I am angry that I listened to my professors and didn't study the neural computation. I started with symbolic, only later switch to the neural computation.

Kathrine

Mm-hmm.

Professor Latecki

So yeah, but I could witness all this transformation as they happened.

Kathrine

Absolutely.

Professor Latecki

And also, the term neural network also was not liked by people before. Before they invented in the year 2012, when the things started changing, they call it deep learning. But we call it now deep learning on neural networks, but term deep learning was invented to hide the neural networks and also artificial intelligence.

Kathrine

Yeah. So finally, what is one realistic step a high school student could take to begin exploring computer vision, machine learning, or artificial intelligence?

Professor Latecki

Yes. Here, in order not to be too high level, you all know now about vibe coding, yes? But it's still not everybody is good at vibe coding.

What vibe coding does, you basically code everything in natural language. So you do not need to learn any programming language. Before, it was a huge barrier.

You needed to first learn programming language. This is like a foreign language. Only like a master of this foreign language, you could do something with computers.

Now this barrier is gone, so you can use English and get things done. But still, it is not trivial. The reason is, learning foreign language, most of us can do after some effort, but we still need to do some effort.

But express our idea and structure them well. This is basically the main benefit of school. This is why high school is good for this.

And this is not a topic everybody teaches you explicitly. Not a single subject teaches you explicitly to structure your thoughts and put them into paper. Also, even if you learn how to write essays, they are in kind of difficult.

For other humans to understand, they do not need to be structured well. So they need to be interesting and so on, but they do not need to be structured well. But you know, computers understand you in English, but you need to express your ideas clearly.

And this is not so trivial to structure your thoughts and express your ideas clearly.

Kathrine

Yeah.

Professor Latecki

And this is basically, you can have, you can do it now interactively with ChatGPT, which is much better because it will ask you questions if you are not clear. Because often if you ask ChatGPT to do something for you, which can slowly do, it will do something very different. Not because ChatGPT is so stupid, but mainly because you did not explain clearly what you want.

So this is the best analogy I view it as like baking a cake. Explain to your friend your cake recipe and see whether your friend really bakes a good cake. If you explain nicely, the cake will be well done.

And this is basically, and you can see even if you read some recipes online, how to do things, they may not be very clear or they make some assumption about your background, which you don't have. They assume that it's obvious for you, but it's not obvious. So basically practicing and you can do with vibe coding is really what vibe coding is to be able to express your idea clearly.

And this is basically what is mostly research about, not only computer science, but research in many other fields is to express your idea very clearly and go into every detail. And kind of we have all background knowledge, which we assume some things are very obvious to us, but they may not be obvious to other people because of a different background. And this is the issue with ChatGPT.

ChatGPT read all stuff which is online, but still it may have still different background that we do. And so this is basically clear communication and expressing your ideas clearly. This is something which would need to be practiced the most.

And for this, it's really important to communicate also with other humans, not only on just very high level where you know, basically you have the same background, which is easier, but try to communicate with people with a different background also. And yeah, all one way to do it is really practice vibe coding. So simple tasks.

Kathrine

Yeah, that sounds like an exciting project for any high school who is interested in this field. And yeah, this is all of my questions. That was a great note to end on.

Thank you so much for sharing your work, your experiences and advice with me. I really appreciate your time. And I think students will learn a lot from hearing your unique perspective.

Professor Latecki

OK, I view your undertaking as a very interesting one. So good luck with this.

Kathrine

Thank you.