Interview with Wilka Carvalho

by Muhammad Khalifa (PhD student, Michigan AI)

Introduction: Wilka Carvalho is a 5th year Ph.D. student in Computer Science and Engineering at University of Michigan. His research interests lie at the intersection of deep reinforcement learning, cognitive science, and neuroscience. Wilka is almost at the end of his Ph.D. and so I took the opportunity to ask him questions about his Ph.D., AI research, and advice for aspiring AI researchers.

Muhammad: Hi Wilka, I am super excited to do this interview with you, and thank you for your time. 

Wilka: Hi Muhammad, I am happy to be here. 

Muhammad: Wilka, could you introduce yourself to our readers?

Wilka: Sure, my name is Wilka Carvalho, I am a fifth-year Ph.D. student. I’m co-advised by Profs. Honglak Lee and Satinder Singh here at U-M Computer Science, and by Prof. Richard Lewis in the Psychology department. You will probably see me walking around in between breaks and if you’ve ever talked to me, you’ll learn that I love to talk about how the brain works. I’ll be a neuro-AI research fellow at Harvard after I graduate, where I’ll try to leverage machine learning to advance theories for how the brain works.

Muhammad: You are almost at the end of your Ph.D. journey. How would you describe this process for yourself? 

Wilka: Definitely hard. It is definitely the hardest thing I’ve ever done. Similar to many students here, I used to think I was really good at critical thinking and that my logic was so good. Then I met the standards of my advisors and I learned that my logic was not that good and that my critical thinking had a lot of room for growth. I think the key thing that I’ve learned here is what it means to have thorough or rigorous evidence. 

Muhammad: Could you elaborate on that?

Wilka: I think the basic answer is scale for computer science. Let’s say if an algorithm has property X, it leads to behavior Y. Either I have a [mathematical] proof that says that’s true, or I have to show empirical evidence. For empirical evidence, you want to show it in as many different data sets or environments that test different parts of the algorithm under different conditions. 

Muhammad: I am sure your interests shifted across your Ph.D. How would you describe this transition? 

Wilka: When I began the Ph.D., I was interested in which algorithms and function classes I could use to characterize human cognition and behavior. And for that interest, I have been very influenced by current characterizations of cognition and behavior. For example, we have object-centric representations of the environment. How to get that [in an AI system] in a way that’s general is quite challenging. In the past, I may have been interested in [manually] building this in and seeing what we can do with that. But if you want a really general algorithm, which humans must have, you want to find mechanisms that are very general and that make very few assumptions, and then allow for [object-centric representation] to be like an emergent ability. How to do that is quite hard, and it forces you to think at another level of abstraction.

To answer your question, the shift [over my Ph.D.] has been to be more abstract and to aim for more abstract [methods] that induce what I care about, as opposed to building it in.

Muhammad: How would you describe the ultimate goal of your research? 

Wilka: Generally speaking, I want to understand the brain’s learning algorithms. And at some level, it is going to require working to replicate them.

Muhammad: Many of the readers of the Michigan AI blog are usually either undergrad students or master’s students with an interest in AI or early Ph.D. students. If there is one piece of advice you would give to someone who is considering doing a research career, what advice would you give?

Wilka: It depends on whether you want to go into industry or academia. If you want to go into the industry, then try to get exposure to different topics. So, just work on projects that you think are fun in your Ph.D. or master’s or collaborate in a lab if you are an undergraduate student. Then, you can intern at a company to see what happens in the real world or at least at those companies that you care about.

But if you want to go into academia or a place like DeepMind or Microsoft Research — that’s a bit more academia-like but in the industry — you probably want to develop a vision and a narrative for the kind of research that you want to do. It’s not a commitment, but you want to really go in-depth with a particular topic and persist with it over multiple projects. Soon enough, as you keep working on that particular topic, you start to go beyond the low-hanging fruits and focus on the harder problems.

Muhammad: Looking at the landscape of the current AI research, what are you most excited about at the moment? On the language side, for example, there are Large Language Models like ChatGPT and GPT-4. There have also been some advances in vision like diffusion models and all of that. What excites you the most at the moment?

Wilka: I’m really interested in behavior more than either vision or language. While these recent advances are exciting, I’m not personally excited by them. On another side, I am also concerned that legislation laws are quite slow to enact and I think the rate at which AI is progressing will certainly change the economy faster than the government can act or is willing to act. I feel like we are going to have a big reduction in the workforce.

On another note, I’m personally really interested in successor features. They’re basically this predictive representation of the environment. You represent where you are now as a prediction of where you would go from that place. To give an example, let’s say I’m in the lobby of the Beyster building. I can possibly represent the lobby with a mixture of the auditorium that’s right next door and the CAEN lab and other possible locations. This lets you transfer knowledge across behaviors, and it’s also been implicated in neuroscience a lot. Interestingly, we used to think that dopamine was learning prediction errors for learning to predict rewards, but it actually seems like it’s learning prediction errors for learning successor features instead. 

Muhammad: Is there anything that you don’t particularly like or would change about how AI research is being done at the moment, whether in the industry, big labs, or academia?

Wilka: That’s a good question. There are a few things I don’t like. For instance, I see the importance of benchmarks– you have a standard to compare against, it’s great—but I think then there’s sometimes too large an emphasis on benchmarks. Therefore, they end up being over-optimized and they end up losing their utility, but I don’t know the solution for that. 

Another thing I’m apprehensive about, right now in AI, is that if you are aiming for AI conferences such as ICLR, Neurips, or ICML, you will have a deadline basically every four months or so. I think that this creates an incentive structure where a lot of academic labs encourage students to try to publish frequently at these conferences. And these end up being more often than not smaller projects. And this is not a critique of incremental research. I think incremental research is really important. 

The problem appears when you combine that with the incentive structure of academia, where there’s a large emphasis on things like the h-index and the number of citations you have. This strategy leads us to lose out on bigger, more impactful projects. I think that DeepMind does a good job, where you have these 30, 40-person projects, and they take three years to complete, but then you truly make progress on an important topic. There was a recent article in Nature about how publications generally have a lower impact now or are less groundbreaking, and I would guess, at least for AI, that [our publication culture] is part of the problem.

Muhammad:  What is next for you and your research?

Wilka: I’ve been working on successor features for a little bit now, and it is a nice setup to think about behavior and transferring knowledge. I’ve accepted a position as a postdoc with Dr. Samuel Gershman at Harvard,  in a psychology lab. I’m hoping my future research can connect cognitive theories with successor features. Interestingly enough, the two have not been connected anywhere in literature, not in neuroscience, AI, or in psychology.

Muhammad: What do you plan to do after your postdoc? Do you want to join the industry, or join academia?

Wilka: Right now I’m still pursuing the academic route. I definitely want to look for a healthy department with good culture, good values, and good leadership. I’m not sure yet if it’s going to be psychology or computer science.

Muhammad: Before we wrap up, is there any final advice that you would like to conclude the interview with?

Wilka: Try not to lose sight of what is exciting and enjoyable, and fun about research. Try to work on topics that, again, are exciting and fun.

Muhammad: Thank you Wilka for this interview. We enjoyed having you here.

Wilka: Thank you for having me!

More about Wilka Carvalho: https://cogscikid.com/