Fulvio Sanguigni

Fulvio Sanguigni

PhD student at University of Modena and Reggio Emilia

Phd Kickstart: My advice and my resources

19 minute read

Published:

Before the start, I introduce myself. I am Fulvio Sanguigni from Italy, and I have just started a Phd at University of Modena. I am an athlete as well, and I have some dreams to realize. But not today. Today we will talk about the Phd. Today it’s time to start giving back, since I received a lot in these years. This is my first blog post. This is my little space in the internet where I will tell my story and share my path towards my goals; this is my little corner where I will welcome you and let appreciate the craving path towards excellence, in all the fields where I play. I am not pretending to be a master at something. But I am working to be that in the future. Let’s start the journey!

Advice: A journey to Phd application

I leave you two main sections: the application phase, with a detailed story of my journey so far, and the resources section, with the best resources I follow for creativity and research advice

The Application

Just to be clear: there is not fixed path or method to get admitted to a Phd. Rather, it’s useful to follow some guidelines and sanity principles. And please talk a lot with others who did this before you. In this way, you collect a database of interesting stories, one different from the other; hopefully, some patterns emerge from these, like the guidelines I mentioned before.

For my application, I followed these principles:

  1. Lifetime curiosity: I didn’t start looking for labs just some months before finishing my studies. I was already doing it because of my curiosity. I kept track of my favourite labs in a notion file and read their works.

    This step lasted months and years, because I just accumulated this knowledge and database following my own taste, starting from my late undergrad years. For example, I read an interesting research work for video character animation from Willi Menapace, I saved his research lab into my database and backtracked other works both from Willi and other related researchers in Italy or foreign institutes collaborating with him. And, by the way, watch out this guy, he is an amazing researcher and very available to discuss further his ideas.

    In that way, I spotted both my actual lab (AImageLab) because of its expertise on Multimodal LLMs and other interesting realities both in Italy and in Europe. I was not interested in USA or other because I believe taking a Phd is a big step enough and Europe has the top clusters of excellence for my topics.

  2. Preferences: I was very clear about my preferences for the labs. I set few strong constraints, namely the geographical place, the people you work with and an international presence. Stop.

    All the rest (GPUs facilities, money, number of Phd years etc…) vary a lot across the labs, it’s important but it’s not as foundamental as these key principles. For me at least.

    I like a lot to live in medium-sized towns near the mountains and/or water areas (sea, lakes). I believe that density of talent beats the absolute numbers in the long run. In Europe we have a few examples of that, like the cyber valley in Tubingen (Ger), some key spotlights in the Netherlands like University of Twente in Enschede, Grenoble in France, and the two swiss biggest universities (ETH, EPFL).

    At the same time, I was looking for motivated and hungry research labs who cared about the growth of their Phd students. Better to be advised by a very good solid professor rather than ending up with the biggest fishs and not being followed at all. Of course, you can achieve both, if you know where to look for. See point 1 and next points.

  3. Get help from others, build your network: In this case, I was lucky enough to spend many months at European Space Agency (Phi-Lab). This helped me to calm down the frantic search for a lab, because I knew I had a powerful network of contacts at my disposal.

    In any case, I asked a lot of advice to my older colleagues (thanks Ale Sebastianelli 🙂 ) and I compared my favourite labs list with those known/suggested by my ESA supervisor. Moreover, I could rely on my Sapienza advisor too and to the network of friends I built in Lead The Future Mentorship (LTF) for further advice and connections.

    LTF is an italian community where he mentor young students (undergrads mostly) to make them reach their desired career path, anything between academia, big tech, startups. And, most of all, trying to give back and returning to Italy with this bag of experience.

  4. Approaching the deadline: few months left We are talking about Spring 2023. In this case I just tried to speedup the process. I started looking closely at the deadlines of the labs, I started reading better their works to get a taste for their research, weather I liked it or not, and I resumed some contacts I made in the past.

    Moreover, I had other nice chats with Phds from IIT (Genoa), EPFL, ETH, Max Planck. Even though I did not end up in these places, I am happy to have gathered more insights and human contacts by having all these chats.

    I almost ended up burnt out at this stage, because I was trying to contact some labs pretending to propose an innovative research line without looking at previous Phd openings, I firmly wanted to propose something mine, maybe an intersection of topics, maybe a totally unexplored one.

    I had a sort of writer block because I tried to look at all possible lab topics and create one on my own, based on my personal taste. This didn’t work, because the literature is too wide and ideas just come up every day, you end up lost.

  5. Approaching the deadline: few weeks left We are talking about June and July. This helped me a lot to focus, because I was both finishing my master thesis and applying for Phd positions. I was more focused for two reasons:

    1) I relied on the Phd opening requirements, using those constraints to my advantage. Maybe I could not make up the perfect topic for me, but I was way more efficient at doing it. You can always refine your proposal during your Phd journey.

    2) I was in a rush with my master thesis, and really I couldn’t miss a single day of work. I enjoyed a lot the challenge of compressing a ton of things in the space of 24 hours. In fact, these have been the two best months of my ESA experience because I did all the possible activities available.

    I kept my training schedule (and possibly, increased). I started hanging out more frequently. I started a dance course. I applied for 4-5 Phd positions + had other chats. I finished my master thesis and discussed it on 21st of July. I joined multiple bike races, including the CNUs (Italian University National Championships). On 31st of July I was finished, but incredibly happy.

  6. Phd application: This deserves another blog article. To keep it short, just remind that usually you have to write both a short statement of purpose and a Phd proposal. Plus, you will need 1-2 reference letters, for example from your master thesis advisor and your co-supervisor (if you did a master thesis internship).

    Many times, they may be more important than your grades. In fact, I scored maximun grades from my bachelor’s but I undeperformed during my master studies, getting just a decent score. Still, I had the privilege to be selected in multiple institutions. You find a couple of good resources here (application templates) and here (general resources on github).

Onboarding

Concerning the onboarding phase, I let you a (non-exhaustive) list of things that I did:

  • Get to know your labmates as fast as possible. Hang out, join some activities, stop with them for lunch. Sometimes I have to force myself doing it, because I spend lunch break doing sport. But it’s foundamental.
  • Get to know the research of ALL your lab mates. You will live with them for 3 years. It’s highly likely that you will end up doing some cross-collaborations. Maybe you work with images and videos, someone else works with NLP. At some stage, inevitably, ideas will start to pop up. But only if you did your homeworks 😉
  • Build a strong foundation for your Phd topic. After that, be balanced between depth and breadth. Be curious and always aim at studying works from different fields. Some of the best works out there took inspiration elsewhere. Stable Diffusion is the result of thermodynamics laws re-used to model the corruption of images and their reverse process, by means of solving a Sthocastic Differential Equation.
  • Be familiar with the lab tools and resources as fast as possible: learn fast how to use your GPUs, if there are some strange commands to get access to them; learn fast how to debug properly (vscode in my case) and how to build reliable GPU-supported enviroment (I use conda). And finally, learn fast to code all the “utility” functions you will need later (dataset classes, visualization tools, download data tools etc…)
  • Relationship with errors and setbacks: your Phd (and mine) will be full of these. What makes the difference is 1) on which tasks do you make errors and 2) never repeating the same error twice.

Reason 1) means that you get a LOT of information by making research-related mistakes; some examples: you test several ways to change your model architecture, you test several ways of creating new data, or you try hard to implement a complex math theory.

Instead, it’s not great to be blocked by errors on your tools, like the debugger, the GPU settings etc…, if these are not the core of your research question. Reason 2) means you have to learn from your errors, move on and never do it again.

3) Bonus: never get emotionally attached to your mistakes. Both stupid ones and research-oriented. Never let errors define who you are. Rather, look at them with the eyes of a child. Every fall is a feedback for you, and teaches to how to walk. Better to get hard feedback than none at all

Conclusion

Of course this is a personal journey towards a Phd, and yours will likely have something different. Take 100 people, and you will have 100 different stories. My hope is to give you a reference and some guidelines, together with some anectodes and concrete examples I provided you along the way. I hope you enjoyed it. I leave you my resource feed!

Resources: My Feed

I leave you my favourite resources, both in terms of blogs and concerning books, lectures, advice…

Best resources (general AI and Math)

Christopher Olah is perfect if you wants to dug deep into the architectures which are the basic foundations of our latest ML models. In particular, he works at anthropic, so he got his hands immersed into LLMs and has a lot of nice insights in his articles. He was one of the main authors at DISTILL, an online journal with well written articles, in-depth analysis of ML architectures and even a peer-review process to evaluate their articles. It was an immense effort, for now it’s put on hold. Read them all as long as they stay online.

Terence Tao is simply the best matematician of this era. A lot of material both on generic career path and on specific math material. Not exactly my area of study, but it’s weel worth it!

Lilian Weng + The AI Summer - If you have ever tried to understand the math behind any ML architecture, you have ended up into one of these two. Definetly a lot of material spanning different areas, from Generative Models to Self Supervised Learning, multiple modalities, training tricks, and a lot of math. I have still to finish them all.

Nathan Lambert and Sebastian Ratschka - Both dig deep into the LLM world, in particular concerning Reinforcement Learning with Human Feedback (RLHF). Nathan has a ton of excellent blog posts, both written for huggingface and some on his personal blog. You find also career-related articles, like this very good one. Sebastian Ratschka has a massive presence on twitter, definetly have a look at his profile! Big plus, both have very dense newsletters full of technical details and insights. Despite many other “Generative AI” newsletters, these are more focused on technical details and much less about buisness speculations and market analysis. A joy to read them both!

Frank Dellaert is one of the top voices in robotics, he is one the minds behind gtsam and it has a lot of articles on NERFies approaches (3D reconstruction techniques). I suggest also the original NERF

Sticking to the robotics field, Davide Scaramuzza and Raffaello D’Andrea are amongs the best researchers in the world concerning automous drones. Back in my robotics days at Sapienza, I was fascinated by these two figures, both Italians and both working in Switzerland. They don’t have blogs, but they followed non-common paths to become leading experts, and they are worth reading. Moreover, both have a lot of talks/interviews online, and prof. Scaramuzza has 2-3 interesting interviews with also general advice for the students.

Michael Bronstein and Petar Velikovic - These two are amongst my favourite researchers, because they combine a huge expertise in Deep Learning architectures and a strong mathematical background. Their expertise lie in the field of Graph Neural Networks (GNNs), but they can easly span all over the architectures. Definetly visit their blogs and dig deep into their book geometric deep learning. I love this book because it builds a unifying framework for every DL architecture, from GNNs to CNNs

Gabriel Peyre is a french matematician which displays a lot of nice math and ML visualizations on his twitter account and he has a lot of free resources for applied math on his website. Such an incredible researcher! A lot of resources on optimal transport. For the Italians, remember that we have an amazing expert of this topic, he is the fields medal Alessio Figalli.

Bounded Rationality (bjlkeng.io) is one of my favourites blogs because I always find a lot of stuff about math foundations of DL architectures. Just as an example, read his Variational Autoencoders article. You will want more of it!

Chip Huyen (huyenchip.com) has both technical DL related articles and research advice. I come back there couple of times in the year.

Jeremy Jordan - From the basics (backpropagation) to Transformers. He is another one who put a lot of material on his blog. His blog takes you by hand and guides you through all the stages of a certain ML framework.

Best Books and Lectures

Best Theory books:

Best theory and interactive exercises books:

Lectures: You go safe with stanford lectures. From the classic CS229 to the more advanced ones, like CS228 from Stefano Ermon (one of the fathers of diffusion models) about Probabilistic Graphical Models

Best sources for Phd and Research advice

A Happy PhD: A specific webpage of an (ex) Phd student, with a lot of structured guidelines both for your well-being and for your productivity.

Andrej Kharpathy: do I really need to introduce him? Career advice on his blog + a TON of technical material on youtube

Michael Black (Max Planck Institute for Intelligent systems, Tubingen): I find somehow refreshing how one of the top researchers in the field writes about science from a different angle, aiming to provide both clarity and several anectodes. One of my favourite articles is about novelty in science, highlighting some common flaws in the scientific reviewing system

Jan Van Gemert: professor from Tu Delft. He has a lot of pratical guidelines for his Phd students in order to collaborate with him. Definetly woth reading and re-reading them!

Sebastien Ruder: He has plenty of technical NLP material on his blog. But if you are just interested in career advice, this is one of the best articles I read so far on the web: 10 Tips for Research and a PhD (ruder.io)

Bastian Grossenbacher Rieck: He has a lot of advice for productivity in his blog. I liked a lot his reference to the ideas diary file

Top 5 articles: Creativity in Academia — Why you are more creative than you think (timdettmers.com) Inspiring. No more to add. Just read it. Tim Dettmers is a guarantee

Lessons learned from a PhD in Machine Learning by Vincent Fortuin 3 Phd series articles from Vincent Fortuin (Now at Helmotz AI). A lot of information out there, very detailed and well explained!

An Opinionated Guide to ML Research (joschu.net) Old But Gold read also the other 2 linked in this article (see next 2 items).

From Richard Hamming (You and Your Research (virginia.edu))

From Michael Nielsen (Principles of Effective Research – Michael Nielsen). No need to introduce them.

(Bonus): Mental Health and Academia By Riccardo Marin, Tuebingen

(Bonus 2): How I Got a Job at Google DeepMind (No ML Degree) from Aleksea Gordic. A source of hope and inspiration, expecially because it shows that hard work, drive and curiosity lead to unexpected and amazing results. And my goodness, Aleksa worked his ass off to get there, no doubt!

Best Newsletters (Goal: keep up with research literature)

  • Alphasignal: weekly reviews, top-5 papers of the week, some in-depth insights
  • Ak: Daily feed
  • The already mentioned newsletters from Sebastian Ratschka (Ahead of AI) and Nathan Lambert (Interconnets.ai)
  • Scholar Inbox: this has the greatest potential. Developed by the researchers at University of Tubingen, it’s a LLM-powered reccomendation system tailored on your preferred topics and authors.

Best communities and creators