Boyd's Blog

how-to-get-into-ais-fellowships

26 min read

— title: “How to get into an AIS fellowship” tags: [essay, mats, advice, retrospective] unlisted: true

I’ve recently finished the 3-month MATS AI safety fellowship and am continuing with the 6-month extension in London. To get in, I put in quite a bit of effort into applying to various AI safety fellowships. I think there’s a small art in doing this, so thought I’d make most of what I can suggest public knowledge.

Apply to lots of streams

Fellowship applications are noisy, so you’ll be rejected even though you shouldn’t have, and someone’ll be accepted even though they should have been rejected. There’s some luck involved, and applying to more fellowships gives you more chances at being lucky.

Think of it like flipping a coin trying to get heads. You can both flip the coin more times (e.g. apply to more fellowships) and you can change the odds of the coin (e.g. improve yourself to be a better applicant). Flipping the coin more times is a bit hacky, but it does work.

Improving yourself is what you should do between applications, but if there’s lots of fellowships available, you should try apply to all of the relevant ones.

Different streams are more/less competitive

I can’t speak for every fellowship, but generally you don’t apply to The Fellowship, you actually apply to The Stream within The Fellowship. This means you’re not competing with everyone who applied to The Fellowship, you’re just competing with the people who applied to The Stream. So while MATS is extremely competitive, your preferred stream within MATS might not be as competitive as is often cited.

The mentors (usually) make the final decision

At the end of the day, you don’t apply to the fellowship, you apply to a mentor’s stream. If possible, you should heavily tailor your application to your mentor’s interests. Don’t be fake! But if you’ve got experience in some niche field, and your mentor happened to have done a MSc in that field, you should consider highlighting this in your application (assuming it’s relevant). You should read your mentor’s website/twitter/papers, and ensure you highlight experience/projects that they’re likely to be interested in, and reduce the emphasis of everything else.

Don’t make the mistake of faking alignment with your mentor’s values. At best this will get you into the fellowship you want but then you’ve got to spend three months doing something you dread, and there’s better things to do than waste your own time pursuing a goal you don’t care about.

Ensure your CV is skim-able

If I look at your CV for 5s, what information do I see? I should see things that are directly relevant to the application, like LLM/ML experience and research experience.

If I look at it from 5m away, what stands out? I sure hope it’s things like “Intern @ Impressive Company” or “Build Cool Project” and not “The internship was from January to March 2025”.

Is ~every word on your CV pulling it’s weight? (I do mean every word). CV’s are particularly prone to tabular replication (e.g. repeating some phrase because it’s consistent with the formatting). Sometimes this is fine and adds clarity, sometimes this is just bloat that detracts from the core parts. Think about removing a word, and then ask “Will a mentor think less of me because they don’t know this information?”

Differentiate yourself from Average Joe Programmer

There will be lots of Average Joe’s applying alongside you, and you want to make sure you’re not mistaken for an Average Joe. You’ve done cool and interesting things, Average Joe hasn’t. But make sure that when you’re describing your experience or projects or accomplishments, that no reviewer could mistake you for an Average Joe.

e.g. Average Joe says

“During the internship at The Company I submitted pull requests to fix bugs.”

You should say

“Intern at Company: increased performance on hot path by 12% and bug fixes”

Note that there’s a number. Numbers are great. They’re like reward hacks for humans. Do you not have any numbers to include in your CV? You should actively work on getting more numbers to include in your CV (e.g. actively pursue tasks that make your skills legible and understandable to low-context outsiders)

You’re good, but are you legible?

Claude describes Seeing Like A State-legibility as such:

Legibility is the property of being readable and measurable by some external system. A skill or resource only “counts” — only has power or gets allocated to you — if it can be seen and quantified by whoever controls the system. If the state (or employer, or algorithm) can’t measure it, it effectively doesn’t exist from their perspective, regardless of its real-world value.

This has an impact on how you present yourself online, thought your CV, and through your references. Listen carefully: It’s not enough to be good. If low-context outsiders (e.g. potential mentors) can’t understand how good you are, then you might as well be another Average Joe.

Certificates, PhD’s, degrees, references, publications are all examples of legible skill. Public Open Source projects are okay, but significantly less legible. If you’ve done something really impressive, but haven’t written about it or published it somewhere, it might as well not exist. I’m not exaggerating.

Strongly consider writing more, and writing publicly (especially if you feel you are skilled but lack credentials to indicate those skills). Writing technical English and descriptions of your projects (not LLM-slop) is a surprisingly high-signal way of expressing your skill to the world.

In general when completing your application and writing your CV, you need to be incredibly pessimistic and assume that whoever is reading your CV/application won’t read the full thing and won’t follow up on everything you want them to. If you’ve done something that Average Joe hasn’t (published a paper, received funding for a cool project, etc) this should be very obvious to see.

I could write a lot more about legibility, but alas, I’ve got a deadline.

Consider how you can improve your CV

Notice the little seed of doubt that appears in your mind when an application implicitly asks for something you don’t have. Google Scholar? PhD? Which Ivy League? Not all of these are solvable, but some of them are, and you should listen to what the applications are telling you they want. If you see lots of applications asking if you’ve done ARENA, then you should try to do ARENA. If you see lots of applications asking “Share things you’ve done related to AI Safety” then you should try to do lots of things in AI safety.

Basically, you should look at each application as both a potential for you to get in, but also a very strong signal for what you need to do next time round to better your chances. If you’re applying to the fellowships over and over again and answering the questions in basically the same way each time, you’ll probably continue to get rejected. Take the questions as a rubric against which you should improve yourself.

There’s a bit of a hierarchy to the fellowships

I’m not entirely sure what the hierarchy looks like, but there’s definitely a progression of “more prestigious” and “less prestigious” fellowships. At the top is probably the Anthropic Fellows Program (they mostly take applicants from MATS I believe), below the AFT is MATS/Astra, and below MATS/Astra would be most of the other fellowships.

Getting into MATS/Astra without any prior experience is hard. Getting into SPAR/MARS/ARENA/LASR/ERA and then getting into MATS/Astra is less hard. If you feel like you don’t have any credentials and this is holding you back, consider getting some credentials through the other fellowships.

If you’re a “risky” option, try start with low-commitment options

If you think you’re skilled but can’t seem to get anyone’s attention, it might be that you’re too high-risk at the moment. Mentor’s might be looking at your application and thinking “this person will either be great or be atrocious, I can’t risk 3 months of my time on that”. This is a rational decision on their part, and you can change their mind by becoming less risky. Things that make you less risky include doing other fellowships or getting other credentials.

But this creates a loop! How do you get into a fellowship if this requires having already gotten into a fellowship? The answer is that you start with the low-stakes fellowships (online, shorter, not funded) and progress towards higher and higher stakes fellowships (in-person, longer, more funding). And you can always find a lower-stakes option, it might be something like doing part of ARENA in your own time on the free Google colab GPUs or writing a public review of a paper you read (bonus points if you disagree with the author’s decisions).

Consider an 80k hours advising call

I applied and got a call, and found this very helpful in getting a third party to highlight my weaknesses and confirm my strengths. The advisor also offered various follow ups with different industry professionals who would have been extremely useful, had I not been accepted into MATS shortly afterwards.

Fellowships are ~constantly accepting new applications

While they don’t literally have rolling applications (yet) the frequency for the big fellowships are much more than annual. If you want to apply, there’s no need to wait until the applications actually open for you to prepare your CV.

A little love letter for typst

typst is great, it’s like LaTeX but significantly easier to use and faster. I spent some time migrating my CV to Typst and it’s paid off massively. I can strongly recommend.

Part of the benefit is that it’s programmable, so my CV looks something like:

 
= Experience
 
#from-yaml("snippets/2026-mats-extension.yaml")
#from-yaml("snippets/2026-mats.yaml")
#from-yaml("snippets/2024-aisct.yaml")
...

And each of those #from-yaml(...) functions reads a YAML file that looks something like:

title: AI Safety South Africa
title-url: "https://www.aisafetysa.com/"
subtitle: Expanding AI Safety in RSA
subtitle-url: null
from: "Jun 2024"
until: "present"
desc: |
  - AI Safety South Africa (AISSA) runs weekly paper discussions and meetups to discuss the latest AI safety research.
  - Through regular attendance and engagement I am one of the most active members and regularly contribute discussion points.
  - With input and funding from AISSA, I have started another chapter in Stellenbosch, a local university town.
  - AI Safety Stellenbosch has similar goals to AISSA, but hosting events in Stellenbosch (about 40 minutes drive from Cape Town) enables the discussions to reach further and engage new people.
pillboxes: ["AI Safety", "Community building", "Organisation"]

And then when I compile the .typ doc, it gives me a CV listing that looks like this:

This is great. I can easily add/remove items from my CV without messing with the formatting. And as a side-effect, I’ve got an LLM-friendly log of every project, job, internship, talk, or thing of interested I’ve ever done. Which brings me to:

Keep an LLM-friendly log of your experience

Not because you’re going to get claude to write your responses. Bad. Don’t do that. The LLMs are not good at integrating the context you have about your experience and the fellowship application. But I did find them extremely useful at reminding me when I was forgetting some extremely relevant piece of experience. If I gave claude the question and my response, claude was very good at noting “You forgot to include “. I’m pretty sure getting an LLM to do a sanity check on every answer I gave significantly improved my applications, and having an LLM-friendly log of experience (and previous questions, see below) is the first step towards doing that.

Keep a text log of your questions/responses

If you go hard, you’re going to end up answering many many somewhat similar questions. You should keep a log (mine was just a faq.md file) of each question you got asked and what you answered (see the bottom of this page). This is immensely helpful for 1. when you get through to the next round and suddenly you’re being interviewed about the responses you gave, and 2. for reducing the burden when fellowships ask word-for-word identical questions.

To be clear, do not reuse your answers if they’re irrelevant to the question. The reason you’re applying to fellowship 1 should not be the same as the reason you’re applying to fellowship 2. If you think they’re the same, you need to go deeper and figure out why they’re different and what makes each fellowship a unique little snowflake.

But sometimes fellowships ask word-for-word the same generic question and then it’s useful to have some text at hand that you can copy (e.g. “what’s your GitHub/LinkedIn/Website”).

If in doubt, you should probably default to writing the answer de novo. Even for something generic-sounding like “Describe your current career plans and aspirations”, you should phrase your career plans in a way that emphasises your goals as relevant to the fellowship you’re currently applying for. Don’t reward hack! Bad! Don’t make up career plans just to match with the fellowship. Do prioritise the aspects of your existing career plans that the mentor would be most interested in seeing. Do not bend the truth about what you want to do. But if you’re career plan involves something that’s irrelevant to the fellowship, don’t go yapping about it for ages and ages.

Your success is roughly proportional to your effort per application

You can definitely always put in more effort into manually (not with an LLM) looking through everything about an application. You should heavily study the mentors on the streams you’re excited about, and read their recent papers. If you can be critical while doing so (“why did they choose X hyperparameter?”, “why didn’t you consider Y?”) this is even better, most mentors appreciate thoughtful critique and discussion of their work.

This is definitely a rabbit hole for some mentors, there’s always more you can do and more you can learn about them. I’m not sure when is the right place to stop, but you should be trying to discover if the work they do is something you’d like to do yourself. If you’re not excited by their research, you probably don’t want to apply to work on their stream, even if they’re “famous”.

Looking through all the mentors pages

This is a bit of a nightmare. For the bigger streams there’s often dozens of mentors. This is a process you can start in advance, before applications for that fellowship technically open. But it does take a long time. I relied on heavily filtering early on (e.g I wasn’t interested in governance or non-Berkeley options) and then going deep on the remaining mentors.

Use the LLMs for motivation, but not for writing

I was working full time while applying to the fellowships, and it was pretty tough to get motivation some nights to actually go through yet another application process. Basically every question would give me significant imposter syndrome and I’d get writers block trying to answer most questions, questioning why I should even bother.

The LLMs were very useful in getting over this. As mentioned above, I concatenated text documents describing everything I had ever done into a single file, and put the file in context with the application’s question and a prompt like:

Here’s my experience and a question for an AI safety fellowship. Do not answer the question for me, but give a bullet list of relevant parts of my background that I should mention in my response.

This would remind me about what I had done that was kinda cool, and from there I could finish the question. Doing this probably improved the quality of my answers, and enabled me to get through more applications.

It’s probably worth taking a day off work when things get going

I forget when, but at some point I had a crazy number of stream applications due in a crazy time window. I was working full time, and most of the applications assume you’re able to answer their questions 24/7. I ended up taking a sick day off of work and just read & wrote applications the whole day. This ended up being worthwhile I think. I managed to get a lot more done than I would otherwise, and that particular day at work wasn’t very special or unique.

Rejection feels like shit

It’s really not fun. But I’m not going to say it’s all alright and you’ll get it next time, because platitudes do not lead to improvement. I don’t think you should beat yourself up about being rejected. But (after taking a break) you should critically review your answers to all your questions, your CV, your references, your online presence. And ask how you can change these things to be closer to a promising AI safety researcher. Try to get involved with a local AI safety org (or think about starting one). Make your work more legible.

ARENA is (surprisingly) high signal

I knew ARENA was good, but if you’ve got some time to work through the content on the ARENA schedule, it’s seen as high signal and many applications ask for it. If I hadn’t gotten in to MATS, working through the ARENA coursework on my own would have been my top priority.

A long list of application questions

Here are all the application questions I answered, slightly deduplicated and anonymised. If you’re wanting to apply for an AI safety fellowship but there’s none available, I strongly recommend going through this list and either answering each question, or considering what you can do to make your future answer to each question the best it possibly can be. Consider this list the dataset on which you should overfit.

  • (optional) Please talk briefly about an area of technical work right now you’re most interested in or excited about, and why. (~3 sentences)
  • Are there any previous projects or experiences you’d like to highlight as especially relevant? For e.g. you can highlight if you’ve previously participated in fellowships such as PIBBSS, MATS, ERA, Pivotal, Talos, or similar programs and include the dates in which you did so.
  • Are you interested in working in a team of four talented, conscientious people (plus us as mentors)? We think people who answer “yes” to this question are probably a better fit for our stream, but it is not a strict requirement.
  • Could you give us a sense of what your longer-term career paths might be? This will help set the agenda for our call, so think of at least two different career paths you could see yourself pursuing and list some pros and cons for each.
  • Describe a time when you faced a non-technical challenge collaborating with other people on a complex project. How did you address it? (Suggested length: ~250 words) Feel free to share other information that would help us understand what it’s like to be your teammate.
  • Describe a time where you had to demonstrate high agency as described here? (100 - 200 words) This refers to a time where you took initiative to solve a problem, create something new, or drive a project forward despite obstacles or uncertainty. This could be academic, professional, or personal.
  • Describe your current career plans and aspirations
  • Explain your strongest disagreement with other alignment thinkers. Consider only your own inside-view understanding, and don’t defer to others’ expertise. (Suggested length ~250 words, but it’s okay to write more)
  • For [Multi-Agent safety, AI for Facilitating Human Cooperation, Gradual Disempowerment], describe the research you’d like to undertake during the fellowship and why you think it is important?
  • Give an example of an empirical AI safety research project you’d be interested in working on, and why? Please mention any relevant experience if applicable. Max 300 words
  • Have you completed Chapter 0 of the ARENA curriculum, or, if you join our stream, would you commit to completing it before the start of the program (Jan 5)?
  • Have you done surveys/literature reviews? Or show some evidence on how fast you can read papers, and what kind of useful takeaways you usually obtain from papers. Put “N/A” if this question is not applicable.
  • How experienced are you with tuning hyperparameters: Write about your hyperparameter tuning experiences. For LLMs, what tricks do usually try to make them work at a target task?
  • How long do you configure the deep learning environment: Describe one of your past experience for the most complicated server that you’ve set up the deep learning environment on. What did you do, and how long does the entire configuration take? (Basically configuration here means everything before you can start to run the program.)
  • How will [Fellowship] help you to achieve your long-term career aspirations? What are you hoping to get out of the program? (max. 800 characters)
  • If you’re open to working in a team, briefly reflect on how you might make this go well. What would a great collaboration look like to you? In your answer, you might draw on prior experience in teams, but you don’t have to. (max 250 words).
  • Imagine it’s 2027 and you completed [Fellowship] a year ago. What specific role or project are you now working on that represents your highest-impact contribution to AI safety or governance? (100-200 words)
  • In 1-2 sentences per project, please provide a short summary of the project and what you contributed to it.
  • Is there anything in particular you’re looking to get out of our potential collaboration? It’s okay to leave this blank. In that case, we’ll assume your answer is “I want to grow as an ML researcher and make contributions to AI safety.”
  • Key factors of AI progress in the next ~5 years? (max. 1500 characters)
  • Link 5 papers or blogposts relevant to AI control, oversight or model evaluations that you’re excited about (they don’t have to be your own). We may ask you about these in interviews.
  • Link to a personal blog post
  • Link to code samples from past projects (e.g. GitHub links or uploaded zip files), ideally a substantial project and (if possible) a machine learning project (these could be the same or different projects).
  • Links to things you’ve done or your experience
  • Mitigating Gradual Disempowerment. Develop concepts and tools that will allow us to develop mitigations to preserve human agency and ensure that our institutions serve us. (500 - 1000 words, 3 hour proctored time limit)
  • Other notable skills or experience
  • Please add a link to 1–2 writing samples, code repositories, or similar that are illustrative of your knowledge or skills relevant to the fellowship, and then briefly give any important context on this piece of work (e.g. whether other people helped produce it).
  • Please describe a hard technical or research problem you solved. What was the bottleneck and how did you overcome it? What impact did solving the problem have? (200 words)
  • Please link impressive AI projects that you’ve built.
  • Please provide 1-2 code samples from past projects (e.g. GitHub links or files).
  • Please provide links to any of your projects or posts related to AI safety.
  • Please share any academic, professional, or personal accomplishments you are proud of. (max 200 words)
  • Please share any other information about your background in ML that you would like us to know.
  • Propose a follow-up experiment to section 3 of this paper and explain the relevance of the experiment to AI safety efforts. (Suggested length: ~250 words for experiment, 1-4 sentences for relevance.)
  • Provide two reference contacts. These contacts should ideally relate to your past AI, AI safety, research, engineering, and/or other relevant experience.
  • Question: What do you consider to be your top achievement(s)? (Max. 50 words) Please share 1-3 achievements that you are especially proud of. These could be scholastic, like your academic performance at university or in an international olympiad, and extracurricular, like winning a competitive sporting event.
  • Read the following report by Model Evaluation & Threat Research (METR): Measuring AI Ability to Complete Long Tasks. For additional technical details, you can see the full-length paper here. In section 7.2.1 of the full paper, the authors discuss “systematic differences between our tasks and real tasks”, including automatic scoring, no interaction with other agents, lax resource constraints, unpunishing, and static environments. Which one of these systematic differences do you expect to have the largest impact on the generalizability of the study’s results? Explain your reasoning. (1600 characters)
  • Take a look at the research projects for [Mentor]‘s stream that I’m interested in (Reasoning Model Interpretability, Red-Teaming and Elicitation, and Model Organisms). What are 1-3 pieces of evidence that you’d be able to do good research in this stream? They don’t have to be standard credentials! Please describe them and why they’re relevant. Aim for conciseness (e.g. a single page is likely sufficient). Feel free to write up a high level plan for a research project you’d want to work on as evidence that you are able to generate and/or flesh out interesting directions that we could work on together.
  • Tell us about a recent achievement where you displayed proactivity, grit, and self-motivation
  • Tell us about any interest or involvement you’ve had in effective altruism.
  • Tell us about your most significant technical accomplishment.
  • What (if any) relevant experience do you have for working in the track you have selected? (750 characters)
  • What NLP/general AI tasks do you usually read papers about, e.g., AI safety, LLM interpretability, LLM for coding, computational social science, etc. (and how many papers for each task, as a rough estimation)?
  • What are 1–2 reasons why you would be a good fit to do empirical AI research? Please give a concise description of each piece of evidence and explain why it’s relevant. Max 200 words
  • What are some (possible) long-term career goals for you (e.g., a bit like “where do you see yourself in your 40s”)?
  • What are your career aspirations? (100 - 250 words) Please outline 2-3 potential career paths you’re considering and explain how this fellowship fits into your trajectory.
  • What are your long-term career aspirations? (max. 800 characters) We ask this to better understand your interests and motivations. We know that career plans often evolve and might be broad at this point, so feel free to express uncertainty.
  • What are your long-term career goals
  • What attracts you to [Company]‘s mission of protecting humanity from AI risks?
  • What concerns you the most about AI development?
  • What deep learning models have you coded before (e.g., adaptations of open-weight LLMs, VLMs, etc)? If applicable, support your answer with your GitHub repos
  • What do you hope to gain from participating in [Fellowship]? (200 words)
  • What do you think is the most promising way to gain helpful information about the probability that advanced AI’s are scheming right now? (<300 words)
  • What do you want to work on? Please give a 3-5 paragraph pitch for your research idea that fits this stream.
  • What have been your preferred sources for learning about AI Safety topics? Please be specific (e.g., name particular researchers, papers, podcasts, courses, books rather than general categories).
  • What is a result in AI Safety, Governance, and/or ML that you have found interesting/surprised you? Did it make you change your mind about something? (max. 1000 characters)
  • What is an important opinion or idea about AI that you wish more people understood? Please explain the opinion or idea in 200 words or fewer. Separately, in 150 words or less, explain why you wish more people understood it and why that might be important.
  • What is one potential project or research question you would be interested to work on as part of [Fellowship]? (2000 characters) You don’t need a fully scoped research idea at this stage, and nothing you write here is a commitment. If you’re accepted to the programme, your project can evolve substantially (and may differ entirely) after you meet your research manager and mentors. However, it is a useful exercise to brainstorm what you could actually work on, and it helps us understand your research interests. Please make an effort to highlight key sub-questions and the theory of change (how would the research ultimately lead to reducing large-scale catastrophic risks from advanced AI?).
  • What is your aim for this research collaboration/internship? Answer using bullet points the following:
  • What is your favourite piece of research/theory? Why?
  • What is your motivation for applying to [the Empirical Research] workstream? Max 150 words
  • What is your motivation to apply to this programme? What do you hope to get out of it?
  • What knowledge areas are you familiar with, or do you like to spend your time thinking about? If applicable, share a link of your notes etc.
  • What previous research experience do you have? Please feel free to provide references to your work. (Max. 100 words)
  • What previous research experience do you have? Prior research experience in fields like ML/CS/math/philosophy may play a role in your ability to do alignment research. That said, if you have done any research in any field that you are proud of, we would also like to hear about that. Feel free to refer to your LinkedIn/resume.
  • What’s the most awesome thing you’ve ever done?
  • What’s your experience with AI safety? If you have any projects or posts, please provide links when able. If you have no experience with AI safety research in particular, please feel free to leave this section blank.
  • What’s your motivation for applying to this stream and project? (max 100 words)
  • Why do you want to be part of [Fellowship]? (150 - 300 words) Here you can mention (but are not required to) your personal motivation for AI safety work, why cooperative AI research specifically interests you, and how participating now fits your longer-term goals for contributing to the field.
  • Why do you want to work at [Company]?
  • Why do you want to work in AI Safety?
  • Why do you want to work on reducing risks from advanced AI through this fellowship? (1200 characters)
  • [50-100 words, max 300] What are 1-3 pieces of evidence that you’d be able to do good research in these streams? (These don’t have to be standard credentials!) Please concisely describe them and why they’re relevant
  • [Your Previous Papers & Coding Skills] If applicable, list your past projects/papers, where each line includes (1) link to the paper (published and unpublished writeup), (2) conference and year (if applicable), and (3) part of the code that you programmed (with GitHub links, and notes about which parts you implemented)
  • [≤ 150 words] Which topic(s) would you be excited to work on as part of these streams and why? Feel free to select one of the example project ideas provided in the stream descriptions or propose your own.
  • [≤ 200 words] Describe a research project that you have done, what went well and some takeaways for improvement.

Graph View