Research scientist at Meta in Berlin working on multilingual models and evaluation; led the multilingual team at Cohere and was a research scientist at Google DeepMind before that.
It collects the places they publish and what they have said there, each linked to the source. They have no account here. Is this you? Claim it, correct it, or ask us to remove it from ppll.
Recent
Link verified 20 Sept 2026. Recent items update automatically from the channel.
Each is a — compiled by korrents.com, not by them: the one-line wordings are korrents', the quotes are theirs.
Recent
The time when benchmarks lasted multiple decades has passed. Going forward, we will rely less on public benchmark results.
The Evolving Landscape of LLM Evaluation Said 13 May 2024
GPT models perform much better on coding problems released before their pre-training data cut-off.
The Evolving Landscape of LLM Evaluation Said 13 May 2024
Retrieval-augmented generation (RAG; Lewis et al., 2020), which conditions on the LLM's generation on retrieved documents is the most practical paradigm IMO.
Command R+ Said 15 Apr 2024
I'm excited about what this means for the open-source community and research, with the gap between closed-source and open-weight models closing and SOTA-level conversational models being more easily accessible.
Command R+ Said 15 Apr 2024
many under-represented languages are primarily spoken; they do not have a written tradition or a standardized orthography. Text-based NLP technology is thus of limited use to them.
True Zero-shot MT Said 27 Feb 2024
the embodied, interactive, and multi-modal nature of first language (L1) acquisition is challenging to replicate with current models.
True Zero-shot MT Said 27 Feb 2024
This lack of knowledge sharing may impede progress in AI development.
Thoughts on the 2024 AI Job Market Said 12 Feb 2024
In light of the applied nature of current research problems, there is another path that exposes you to cutting-edge AI work: joining a startup.
Thoughts on the 2024 AI Job Market Said 12 Feb 2024
Such size poses challenges for the effective execution and prioritization, increasing friction and making it more difficult to quickly make decisions.
Thoughts on the 2024 AI Job Market Said 12 Feb 2024
At its best, research is a collaborative and sometimes argumentative conversation.
The Big Picture of AI Research Said 18 Jan 2024
while massive compute often achieves breakthrough results, its usage is often inefficient.
NLP Research in the Era of LLMs Said 19 Dec 2023
In the near term, the largest models using the most compute will continue to be the most capable.
NLP Research in the Era of LLMs Said 19 Dec 2023
recent LLMs are reaching the limits of text data online and repeating data eventually leads to diminishing returns
NLP Research in the Era of LLMs Said 19 Dec 2023
In light of the increasing number of closed-source LLMs, it is important to continue to promote an open culture of sharing knowledge, data, and software, from which the NLP community has benefited greatly.
EMNLP 2023 Primer Said 5 Dec 2023
LLMs are still limited in non-English settings and that making LLMs more multilingual is an important direction.
EMNLP 2023 Primer Said 5 Dec 2023
Full fine-tuning of LLMs has become prohibitive and requires parameter-efficient methods instead.
EMNLP 2023 Primer Said 5 Dec 2023
In sum, whenever we don't have infinite amounts of pre-training data, we should train smaller models for more (up to 4) epochs.
NeurIPS 2023 Primer Said 1 Dec 2023
Overall, current LLMs still struggle with composing operations into correct reasoning paths.
NeurIPS 2023 Primer Said 1 Dec 2023
Overall, while initial plans of LLMs can be useful as a starting point, LLM-based planning currently works best mainly in conjunction with external tools.
NeurIPS 2023 Primer Said 1 Dec 2023
Models that are instruction-tuned on ChatGPT-generated data mimic ChatGPT's style (and may thus fool human raters!) but not its factuality
An Overview of Instruction Tuning Data Said 15 Nov 2023
training on a small set of high-quality data outperforms instruction-tuning on larger, noisier data.
An Overview of Instruction Tuning Data Said 15 Nov 2023
Given the trend of pre-training larger and larger models, we believe modularity will be crucial.
Modular Deep Learning Said 23 Feb 2023
But modularity may also facilitate a shift away from a concentration of model development in a few institutions and to distributing the development of modular components across the community.
Modular Deep Learning Said 23 Feb 2023
What is a korrent?
A korrent is a belief a person has stated in their own words: one sentence stating the claim, backed by a quote and a source, kept at korrents.com.
Under a name here, the quoted block is what they actually said. The korrent beneath it is the claim those words support, in korrents' wording — tap it to see the record, its source, and who else holds it.
Nobody here wrote their own korrents. They are compiled from public statements, and a person can change their mind, which is recorded too.
Hiding
From one piece The Evolving Landscape of LLM Evaluation 2 beliefs · ruder.io
Their words
GPT models perform much better on coding problems released before their pre-training data cut-off.
Their words
The time when benchmarks lasted multiple decades has passed. Going forward, we will rely less on public benchmark results.
From one piece Command R+ 2 beliefs · ruder.io
Their words
I'm excited about what this means for the open-source community and research, with the gap between closed-source and open-weight models closing and SOTA-level conversational models being more easily accessible.
korrents.com
Retrieval-augmented generation is the most practical approach for reducing LLM hallucinations.Their words
Retrieval-augmented generation (RAG; Lewis et al., 2020), which conditions on the LLM's generation on retrieved documents is the most practical paradigm IMO.
From one piece True Zero-shot MT 2 beliefs · ruder.io
Their words
the embodied, interactive, and multi-modal nature of first language (L1) acquisition is challenging to replicate with current models.
Their words
many under-represented languages are primarily spoken; they do not have a written tradition or a standardized orthography. Text-based NLP technology is thus of limited use to them.
From one piece Thoughts on the 2024 AI Job Market 3 beliefs · ruder.io
korrents.com
Large research team sizes on frontier AI projects create friction that makes execution and decision-making harder.Their words
Such size poses challenges for the effective execution and prioritization, increasing friction and making it more difficult to quickly make decisions.
Their words
In light of the applied nature of current research problems, there is another path that exposes you to cutting-edge AI work: joining a startup.
Their words
This lack of knowledge sharing may impede progress in AI development.
korrents.com
At its best, research is a collaborative and sometimes argumentative conversation, not just its published output.Their words
At its best, research is a collaborative and sometimes argumentative conversation.
From one piece NLP Research in the Era of LLMs 3 beliefs · ruder.io
Their words
recent LLMs are reaching the limits of text data online and repeating data eventually leads to diminishing returns
Their words
In the near term, the largest models using the most compute will continue to be the most capable.
korrents.com
Breakthroughs achieved with massive compute are typically achieved using it inefficiently.Their words
while massive compute often achieves breakthrough results, its usage is often inefficient.
From one piece EMNLP 2023 Primer 3 beliefs · ruder.io
Their words
Full fine-tuning of LLMs has become prohibitive and requires parameter-efficient methods instead.
Their words
LLMs are still limited in non-English settings and that making LLMs more multilingual is an important direction.
Their words
In light of the increasing number of closed-source LLMs, it is important to continue to promote an open culture of sharing knowledge, data, and software, from which the NLP community has benefited greatly.
From one piece NeurIPS 2023 Primer 3 beliefs · ruder.io
Their words
Overall, while initial plans of LLMs can be useful as a starting point, LLM-based planning currently works best mainly in conjunction with external tools.
korrents.com
Current large language models still struggle to compose operations into correct multi-step reasoning paths.Their words
Overall, current LLMs still struggle with composing operations into correct reasoning paths.
Their words
In sum, whenever we don't have infinite amounts of pre-training data, we should train smaller models for more (up to 4) epochs.
From one piece An Overview of Instruction Tuning Data 2 beliefs · ruder.io
korrents.com
In instruction tuning, a smaller set of high-quality training examples outperforms a larger, noisier dataset.Their words
training on a small set of high-quality data outperforms instruction-tuning on larger, noisier data.
korrents.com
Models fine-tuned to imitate another model's outputs copy its style without acquiring its factual accuracy.Their words
Models that are instruction-tuned on ChatGPT-generated data mimic ChatGPT's style (and may thus fool human raters!) but not its factuality
From one piece Modular Deep Learning 2 beliefs · ruder.io
Their words
But modularity may also facilitate a shift away from a concentration of model development in a few institutions and to distributing the development of modular components across the community.
Their words
Given the trend of pre-training larger and larger models, we believe modularity will be crucial.
Nothing matches.
About the English under a post
Some people here publish in a language other than English. Where they do, this site shows a machine translation beneath the post, in this typeface — the site's own, not theirs.
The post itself is never changed, moved or hidden: what is set in the serif above is exactly what the person published, and it is what to quote them on. A translation can be wrong in ways that matter, especially about tone.
Only the post's own words are translated. A quoted post, a linked article and a belief on korrents.com are left in their original language.