Everything, newest first — across every channel. Their profile →
Hiding
From one piece The Evolving Landscape of LLM Evaluation 2 beliefs, in the piece's order there
Their words
GPT models perform much better on coding problems released before their pre-training data cut-off.
Their words
The time when benchmarks lasted multiple decades has passed. Going forward, we will rely less on public benchmark results.
From one piece Command R+ 2 beliefs, in the piece's order there
Their words
I'm excited about what this means for the open-source community and research, with the gap between closed-source and open-weight models closing and SOTA-level conversational models being more easily accessible.
korrents.com
Retrieval-augmented generation is the most practical approach for reducing LLM hallucinations.Their words
Retrieval-augmented generation (RAG; Lewis et al., 2020), which conditions on the LLM's generation on retrieved documents is the most practical paradigm IMO.
From one piece True Zero-shot MT 2 beliefs, in the piece's order there
Their words
the embodied, interactive, and multi-modal nature of first language (L1) acquisition is challenging to replicate with current models.
Their words
many under-represented languages are primarily spoken; they do not have a written tradition or a standardized orthography. Text-based NLP technology is thus of limited use to them.
From one piece Thoughts on the 2024 AI Job Market 3 beliefs, in the piece's order there
korrents.com
Large research team sizes on frontier AI projects create friction that makes execution and decision-making harder.Their words
Such size poses challenges for the effective execution and prioritization, increasing friction and making it more difficult to quickly make decisions.
Their words
In light of the applied nature of current research problems, there is another path that exposes you to cutting-edge AI work: joining a startup.
Their words
This lack of knowledge sharing may impede progress in AI development.
korrents.com
At its best, research is a collaborative and sometimes argumentative conversation, not just its published output.Their words
At its best, research is a collaborative and sometimes argumentative conversation.
From one piece NLP Research in the Era of LLMs 3 beliefs, in the piece's order there
Their words
recent LLMs are reaching the limits of text data online and repeating data eventually leads to diminishing returns
Their words
In the near term, the largest models using the most compute will continue to be the most capable.
korrents.com
Breakthroughs achieved with massive compute are typically achieved using it inefficiently.Their words
while massive compute often achieves breakthrough results, its usage is often inefficient.
From one piece EMNLP 2023 Primer 3 beliefs, in the piece's order there
Their words
Full fine-tuning of LLMs has become prohibitive and requires parameter-efficient methods instead.
Their words
LLMs are still limited in non-English settings and that making LLMs more multilingual is an important direction.
Their words
In light of the increasing number of closed-source LLMs, it is important to continue to promote an open culture of sharing knowledge, data, and software, from which the NLP community has benefited greatly.
From one piece NeurIPS 2023 Primer 3 beliefs, in the piece's order there
Their words
Overall, while initial plans of LLMs can be useful as a starting point, LLM-based planning currently works best mainly in conjunction with external tools.
korrents.com
Current large language models still struggle to compose operations into correct multi-step reasoning paths.Their words
Overall, current LLMs still struggle with composing operations into correct reasoning paths.
Their words
In sum, whenever we don't have infinite amounts of pre-training data, we should train smaller models for more (up to 4) epochs.
From one piece An Overview of Instruction Tuning Data 2 beliefs, in the piece's order there
korrents.com
In instruction tuning, a smaller set of high-quality training examples outperforms a larger, noisier dataset.Their words
training on a small set of high-quality data outperforms instruction-tuning on larger, noisier data.
korrents.com
Models fine-tuned to imitate another model's outputs copy its style without acquiring its factual accuracy.Their words
Models that are instruction-tuned on ChatGPT-generated data mimic ChatGPT's style (and may thus fool human raters!) but not its factuality
From one piece Modular Deep Learning 2 beliefs, in the piece's order there
Their words
But modularity may also facilitate a shift away from a concentration of model development in a few institutions and to distributing the development of modular components across the community.
Their words
Given the trend of pre-training larger and larger models, we believe modularity will be crucial.
Nothing matches.
What is a korrent?
A korrent is a belief a person has stated in their own words: one sentence stating the claim, backed by a quote and a source, kept at korrents.com.
Under a name here, the quoted block is what they actually said. The korrent beneath it is the claim those words support, in korrents' wording — tap it to see the record, its source, and who else holds it.
Nobody here wrote their own korrents. They are compiled from public statements, and a person can change their mind, which is recorded too.
About the English under a post
Some people here publish in a language other than English. Where they do, this site shows a machine translation beneath the post, in this typeface — the site's own, not theirs.
The post itself is never changed, moved or hidden: what is set in the serif above is exactly what the person published, and it is what to quote them on. A translation can be wrong in ways that matter, especially about tone.
Only the post's own words are translated. A quoted post, a linked article and a belief on korrents.com are left in their original language.