-
Their words
In short, RLVR gives us no reason to expect the normative representations learned in pre-training will acquire motivational force over a given action, especially when post-training repeatedly selects trajectories for terminal task success.
Related posts
Christopher Olah GitHub
The the words it uses that this site has seen least often elsewhere. Posts are matched on those words alone — nothing here is a summary of this one.
Top people
Showing Profile →
Hiding
Hiding
11 August
29 June
-
Their words
To shout “temporary” and pull the plug essentially overnight after immigrants have relied on the government’s representations that they should put their efforts and future into establishing a life in the United States wreaks unfair havoc on the lives of immigrants as well as the welfare of their communities.
SCOTUS Whitewashes Otherwise Racist Decisionblog.simplejustice.us
17 October 2025
-
Their words
And it just so turns out that um this was extremely early, way too early. so early that we shouldn't have been working on that, you know, uh because um if you're just stumbling your way around and keyboard mashing and mouse clicking and trying to get rewards in these environments, um your reward is too sparse and you just won't learn and you're going to burn a forest uh computing and you're never actually going to get something off the ground.
12 September 2025
From one piece Fully autonomous robots are much closer than you think – Sergey Levine 2 beliefs, in the piece's order there
-
Their words
I think that it's optimistically that it's actually the other way around that the robotics uh element of the equation will make all the other stuff better. And there are two uh reasons for this that that I could tell you about. One has to do with representations and focus.
-
Their words
Whereas with text, it's already sort of been abstract into those bits that we as humans care about. So the representations are already there and they're not just good representations. They actually like focus in on what really matters.
17 July 2025
From one piece Assessing Adaptive World Models in Machines with Novel Games (with 13 co-authors) 3 beliefs, in the piece's order there
-
korrents.com
Human adaptability is fundamentally linked to the efficient construction and refinement of internal world modelsTheir words
We argue that this profound adaptability is fundamentally linked to the efficient construction and refinement of internal representations of the environment, commonly referred to as world models, and we refer to this adaptation mechanism as world model induction.
-
Their words
However, current understanding and evaluation of world models in artificial intelligence (AI) remains narrow, often focusing on static representations learned from training on massive corpora of data, instead of the efficiency and efficacy in learning these representations through interaction and exploration within a novel environment.
-
korrents.com
An adaptive agent's world model cannot be a static representation learned once and fixedTheir words
For adaptation to complex and changing environments, an agent's world models cannot be static representations learned once and fixed.
29 June 2025
11 October 2018
From one piece BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding 2 beliefs, in the piece's order there
-
korrents.com
Pre-trained representations reduce the need for many heavily-engineered task-specific architectures.Their words
We show that pre-trained representations reduce the need for many heavily-engineered task-specific architectures.
-
korrents.com
Current techniques restrict pre-trained representation power because standard language models are unidirectional.Their words
We argue that current techniques restrict the power of the pre-trained representations, especially for the fine-tuning approaches. The major limitation is that standard language models are unidirectional, and this limits the choice of architectures that can be used during pre-training.
From one piece BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding 2 beliefs, in the piece's order there
-
korrents.com
Current techniques restrict pre-trained representation power because standard language models are unidirectional.Their words
We argue that current techniques restrict the power of the pre-trained representations, especially for the fine-tuning approaches. The major limitation is that standard language models are unidirectional, and this limits the choice of architectures that can be used during pre-training.
-
korrents.com
Pre-trained representations reduce the need for many heavily-engineered task-specific architectures.Their words
We show that pre-trained representations reduce the need for many heavily-engineered task-specific architectures.
From one piece BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding 2 beliefs, in the piece's order there
-
korrents.com
Pre-trained representations reduce the need for many heavily-engineered task-specific architectures.Their words
We show that pre-trained representations reduce the need for many heavily-engineered task-specific architectures.
-
korrents.com
Current techniques restrict pre-trained representation power because standard language models are unidirectional.Their words
We argue that current techniques restrict the power of the pre-trained representations, especially for the fine-tuning approaches. The major limitation is that standard language models are unidirectional, and this limits the choice of architectures that can be used during pre-training.
From one piece BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding 2 beliefs, in the piece's order there
-
korrents.com
Pre-trained representations reduce the need for many heavily-engineered task-specific architectures.Their words
We show that pre-trained representations reduce the need for many heavily-engineered task-specific architectures.
-
korrents.com
Current techniques restrict pre-trained representation power because standard language models are unidirectional.Their words
We argue that current techniques restrict the power of the pre-trained representations, especially for the fine-tuning approaches. The major limitation is that standard language models are unidirectional, and this limits the choice of architectures that can be used during pre-training.
Nothing matches.
What is a korrent?
A korrent is a belief a person has stated in their own words: one sentence stating the claim, backed by a quote and a source, kept at korrents.com.
Under a name here, the quoted block is what they actually said. The korrent beneath it is the claim those words support, in korrents' wording — tap it to see the record, its source, and who else holds it.
Nobody here wrote their own korrents. They are compiled from public statements, and a person can change their mind, which is recorded too.
About the English under a post
Some people here publish in a language other than English. Where they do, this site shows a machine translation beneath the post, in this typeface — the site's own, not theirs.
The post itself is never changed, moved or hidden: what is set in the serif above is exactly what the person published, and it is what to quote them on. A translation can be wrong in ways that matter, especially about tone.
Only the post's own words are translated. A quoted post, a linked article and a belief on korrents.com are left in their original language.