Łukasz Kaiser
One of the eight authors of the 2017 transformer paper "Attention Is All You Need", then at Google Brain; per the paper's own contribution note, designed and implemented parts of tensor2tensor, the codebase that replaced the team's earlier one.
Łukasz Kaiser did not write this page. ppll compiled it from their own words, each linked to the source. Is this you? · Report a mistake
What they believe
-
Sequence transduction can use a simple architecture based solely on attention, without recurrence or convolutions.
We propose a new simple network architecture, the Transformer, based solely on attention mechanisms, dispensing with recurrence and convolutions entirely.
arxiv.orgSaid 12 Jun 2017This on korrents
The bold line is korrents’ wording; the quote is theirs. These one: what they wrote before what they said aloud, one per source, newest first.
Beliefs
Korrents What they believe 3 beliefs — each backed by an exact quote.
Each is a — compiled by korrents.com, not by them: the one-line wordings are korrents', the quotes are theirs.
Recent
Sequence transduction can use a simple architecture based solely on attention, without recurrence or convolutions.
We propose a new simple network architecture, the Transformer, based solely on attention mechanisms, dispensing with recurrence and convolutions entirely.
Attention is all you need Said 12 Jun 2017
A transduction model can rely entirely on self-attention for input and output representations without sequence-aligned RNNs or convolution.
To the best of our knowledge, however, the Transformer is the first transduction model relying entirely on self-attention to compute representations of its input and output without using sequence-aligned RNNs or convolution.
Attention is all you need Said 12 Jun 2017
Self-attention can yield more interpretable models.
As side benefit, self-attention could yield more interpretable models.
Attention is all you need Said 12 Jun 2017
Beliefs others hold too
Sequence transduction can use a simple architecture based solely on attention, without recurrence or convolutions. 8 hold this
We propose a new simple network architecture, the Transformer, based solely on attention mechanisms, dispensing with recurrence and convolutions entirely.
Attention is all you need Said 12 Jun 2017
A transduction model can rely entirely on self-attention for input and output representations without sequence-aligned RNNs or convolution. 8 hold this
To the best of our knowledge, however, the Transformer is the first transduction model relying entirely on self-attention to compute representations of its input and output without using sequence-aligned RNNs or convolution.
Attention is all you need Said 12 Jun 2017
Self-attention can yield more interpretable models. 8 hold this
As side benefit, self-attention could yield more interpretable models.
Attention is all you need Said 12 Jun 2017
What is a korrent?
A korrent is a belief a person has stated in their own words: one sentence stating the claim, backed by a quote and a source, kept at korrents.com.
Under a name here, the quoted block is what they actually said. The korrent beneath it is the claim those words support, in korrents' wording — tap it to see the record, its source, and who else holds it.
Nobody here wrote their own korrents. They are compiled from public statements, and a person can change their mind, which is recorded too.
About the English under a post
Some people here publish in a language other than English. Where they do, this site shows a machine translation beneath the post, in this typeface — the site's own, not theirs.
The post itself is never changed, moved or hidden: what is set in the serif above is exactly what the person published, and it is what to quote them on. A translation can be wrong in ways that matter, especially about tone.
Only the post's own words are translated. A quoted post, a linked article and a belief on korrents.com are left in their original language.