What public figures publish and believe, in their own words.
About this feed
Highlights: posts that did unusually well for the person who wrote them, everything
they published at length, each release and new project, and every belief — at most
two a day from anyone. Day by day, newest
day first; within a day, the people with the most beliefs on this site come first. Nothing
else orders it. Show everything instead.
The quoted blocks are what people actually said; a beneath one is
the belief those words support, in korrents' wording. Nobody here wrote their own page.
Top people are the people in this feed with the most beliefs on this site, then the
most here. Choose an area and the row leads with the people whose beliefs are about it;
tap a face for their feed.
Hopefully I've convinced you that chatbots are a terrible interface for LLMs. Or, at the very least, that we can add controls, information, and affordances to our chatbot interfaces to make them more usable.
I want to see more tools and fewer operated machines - we should be embracing our humanity instead of blindly improving efficiency. And that involves using our new AI technology in more deft ways than generating more content for humans to evaluate. I believe the real game changers are going to have very little to do with plain content generation.
Good tools let the user choose when to switch between implementation and evaluation. When I work with a chatbot, I'm forced to frequently switch between the two modes.
But modularity may also facilitate a shift away from a concentration of model development in a few institutions and to distributing the development of modular components across the community.
The biggest issue is probably that we don’t control neutral networks enough to be able to ensure AI doesn’t harm humans. We can’t even control AI to not reveal internal prompts.
This book is my best crack at explaining what I think is an existential risk to liberal societies and what I think we need to do to get to that awesome future I used to be so excited about.
What ChatGPT is, in this instance, is replication as travesty. ChatGPT may be able to write a speech or an essay or a sermon or an obituary but it cannot create a genuine song.
I think this matches the general finding that AI progress is faster than expected, and increases my certainty that scale and normal progress can sometimes be enough to solve even very difficult problems.
But in terms of what we value the most as humans, which is to say our feelings, our emotions, our sense of what the world is in a very personal way that I think means as much or more to people than their information processing. And that’s where I don’t think that AI necessarily will become conscious because I think it’s the property of life.
The best and easiest-found-by-optimization algorithms for solving problems we want an AI to solve, readily generalize to problems we’d rather the AI not solve; you can’t build a system that only has the capability to drive red cars and not blue cars, because all red-car-driving algorithms generalize to the capability to drive blue cars.
For example, large language models can generate outputs that are untruthful, toxic, or simply not helpful to the user. In other words, these models are not aligned with their users.
This is because the language modeling objective used for many recent large LMs-predicting the next token on a webpage from the internet-is different from the objective "follow the user's instructions helpfully and safely" (Radford et al.,, 2019; Brown et al.,, 2020; Fedus et al.,, 2021; Rae et al.,, 2021; Thoppilan et al.,, 2022). Thus, we say that the language modeling objective is misaligned.
Even though InstructGPT still makes simple mistakes, our results show that fine-tuning with human feedback is a promising direction for aligning language models with human intent.
Science journalist Jacob Ward warns about a future in which AI technology forms a tight cybernetic feedback loop with the biases ingrained into the human brain, pushing our day-to-day existence in frightening directions. It's sort of like a version of The Terminator in which Danny Kahneman plays a starring role. As longtime readers and listeners of mine know, a big theme in my techno-criticism is the under-appreciated degree to which technologies exert powerful, unintended consequences on our personhood and culture, so Ward's warning hits a sweet spot with me.
I I would I would not say that about Google. I would just say that more generally like that is the incumbent temptation. Um it is is basically is basically go from attack mode to defend mode, right? Go from being the pirate to being the navy.
I wouldn’t say that this is the most compelling book I’ve ever read in terms of the prose style or storytelling, but it does provide a very helpful, almost quantitative overview of all the potential threats looming out there
AI was in the air in the mid 1980s, but there were two things especially that made me want to work on it: a novel by Heinlein called The Moon is a Harsh Mistress
If you're curious how Google built it's culture and the processes thanks to their extremely data & engineering driven nature, this is a great book. What held it back was being ~100 pages too long and Bock didn't always understand where what they did only fit at a $100Bn+, mega-profitable company. Still, the insights on the studies on their 50,000+ employees is well worth the read since so few of us can get similar statistical significance to apply to our teams.
The interests of a Venture Capitalist are different than those of the entrepreneurs building a company they've invested in. Jeff does an awesome job of helping explain how you can get misaligned in your goals versus your investors. Fortunately, he also covers how to avoid it.
this channel is full of high production quality and well-written videos. There are also several series on machine learning, the topics underlying self-driving cars, so this is certainly one of the more applied math channels out there.
Here we show that scaling up language models greatly improves task-agnostic, few-shot performance, sometimes even reaching competitiveness with prior state-of-the-art fine-tuning approaches.
This book stretched my mind more than almost any other book I’ve read. It’s tough at parts, it’s long, but you’ll come out of it thinking about brains, minds, intelligence, and AI in an entirely new way.
As generative drug analoging grows in importance, it's going to be crucial for people to make the entire training set available in detail when such work is published.
spending an increasing amount of my free time working with tools like Google Colab, Runway ML, TensorFlow, and Hugging Face to play around with machine-generated text, audio, imagery, and video.
We argue that ARC can be used to measure a human-like form of general fluid intelligence and that it enables fair general intelligence comparisons between AI systems and humans.
When I'm writing articles etc that need to be shared with an editor, I typically write in Microsoft Word. I hate all word processing software, and anyone who insists that Google Docs is so much better than Word is a liar.
It's increasingly coming to the fore and Marcus deals with it in a really interesting way. He takes us through lots of different areas where AI is being used to mimic creativity, which some people would argue is the essence of being human. It's interesting to see how close to being human AI is getting.
so we use Google Meet speakermics, which allow for full-duplex conversations, which makes conversation far more organic. That device turned Google Meet from something exhausting to something really good.
In terms of iOS apps, I use PDF Expert, which is a great PDF reader and annotator (using Apple Pencil), Notability as a virtual whiteboard (again with Apple Pencil), Google Docs for document editing, Google Calendar, and Gmail.
Eric Schmidt, Jonathan Rosenberg and Alan Eagle, Trillion Dollar Coach: The Leadership Playbook of Silicon Valley’s Bill Campbell: I understood Bill Campbell was a behind-the-scenes guy in Silicon Valley, but I had no idea just how influential he was. Bill Gurley of Benchmark noted “I would argue that Bill has had a bigger impact on Silicon Valley than any other single person simply because his reach was so amazingly wide.” That’s a story I want to read.
For coding, I use Visual Studio Code. I use Git for source control, Photoshop for 2D art, Audition for audio editing, Premiere for video editing, and our team coordinates on Google Docs and Skype.
What we are actually listening to is human limitation and the audacity to transcend it. Artificial Intelligence, for all its unlimited potential, simply doesn’t have this capacity.
Now, almost everything happens in Google Docs, except for when I write straight into the Axios or Substack CMS. (Both of which are lovely, WYSIWYG, very simple and easy to use.)
Such restrictions are sub-optimal for sentence-level tasks, and could be very harmful when applying fine-tuning based approaches to token-level tasks such as question answering, where it is crucial to incorporate context from both directions.
We argue that current techniques restrict the power of the pre-trained representations, especially for the fine-tuning approaches. The major limitation is that standard language models are unidirectional, and this limits the choice of architectures that can be used during pre-training.
We argue that current techniques restrict the power of the pre-trained representations, especially for the fine-tuning approaches. The major limitation is that standard language models are unidirectional, and this limits the choice of architectures that can be used during pre-training.
Such restrictions are sub-optimal for sentence-level tasks, and could be very harmful when applying fine-tuning based approaches to token-level tasks such as question answering, where it is crucial to incorporate context from both directions.
Such restrictions are sub-optimal for sentence-level tasks, and could be very harmful when applying fine-tuning based approaches to token-level tasks such as question answering, where it is crucial to incorporate context from both directions.
We argue that current techniques restrict the power of the pre-trained representations, especially for the fine-tuning approaches. The major limitation is that standard language models are unidirectional, and this limits the choice of architectures that can be used during pre-training.
We argue that current techniques restrict the power of the pre-trained representations, especially for the fine-tuning approaches. The major limitation is that standard language models are unidirectional, and this limits the choice of architectures that can be used during pre-training.
Such restrictions are sub-optimal for sentence-level tasks, and could be very harmful when applying fine-tuning based approaches to token-level tasks such as question answering, where it is crucial to incorporate context from both directions.
To me, this book is an illustration of the power of names. Today, in the era of Google, if you know the name of something, you can find out about it with a simple search. But if you don't know of what you're looking for, it suddenly becomes much harder to find it. Having in the back of your head the names of common algorithms that help you solve problems is really powerful.
Google Chrome: What's a Web Developer without Google Chrome browser? The first thing I do with Safari is to get Google Chrome and install the latest version.
Neural network pruning techniques can reduce the parameter counts of trained networks by over 90%, decreasing storage requirements and improving computational performance of inference without compromising accuracy.
Neural network pruning techniques can reduce the parameter counts of trained networks by over 90%, decreasing storage requirements and improving computational performance of inference without compromising accuracy.
In particular, this means that incredibly powerful AI will emerge in a world where crazy stuff is already happening (and probably everyone is already freaking out).
The main disagreement is not about what will happen once we have a superintelligent AI, it’s about what will happen before we have a superintelligent AI.
One of the most commonly emphasized points in the AI safety literature is that one must never anthropomorphize AI—that is, don’t project human mental properties onto artificially intelligent systems.
A superintelligence whose goal system is even slightly misaligned with ours could, being far more powerful than any human or human institution, bring about human extinction for the very same reason that construction workers routinely slaughter large populations of ants.
For writing code, I use emacs and Google's internal distributed build system (a version of which was open sourced as Bazel) and our version control system
A korrent is a belief a person has stated in their own words: one
sentence stating the claim, backed by a quote and a source, kept at
korrents.com.
Under a name here, the quoted block is what they actually said.
The korrent beneath it is the claim those words support, in
korrents' wording — tap it to see the record, its source, and who
else holds it.
Nobody here wrote their own korrents. They are compiled from public
statements, and a person can change their mind, which is recorded too.
About the English under a post
Some people here publish in a language other than English. Where they
do, this site shows a machine translation beneath the post, in
this typeface — the site's own, not theirs.
The post itself is never changed, moved or hidden: what is set in the
serif above is exactly what the person published, and it is what to
quote them on. A translation can be wrong in ways that matter,
especially about tone.
Only the post's own words are translated. A quoted post, a linked
article and a belief on korrents.com
are left in their original language.