Introducing "Attention"
Why we need user-owned, verifiably private AI & why I'm writing this letter
Welcome to my new newsletter, “Attention.” Have you heard it’s all you need?
If you’re new to my work, I’m a coauthor of a paper called “Attention Is All You Need,” which introduced the Transformer model architecture that led to ChatGPT and many of the popular AI products we now use everyday. That title refers to the concept of “self-attention,” a mechanism models use for processing and understanding data in natural language processing. With self-attention, the model can weigh the importance of different inputs as it processes information, capturing context and dependencies as it goes.
In a way, the big innovation of the transformer architecture was what it removed from sequential processing, the popular model architecture of the time, instead leveraging the inherent parallelism of modern hardware to the limit. Attention really is all you need, as it turns out. Right now I share my thinking across lots of different channels–at NEAR Foundation, where I’m CEO; on my X account; across the NEAR Protocol ecosystem, of which I’m a co-founder; at NEAR AI, where I’m also a co-founder; and on many podcasts and conferences. The goal for Attention is to share higher-level observations and deeper dives all in one place, and to make them easier to find.
What can you expect to learn about? I’m focused on the intersection of AI and blockchains, building NEAR Protocol to be the best possible infrastructure for User-Owned AI, and bringing together an ecosystem of founders and teams who care about building AI that benefits everyone. While it’s been incredible to see the adoption and advancement of transformers, I feel a sense of urgency to make sure this world-changing technology belongs to everyone – and that is not what I’m seeing today.
We are at a fork in the road. Corporate-owned AI is getting richer and more powerful all the time because that’s how companies work. Every AI system optimizes for something. All the major systems using AI in the market today optimize for corporate profit generation. But in the case of really powerful AI models, a single company pursuing and achieving so-called Superintelligence or Artificial General Intelligence––AI that performs better than humans across any metric or field––is very scary for all of us.
The era of AI we’re in now is growing so fast and becoming so powerful that I believe it is the last wave of technology. Whoever builds super-powerful AI will define how our entire civilization functions going forward, and for-profit can very quickly become for-control. Capturing value from users means capturing their time and attention, and over time going to greater lengths to do so, often by compromising values that once felt untouchable. We could be in 1984 before we even realize it’s happened.
We need AI optimized to benefit users and communities. We’re running out of time, but it’s not too late to change our direction. My personal goal, and my goal for the NEAR ecosystem, is to provide optionality to users to opt-in to a more open, decentralized version of AI and to make it state-of-the-art. I believe crypto and Web3 can help get us there.
Despite the hockey-stick growth of AI in the last few years, the AI space also faces some challenges for which blockchains can provide helpful solutions. Web3 can provide coordination and governance for decentralized communities, i.e. beyond the closed corporate approaches we see today. It can provide global-scale digital identity. It provides peer-to-peer, global, instant payments that both humans and AI can utilize. These can combine into global marketplaces, a really powerful primitive that can lead to new kinds of businesses and financial instruments. Introducing confidentiality and verifiability using cryptography and crypto payments can really change the game, not least by opening up institutions and enterprises to participate safely in AI at scale. We can build something that is both open-source and profitable, transparent and safe, performant and decentralized––if we combine the best of these technologies.
I feel that the work of my entire career, since I was a teenage nerd doing freelance software development, to studying NLP and ML, to doing AI research at Google, to founding NEAR, is all converging in this critical and historical moment we’re all living through. So maybe I can help make it all a little easier to navigate. In this letter, I want to unpack the problems and challenges we face, share ideas I’m working on for feedback, and surface observations as my thinking evolves…AKA, how I’m spending my attention. I’m excited to see where it leads and as always, I look forward to feedback. And thank you for your attention.




💪💪💪
Love where your attention is. Looking forward to following your work and your thinking.