Ashish Vaswani

Co-Founder — Essential AI

Profile

Ashish Vaswani is the lead author of “Attention Is All You Need” — the 2017 paper that introduced the Transformer and quietly rewired the entire field of AI. If you’ve used GPT, Claude, Gemini, Stable Diffusion, or virtually any modern model, you’ve used his architecture. The paper has accumulated over 200,000 citations and counting, making it one of the most influential pieces of computer science research of the century.

Vaswani earned his PhD at USC under David Chiang working on statistical machine translation, then spent five-plus years at Google Brain where the Transformer was born. He left Google in 2021 to co-found Adept AI with fellow Transformer co-author Niki Parmar, then walked away from that company in late 2022 — reportedly over disagreements with investors. In 2023 he and Parmar started over with Essential AI, where Vaswani is now CEO.

Essential raised $56.5M in 2023 from a who’s-who of AI infrastructure — Nvidia, AMD, Google, and Thrive Capital among them. The pitch was building enterprise AI agents, but the company stayed quiet for two years before shipping anything substantial. In December 2025 they finally released Rnj-1, an 8B open-weights base + instruct pair trained on 8.4T tokens with the Muon optimizer (not AdamW), released under Apache 2.0. It hits ~20.8% on SWE-bench Verified — competitive with Gemini 2.0 Flash and Qwen2.5-Coder 32B at a fraction of the size — and was deliberately built with minimal post-training and no RL, betting that disciplined pretraining beats clever fine-tuning.

For developers learning AI: Vaswani is interesting precisely because he’s not a celebrity researcher. He doesn’t tweet much, doesn’t podcast, doesn’t evangelize. He shipped one paper that changed everything, then put his head down and started a company. Watch what Essential AI ships — when one of the people who invented the Transformer bets against the post-training-heavy industry consensus, it’s worth paying attention.

Key Articles & Papers

Attention Is All You Need 2017 — The paper that introduced the Transformer architecture. Every modern LLM descends from this. Self-Attention with Relative Position Representations 2018 — Follow-up work on positional encoding that influenced later Transformer variants. Image Transformer 2018 — Early work extending self-attention beyond text into image generation — a precursor to vision transformers. Tensor2Tensor for Neural Machine Translation 2018 — The open-source library where the original Transformer reference implementation lived. Stand-Alone Self-Attention in Vision Models 2019 — Replacing convolutions entirely with self-attention in vision models — pre-ViT exploration. Announcing Rnj-1: Building Instruments of Intelligence 2025 — Essential AI's first open release. 8B coding model, Apache 2.0, trained with Muon instead of AdamW.

Spotify Podcasts

Attention is all you need by Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan Gomez, Lukasz Kaiser, and Illia Polosukhin

36 - Attention Is All You Need, with Ashish Vaswani and Jakob Uszkoreit

EP7: Attention Is All You Need by Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser and Illia Polosukhin