AI is all you need

2025/01/19

Reading time: 5 minutes

The AI present and future

There’s no escaping the buzz around AI(artificial intelligence) these days. And yet, it’s hardly a couple of years before the pandemic, that Big Tech was relying heavily on human-input to fine-tune their models, something that used to be called the last mile problem of AI. e.g. Amazon was using its infamous Mechanical Turk to crowdsource cheap human labor for tasks like verifying AI inputs or labeling data.

Attention is all you need
The pivotal moment came in 2017 yet again from Google (a pioneer in big data/MapReduce/Hadoop research) when it published its transformer architecture paper. True to its engineering roots, Google was able to process data in parallel using the transformer’s neural network architecture, an approach which was made possible by the self-attention innovation of transformer. Essentially, transformer allowed the models to take into account the entire context of the sentence, rather than just the preceding words. This vastly improved the contextual understanding of the models. This led to a revolution in natural language processing (NLP) by AI models, helping them understand context and meaning in language by increasing orders of magnitude.

ChatGPT
AI fully burst into the hype cycle with the announcement of ChatGPT, by the end of 2022, developed by OpenAI. For the first time in human history, an AI model was able to generate coherent, human-like text, engaging in useful conversations and even creating art/literature/music on-demand from simple text-based prompts. The full-form of ChatGPT is Chat Generative Pre-trained Transformer, it being trained from a vast corpus of publicly available text, images and other media (audio, video) across the internet, including but not limited to the wikipedia, books, public social media. The phenomenon of ChatGPT highlighted the potential and importance of AI as a key technology differentiator not only in business but also across countries e.g. US vs China.

The (Generative) AI race
It can definitively be said that ChatGPT spurred the AI race (Generative AI and more), similar to how the iPhone circa 2007 led to the development of the smartphone/touchscreen technologies. More advanced language representation models (encoders as well as decoders) continue to be developed. e.g. BERT in 2018, Modern BERT in 2024. All the key big tech players have jumped onto the bandwagon, as well as notable startups:

OpenAI: The stable which provided ChatGPT has been consistently setting benchmarks with newer models like GPT-4/o/mini. Has a working partnership with Microsoft.
Microsoft: Has been a fast and nimble mover in the AI space, bringing its CoPilot suite of products into its Office, M365, Azure platforms in record time-to-market.
Google: From the behemoth, Google Gemini quickly reversed the gaffes of its predecessor Bard and is developing multi-modal AI.
Anthropic: US based AI competitor of OpenAI, known for its AI models Claude.
Mistral: Based in Paris, founded by former engineers from Google DeepMind and Meta, its models have been at the forefront of the AI race.

Of chips and shovels
The shortage of chips had already been a major issue since the pandemic slowdown, and the evolution of AI led to increasing demands in advanced hardware, notably AI chips. With keen anticipation and deft execution, Nvidia has emerged as the tearaway dominant force in this space: its GPUs powering almost all of recent AI applications, supercomputers and applications.
Chips are shovels in the AI gold rush
As the software companies have raced to build even more capable models, trained on yet more millions and billions of features and parameters, Nvidia continued selling shovels for this gold rush.

DeepMind + Google v/s. OpenAI + Microsoft
The London based DeepMind was acquired by Google in 2014, almost a decade earlier and soon after taking the reins of Google, CEO Sundar Pichai focused on the company becoming AI first in its approach. Yet for all its foresight, Google was caught offguard by the ChatGPT development, and the rapid pivot of Microsoft in sync with OpenAI to launch its CoPilot products integrating AI into Office and daily products used by the business world. True to Satya Nadella’s words, it would be interesting to see if the elephant (Google) can dance. Whether it can or not, Google surely has a trick up its sleeve with DeepMind. As the co-founder and CEO, and 2024 Nobel laureate in Chemistry (for protein structure prediction using AI models), Demis Hassabis talks about in this interview with Dr. Hannah Fry, the future goals of DeepMind are certainly commercially applied research, leading to AI products for Google.

Charismatic leaders or not
The following leaders, charismatic or not, are key to look out for in the AI race:

We need to also look out for the AI tigers or dragons of China which is rapidly advancing as an AI superpower. As always, innovation can come from anywhere in the world!

Towards AGI?
While the definition of AGI(artificial general intelligence) still remains fuzzy, for many (including Hassabis), we’re on the cusp of developing proto-AGI in the AI race. Discounting for AI model hallucinations, which should progressively become rare as models and engineering improve even further, and internet search plugged in, it would be a wonder if the AI models of today won’t be able to pass the Turing test at least. Whether we attain AGI in the next few years or not, it’s certain that the AI race will certainly have massive and far-fetching impacts not only on the tech and business worlds but also societal and economic implications, with key questions about productivity, jobs, pensions, meaning which need to be answered.

References:

The rapid evolution of AI