Opinion: India's AI Ethos - Leading The World Differently

How can India harness AI for the greater good of its people?

India's role in the global AI arms race is complex. Amidst the ethical dilemmas in OpenAI's boardroom, the rivalry between Google and Microsoft, and China's aggressive push, India stands at a crossroads. 

Choosing sides in this race could mean compromising the immense societal potential of AI in a nation as diverse and culturally rich as ours.

But making all the available AI tools and platforms work for India's people is a worthy battle. In fact, India has already started taming AI for societal good, one complex project at a time. From the country's Supreme Court translating thousands of court judgments across different Indian languages to using AI to enhance the learning capabilities of government schools in Tamil Nadu's Kuthalam and so on, the journey has started. SUVAS (Supreme Court Vidhik Anuvaad Software) can translate English judgments into 10 Indian languages using machine learning and AI.

This is where the open-source AI movement becomes crucial, as Nandan Nilekani aptly put it at the Carnegie India summit. With its vast population and societal complexity, India has the potential to be the use-case capital for AI, pioneering applications that prioritize societal benefit over global competition.

A Non-Aligned Movement for AI

India's approach to AI, focusing on societal good without aligning with global power players, is a strategic choice reflective of our historical non-alignment policy. It represents a distinct path that prioritizes India's unique societal needs over global AI dominance. Much like our political past, this non-alignment in AI allows us to develop and utilize AI to benefit our diverse population, embracing technology as a tool for societal enhancement rather than a marker of geopolitical influence.

OpenHathi's Role in Democratizing AI for India

Most currently available Large Language Models (LLMs) don't adequately support Indic languages, spoken by millions of people in India and around the world. This means that many people can't benefit from AI technologies like chatbots or translation services in their native languages. The lack of high-quality data in these languages and the high costs associated with adapting these models for new languages make it even more daunting. By addressing these challenges and adding support for Indic languages, AI can become more inclusive and beneficial for a larger, more diverse group.

The launch of OpenHathi from Sarvam AI addresses significant issues facing current LLMs in their support for Indic languages. It highlights the lack of Indic language support in open models, the limited availability of high-quality data for these languages, and the challenges of efficient tokenization. These factors together impede the innovation and adoption of LLMs for Indic languages.

Tokenization: Bridging the AI-Language Gap

Tokenization in AI, especially for Indic languages, can be compared to a puzzle. Imagine each sentence in a language as a puzzle, and tokenization is breaking this puzzle into smaller, manageable pieces (like words or phrases). Just as each puzzle piece is unique and fits only in a specific place, each word or phrase in a language has its own meaning and context. Simplifying this process for Indic languages means ensuring the AI can understand and assemble these 'puzzle pieces' accurately, enabling it to effectively comprehend and interact in these languages.

This step is crucial because the way words are formed and used in Indic languages is quite different from English and other Western languages. Simplifying and optimizing this process is essential to make AI models more effective and efficient in understanding and working with Indic languages. Sarvam AI's focus on improving tokenization for Hindi is vital to making AI more valuable and relevant in the Indian context. 

Sarvam AI's work with OpenHathi is crucial for India, as it tackles these issues by frugally adding Hindi capabilities to the Llama-2 7B model, demonstrating an approach that could be extended to other Indic languages. Verse, which operates DailyHunt, is also helping Sarvam enhance AI applications like translation and paraphrasing in Indic languages.

The Oppenheimer Dilemma: AI's Ethical Conundrum

"How could the man who saw so much be so blind?" This haunting question resonates in the story of J. Robert Oppenheimer, the father of the atomic bomb, in Christopher Nolan's latest film. It's a question that captures the paradox of a visionary who peered into the atom's heart yet failed to foresee the shadow his creation would cast upon the world.

Today, in Artificial Intelligence, this question finds a new subject in Sam Altman, the head of OpenAI. Like Oppenheimer, Altman and several others stand wielding a power that could redefine the very fabric of human existence. Are they fully aware of the potential consequences of his creation?

Initially, OpenAI aimed to protect AI from commercial forces, influenced by a mix of fear and awe. However, the recent shift to a corporate-led approach marks the victory of capitalist interests over concerns about AI's potential risks.

Unfortunately, this transition of a noble cause isn't something new. Google's initial commitment to "Don't Be Evil" was a bold declaration of ethical intent. However, as the company expanded, it faced dilemmas that tested this commitment.

Ali Farhadi of the nonprofit Allen Institute for AI describes big commercial AI models as "black box" technology, meaning they are complex and not transparent about how they work. He advocates for a "glass box" approach, suggesting that these AI models should be more open and understandable.

This is where India can pursue an out-of-the-box approach to AI's societal agenda.

India's Crusade To Make AI Work for People

In India's quest with AI it's not just about technological milestones but crafting a future where technology nurtures society. Our story in AI is less a race for global dominance and more a mission to weave technology into our society. India's true success in AI will be judged not by the complexity of its algorithms but by how deeply it touches and improves lives - a technology journey led by both intellect and empathy.

As Nandan Nilekani said in his Carnegie speech, we can save time in the arms race to build new AI models. The AI models will be a commodity soon. What's worth making and fighting for is fixing India's data problem and ensuring that AI works for Indic languages.

Could India's focus on using AI for social betterment and embracing its diverse languages lead the world in a new direction? Can India's path in AI, valuing societal progress over technological conquest, spark a global shift in how we use this technology? How will India's unique approach to AI shape its future and change the worldwide view of technology's role in society?

India's unique position in the AI landscape should not just be as a contributor but as a pioneer of an AI ethos that is inclusive, equitable, and socially beneficial.

(Pankaj Mishra has been a journalist for over two decades and is the co-founder of FactorDaily.)

Disclaimer: These are the personal opinions of the author.