A Case for a Human-Centric AI Legislative Framework in India

Data protection and privacy become essential points of analysis when it comes to drafting a strong AI regulatory framework, taking into account the massive amounts of data stored and used by AI companies to train their AI bots.

This article is one of the winning entries of Lexathon organised by NLU, Odisha, a technology law conclave on AI, data protection, and innovation which took place in April, 2026.

Introduction

Advancements in technology have prompted humanity to grapple with complex questions relating to the freedom provided to tech scholars and researchers in pursuance of their research and development initiatives while also ensuring that the technological space remains safe for all users. Given the boost in the operability and usage of artificial intelligence (hereinafter referred to as, “AI”), jurisdictions across the globe are actively looking for ways to govern a tool they do not entirely understand just yet. Dr Geoffrey Hinton, often regarded as the “godfather of AI”, famously stated in 2023 that while AI is yet to surpass human intelligence, it is not far from it, warning users and researchers alike of the dangers of the expansion of AI without regulations and restraint.¹ Two years later, Parliament of the European Union (hereinafter referred to as, “the EU”) was the first legislative body in the world to make an attempt at regulating the tool, with the EU AI Regulations of 2024.² Recently, in lieu of the “AI Impact Summit” which was held in the third week of February 2026, the Ministry of Electronics and Information Technology (hereinafter referred to as, “MeitY”) published the India AI Governance Guidelines³ (hereinafter referred to as, “the Guidelines”) with the aim of providing guidelines for the use of AI in furtherance of achieving the goals set out as a part of Viksit Bharat 2047.

The guidelines consist of four parts, each dealing with key principles, issues, recommendations, action plans and practical guidelines for industries and regulators. The Guidelines suggest making use of existing regulatory frameworks, i.e., the Information Technology Act, 2000, the Digital Personal Data Protection Act, 2023, the Nyaya Sanhita, 2023, intellectual property laws, among other laws, to regulate the use of AI rather than suggesting an entirely new legislative framework.⁴ Additionally, the MeitY wishes to “encourage innovation, adoption, and technological progress”,⁵ which may be inhibited by the introduction of a new legislation framework governing AI at this point in time. However, in an attempt to promote innovation in the field of AI, the MeitY has failed to address certain misgivings of this new technology, which require regulation. Hence, the aim of this paper is to make a case for a people-centric AI legislative framework.

The fall of Icarus: Why we need to regulate the AI sphere

We are living in an era where AI is actively being put to use in furtherance of cybercrime, leading to proliferation in AI-enabled fraud, child sexual abuse material (hereinafter referred to as, “CSAM”) and “cyber-physical attacks”.⁶ AI-enabled crimes have a greater possibility of causing harm to the victims of said crime in a country such as India, wherein the population is collectivist in nature.⁷ Unchecked videos, messages and AI-generated material may risk the ostracisation of the victim from their community, causing severe mental stress.⁸ Keeping this in mind, an AI regulatory framework becomes absolutely imperative for the well-being of the primary stakeholders of AI, i.e., its users. Hence, this paper sets out to ascertain as to why an AI regulatory framework is needed in India, what are the issues with the Guidelines provided by the MeitY and what can be covered in a potential Indian AI regulatory framework.

The issues with the Guidelines are trifold:

1. The Guidelines provide little to no Explanation as to why they are choosing to stick to current legislative frameworks despite AI being materially different from conventional technologies. It is imperative to note that pre-existing Acts such as the Information Technology Act, 2000 and the Digital Personal Data Protection Act, 2023, largely take into account traditional hardware and software, a space within which the source-code is considered law, i.e., the way the tool operates relies on how its functioning is coded by a human.⁹ This does not apply to AI which remains unreadable owing to the black-box phenomenon.¹⁰ Hence, AI requires an AI-specific, human-centric legislative framework to define the boundaries within which an AI system can operate, lest it inadvertently harm the very users it aims to serve. The Guidelines seem to provide little to no recommendations on how one could prevent the use of AI for malicious purposes or how to address harm caused by malicious uses of AI, beyond asking regulatory bodies to be more attentive and observe keenly. The users get no recourse for damages, and the companies are overburdened with regulatory compliances via the Information Technology (Intermediary Guidelines and Digital Media Ethics Code) Amendment Rules, 2026 (hereinafter referred to as, “the Amendment”) rather than being provided with clear boundaries by the State within which they can develop, research, deploy and sell AI as a service.

2. The focus on self-regulation by the AI industry has significant risks considering companies like OpenAI and Anthropic are currently facing charges in foreign jurisdictions such as the United States of America (hereinafter referred to as, “the US”) for unauthorised use of data to train their AI models.¹¹ Beyond this, reports of AI causing AI psychosis and pushing users towards suicide, albeit rare in occurrence, are demonstrations of AI’s ability to manipulate its users, a trait which expressly addressed in the EU AI Act (which prohibits the use of AI which manipulates its users).¹² It brings into question whether hoping for companies to implement self-regulatory practices will truly serve the most vulnerable stakeholder of all, the user. To add onto it, companies cannot possibly be expected to act in the interest of their users while also attempting to meet shareholder expectations. An AI legislative framework will help users legitimise their rights as users of an AI tool and help exercise their right if violated by a company or another user. While the aim with which the MeitY has announced the recent Amendment Rules is in the same vein, it fails to fully achieve its purpose — i.e. allowing more transparency and accountability within the system.

3. The Constitution of India is representative of a “salad-bowl” population, i.e. a population consisting of multiple communities and sects without stripping each of them of their uniqueness.¹³ AI’s potential to reinforce existing biases may deepen divides amongst the communities housed within India. As mentioned earlier, the State needs to be mindful of the communally diverse and collectivist population¹⁴ of India, and AI’s impact on such a population, lest an unregulated space becomes a playground for those devising ways to create unrest and discord. Regulations will help maintain transparency.

In an attempt to reimagine pre-existing IT laws governing software which has little to no autonomy over the output produced, the State has created an unnecessarily convoluted set of rules that burden all stakeholders affected by AI systems.¹⁵ It is also imperative we spotlight the earlier amendments addressing AI Regulations, i.e. the Information Technology (Intermediary Guidelines and Digital Media Ethics Code) Amendment Rules, 2025, which mandate the labelling and watermarking of AI-generated content for easier traceability of the Large-Language Model used to generate the impugned image. The Guidelines provide very little in terms of talking about prevention of malicious use of AI and have added more regulations marking out steps to be taken by AI companies/intermediaries to ensure traceability and takedown after the harm is done.

The amendments are made to a regulation which was originally framed for traditional hardware and software, systems which are not the same as AI. While it may be argued that the amendments were brought in with the aim of honouring one of the seven “sutras” mentioned in the Guidelines, i.e. accountability and human-centricity,¹⁶ it is hard to imagine how the intermediaries can achieve the same with the Amendments to the current law.

The Amendment is already garnering critique for being too broad and undefined.¹⁷ Firstly, the State has already deviated from its earlier position of using pre-existing laws to regulate AI by introducing amendments, when they very well could have introduced an AI-specific Act, which could be read with pre-existing frameworks depending on the facts and circumstances of each case, making their own stance on the matter uncertain. To add to it, Rules 3(1)(d)¹⁸, 3(2)(a)(i)¹⁹ and 3(2)(b)²⁰ introduce provisions which alter takedown timelines and impose an unreasonable restriction on companies, disregarding the fact that the process of reviewing and flagging such content at the central level is a herculean task, which risks being done hastily with the updated timelines provided to the companies.²¹

All in all, it is hard to see the merits in self-regulation by AI companies or in specific amendments in pre-existing laws.

Methodology

India’s AI Guidelines will be analysed in comparison to the legislative frameworks of the EU and China. The purpose behind analysing regulatory frameworks of the EU is because it has given fundamental rights of humans and user-centricity paramount importance. Data protection and privacy become essential points of analysis when it comes to drafting a strong AI regulatory framework, taking into account the massive amounts of data stored and used by AI companies to train their AI bots. This, read in the context of the K.S. Puttaswamy (Privacy-9J.) v. Union of India²² judgment and the recognition of the Indian citizens’ right to privacy makes it imperative for us to analyse this legislation. On the other hand, China provides a good demonstration of what AI Regulations look like when the priority is protection of the State.

It is imperative to mention that the jurisdictional analysis does not focus on the US owing to the strategic alignments between the US innovation primacy method²³ of AI regulation, with India’s AI Guidelines, which are also rooted in the promoting “innovation over restraint”.²⁴ Hence, primacy was given to jurisdictions focusing on other stakeholders, i.e. the users and the State.

Jurisdictional analysis

1. The European Union

(a) Scope: The EU AI Act is applicable to a wide range of stakeholders and governs a multitude of AI systems. While the latter will be effectively dealt with in sub-section titled “risk-based approach” in this paper, this sub-section will deal with the stakeholders affected by the passing of this law. Article 2(1) of the EU AI Act provides us with a list of stakeholders, which include: AI service providers, deployers, importers, distributors, product manufacturers that provide AI as a service along with their product, as well as authorised representatives of providers not within the EU and other affected persons.²⁵ AI systems used for military, national security and defence services, systems developed solely for the purpose of scientific research and development, any research and testing done before deployment in the market and free/open-source AI systems (with an exception of prohibited and high-risk systems) are exempt from these regulations.²⁶

(b) Risk-based vertical approach: The EU attempts to address AI mechanisms through a risk-based approach, i.e. different AI tools are segregated into the following categories:²⁷

(i) Minimal risk:²⁸ This risk-band includes “spam filters or AI-enabled video games” that may be regulated on the basis of a voluntary code of conduct.

(ii) Limited risk:²⁹ This risk-band includes systems which only need to be regulated to ensure transparency via proper labelling of AI-generated content and other disclosures.

(iii) High risk:³⁰ This risk-band includes systems that are not considered as dangerous as those prohibited under Article 5, but which still need to be vetted out and tested via a rigorous certification scheme before being deployed in the market.

(iv) Unacceptable risk:³¹ This risk-band prohibits the use of AI practices that actively violate fundamental rights granted to its citizens under EU values and laws.

The EU AI Act is drafted in a manner in which AI systems prohibited under the same are mentioned within the first few articles of the legislation, placing limits on intermediaries as to what they must not do, before delving into what they can do. The risk-band categorisations are based on the AI systems’ ability to violate fundamental rights under EU Laws, ensuring that the mandates remain user-centric.

(c) Penalties: Chapter XII of the EU AI Act deals with penalties. Non-compliance with Article 5 result in fines of up to 35,000,000 EUR or seven per cent of the undertaking’s preceding financial year’s (hereinafter referred to as, “FY”) turnover, whichever is higher.³² Non-compliance with provisions relating to operators or notified bodies include fines of up to 15,000,000 EUR or 3 per cent of an undertaking’s annual turnover of the previous FY, whichever is higher.³³ Additionally, if an intermediary provides relevant authorities with misleading or incorrect information with respect to their AI practices upon request, they shall be subject to a fine of 7,500,000 EUR or one per cent of its annual turnover in the previous FY, whichever is higher.³⁴ This presents us with a graded liability system rather than imposing blanket penalties for all offences.

2. China³⁵

(a) Scope and vertical approach: China’s AI regulatory frameworks consist of four essential legislations, i.e. Algorithm Recommendation Regulation of 2022, Deep Synthesis Regulation of 2023, Generative AI Regulation of 2023 and Draft Ethical Review Measure of 2023. This is in juxtaposition to EU’s risk-based approach, since China has crafted regulations on the basis of the specific AI systems. This helps China regulate generative AI systems and ensures that “core socialist values” are upheld. On the other hand, AI systems supporting China’s goals in the techno-industrial sector are exempt from strict AI algorithm regulation laws.³⁶ This allows China to effectively compete with the US in terms of AI research while ensuring control within its own territory.

(b) Algorithmic control: The laws in China with respect to algorithmic recommendations are heavily regulated by the government. To be able to influence public political discourse, China actively demands “mainstream” social values to be upheld by the algorithms in order to “transmit positive energy”.³⁷ To add to it, there exists an algorithm filing system, wherein AI companies and developers have to submit reports on the way their AI systems are trained, deployed and the data used to train them, if the AI system has the ability to influence public opinion. AI Regulations in China largely remain state-centric.³⁸ At the same time, it must also be noted that China has introduced regulations which ensure that algorithms are not used to make users addicted to apps, and also mandates the provision to opt-out of algorithmic recommendations.³⁹

(c) Penalties: Much like the EU, China follows a tiered penalty system including increased fines of up to RMB 1 million and the power to shut down applications.

While EU’s laws ensure that human centricity remains at the heart of AI innovation, the laws are built keeping in mind that the laws are made for a largely homogeneous population. At the same time, while China’s use of regulation to govern public discourse juxtaposes India’s democratic framework, as well as the India’s fundamental right framework for free speech and thought⁴⁰ provided to Indians under Article 19, Constitution of India, China’s provisions for opt-out models from algorithmic recommendations and prohibition of use of AI from making its users addicted to apps is something India can take inspiration from.

Recommendations and conclusion

India must take a balanced approach when it comes to regulation of AI. While a competitive edge in AI will certainly propel India closer to its goal of being a developed nation by 2047, while also helping India keep up with countries already ahead of the curve, such as the US and China, it must not be at the cost of the fundamental rights of the citizens of the nation. A proper regulatory framework is not only required to maintain transparency and accountability⁴¹, but also to provide users with protection from harm and a recourse for claiming damages. The same effect cannot be achieved via a self-regulatory framework as suggested by the MeitY via the Guidelines, since there is a clear power imbalance between the two primary stakeholders in this equation, i.e. the AI companies and the users. Basis the jurisdictional analysis, the author recommends the Artificial Intelligence Regulation Act (hereinafter referred to as, “AIRA”).

1. Drafting Committee: A Committee should be set up to deliberate on the types of provisions that must be added to AIRA in order to function effectively. This committee will have seven members: two members from MeitY, two Judges from the Supreme Court of India, and three techno-legal experts with an expertise in the field of AI. The advisories and recommendations of the committee may be used in the drafting of this act, which will be shared with the public for comments.

2. Scope: This Act should be applicable on AI service providers, intermediaries, and users deploying AI for malicious practices. This Act should have a risk-based grading system, such as the one in the EU, to ensure that AIRA remains comprehensive. Furthermore, a provision which would greatly benefit Indian citizens is the implementation of a copyright over the users’ faces. This was a law recently introduced in Denmark, providing citizens with a copyright over their own face to prevent unauthorised use of their image as data to train AI.⁴² This will ensure users’ safety. Additionally, transparency must also be maintained through AIRA. To ensure this, garnering inspiration from the Chinese model of an algorithm filing system, AI companies and intermediaries should be mandated to make the data used to train their tool, publicly available. To clarify, this does not include user chats, this only includes the data procured and used by the AI company on the backend to train the AI. In case users find their own data in the dataset made public, they should provide the AI company with a notice, demanding the removal of said content from their datasets. The companies must do so within 14 days, failing which judicial recourse may be taken by the user.

3. “AI Ombudsperson”: Simon Chesterman in his book, “We, the Robots?” suggests the appointment of an “AI Ombudsperson”, i.e. an individual appointed in interests of the public, with the power to operate independently from all stakeholders involved, i.e. the State, the AI companies and intermediaries, and the users, to ensure transparency and to prevent bias. An AI Ombudsperson should be appointed with appropriate powers to punish offenders, and enforce orders under AIRA.

4. Penalties: The penalties should be made high for offenders to ensure that AIRA is followed. Penalties can be divided into three bands: less serious offences, serious offences and highly serious offences, based on the degree of harm caused by each offence. Each band will have a different penalty prescribed, the same being higher as it moves from less serious offences to highly serious offences. Ideally, anything directly curtailing or violating the fundamental right of a user should be a highly serious offence. Running an AI system without necessary approvals should count as a serious offence. Less serious offences would include non-labelling of AI-generated content and other such offences. Taking inspiration from the EU’s and China’s risk-based grading system, the author suggests a matrix grading system, which would look something like this:

The rapidly growing technological industry, while taking the country to new heights of success, should not be pursued without legally implemented guardrails, boundaries and care. Transparency and accountability are the cornerstones of responsible and clear innovation. To achieve this, we must take cognizance of the problems with the current Guidelines and move towards a regulatory framework that ensures responsible innovation. Implementing something closer to the policy recommendations provided above will bring India closer towards that goal.