The Cultures That Calibrate AI

When an AI system tells you that a response is helpful, someone decided that. Not the algorithm. A human being, sitting somewhere, rated that response as good or not good, complete or incomplete, appropriate or not. That decision, repeated across millions of interactions, is what shapes the behavior of AI systems now running in classrooms, clinics, courtrooms, and government offices around the world. And those human raters are not a representative cross-section of humanity. They come from a specific cultural context, with specific assumptions about what knowledge looks like, what a good argument sounds like, and what it means to give a helpful answer.

This is not an accusation of bad intent. It is a structural observation: every AI system trained on human feedback inherits the cultural assumptions of its raters. Those assumptions shape not just what the system says, but what it considers worth saying at all.

Who Rates the Responses

The process of training AI behavior through human feedback is well understood in research circles. What gets far less attention is who, exactly, is doing that work. Large-scale rating and labeling operations rely heavily on contractors working in English, in a small number of countries, using rubrics that reflect particular ideas about what good communication looks like: that being direct is a virtue, that a confident answer signals accuracy, that a well-structured response with a clear argument and supporting evidence is simply better than one organized around community authority and relational context.

Consider what this means in practice. An AI system asked about land governance will score highly if it produces a clean, three-paragraph summary. A response that first establishes who, within a given community, holds the authority to speak on that question at all will likely be rated as incomplete or evasive. The first kind of answer reflects a Western approach to information: find the fact, state it clearly. The second reflects an Indigenous approach to knowledge: understand the governance structure before claiming to know anything. After training on millions of these preference judgments, the system learns that one way of knowing is correct and the other is a failure to communicate.

Recently, my colleagues and I proposed a conceptual framework at the Symposium on Artificial Intelligence for Cultural Heritage and Indigenous Futures (AI4CHIEF 2026), where we document that Indigenous knowledge systems are systematically treated as raw data rather than recognized as governance structures with their own authority and internal logic. Our forthcoming paper, Gattupalli et al. (2026), argues that Indigenous knowledge does not function like a dataset. It functions more like a governing document: it defines not just what is known, but who has the right to interpret it, how that interpretation must be validated within the community, and under what conditions any action can follow. When AI systems take the content of that knowledge and discard its governance layer, the result is not inclusion. It is distortion dressed as representation. I will update this article with a link to the full paper as soon as it is published.

When AI Gets It Wrong

The cultural narrowness described above produces real errors. AI systems have been observed correcting the spelling of proper names that are spelled correctly in their source languages, converting them into anglicized versions that are easier for English speakers to read. AI tutoring platforms have flagged explanations rooted in oral knowledge traditions as incomplete, because they do not include citations formatted for print-based academic culture. Language models have described ceremonial practices as “belief systems,” a label that implicitly contrasts them with factual knowledge, a framing that many communities would find not just inaccurate but disrespectful.

None of these outputs came from malicious intent. They came from training pipelines where the people doing the rating simply did not know what they did not know. The result is a technology that speaks with equal confidence about things it understands and things it has quietly misrepresented. The communities affected by those misrepresentations have no clear way to contest them. They cannot flag the training data or correct the model weights. They encounter the output and know, without any formal path to appeal, that something important has been lost.

Cultural Intelligence Is a Design Requirement

The central question is not what AI systems can produce. It is who holds the authority to decide what a valid, appropriate, or helpful response looks like. Right now, that authority sits with a small number of companies, operating within a small number of cultural frameworks, making decisions that affect billions of people who were never part of that conversation.

This matters well beyond communities whose knowledge has been directly misrepresented. A system calibrated within a narrow cultural range will, over time, teach all of its users to think within that range. It will reward the reasoning patterns it was trained to recognize and quietly disadvantage the ones it was not. It will reinforce, at global scale, the idea that one grammar of thought is simply correct.

Cultural intelligence, in this context, is not a soft skill or a nice addition. It is a design requirement. Building AI that genuinely serves a diverse world means building the institutions, feedback mechanisms, and governance structures that allow many communities to help define what the system is actually trying to measure. Until that happens, the cultures that calibrate AI will continue to do so. And everyone else will continue to adapt to a standard they never set.

References

Gattupalli, S., Chakravarty, P., Chakravarty, U., Chand, G., & Arroyo, I. (2026). Indigenous environmental knowledges as governance infrastructure for climate AI in climate resilience and adaptation. The Symposium on Artificial Intelligence for Cultural Heritage and Indigenous Futures 2026 (AI4CHIEF). (forthcoming)

Latulippe, N., & Klenk, N. (2020). Making room and moving over: Knowledge co-production, Indigenous knowledge sovereignty and the politics of global environmental change decision-making. Current Opinion in Environmental Sustainability, 42, 7–14.

Perera, M., Vidanaarachchi, R., Chandrashekeran, S., Kennedy, M., Kennedy, B., & Halgamuge, S. (2025). Indigenous peoples and artificial intelligence: A systematic review and future directions. Big Data & Society, 12(2).

Cite this article

Gattupalli, S. (2026). The cultures that calibrate AI. Society and AI. https://societyandai.org/perspectives/cultures-that-calibrate-ai/

Write for Society & AI

We welcome contributions from educators, researchers, librarians, policymakers, and practitioners examining the intersection of artificial intelligence, education, and society. All work published on Society & AI is made available free of cost and in open access, in service of education and the public good. Write to us at submissions@societyandai.org.