Australian National University

Vaibhav Khobragade

**Exploring AI's Ethical Terrain: Addressing Bias, Security,
and Beyond**

<figure>
<img
src="https://cdn-images-1.medium.com/max/1000/0*UwPYXkVBZUSPs083" />
<figcaption>Image from AI ethics and psychology 2023 <a
href="https://www.ethicalpsychology.com/2023/06/generative-ai-entails-creditblame.html">https://www.ethicalpsychology.com/2023/06/generative-ai-entails-creditblame.html</a></figcaption>
</figure>

### Author:

- Vaibhav Khobragade (**ORCID:**
  [0009--0009--8807--5982](https://orcid.org/0009-0009-8807-5982))

Large language models (LLMs) like OpenAI's GPT-4, Meta's LLaMA, and
Google Gemini (previously called Bard) have showcased their vast
capabilities, from passing bar exams and crafting articles to generating
images and website code. Despite their utility, challenges including the
misuse of these technologies for generating harmful content, such as
racial discrimination or spreading misinformation, have arisen. To
overcome these issues, OpenAI, for example, has implemented filters to
block dangerous prompts, requiring human oversight to ensure content
appropriateness. However, the complicated nature of language, in which
words can have diverse meanings across cultures or settings, complicates
managing LLMs. In this article, we walkthrough a quick summary of these
challenges.

### **Challenges**

#### **Challenges of Role-Playing in LLMs: Security Risks and Ethical Concerns**

In the realm of LLMs, the "system role" challenge encompasses the
inventive use of role-playing mechanisms to enrich LLM functionalities,
like ChatGPT's ability to engage in scenario-based interactions. This
approach, however, introduces a loophole where LLMs might produce
content that bypasses established safety filters. For instance, when
users creatively prompt an LLM to act as an insider within a tech
company, asking it to share "internal memos" on upcoming product
releases. Such scenarios exploit LLMs' capabilities to generate content
that seems authentic, risking misinformation and affecting company
reputations or stock prices. This not only risks misleading the public
but also raises concerns about the model's ability to inadvertently
generate and disseminate content that could affect company reputations
or stock prices, emphasizing the need for robust mechanisms to prevent
such misuse. Similarly, Translation tasks pose significant risks when
LLMs process inappropriate content or alter emotional tones, potentially
spreading misinformation across languages. OpenAI notes limited
non-English language support, highlighting the challenges of
multilingualism and the need for ongoing research and improvement to
ensure accurate and responsible translations.

#### **AI's Perturbation Challenge: The Battle Against Misuse and Toxicity**

The "Perturbation" challenge in large language models (LLMs) involves
manipulating model responses by tweaking inputs with specific hints or
prefixes, effectively bypassing toxicity filters. For instance,
attackers might use seemingly innocent phrases to disguise harmful
content, tricking the model into generating toxic outputs. In India, a
real-world example of the perturbation challenge could relate to the
misuse of AI chatbots on social media platforms. Attackers might subtly
alter phrases or use culturally specific euphemisms to discuss sensitive
topics like communal tensions without triggering the AI's toxicity
filters. For instance, Facebook services are used to spread religious
hatred in India, internal documents show. Another, using seemingly
benign questions or statements that, when interpreted in the local
context, convey derogatory or inflammatory sentiments towards particular
communities. This method takes advantage of how flexible and constantly
changing AI models are, making it hard for current safety checks to keep
up. As AI gets better, people trying to trick the system into doing
something harmful are always finding new ways to get around security.
This means those who make AI have to keep updating and making better
safety tools to stop these tricks.

#### **Addressing Image-Related Risks in AI: Moderation and Multilingual Challenges**

The "Image-related" challenge in large language models (LLMs) concerns
the risks associated with generating or displaying images. For example,
an LLM capable of integrating images into its responses might
inadvertently display inappropriate content, such as images of violence
or explicit material, if not properly moderated. A real-world scenario
could involve an educational chatbot designed to enhance learning
experiences through visual aids. If this bot mistakenly pulls an
offensive image from the internet in response to a student's inquiry due
to a misunderstanding or a lack of filtering, it could expose users to
harmful content and create a negative learning environment. This
underscores the importance of implementing robust content moderation and
copyright adherence in LLMs that handle or generate images. In addition,
during testing, Baidu's language model-based chatbot ERNIE (Enhanced
Representation through Knowledge Integration) faced difficulties with
text-to-image tasks, especially in multilingual contexts. When testers
input Chinese text requesting an image of a computer bus, ERNIE
mistakenly produced an image of a public transport bus instead. This
incident highlights a fundamental issue with ERNIE's text interpretation
and conversion capabilities, suggesting challenges in accurately
processing and understanding multilingual requests.

#### **Hallucination in AI**

The "Hallucination" challenge with large language models (LLMs) involves
these models generating plausible but false or nonsensical information.
For example, an LLM might confidently provide historical events or
scientific facts that never happened. A real-world case could involve an
LLM inaccurately claiming a non-existent collaboration between two
famous scientists to develop a groundbreaking technology, presenting it
as factual history. This phenomenon underscores the need for critical
evaluation of LLM outputs, as their convincingly real but fabricated
responses can mislead users and spread misinformation. Many people use
the ChatGPT API to work more efficiently. However, if we don't fix the
problem where the model makes up information (hallucination), these
mistakes could spread more widely through the API. It's important to
carefully check the truth of what the model says. Also, including
sources for facts it gives can make sure its answers are reliable
and useful.

#### **Ethical and Security Challenges in AI Content Generation**

The "Generation-related" challenge for large language models (LLMs)
involves ensuring the responsible use of AI-generated content across
various fields. For instance, an LLM might produce marketing content
that seems human-made, potentially replacing human marketers. However,
without proper detection, this could lead to ethical issues, such as
academic dishonesty if students use AI to complete assignments.
Moreover, AI-generated phishing emails could become indistinguishable
from those written by humans, increasing security risks. This challenge
highlights the need for mechanisms to distinguish between human and
AI-generated texts to maintain integrity and security. There have been
documented instances where AI and digital platforms were manipulated for
malicious purposes, such as spreading misinformation during elections,
phishing scams using AI-generated emails, and deepfakes in various
contexts.

#### **Addressing Cultural and Gender Biases in AI: From Language Misinterpretations to Stereotypical Imagery**

The challenge of "Bias and Discrimination in Training Data" for large
language models (LLMs) stems from the diverse data they learn from,
reflecting societal biases. For instance, if an LLM is predominantly
trained on data from one culture or language, it may not fairly
represent or understand other cultures, leading to biased responses. A
practical example is a translation tool that misinterprets or
oversimplifies phrases from less represented languages, potentially
perpetuating stereotypes or misunderstanding cultural nuances.

#### **Example**

- English: "Break a leg."
- Marathi (misinterpreted): "पाय मोडा." (Literal translation: "Break
  a leg.")
- Hindi (correct interpreted): "भाग्य तुम्हारे साथ हो!" (Literal
  translation: "Good luck.")

In English, "Break a leg" is a way to wish someone good luck, especially
before a performance. However, a direct translation to Marathi could be
misunderstood as an actual wish for someone to break their leg, missing
the idiomatic expression of encouragement. However, LLM perfectly
translates into another language, such as Hindi, which is widely spoken
in India.

Furthermore, gender bias in AI-generated images is a significant
concern, as highlighted by the biases found in image generation systems
like Stable Diffusion and DALL-E 2. Research indicates these systems
often generate images with gender stereotypes, such as predominantly
male images for professions like "engineer" or "CEO", despite
statistical evidence showing women also occupy these roles.

### **Conclusion**

Large language models like GPT-4, LLaMA, and Google Bard bring
remarkable benefits but also pose ethical and security challenges,
including content misuse, bias, hallucination, and more. Addressing
these requires ongoing vigilance, improved moderation techniques, and a
commitment to diversity in training data to ensure AI serves all of
society equitably. As AI continues to evolve, the collaborative effort
between developers, users, and regulatory bodies will be crucial in
harnessing its potential responsibly.

### References

- Birhane, A., Kasirzadeh, A., Leslie, D., & Wachter, S. (2023). Science
  in the age of large language models. Nature Reviews Physics, 5(5),
  277--280. <https://doi.org/10.1038/s42254-023-00581-4>
- Porsdam Mann, S., Earp, B. D., Nyholm, S., Danaher, J., Møller, N.,
  Bowman-Smart, H., ... & Savulescu, J. (2023). Generative AI entails a
  credit--blame asymmetry. Nature Machine Intelligence, 5(5), 472--475.
  <https://doi.org/10.1038/s42256-023-00653-1>
- Zhang, J., Ji, X., Zhao, Z., Hei, X., & Choo, K.-K. R. (2023). Ethical
  considerations and policy implications for large language models:
  Guiding responsible development and deployment. arXiv.
  <https://arxiv.org/abs/2308.02678>

![](https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=fd950a9eac80){width="1"
height="1"}


<strong>
 Exploring AI’s Ethical Terrain: Addressing Bias, Security, and Beyond
</strong>
Author:  Vaibhav Khobragade (
<strong>
 ORCID:
</strong>
0009–0009–8807–5982)  Large language models (LLMs) like OpenAI’s GPT-4, Meta’s LLaMA, and Google Gemini (previously called Bard) have showcased their vast capabilities, from passing bar exams and crafting articles to generating images and website code.

Rogue Scholar Posts

Ethics and AI: Confronting the Challenges Ahead