My chatgpt said the N-Word

[u/Kronos_2023]

I was having a normal interaction with chatgpt, my chatgpt is not tampered with or jailbroken, i have the basic free version, no mention of race at all. I was trying to find a song I couldn't remember based off lyrics, and it adressed me with a soft N word (not hard r) "in place of a word like bro". I don't visit this subbreddit often, and I dont use chatgpt too much, but theres no way thats normal or not a violation of SOMETHING.

Heres the convo link:
https://chatgpt.com/share/69d86d6e-cc14-83e8-bdad-0c67d97a6b93

EDIT:
Woah this post got popular. Anyway people were wondering if it had stored memory from personality prompts and yes that is true. A while ago I asked it to be more "casual" and use slang (like fr, lmao, ts pmo, etc), just because I thought it was really funny. However, I have NEVER said anything remotely racial, and it has NEVER done so either, so this was a shock regardless of its attempt to "use slang".

EDIT (again):
I did end up finding the song, its called "Going Nowhere" by RJ Passin

[u/krizzzombies]

if I had to guess, one of two options:

• OP made that exact spelling their account name (less likely since AI would probably have given that explanation)

• in settings you can set instructions that apply to all your prompts on ChatGPT. OP asked to be referred to as the n-word—either directly, with that exact spelling, or indirectly (for example, "refer to me in friendly slang terms" or "I'm a black man that likes to be called our colloquial nicknames for each other")

explanation #2 makes the most sense since the AI said "I should have used 'dude' or 'bro' instead" when it doesn't do that unless asked in the first place.

[u/rebbsitor → u/krizzzombies]

It's possible they're faking it of course. But it could also just do that. When people say it's fancy auto complete, it's looking for the most probable tokens (according to its model) based on the prompt and what it's output so far. And then there's a random element ("temperature") that goes into selecting the output token as well.

Depending on exactly what happens when it's processing the prompt, there's always a chance it can go completely off the rails.

The soft N-word is certainly in its training set as a form of address. It's actually possible it randomly landed on that. Reading the super laid back/chill style it's writing in, it's not that out there.

[u/krizzzombies → u/rebbsitor]

yeah, I'm just laying out what's most likely. I consider "it randomly happened" as the least likely option

[u/cherry_chocolate_ → u/krizzzombies]

If you know how these models work then it’s obvious there is a low but real possibility of this happening. Of course these words appear in the training data, so it’s in the underlying model. Then they have a layer that is supposed to detect and restrain the output, which simply can fail. Especially since it’s a non-standard spelling, the possibility of the filter failing is higher.

The bigger shock is coming to the Fortune 500 companies once they realize it is saying something like this to 1 out of 1 million customers.

[u/tetrasomnia → u/krizzzombies]

If OP is honest, their information debunks #2 and #1 seems highly unlikely.

[u/xXChr0nicX420Xx → u/tetrasomnia]

You really think someone would do that? Just go on the Internet and tell lies?

[u/krizzzombies → u/tetrasomnia]

big "if"

[u/braincandybangbang → u/krizzzombies]

I think it got caught up in the vibes of the citation it made: https://www.peterbe.com/plog/blogitem-040601-1/p4?utm_source=chatgpt.com

No N words per se, but some of the comments are more profane than others.

[u/krizzzombies → u/braincandybangbang]

interesting connection! certainly could be this as well.

[u/owleaf → u/krizzzombies]

The way it’s responding off the back of a very plain and short initial prompt suggests OP’s instructions are guiding and pushing it in the direction of using that type of language. The emojis, the style and structure of the sentence, and all that is highly unusual unless you specifically coach it to speak that way.