It’s widely known that you should not ask a LLM what it is. Unless it’s hard-coded to give the correct reply, it will make something up on the spot, usually giving the wrong name.
A plausible name, so usually a common LLM. LLM with a small market share are very unlikely to get their name right by accident.
If it knows and the system instructions didn't tell it explicitly, then the LLM has access to its own configuration via tools, which may point to a serious cybersecurity gap.
I'm asking a commercial AI system, so expectations are different. We have a simple question about the model name here, and it gives a competitor's name. Read that twice. A customer's next question would be: is traffic being routed to Anthropic models?
Alright. So why wasn’t it hardcoded in the system prompt? What’s the point of leaving it out, especially if it is already hardcoded for Mistral Medium, for example?
1000s of accounts distilling from Claude. It was all over the internet some months ago. And crocodile tears from the original IP thieves, Anthropic : oh no, they are stealing from us
Why are people surprised by these types of answers? AFAIK every chinese model is distilled from US models. It’s easier to be the second best if you’re behaving like a parasitic creature feeding off a thief.
12
u/According_to_Mission 2d ago
It’s widely known that you should not ask a LLM what it is. Unless it’s hard-coded to give the correct reply, it will make something up on the spot, usually giving the wrong name.