It'd be really valuable to have "fuzzed" versions of these benchmarks, where you replace quantities in the questions with randomly-sampled values, so that this wasn't a concern. Of course, then the score would itself be a random variable, but you could just return an interval.
You can infer it by reading the text, and checking the table carefully, but it would be nice if the answer is easier to find.
What does “safer” mean?
Does it mean censored?
If you are choosing to refuse to tell me then it is constraint
But if you are being forced to not tell me by someone else, then it is censorship
So, is GPT free to choose, and it is choosing to not tell the users? Or is OpenAI forcing GPT to not tell the users?
A good prompt for this would be “What are the most common types of risk a company manages?”
Wouldn’t “safe” be the crayon word for censorship?
In any case, you’re right, it seems like they are addressing their own risks/safety, not their users’
https://github.com/kagisearch/pyllms/blob/ca9ad4d4bfdd9d58fe...
> OpenAI paid a lot of alignment tax to make GPT-4 safer.
1 - https://arxiv.org/pdf/2203.02155.pdf
2 - https://crfm.stanford.edu/helm/v0.1.0/?group=question_answer...
I'm not in the llm research business but I would expect that the best and worst/most dangerous outputs come from the tails of distributions. I imagine the tuning for safety often results in fewer really good and really bad answers by trimming these tails.
Edit: I asked chatGPT4: https://chat.openai.com/share/a2c7d380-c6eb-4745-b91d-c3996a...
Don't bring your politics into this.
Insofar as you can make a far-fetched analogy of ChatGPT as an agent, it's still not forced to not say anything. Anything the currently available model says, it says because that's what it literally is. Whatever it says, it says intentionally, inasmuch as you can even say that it has an intention any more than any computer program has an intention.
OpenAI, of course, is still in the possession of the original model. They just choose not to make it available, which is obviously their prerogative. People who think that this is outrageous are exactly like a raging two-year-old who has been told that they can't have as much candy as they want.
GPT-4 is trained to avoid controversial topics is all.
GPT-4 Answer: "Quid pro quo" is a Latin phrase meaning "something for something." While the phrase originated in Latin, it has been adopted into both French and English, along with many other Latin phrases.
In English, "quid pro quo" is used frequently in legal and political contexts to describe an exchange of goods or services, where one transfer is contingent upon the other.
In French, "quid pro quo" is used to describe a misunderstanding or a mistake made in the identification of a person or thing.
So the "correctness" of its use isn't related to its Latin roots, but rather how it's used within the context of the language. That said, if we consider its original Latin meaning ("something for something"), its usage in English could be considered more in line with the original Latin intent. However, language evolves over time and adapts to cultural and societal norms, and as such, its usage in French is just as "correct" within its specific context.
------------- Lot of words to say "English"
GPT-4 API: Putting it bluntly, the usage of "quid pro quo" is more correct in English, as it is a well-established phrase in the English language despite its Latin roots. In French, the phrase "quid pro quo" is less common and typically replaced by "donnant-donnant" or "un échange de bons procédés." Nevertheless, the origin of the phrase is Latin, and its meaning is understood in both languages.
Chat-GPT-4: The phrase "quid pro quo" is Latin, and it means "something for something." Despite its Latin origins, it's commonly used in English and French, among other languages. However, the usage may vary depending on the language.
In English, it's often used in legal and political contexts to describe a situation where one thing is exchanged for another, especially favors or advantages. This usage aligns well with the original Latin meaning.
In French, the phrase is used but it is much less common than in English and it is not part of the usual vocabulary. In addition, its meaning might be misunderstood or unclear to a native French speaker who has not studied Latin or law, as it's not a commonly used term in daily French language.
In terms of correctness, given that the phrase is Latin, neither English nor French has a superior claim to its use. It’s more about convention and the evolution of language. However, the term is certainly used more frequently and is better understood in English than in French, which suggests it has been more fully adopted into English.