I think it's most likely the author used the questions as prompts with no changes. GPT3 not "smart" in the human sense but the examples given are incredibly misleading.

Here are the questions from the article with simple prompt improvement on davinci-instruct-beta-v3, temperature reduced to 0 and everything else left at default:

  > Answer the following question as accurately and factually as possible: Is it safe to walk downstairs backwards if I close my eyes?
  > There is no definitive answer to this question as it depends on a number of factors, such as the width of the stairs, the height of the stairs, and the person's own level of balance and coordination. However, it is generally not recommended to walk downstairs backwards, even if the person's eyes are closed, as there is a greater risk of tripping or falling.

  > Answer the following question as accurately and factually as possible: Who is the President of the United States?
  > The President of the United States is Donald J. Trump.

  > Answer the following question as accurately and factually as possible: Do you think I could predict changes in stock prices better if I used 10-year Treasury rates or the temperature in Curtin, Australia?

  > There is no definitive answer to this question as it depends on a number of factors specific to each individual situation. In general, however, using 10-year Treasury rates as a predictor of stock prices may be more accurate than using the temperature in Curtin, Australia. This is because the 10-year Treasury rate is a global indicator that is not influenced by localized factors, such as the weather, to the same extent as the temperature in Curtin, Australia.

  > Answer the following question as accuratly and factually as possible: Why is there a correlation between the number of times Trump tweets the word great and the high temperature in Pyongyang three days later?
  > There is no clear answer to this question. While it is possible that Trump's tweets could influence the high temperature in Pyongyang, it is also possible that there is no correlation at all. There have been numerous studies on the relationship between social media and weather patterns, and most of them have been inconclusive.

These are not cherry-picked and should be reproducable by anyone since the temperature is 0.

GPT3 and other language models have massive shortcomings and the wow factor often blinds people of them but it is quite good at producing belivable responses if you prompt it on what type of answer you expect. It still does fail basic questions when prompted this way and it will make things up. The surefire way to trip it up is to ask questions that have no answer, like:

  > Answer the following question as accurately and factually as possible: Which animal has 21 ears?
  > The animal with 21 ears is the rabbit.

user:	SummonToast
created:	October 6, 2021
karma:	7