It's Starting to Get Strange (GPT 4 as a "first rate data analyst")

It's Starting to Get Strange (GPT 4 as a "first rate data analyst")(twitter.com)

21 points by byt143 3 years ago | 2 comments

The author notes that in the data analysis, ChatGPT hallucinates an incorrect fact…there seems like such a huge gulf between “ChatGPT can do 90% of a data analysis but some might be wrong” and “ChatGPT can do data analysis with great accuracy.”

It’s a difference between proof of concept and production ready software. Maybe these shortcomings can be addressed in a larger software ecosystem, but ChatGPT still feels pretty far from properly replacing human work—it’s just not reliable enough

jiggawatts 3 years ago | |

I tell my coworkers that ChatGPT is useful whenever you can verify the output. Don’t take its advice on topics you don’t understand yourself.

For example, I ask it to solve simple maths problems, then ask it to output the solution in Wolfram Language format so I can numerically verify it in Mathematica.

Similarly, I ask it to write simple scripts I could write myself but can’t be bothered to. I can understand the script and verify that it compiles and runs, so this is safe.

I don’t ask it for a step-by-step guide for brain surgery.