It's not getting close to GPT-4. It's getting closer according to synthetic benchmarks but spend any non-trivial time with both and you'll quickly realise GPT-4 is still leagues ahead, especially for writing code and more complex reasoning. Which makes sense since the model is orders of magnitude larger parameter count wise.

Don't get me wrong, it's still remarkable that we already have LLMs that can be ran on consumer grade hardware that are anywhere near GPT-3.5/4 levels. But if you want the absolute highest quality of output GPT-4 is still way to go.

I have found it pretty decent at explaining math and physics concepts, and generating some basic code. It seems over-tuned for code generation (on purpose) as sometimes it inappropriately generates code when asking non-code questions.

Overall, it performs better than GPT-3.5-turbo in many use cases. Harder to quantify the GPT-4 comparison, as there are multiple versions of GPT-4 which are rumored to have significantly varied outputs.

It's definitely worth giving a shot. The parameter size of 34B makes a big difference, and it's been found that you're still better off extremely quantizing a larger-parameter model than using unquantized smaller models.

This post from today says that Llama 2 7b performs at the level of gpt-4, but 30x cheaper.

https://www.anyscale.com/blog/llama-2-is-about-as-factually-...

This is the proper way to use ChatGPT. Relying on it as a source is a bad idea. However, it's handy for finding reliable sources to read further.

It's not the tool that is dystopian, it's the mindset expressed in the post.

You can spoof your user agent from what I understand.

Yeah but if I'm doing research online I'm going to stick with sources I consider reliable, written by real people and not a text generator. So I'm not sure I understand the comparison you're trying to make.

you read sources and original works, did you actually think this was a gotcha? you are depriving yourself of proper learning and knowledge lol

The simple answer is don't rely on information you find on the internet, SEO killed that years ago. And chatgpt was trained on that awful mess. Go read an actual book

Stochastic parrots don't give you the original source. In fact they make up sources on the spot.

That is worse than a search engine.

> have you ever considered devoting your full attention to what you're reading

have you ever considered being less assumptive and judgemental? you have absolutely no insight into my reading comprehension ability, and have no idea what my process is like.

> you don't need to rely on chatgpt mangling the information into nonsense

except that doesn't happen? It only strengthens my understanding by allowing me to ask questions?

you really need to look at how you're approaching this conversation and calibrate. instead of this mess of assumptions and loaded questions, ask a real, open-minded question such as "what does your workflow look like? what are the pros/cons of this approach?"

if my system works for me, I don't need to prove it to you, however you yourself are missing out on a new style of research which will become incredibly common.

Imagine that the calculator breaks, and you need to do some quick math. (Yeah, yeah, I know. Cell Phones and such.) It is good to know how to do math without a calculator for those edge cases where one is not available, or when you can't do the equations that you need with it.

Calculators are good. Calculators are useful. Calculators accelerate your workflow beyond what your ancestors could do. Not knowing how to do math without one is still a hindrance, hence why we still need math classes. You need to know the underlying theory of why the calculators do what they do in order for them to be useful.

It's the same with ChatGPT. ChatGPT is a fantastic tool that can benefit your workflow. I use it all the time myself. However, being 100% dependent on it for work is a dangerous game. If it goes down(like today) or the company behind it makes a change that makes it less useful, you still need to know how to do your job without it. That's why OP's comment is worrying. They said that they feel unable to work without it. It reminds me of that Avengers quote: "If you are nothing without the Iron Man suit, you shouldn't have it."

These tools should be an extension of you, not a reliance.

See, I’d still be able to do my math without a calculator (to some extent)

Your comment above sounds like you get paralyzed when ClosedAi (OpenAi)s service have outage.

Why would you want to depend on a private corporation to this extent?

It blows my mind!

>The conceptual level at which I work

Oh come off it lol

The calculator is not a service but a tool. Until LLMS don't become just a tool don't rely on them too much or expect it be broken and have a workaround for that.

If your calculator breaks, it isn't hard to buy a new one, or borrow one, or revert to pen/paper.

the fact you even think they are comparable to a calculator is scary. they're not in any way comparable.

At least calculators don't give you plain wrong information

You're just projecting your own ignorance on the process.

I constantly learn new maths or programming while working alongside LLMs. They increase and augment my experience, they don't subtract from it.

You need a developed strategy when it comes to work and research, and if you already have a good one, adding GPT as a resource only helps you.

> But you should still know how to do it regardless. Because of situations like this. That is the point of my comment. If you can't do your job without ChatGPT, you don't have any business working in your field to begin with. Even if it's at a reduced speed, you still need to know how to do your job.

I'm currently using ChatGPT for a bunch of AI/ML that I don't know how the insides are working. But I'm able to build models from scratch that does exactly what I want, with 99% accuracy in my test cases, without actually knowing what the model does, but together with GPT4 + automatic hyperparameter tuning, I'm able to build models I can use in production.

Does it matter if I know exactly how everything inside in the model works, if I can get it to work exactly to my specification without it?

This is essentially how I started programming as well way back in time. I didn't know exactly what the Perl code I copy-pasted did, but if it solved the problem, it solved the problem. It brought me and my family out of poverty, and at that point I couldn't care less about how the magic actually was done, just that it did work.

Obviously now I have more knowledge about web field in general and 10+ languages that I no longer have to use any docs to be productive with, and maybe that'll happen with AI/ML eventually as well, but for a person who is starting with something new and wanna be productive quickly, GPT4 is a godsend.

> Not really. To do advanced stuff you have to understand the basics.

Doesn't that defeat the whole purpose of these abstractions? Can you read the machine code that your C compiler produces? How much of electrical engineering do you need to know to write a bash script? The physics of how a NAND gate is implemented?

It's obviously in the early stages, and I don't disagree with you completely today -- but this will just be one more layer on top of an already deep stack of abstractions that underlie all of computing.

There are so many things wrong with this argument that it will take me a while to list every one of them.

>And if you eventually can solve a problem by treating it as a black box, who cares

When the inevitable security bug gets introduced because you decided to be lazy, you will care. Black boxes make spaghetti code that is inherently buggy and insecure from the get-go. My current job is to clean up from programmers like you who don't give a crap about the code as long as it "just werks." Am I glad to have the job? Yes. Does it piss me off to see such blatantly terrible code? Also yes.

>It's not though, it's to illustrate that you can have a real-life impact using code that you don't understand at the time

Guess what? Programming lifted me out of poverty as well. I never use black boxes and never have. If I can't understand the stack overflow post, I don't use it. I find a solution I can understand. It's irrelevant and a logical fallacy. "well using shitty code helped me not be poor so it's good lmao!" Just no.

> That is irrelevant though, because no matter if it sucked or not, it worked and solved a real problem, which is the reason we (I at least) write code in the first place.

If writing spaghetti code causes more problems than it solves, it's not irrelevant. And guess what? It does. It might be years before these problems are revealed, but when that code gets exploited over something simple that a cursory understanding could have solved, then yeah. That's on you. It "just werks" is not a valid excuse.

> You seem to fall into the classical programmers trap of thinking that code has to be beautiful just to look at in order to be valuable, and anyone who disagrees is a shitty programmer and it's their fault you have to refactor some shitty code right now. It's not, they're not, and it's not their fault.

For one, ad hominem. Second, Beautiful code =\= good code. I have seen terrible code that was written beautifully sticking to a single programming style. I have seen great code that looked a little messy. I can tell when the programmer behind the code knew what they were doing or not. I like beautiful code, but I prefer secure code.

Also

>and it's their fault you have to refactor some shitty code right now.

It...literally is. They wrote it. If they write shitty code, and I'm the one that has to fix it, the blame falls on them for writing it in the first place without any quality in mind.