BLOOMChat, a 176B parameter, Multi-lingual, fine tuned chat(huggingface.co) |
BLOOMChat, a 176B parameter, Multi-lingual, fine tuned chat(huggingface.co) |
When I asked which is more 1.7 or 120, it said 1.7 is greater number and then started spewing complete garbage math how 1.7 - 120 = 60 and since 60 is more than 0 then 1.7 is more than 120.
Utter garbage
>>> Write a program that calculates if 120 is greater than 0.7.
>Sure, here's a program in Python that calculates if 120 is greater than 0.7:
if 120 > 0.7:
print("Yes, 120 is greater than 0.7")
else:
print("No, 120 is not greater than 0.7")
I would expect the models to be bad at, say, division of long numbers in the same way humans are bad at doing the same calculations in their head!
I'd be more interested in having the thing "study maths" in the sense of seeing many examples of Wolfram Language being used in matching English context. That way, it would learn the English->Maths mapping in a format that it can then feed into an industrial-strength mathematics engine. Apparently, Stephen Wolfram is working on this now, but the "full" fine tuning training would require significant funding and time. I suspect OpenAI has other priorities right now, but this type of thing will eventually become a routine way of making specialised and/or more capable large language models.