Ask HN: How did LLMs change your products?

14 points by scoresmoke 2 years ago | 8 comments

Since the LLMs and other generative models had been around for a while, many companies claimed to build products on top of them or launch new AI features based on them.

What happened at your company? How did these features impact your users, your product, and your management?

Paul-Craft 2 years ago |

I'm currently unemployed and looking for work, but I can say I have seen more than one job listing that mentions LLMs, and even one that specifically mentioned GPT-4 as part of the company's tech stack. So, people definitely are integrating LLMs into their products.

scoresmoke 2 years ago | |

It's all about timing and communities. In my echo chamber, everybody is currently making them a part of the product, but I'm interested in a more representative sample. Hiring usually precedes shipping.

Every success in your job search!

Paul-Craft 2 years ago | | |

Thanks! I appreciate it. :)

crazy_marksman 2 years ago |

Made it easier to document code. You can send the function as context, and it will generate a well-formatted docstring that accomplishes 90% of the task!

wscp-dev 2 years ago |

well... My company was aiming to do codegen to fix errors. We took quite some time to figure out what to do with AI in our product. When we finally implemented something, we found apis like gpt4 to be quite expensive, though GPT4 did work well for the task. We later turned towards llama2 and codellama, which worked decently but had a higher "wrong output" rate.

And then they laid off 60% of the workforce (We were only 30 people)... I am not sure LLMs are going to affect their outlook very positively in the long term.

scoresmoke 2 years ago | |

I have seen that many teams are trying to perform model routing by sending simpler queries to cheaper models and complex queries to more complex and expensive models, e.g., https://www.anyscale.com/blog/a-comprehensive-guide-for-buil...

Were these people laid off because of LLMs? It sounds pretty dystopic. Who were these people?

shortrounddev2 2 years ago |

Made it way easier to generate the hundreds of lines of boilerplate crap you have to write with typescript backends

scoresmoke 2 years ago | |

I found code LLMs to be very useful for rewriting nested for-loops in Python as nice vectorized operations in NumPy. One has to be careful about unexpected array materialization, but generally it works really well and saves a lot of time.