Show HN: Prompt Engineering Studio – Toolkit for deploying AI prompts at scale

2 points by retrovrv 1 year ago | 4 comments

Hey HN! Long-time reader, occasional commenter here. I wanted to share something my team and I have been building to solve our own frustrations.

We've all seen the explosion of prompt engineering tools lately. While playing around in playgrounds is fun, when we tried to take our AI prompts to production, we hit a wall. I'm guessing many of you have experienced similar pain points.

We kept hitting questions nobody seemed to be answering: How do you version control thousands of prompts? How do you handle multiple production deployments? How do you scale from prototype to millions of requests per day? How do you collaborate across hundreds of engineers without stepping on each other's toes?

So we built Portkey's Prompt Engineering Studio - a complete toolkit designed specifically for productionizing AI prompts across 1600+ models.

Some technical details that make our approach different:

- High-performance infrastructure: We've deployed prompts as large as 500,000 tokens with production-level latency - Git-like version control with instant rollbacks for prompt deployments - Mustache templating system for parameterization and reusable snippets - Publish/release flow with proper dev/staging/prod environments - Real-time analytics tracking prompt performance, latency, and token usage - Native integrations with Langchain, Llamaindex, and Promptfoo

The scaling capabilities have enabled some impressive use cases:

- A content company running 500+ prompts across 700+ websites - A tech firm that cut deployment times from 3 days to near-instant - Education platforms with hundreds of non-technical creators building AI workflows

Our platform has processed hundreds of millions of prompt completion requests already, with over 10,000 prompts deployed to production environments.

We think the HN community will especially appreciate our approach to bringing software engineering best practices to AI development!

You can try it yourself at prompt.new

I'd genuinely love to hear how others in the community are handling these challenges, what you think of our approach, or any other feedback you might have. This community has been invaluable in shaping how we think about developer tools.

veilgen 1 year ago |

This looks like a powerful step toward making prompt engineering more scalable and production-ready. The version control approach, along with staging environments and real-time analytics, seems particularly useful for teams handling high-volume AI workloads.

One question: How do you handle prompt drift over time? As models evolve, prompt effectiveness can degrade—do you provide any automated testing or monitoring to detect when a deployed prompt needs adjustment?

Looking forward to exploring Portkey’s capabilities.

retrovrv 1 year ago | |

thank you! we don't have a strong Evals module within our Prompt Studio at the moment. So there's no straightforward way to do that. however, we do have one of the better modules for applying guardrails on live AI traffic and setting up routing based on guardrail verdicts: https://portkey.ai/features/guardrails

JTyQZSnP3cQGa8B 1 year ago |

> How do you version control thousands of prompts?

Kill me now.

> How do you collaborate across hundreds of engineers

What do you mean by that? This only targets a few big companies.

> A tech firm that cut deployment times from 3 days to near-instant

That's a process and maybe CI issue, I don't see how AI would improve any of that but I'll be gladly proven wrong.

> You can try it yourself at prompt.new

All I see is a login page from another company. Don't you have a web site with all those serious prompting you do?

retrovrv 1 year ago | |

lol. i get you - i think the 3rd point you shared - it's not exactly about CI issue - CI is already as fast as it can be. It's just that a prompt is a critical part of your AI app and any change to it needs to go through a few hoops before it's available to all users.

On Portkey, since we decouple prompt templates from your code - you can continue iterating on the prompts on Portkey and just reference the prompt ID in code. Any change you make to Portkey prompts automatically get reflected in the app because the prompt ID keeps pointing to the latest / published version.

does that make sense?