I am trying to understand structured output from LLMs, and as training I am building a text game.

OpenAI is using pydantic to create objects directly in its beta branch, it's quite nice. Anthropic is a bit more involved, as you need to involve tool calling.

LM Studio 0.3.5 introduced a bug for structured output when using pydantic and enums or literals, I created a workaround.

Also Gemini is not using Pydantic at all, instead TypedDict to create the json scheme.

And Gemini does not have a system message.

I am quite proud to have everything working, if you want to check out my code please take a look: https://github.com/HabermannR/Fantasy-Tribe-Game

Here is the backend: https://github.com/HabermannR/Fantasy-Tribe-Game/blob/main/L...

For example, this is how I call Gemini:

completion = model.generate_content(

messages[0]['content'] + ": " + messages[1]['content'],

generation_config=genai.GenerationConfig(

response_mime_type="application/json", response_schema=response_types.typed_dict

),

)

result = response_types.pydantic_model.model_validate_json(completion.text)

Happy for any feedback!