Asynchronous LLM computations specifications with LLM:Graph

Asynchronous LLM computations specifications with LLM:Graph(rakuforprediction.wordpress.com)

5 points by antononcube 277 days ago | 8 comments

antononcube 277 days ago |

Specifications for asynchronous LLM computations with Raku's "LLM::Graph" detail how to manage complex, multi-step LLM workflows by representing them as graphs. By defining the workflow as a graph, developers can execute LLM function calls concurrently, enabling higher throughput and lower latency than synchronous, step-by-step processes.

"LLM::Graph" uses a graph structure to manage dependencies between tasks, where each node represents a computation and edges dictate the flow. Asynchronous behavior is a default feature, with specific options available for control.

librasteve 277 days ago |

that’s very interesting as far it goes, but wouldn’t Mathematica or Python be better choices than Raku for this kind of thing?

antononcube 277 days ago | |

Ah, yes, Raku's "LLM::Graph" is heavily inspired by the design of the function LLMGraph of Wolfram Language (aka Mathematica.)

WL's LLMGraph is more developed and productized, but Raku's "LLM::Graph" is catching up.

I would like to say that "LLM::Graph" was relatively easy to program because of Raku's introspection, wrappers, asynchronous features, and pre-existing LLM functionalities packages. As a consequence the code of "LLM::Graph" is short.

Wolfram Language does not have that level introspection, but otherwise is likely a better choice mostly for its far greater scope of functionalities. (Mathematics, graphics, computable data, etc.)

In principle a corresponding Python "LLMGraph" package can be developed, for comparison purposes. Then the "better choice" question can be answered in a more informed manner. (The Raku packages "LLM::Functions" and "LLM::Prompts" have their corresponding Python packages implemented already.)

ab5tract 277 days ago | |

In that case, when would Raku ever be a better choice than something else?

I personally don’t see what advantages Python as a language (not an ecosystem) would have here.

librasteve 277 days ago | | |

Python is more widely known and a very popular tool for “LLM engineering”, so I’m curious what would be the reason to choose Raku in this case and wondering how the feature benefits of Raku outweigh the general incentive to use more popular tools.