AI tools are the tentacles exposed to AI models in order to allow them to access and drive external functionality. The primary form this takes for LLMs is function calling. We have also seen so-called "computer use" and web browsing designated in similar fashion.
For code generation, we could also consider the ability to integrate dependencies in the form of external packages as a form of tooling. In other words anything that extends the capabilities of the AI/LLM model could be thought of as a tool.
We can then extend this notion into the runtime environment for the code that LLMs generate by generating code that depends on runtime dependency injection.
Late Binding
Dependency injection is accomplished via "late binding", meaning the actual implementation
of a given functionality isn't specified at the time code is written, until at some later point before execution of logic.
Late binding can be implemented at the language level, such as in C++ you have virtual functions that
are eventually bound via dynamic-binding. In Java you have interfaces that can define the API and the actual implementation can be later supplied by some module/jar artifact.
Beyond language specific constructs, late binding can be accomplish in a broad set of ways. If a language supports passing functions as arguments, you can in effect bind functionality at the moment the invoked function that receives the function argument is called.
You can pass whole object instances representing implementations that have been instantiated
via some dynamic mechanism (ex Java beans, reflection,IoC frameworks (ex Spring),OSGi...etc).
There is really no shortage of ways one could implement late binding.
Late Binding Into AI CodeGen
Imagine you have a bunch of utilities that can perform various tasks, you are free to think of this in pretty much any way you want, ie API, SDK, CLI...etc.
To use such utilities in automation, you may wish to wrap additional logic around them in order to accomplish a particular task (ex: to satisfy a specific customer need).
For instance you may have a utility that knows how to parse a given programming language and perhaps provides a way to iterate over the AST and extract information or perform some additional tasks.
You could provide an LLM with the necessary context and hope that it will figure out how to create such a utility by itself using whatever APIs are available, or you could (with the possible help of LLMs) create such utilities in a manner that makes them reusable and then use LLMs to drive reuse of such
utilities in an automated fashion.
The gist of the idea is that when you want to do small scale automation, it defeats the purpose if every time you wish to use AI to do it you have to embark on a journey that involves the usual challenges of working with LLMs in order to get the desired results.
By abstracting away the challenging tasks that LLMs are likely to struggle with into pre-built utilities, you can leave the LLMs to wrap mediocre logic/code around such utilities in order to drive just-in-time automation in a manner that is likely to be consistent,reliable and desirable.
The concrete way to accomplish this is to instruct the LLM to in essence assume the existence of your utilities (ex:as object instances), provide information about the interface they expose, provide an object that represents an implementation instance and tell the LLM to use that object to access the utility. The LLM need not worry about how the object will be instantiated necessarily, it perhaps just needs to know where/how (contextually) it could find this object.
There is nothing new about using dependency injection, we simply observe that if the functionality (ex: domain specific utilities) that are likely to trip LLMs can be abstracted away into pre-built utilities, it can make it very easy to call on AI/LLMs to drive the reuse of such utilities via CodeGen that relies on injected objects.
This approach is a kind of tool use, except it is targeted at the actual code the AI writes as opposed to something the model invokes directly.
From our experiments, this proves to be a nice way to drive small scale automation. We can ask the LLM to write code targeting our utility object and it will do it without having to worry about implementing the functionality of that utility every time we need some variation of automation.
For us this approach is primarily intended to be used to target the Solvent-Botworx environment itself, ie exposing platform capabilities as utilities that we then have AI use to write small programs around, the approach however can be implemented in any other software platform environment.
Demo Video
Comments
Post a Comment