Generative AI has found a niche everywhere, and, whether you like it or not, it’s here to stay. As the hype slowly ebbs away, real-world applications that are actually useful will begin to appear once the shoe-horning hypefest has ended.
It’s already happening, actually, and it’s not even that new – but it *is* under-reported, with one killer feature:
Tool use: The ability for an LLM to actively choose to pick up a tool that you provide it, and then use it to answer a query, or complete a task.
Next-level tooling
Large generative LLMs have had this ability for a while now (first, OpenAI’s ChatGPT 3.5 and then later Claude Sonnet 3.5, while Llama 3.1 and 3.2 has some support, but it is very fiddly to get working).
Critically, we can put tools into AI agents (a way of structuring LLMs to act independently, sometimes as a group, to complete a task over time). Which means you can specialise them to a task while giving them the tools to do so: researchers (with web search and database support), copywriters (with writing guidelines and tone-of-voice guides), testers (with QAtools and sandbox code editors), planners (with project management tools), etc.
Tool-use is what takes an LLM from an interesting (if somewhat untrustworthy) alternative to search, to becoming a useful component in your daily workflow.
I’ll give you a simple example of how we make use of tools at Tyk in our own AI Portal:
- I ask it to list tickets opened in the product tracker in JIRA in the past 12 hours
- The LLM use our JIRA tool and queries it using JQL, and outputs a nice, human friendly list of new tickets
- I notice one of those tickets has been created from our support desk
- I ask it to tell me what client caused that ticket to be opened
- The LLM again uses the JIRA tool to pick up more details on the ticket and gives me the company name.
- I ask it how long Company X has been a customer
- It picks up our CRM tool and does a company search
- It correctly deduces that the company has been with us as a customer for two years
- I ask it for more details about their actual issue
- The LLM picks up our Customer Support Tool to pull the actual conversation thread and provides a summary of their use case that led to the issue.
All of the above could have been done by me with a browser, and assuming all these systems are linked up, the data should only really be a few clicks away.
But it is still faster, and much more direct to ask the LLM for this data rather than logging into 4 different systems with my 2FA keys and half-forgotten SSO login to then have to filter out all that noise myself.
LLMs using tools are a super-power
Under the hood, these LLMs are using APIs to make these tool calls, and here is the crux of why APIs are central to the success of LLMs in the organisation.
You see, AI tool-calling is not direct, it is delegated: specifically it is delegated to the chat client:
- Client sends prompt to LLM
- Client includes list of available tools and parameters
- LLM decides to use tool, send back tool call and params to use
- Client makes tool call and sends data back to LLM
- LLM responds with its response or more tool calls.
Ultimately that means that an LLM is only as smart as the client accessing it, and a good client will make tool-calling easy.
I see tool use as the real first step towards cementing the AI supply chain:
- LLM vendors like openAI provide the models,
- Hyperscalers like Google and Amazon provide the runtime
- Software or SaaS vendors provide the clients (for example Zed editor, LLM Studio)
- Data publishers such as databases and APIs (both functional and data-oriented) provide the context for the models operational environment
This is where APIs and open standards really come into play, and show how important it is to have good, solid API documentation in an open and accessible format.
I’m not just saying that because it’s nice in theory, we put out money where our mouth is…
At Tyk, for our AI Portal, we created what I call the “Universal Client”, it is an app that can take any Open API Specification , and provide a standard way to call any operation in that spec using a standard interface. Bridging the gap between documentation and consumption for an LLM without any custom glue code.
This is nothing new, OAS specs are very commonly used to generate SDKs and client libraries, and the reason we need client libraries is because we need to finagle the returned data in some way to meet our desired use case.
But with LLMs, the LLM can do that *for us*, since the output is traditionally conversational, and LLMs can handle massive blobs of data without needing it carefully structured for a UI component, or a print statement.
This is where I want to commend Anthropic for their recent release of the Model Context Protocol (MCP <LINK>) – a standardised interface for AI clients to implement tool use and context enrichment for upstream LLMs.
By using an open standard transport (JSONRPC), and open-sourcing the protocol definition itself, in conjunction with the stability and wide adoption of the OpenAPI Spec, it should mean that AI clients will (over time) become much more plug-and-play when it comes to interfacing with the context that they are meant to be working with.
The number of MCP servers has exploded since the release of the protocol, with modern agentic systems like Cline <LINK> actively incorporating the protocol into their own interfaces to make their own capabilities as agents highly extensible in a standardized way.
Having open API standards for GenAI interactions means that – for example, my chosen LLM vendor, is piped through my preferred client (lets say an Alexa speaker), and the speaker is aware of my MCP-enabled Home Assistant, or Philips Hue bridge, and makes controlling this context require zero glue-code on the speaker (or my phone’s client app).
This is the start of the supply-chainification of AI: Where vendors, clients, and service providers can each develop and implement specialised and competitive capabilities, without having to deeply integrate with one another. It is a pivotal step towards real productivity and wider utility from GenAI.
To move one level up – there seem to be two distinct strategies targeting the AI market by the big vendors: in the first camp are OpenAI and Google, who want their AIs and supporting tooling to be all-embracing, i.e. a platform that is powered by their own solutions, as Sam Altman famously said:
“I think, fundamentally, there are two strategies to build on AI right now. There’s one strategy, which is to assume the model is not going to get better, and then you build all these little things on top of it. Then there’s another strategy, which is built assuming that OpenAI is going to stay at the same rate of trajectory and that the models are going to keep getting better at the same pace. […] a lot of the startups have been built in the former category when we just do our fundamental job because we have a mission that we’re going to steamroll you.”
And in the second camp is Anthropic, trying to build a composable, toolchain focussed platform that bets on the wider community to build value on top of AI, and treat all AI as a tool, regardless of whether the models become sentient overlords.
To stress the point: There’s more use cases for dumber AI than for AGI. For example, if we can make a light switch smarter, but don’t need full AGI to control it (it’s a light switch!) because we can embed a smaller, composable AI into the switch, then dumb AI will proliferate much faster than the AGI in the universe of human industry and a market economy.
Think electronic components, shipping containers, and third-party car parts – not Apple devices.
Embrace open standards
The key takeaway I would like to give here – and especially if any LLM client devs are listening: embrace open standards – wide adoption puts pressure on the market to grow, to specialise, and to commoditise to an extent where a wide and powerful ecosystem of AI-led capability makes us more productive individuals, creates new and innovative jobs, and widens the value that AI can generate in the economy.
More importantly, by embracing and pushing for open standards that commoditise the AI supply chain, we put pressure against monopolisation, and the trend of “commoditising the cohort” that inevitably leads to competitive stagnation as vendors integrate all value-added activities into their platform, and eventually lock customers into their technology ecosystems with little to no oversight or alternatives.
That is not a future we should wish for with AI, and embracing open standards in the AI supply chain is critical to ensuring a solid foundation for the future of AI adoption, safety, and consumer choice.
