Prompt, Workflow, Agent

Prompts, Workflows and Agents. These three terms are used like their meanings are common knowledge. I do not think this is the case, so what I want to do in this post is lay out some reference points for thinking about and using the technology being referred to in the context of getting work done. The aim is that by the end of this post we have a mental framework in place for choosing the right tools for the job.

Let’s start with what we are familiar with, which is the idea of a Prompt - it’s you asking a system to do a single thing now. For example:

What is the capital of France?
Summarize the attached article.
Create an image of a pirate riding a scooter.

 

Each of these I would suggest are reasonable examples of the kind of single input, single output interactions you have had with Generative AI systems.

At the other end of the scale we have the idea of the Agent, an AI system that can just do things based on a goal:

Plan a trip to Paris for me, 
Use the attached article to select five activities I would enjoy 
Make sure to include the Pirate Scooter tour
I want to go on this trip the second week of October
Make the necessary reservations.

 

You can test the first set of examples with your favorite LLM without too much fuss. To test the second example you need a system that can Plan, has Memory and can perform Tool Use. Each of these contains subsets of technologies and techniques, for example as explored by Lilian Weng in LLM Powered Autonomous Agents. Autonomous Agents have many moving parts besides the LLM engines that ‘power’ them and their ability to reach the goals set for them depends on the clever arrangement of these supporting components.

So let’s put working with Prompts on the first run of a autonomy/complexity ladder, and Autonomous Agents on the top rung of that ladder. What are some of the steps in-between?

Somewhere in the middle we will find Workflows, thinking of them as a collection of many Prompt interactions arranged in ways where they can work together. When we start building systems around LLMs and start using terms like Then or If or While, so we have started to encode logic to create predictable paths between each Prompt, then we are in the region of Workflows.

There are multiple rungs on this conceptual ladder:

• Single LLM calls. The shallow end of the Prompt pool - capital of France etc.
• Chains of LLM calls. Looking more like a Workflow.
• Chains of LLM calls with Routing, that is decisions between pre-defined paths. Still a Workflow, but beginning to feel more Agentic.
• All of the above and add a State Machine. Briefly, a state machine workflow knows what stage a system is at, what the options are for what to do next, and which of the options to choose based on what it is doing now. This is starting to look more like what is imagined when people say Agent.
• Agents with Autonomy. Add the ability to Plan, give it Memory and Tools and the system can act without user input, doing things like ordering inventory automatically.

The industry blurs these distinctions - at the time of writing most of the usual suspect tools for interacting with Generative AI allows users to compose in a ‘Prompt’ complex requests that require multiple steps, displaying chains of LLM calls and branching in workflows the tool composes. There are some tools that allow you to make things that are labelled as ‘Agents’ that are more like a ‘Workflow’ in their behavior, and other tools that really do include the Plan, Memory, Tool Use ingredients to really walk and talk like an ‘Agent’.

It isn’t clear when you sit down to work with one of the mainstream tools (ChatGPT, Claude etc) what rung of the ladder you are on - the tool chooses the rung for you, depending on what you ask it to do. So why make these distinctions between ‘Prompt’, ‘Workflow’ and ‘Agent’?

The reason is that we are interested in knowing how to get Work done with GenAI. This hinges on being able to talk about what kind of work is being done. Having categories simplifies our thinking in that respect.

Part of Simplification is stripping something down to its essentials in order to think about the parts that really matter. We can approach this if we can identify the essential Components of some work we hope to get done with GenAI.

One way of identifying these kinds of atomic tasks is to try and describe in with the following pattern:

Role – Verb Noun → Output (Done-when?)

If we can identify a Role (‘Research Assistant’), action (Summarize’), thing to perform the action on (‘The attached paper’) and a desired outcome with a definition of ‘done’ (‘Five bullet points’), then we have the kind of task we are looking for.

When you look at how to get work done with AI, you can use this pattern in a process that looks like the following:

1. Identify a work product that you ‘ship’ regularly. What we mean by ‘work product’ is something that basically you can save and send, like a presentation, a report, a blog post etc.
2. Decompose the work you do to produce the work product into a series of tasks that you can describe using the Role – Verb Noun → Output (Done-when?) pattern described above.
3. Look for verbs that you know, based on your own experience, are friendly for GenAI. Popular choices include Summarize/Extract/Classify/Rewrite/Translate Spotting these provides an entry point for getting work done with AI.
4. Look at how many of the atomic task patterns you came up with. One simple task means you can use a single Prompt. Several patterns in a row means thats a Workflow. If you end up with branches or loops (e.g ‘Decide’ shows up), then you are getting into more of an ‘Agentic’ Workflow, and open ended goals based on conditions in the moment leads to an Autonomous Agent.
5. Decide if shipping the work product requires a Prompt, a Workflow or an Agent. As discussed these distinctions are somewhat hard to see in the real world, but while thinking try and identify the center of gravity for the work product.
6. Consider the guardrails you would want in place. As with the atomic task pattern come up with a ‘Done-when?’ statement for the work required to produce the work product. And think about what is not allowed - phrases you need to avoid using in writing a blog for example. You don’t have to list them all, just have an idea of what’s potentially going to be required.

When you have done these six steps, ask yourself the following questions. These are not algorithms for decision making, but more guidance on where to focus attention:

• How many Gen AI friendly verbs did I find?
• Based on the number of atomic tasks and how they interact, where is the center of gravity for this work, more at the Prompt end or the Agent end?
• How scary are some of the guardrails I came up with in terms of consequences?

The weighing of whether or not you can get work done with AI is personal. It is language heavy and relies and a sense of how close or how far you are from a solution - kind of like an LLM. But just because its complex does not mean we cannot use simple steps to point us in the right direction.