12-Factor Agents: Patterns of reliable LLM applications — Dex Horthy, HumanLayer
Building effective AI agents often involves challenges when trying to move beyond basic functionality. Many production agents are not fully autonomous but rely on structured software engineering principles. To create reliable agents, it's crucial to manage the control flow, own the prompts, and carefully craft the context provided to the language model. Agents should be small and focused, similar to microservices, and designed to collaborate with humans. The key is to treat agents as software, leveraging existing engineering practices to ensure flexibility and high-quality outputs.
Key Vocabulary
The potential for improvement or positive development, especially in terms of quality or performance.
Examples:
- Investing in new training for employees offers significant upside in productivity and morale.
- The manager highlighted the upside of adopting the new software, pointing to reduced errors and faster processing.
- Despite the initial challenges, the project has a clear upside in terms of market expansion.
To encourage or push someone to do something, often repeatedly or persistently; in a technical context, to test or stimulate a system.
Examples:
- The team leader had to prod the developers to complete their tasks before the deadline.
- Engineers will prod the new server with various requests to assess its stability under load.
- She gently prodded her colleague for an update on the client's feedback.
An abbreviation for 'large language model,' a type of artificial intelligence program designed to understand and generate human language.
Examples:
- The new feature uses a powerful LLM to summarize lengthy documents automatically.
- Researchers are constantly improving the capabilities of LLMs for more nuanced text generation.
- Many applications now integrate LLMs to provide more interactive and intelligent user experiences.
An acronym for Directed Acyclic Graphs, a type of graph where all edges are directed and no cycles exist, commonly used in computer science to model workflows or dependencies.
Examples:
- The build pipeline was visualized as a DAG to clearly show the order of operations and dependencies.
- Data processing tasks are often organized into DAGs to ensure that steps are executed in the correct sequence.
- Understanding the DAG structure is crucial for optimizing complex computational workflows.
A specific URL or network address where an API (Application Programming Interface) can be accessed to perform a particular function.
Examples:
- The mobile app connects to a secure API endpoint to fetch user data.
- Developers need to document each endpoint to ensure other services can interact with the API correctly.
- Accessing the payment gateway requires sending a request to its designated endpoint.
Past tense of 'tweet,' meaning to post a short message or comment on the social media platform X (formerly Twitter).
Examples:
- The company officially tweeted about their new product launch this morning.
- He tweeted his thoughts on the recent industry trends, sparking a lively discussion.
- After the event, many attendees tweeted their positive feedback.
Provided with a basic framework or structure upon which to build, often referring to code or project setup that offers a starting point.
Examples:
- The new project was scaffolded with essential files and configurations to accelerate development.
- The IDE automatically scaffolded the necessary code for a basic web application.
- Learning a new programming language is easier when the initial exercises are well scaffolded.
Systems or tools that manage and coordinate complex workflows, services, or tasks, ensuring they run in the correct order and handle dependencies.
Examples:
- Workflow orchestrators are essential for managing microservices in a distributed system.
- The IT department implemented new orchestrators to streamline their cloud resource provisioning.
- Without proper orchestrators, managing the deployment of multiple interdependent applications becomes chaotic.
A common abbreviation for 'repository,' which is a central location where code, documentation, and other project assets are stored and managed, often using version control.
Examples:
- All the source code for the application is stored in the main GitHub repo.
- New contributors are encouraged to fork the public repo to begin working on features.
- The team regularly pushes updates to the shared repo to keep everyone synchronized.
To combine multiple items or components into a single package or unit, often for distribution or deployment.
Examples:
- The software update will bundle several bug fixes and new features into one release.
- Developers often bundle their application code with all its dependencies before deploying it.
- The marketing team decided to bundle the premium software with a year of support at a discounted price.
To criticize severely or attack verbally; in a technical context, it can refer to the Unix shell program 'Bash' used for command-line operations.
Examples:
- The critics were quick to bash the new movie for its predictable plot.
- It's easy to bash existing frameworks, but building a better one is far more challenging.
- He used a bash script to automate the daily backup process on the server.
To put an application or system into active operation, making it available for use by end-users.
Examples:
- The development team plans to deploy the new website update next week.
- Before we deploy the software, it needs to pass all the integration tests.
- The company decided to deploy their new service on a global cloud infrastructure.
Individuals or entities that provide input, code, ideas, or resources to a project or cause.
Examples:
- The open-source project thrives on the efforts of its many volunteer contributors.
- Acknowledging all the contributors in the annual report is a sign of appreciation.
- New contributors are always welcome to join the development team and offer their expertise.
A newly established business, typically small, that is in its initial stages of development and often focused on innovation and rapid growth.
Examples:
- The tech startup quickly gained traction with its innovative mobile application.
- Many startups rely on venture capital funding to fuel their early growth.
- Working at a startup often involves wearing many hats and adapting to constant change.
To initiate or get started using minimal external resources; in a technical context, to load the initial set of instructions or a basic framework to begin a process.
Examples:
- The entrepreneur managed to bootstrap his business with only his personal savings.
- The operating system uses a small program to bootstrap the entire system when the computer starts.
- We used a pre-built template to bootstrap the design of the new user interface.
Describes a system or component that does not store any information about its past interactions or current state, treating each request as an independent transaction.
Examples:
- RESTful APIs are often designed to be stateless, simplifying server management and scaling.
- A stateless function always produces the same output for the same input, regardless of previous calls.
- Implementing stateless microservices helps improve the overall resilience of the application.
A unit of information or a distinct, meaningful element, often used in the context of natural language processing to refer to words or sub-word units.
Examples:
- The security system generates a unique token for each user session.
- In this language model, each word is broken down into smaller tokens for processing.
- The API request requires an authentication token to verify the user's identity.
The act of putting something into service or active use, especially in a military or technical context; plural of 'deployment'.
Examples:
- The company's continuous deployments ensure that new features are rolled out frequently.
- Managing multiple deployments across different environments can be complex.
- Successful deployments require careful planning and thorough testing.
To improvise or elaborate on a theme or idea, often in a creative or conversational manner.
Examples:
- The comedian began to riff on current events, drawing laughter from the audience.
- During the brainstorming session, team members were encouraged to riff on each other's ideas.
- The jazz musician started to riff on the main melody, adding his own unique flair.
The process of making a system or process operate automatically, without direct human intervention.
Examples:
- Automating data entry can significantly reduce errors and save time.
- The company is investing in new software for automating customer support responses.
- Automating mundane tasks allows employees to focus on more strategic work.