AI agents in business: where to start and how to measure value

An AI agent is an AI system that chooses steps and calls tools to reach a goal. See agent types, how agents differ from chatbots and how to choose a pilot.

By UNIXDEV Team · 12 min read
Original published 2026-09-19 · English edition 2026-11-01

Editorial illustration for AI agents in business: where to start and how to measure value

An AI agent is a system in which an AI model chooses its next steps and calls tools, such as search or a business system, to work toward a goal within the permissions an organization sets.

An AI pilot is useful when it answers a business question: can this task be done reliably enough, at a cost that makes sense? Begin with one workflow and evidence your team can inspect. The examples and pilot criteria below are proposals to adapt, not customer results or a guaranteed return.

Agent, chatbot or workflow?

A chatbot provides a conversational interface. A workflow follows predefined steps. An agent can choose tools or next steps within a defined task. These approaches can be combined, but a fixed workflow is often easier to test when the sequence is already known. Do not add autonomous decisions merely because a model can make them.

Pattern	Suited to	Example
Chatbot / question answering	Answering or summarizing, possibly with document retrieval	Finding the travel-expense policy with a link to the source
Workflow automation	Mostly fixed steps and conditions, with AI in selected steps	Read a request, classify it, route it to a queue by rule
AI agent	Choosing what to look up or which tool to use next, based on what it finds	Investigating a problem across systems and proposing a fix

If fixed rules solve the whole problem, start with the rules and add AI only where text must be understood or input varies widely.

What kinds of AI agents are there?

There is no standard taxonomy of AI agents, and vendors use different names. The grouping below is this article's practical proposal for scoping work inside an organization, not any vendor's classification. It looks at two things: how much the model decides, and the job the system does.

By level of autonomy

Level	Who decides the steps	Example
1. Tool-calling assistant	The user makes one request at a time; the model chooses which tool to call to answer it	Asking for the status of a request, answered from the case system with a reference number
2. Agentic workflow	The main steps are fixed in code; the model handles only the steps that need language understanding	Read a document, extract fields, route to a reviewer by rule
3. Planning agent	The model chooses the sequence of steps and tools within defined scope and permissions	Investigating a problem across several systems and summarizing evidence and options
4. Multi-agent	A lead agent splits the task among other agents and combines their results	Searching several sources in parallel and merging them into one report

Level 1 is close to what Anthropic's Building effective agents calls the augmented LLM: a model enhanced with retrieval, tools and memory, the basic building block of agentic systems. Level 2 is what Anthropic defines as a workflow, where models and tools are orchestrated through predefined code paths. Level 3 is an agent in Anthropic's sense, directing its own process and tool use, and level 4 resembles its orchestrator-workers pattern, which Anthropic groups under workflows. Higher levels are more flexible but harder to test, cost-control and audit, which is why Anthropic recommends starting with the simplest solution and adding complexity only when needed.

By job

Document question answering (RAG): answers from internal documents with sources; level 1 or 2 is often enough. See what RAG is and how to build it for Thai documents.
Customer service: summarizing requests, finding guidance and drafting replies for an employee to review.
IT operations: gathering the logs and metrics around an incident, suggesting likely causes and drafting remediation steps for an administrator to approve.
Document processing: reading forms or contracts, extracting key fields and checking completeness before entry into a system.

Knowing which group a task belongs to, and how much autonomy it needs, makes the pilot choice and permission design easier. The next section gives proposed pilot examples.

Three places to investigate

Customer-service requests

Read a request, find relevant guidance and prepare a sourced draft for an employee to review. If the only need is classification and routing, a workflow may be enough. Measure total handling time, substantive corrections and repeat work.

Internal document search

Answer questions from approved documents with references and a clear response when evidence is missing. Retrieval-augmented generation may solve the problem without a multi-step agent. Test access boundaries as well as answer quality.

Sales meeting preparation

Assemble permitted CRM history, open issues and source links into a briefing. Begin with a draft rather than permission to edit the CRM or send messages. Measure preparation time, completeness and factual corrections; changes in revenue have many causes.

Four checks before choosing the pilot

Does the work repeat often enough? Record task volume and current handling time.
Is the data usable? Identify sources, owners, access rules and quality problems.
Can someone judge the result? Define the facts and conditions a passing output must include.
Can the impact be contained? Start with reading or drafting, a human escalation path and a way to return to the existing process.

If all four can be answered, start with one task, one team and one primary metric. If the team cannot say how correctness will be checked or who is responsible when the output is wrong, prepare that first, however much time the task might save.

How ready does the data need to be?

The whole organization's data does not need to be cleaned first. The data for the pilot task is ready when the team can say where it is, who owns it, whether it is current, who may see it and how the system will reach it.

Area	Question to answer	Warning sign	What to do first
Sources and owners	Which system or folder holds the data, and who is responsible for its accuracy?	Nobody can confirm which version is in use	List sources with owners and pick one reference version per topic
Freshness	Do documents carry an update date, and how are old versions retired?	Several manual versions sit together without status	Mark status and date on in-scope documents and separate retired ones
Format	Is the data searchable text, tables or scanned images?	Key information exists only in scans or images not yet converted to text	Convert and check the most-used documents first
Access rights	Which data should each user group see?	Permissions in the source system are wrong or unmaintained	Summarize pilot user groups and permissions and have data owners confirm them
Personal and confidential data	Is there data that must be masked or must not be sent outside the system?	No one has decided whether it may be used with an external AI service	Have the organization's privacy and security owners set conditions before design
Integration path	Is there an API, an export or a read-only database?	The only way out is a person copying from the screen	Identify the usable channel and estimate integration work separately from the AI work
Test set and reference answers	Are there real examples with outputs agreed to be correct?	Quality is judged by how the trial users feel	Have the task owner select easy, hard and exceptional cases and write pass criteria
Master data	Do customer codes, product codes and statuses match across systems?	The team routinely fixes master data by hand	Put the fix in the pilot plan or choose a task that does not depend on that data

If several areas are not ready, the right pilot may be organizing the data for that task first. For the broader view of data and process readiness, see starting digital transformation and the first 90 days.

Set boundaries before enabling tools

Write the scope as a checkable list: which systems the agent may read, which fields it may change and which actions require human approval. During a pilot, reading and drafting are usually enough; sending to customers, refunds and changes to user permissions should go through the approval step defined in the system.

Give each tool a narrow purpose and only the permissions needed, and enforce those permissions in the target system as well. Require review for consequential actions, limit retries and make repeated requests idempotent (safe to retry without duplicate effects). Set limits on steps, time and cost per task, and stop or hand over to a person when data is incomplete or conflicting, a tool does not respond, or a limit is reached. Actions affecting customers, money or user permissions need human approval. Test unavailable tools, incomplete records and malicious instructions inside documents. Log tool calls, results and approvers in enough detail for review, with controlled access and without unnecessarily exposing sensitive data.

Retrieved documents and messages can contain hidden instructions intended to change the model's behavior, so separate content that is read from trusted instructions. OWASP LLM01:2025 Prompt Injection describes this risk from external sources such as websites and files, and lists mitigations including constraining model behavior, defining and validating output formats, filtering input and output, least privilege, human approval for high-risk actions, clearly marking external content and regular adversarial testing.

Measure accepted work

Use representative tasks and compare them with a baseline of similar difficulty. An initial period can run the AI in parallel without sending output to live systems, so reviewers can judge quality before a small team uses it.

Metric	How to count it
Human time per task	Include review, correction and rework, compared with the existing method
Pass rate	Tasks that meet the criteria ÷ all tasks attempted, counting failures and timeouts
Cost per accepted task	Total operating cost during the trial ÷ accepted tasks, including model, tools and review labor
End-to-end latency	Time from receipt to completion, reported as the median and a high percentile such as p95, kept separate from human working time
Out-of-scope events	Unauthorized access or actions, reviewed case by case rather than averaged away

An average quality score must not hide an access-control failure.

A worked example, not a forecast

Suppose 2,000 eligible tasks a month take 10 minutes each. With AI, review and correction average 6 minutes, including tasks completed manually after failure. The capacity recovered is 2,000 × (10 − 6) ÷ 60 = 133.3 hours a month. At an assumed value of THB 300 an hour, that is THB 40,000 of capacity.

If model, tool and operating costs are THB 15,000 a month, net capacity value is THB 25,000. An assumed THB 150,000 setup would take six months to recover on that basis. This is not necessarily cash saved: staff time must be put to useful work. For a cash return, use costs that actually fall, such as overtime no longer paid, and do not count the same time twice. If review takes eight minutes, net value falls to THB 5,000 and the same calculation becomes 30 months; if net value is not positive, there is no payback under those assumptions. These are illustrative assumptions, not a quote or measured customer result.

A 30-day pilot outline

Week 1: choose one task and owner, gather a baseline with easy, hard and exceptional cases, and define acceptance, cost and access rules.
Week 2: connect a limited set of sources and test a draft-only prototype. Keep a separate evaluation set that is not used to tune prompts.
Week 3: let a small team use it with reviewers; evaluate normal and difficult cases, failures and permission boundaries; record correction time, handovers and permission problems, and test stopping the system and returning to the old process.
Week 4: compare quality, time and cost with the baseline, then decide to expand, revise or stop, and name an owner for the next step.

This is a planning outline for a small task whose data and permissions are already in place. Legacy integration, data preparation or approvals may require longer.

Agree on the expansion conditions before seeing results, for example quality no worse than the current method, human time reduced to an agreed target and cost per task within an acceptable range. Any access beyond permitted scope should stop the affected path for investigation even if other scores pass. Finding no incident in the test set does not prove the risk is absent at larger scale.

What to send a development team for assessment

The aim is enough information to decide whether to start with a workflow, retrieval or an agent, and how value will be proven.

Item	What helps an accurate assessment
Problem and owner	The problem, the team that will use the system, who judges a passing result and who approves the project
Real examples	Permitted, sanitized examples covering easy, hard and exceptional cases, with the outputs considered correct
Baseline	Monthly volume, time per task and correction or rework rate, as measured by the team
Current process	A map or step-by-step description with decision points, approvals and handovers
Systems involved	System names, available access such as an API or export, owners and rate or time limits
Data and security requirements	Data that must not leave the organization, policy on cloud and external model services, and the readiness review above
Acceptance and decision criteria	The main metric for expanding or stopping, and the budget and timeline the organization has set

Do not send in the first round:

Production passwords or credentials. Share them through a secure channel after scope and confidentiality are agreed, with only the access the task needs.
Full unmasked data sets. Sanitized samples are enough for an initial assessment; full data belongs in an environment the organization controls.

Expect back from the assessment:

A recommendation of workflow, retrieval or agent, with reasons.
Gaps in data, permissions and integration to address before or during the pilot.
Pilot scope, human approval points, metrics and expand-or-stop conditions.
Open risks and limitations, and how the pilot will test them.

If the need is general software development rather than AI, see how to choose a software development company in Thailand.

Checklist before approving the pilot

One repeated task that can be checked and contained, with a clear owner.
Data readiness reviewed across the eight areas above, with a written decision on sensitive data.
A written scope of what the system may read, change and do, and which actions need human approval.
A baseline gathered before starting, and an evaluation set kept apart from tuning.
Expand, revise or stop conditions agreed before results are seen.
An assessment pack prepared without credentials or unmasked data.

Prepare the first discussion

Bring the workflow, task volume, sanitized examples, data sources and the people responsible for acceptance. Discuss an AI and LLM engineering pilot before choosing a broad platform rollout, or request a proposal.