From agent architecture and tool calling to reasoning loops, LangGraph, and production failure handling

Introduction

I’ve noticed something interesting while preparing for and interviewing AI Engineers.

A few years ago, the interview questions were mostly:

What is a Transformer?

Explain RAG.

What is vector search?

Now the questions are becoming more architectural.

For example:

How is an AI agent different from a normal LLM application?

When should you not use an agent?

How does an agent know whether to call a tool?

What happens when an agent gets stuck in an infinite reasoning loop?

How would you design state management across a multi-step workflow?

These questions test something different.

They test whether you understand how an AI system behaves after the LLM produces its first output.

The source material for this part focuses on exactly that transition: from single-step LLM applications to systems that plan, act, observe, maintain state, call tools, and recover from failures.

So, continuing my 20-Part AI Engineer Interview Questions series, this part focuses on one of the most important areas of modern GenAI engineering:

AI Agents.

Let’s get into it.

AI Engineer Interview Questions — Part 6

1. How is an AI agent architecturally different from a standard LLM-based chatbot?

A traditional LLM application often looks like this:

User Input
    ↓
LLM
    ↓
Response

The model receives an input and generates an output.

An AI agent introduces a decision loop:

User Goal
    ↓
Understand
    ↓
Plan
    ↓
Choose Action
    ↓
Call Tool
    ↓
Observe Result
    ↓
Decide Next Action
    ↓
Final Answer

The key difference is not simply that an agent uses an LLM.

A chatbot can use an LLM too.

The difference is that an agent can repeatedly decide:

• What should I do next?
• Do I need external information?
• Which tool should I use?
• Did the previous action succeed?
• Should I retry?
• Is the task complete?

This aligns with the source’s distinction between single-step conversational systems and multi-step Plan → Act → Observe loops.

Interview-ready answer

A standard LLM application usually performs single-step inference: input goes to the model, and the model generates a response. An AI agent operates as a multi-step control loop where the model can plan, select actions, call external tools, observe the results, update its state, and decide what to do next until the task is complete.

2. When can adding agent behavior actually make a system worse?

This is one of my favorite interview questions because it tests whether you understand the trade-offs.

Not every LLM application needs an agent.

Suppose you need to classify a customer complaint.

Input
  ↓
Classifier / LLM
  ↓
Category

Adding planning, tool selection, memory, and multiple reasoning steps would probably make the system:

• slower
• more expensive
• harder to debug
• less deterministic

Agent behavior can be a poor choice for deterministic workflows, latency-sensitive systems, high-stakes irreversible actions, and heavily regulated environments where unpredictability creates additional risk.

The rule I use

Simple problem
    ↓
Simple workflow
Complex, uncertain problem
    ↓
Consider an agent

Interview-ready answer

I would avoid agents when the workflow is deterministic, latency-critical, or requires highly predictable behavior. If the action sequence is already known, explicit code or workflow orchestration is usually faster and more reliable than asking an LLM to decide every step.

3. What does autonomy actually mean in an AI agent?

Autonomy does not mean:

“The AI does whatever it wants.”

In production, autonomy means the system can complete a task within a defined decision-making scope without requiring human intervention at every step.

For example:

Goal:
Find the latest sales report and summarize the major changes.
Agent:
1. Search document repository
2. Find latest report
3. Retrieve data
4. Analyze changes
5. Generate summary

A useful way to measure autonomy is through:

• Task completion rate without human intervention
• Error recovery rate
• Escalation rate
• Clarification rate

The source specifically frames autonomy around independent action selection, error recovery, task completion without intervention, and balancing assumptions against asking for help.

Interview-ready answer

I define autonomy as the ability of an agent to independently select and execute an action sequence within predefined permissions and constraints. I would measure it using metrics such as autonomous task completion rate, recovery rate from failures, human escalation rate, and unnecessary clarification rate.

4. How does an agent decide whether to answer directly or call a tool?

Suppose a user asks:

“Explain how RAG works.”

The model probably doesn’t need a tool.

But now the user asks:

“What was our company’s revenue last quarter?”

The answer depends on external and potentially private data.

Now the agent should call a tool.

Conceptually:

User Query
    ↓
Can I answer from available context?
       │
   ┌───┴────┐
   │        │
  Yes       No
   │        │
   ↓        ↓
Answer    Select Tool
             ↓
          Execute
             ↓
          Observe
             ↓
          Respond

The source distinguishes conceptual and explanatory requests from requests requiring external data, state access, or real-time information, with tool routing influenced by system instructions and clear tool descriptions.

Interview-ready answer

The agent should call a tool when the answer requires information or actions outside the model’s available context, such as real-time data, database access, or external system state. The routing decision depends heavily on clear system instructions and precise tool descriptions.

5. Why is decision-making the core problem in agent design?

Because every decision can change the rest of the workflow.

Imagine this:

Goal
 ↓
Agent selects wrong tool
 ↓
Receives incorrect data
 ↓
Builds reasoning on wrong data
 ↓
Calls another incorrect tool
 ↓
Produces wrong result

This is called cascading failure.

An agent must continuously make decisions about:

• action selection
• retries
• alternative strategies
• stopping conditions
• failure handling

The source highlights that wrong tool selection can cascade downstream and that the agent must also decide whether to retry, switch methods, or stop entirely.

Interview-ready answer

Decision-making is central because an agent’s outputs are not just text—they can trigger actions with downstream consequences. A poor decision early in the workflow can cascade through later steps. Therefore, action selection, retry logic, failure recovery, and stopping conditions must all be designed explicitly.

6. What is the difference between reactive and deliberative agents?

A reactive system behaves like this:

Input
  ↓
Immediate Decision
  ↓
Action

It focuses primarily on the current state.

A deliberative agent behaves more like this:

Goal
  ↓
Build Plan
  ↓
Evaluate Options
  ↓
Execute Step
  ↓
Update State
  ↓
Replan if Necessary

Reactive agents are generally faster.

Deliberative agents can handle more complex tasks but introduce additional latency and failure modes.

The source characterizes reactive systems as mapping current inputs to immediate actions, while deliberative systems maintain richer internal state and plan multiple steps ahead.

A practical architecture

Many production systems should be hybrids:

                User Request
                     ↓
              Complexity Router
                 /         \
                /           \
          Simple             Complex
             ↓                  ↓
        Reactive Path     Planning Agent
             ↓                  ↓
           Result             Result

7. When would a reactive agent outperform a planning-based agent?

Planning is not free.

Every additional reasoning step increases:

• latency
• token consumption
• cost
• failure probability

For a low-latency or highly deterministic task, planning can become unnecessary overhead.

Examples include:

• real-time control loops
• high-frequency decisions
• fixed input-to-output workflows

The source specifically contrasts low-latency and highly deterministic environments with tasks that benefit from deeper research, reasoning, and sequencing.

Interview-ready answer

A reactive architecture is better when the decision is simple, latency-sensitive, and based on a relatively stable input-to-output mapping. A planning agent becomes more valuable when the task requires decomposition, multi-step reasoning, research, or adaptation to intermediate results.

8. How would you balance reasoning depth against latency?

This is becoming increasingly important in production AI.

Consider two requests.

Request A

“Summarize this email.”

One LLM call may be enough.

Request B

“Analyze our quarterly financial performance, identify anomalies, verify them against source data, and recommend follow-up actions.”

This may justify:

Planning
  ↓
Data Retrieval
  ↓
Analysis
  ↓
Verification
  ↓
Critique
  ↓
Final Answer

The solution is not always maximum reasoning.

The source recommends adaptive depth—routing simple and complex tasks differently and matching reasoning depth to the stakes of the task.

A production approach

def route_task(query):
    complexity = classify_complexity(query)
    if complexity == "simple":
        return direct_llm()
    elif complexity == "medium":
        return llm_with_tools()
    else:
        return planning_agent()

Interview-ready answer

I would use adaptive reasoning depth. Simple tasks should take the shortest reliable path, while complex or high-stakes tasks can justify planning, verification, and multiple tool calls. The architecture should dynamically match reasoning depth to task complexity, latency requirements, and risk.

9. Why is state management important in multi-step agents?

Consider this workflow:

Step 1: Retrieve customer data ✓
Step 2: Retrieve transaction history ✗
Step 3: Analyze customer behavior

If Step 3 assumes Step 2 succeeded, the agent may generate conclusions using incomplete data.

Multi-step agents need explicit state.

For example:

state = {
    "goal": "Analyze customer",
    "customer_data": {...},
    "transactions": None,
    "step_status": {
        "customer_lookup": "success",
        "transaction_lookup": "failed"
    }
}

The source emphasizes that reactive systems are effectively limited to immediate context, while deliberative systems track goals, steps, and failures—and can fail if later steps assume earlier ones succeeded.

Interview-ready answer

State management is critical because later actions depend on the results and status of earlier actions. The agent must explicitly track goals, intermediate outputs, failures, and execution status so that downstream steps don’t incorrectly assume that previous operations succeeded.

10. Why do deliberative agents need stronger control mechanisms?

Because errors compound.

Imagine:

Step 1 ✓
Step 2 ✗ Incorrect assumption
Step 3 → Uses incorrect assumption
Step 4 → Expands incorrect plan
Step 5 → Calls wrong tool
Step 6 → Retries
Step 7 → Retries again
Step 8 → Burns $50 of API calls

Common risks include:

• goal drift
• infinite loops
• repeated tool calls
• token exhaustion
• runaway costs

The source explicitly calls out error compounding, goal drift, infinite retries, and the need for hard limits on tokens, loop counts, and execution time.

A production agent should have:

max_iterations
max_execution_time
max_token_budget
max_tool_retries
cost_budget

11. What should happen before an agent is allowed to call a tool?

Tool calling should not mean:

LLM generates JSON → execute immediately.

Before executing a tool call, validate:

1. Authorization

Is this user or agent allowed to perform the action?

2. Input validation

Are the arguments valid?

3. Idempotency

What happens if the call is repeated?

4. Confidence or risk level

Should the system require confirmation?

The source specifically identifies authorization, strict input validation, retry safety, and confidence thresholds for actions that modify data.

A safer architecture is:

LLM
 ↓
Proposed Tool Call
 ↓
Schema Validation
 ↓
Authorization Check
 ↓
Risk Check
 ↓
Execution
 ↓
Result

12. How does an agent choose the correct tool?

This is one of the biggest practical challenges.

Suppose an agent has:

search_database()
search_web()
search_documents()

If the descriptions overlap, the agent may select the wrong tool.

Good tool descriptions should explain:

• What the tool does
• Required inputs
• Expected outputs
• When to use it
• When not to use it

The source identifies tool descriptions as a critical routing signal and emphasizes defining inputs, usage conditions, and non-usage conditions to reduce overlap.

Bad:

search(query)

Better:

search_customer_database(
    customer_id: str
)

Description:

Use this tool only when you need information about an existing customer using their internal customer ID. Do not use it for public information or document search.

Clear boundaries improve tool selection.

13. What happens when tool descriptions are ambiguous?

Ambiguous tools create multiple failure modes.

For example:

search()
lookup()
find()
query()

Which one should the agent choose?

Possible consequences include:

• wrong tool selection
• hallucinated arguments
• duplicate tool calls
• unnecessary latency
• cascading failures from bad tool results

These are the exact categories highlighted in the source material.

Interview takeaway

Tool descriptions are not just documentation.

They are part of the agent’s control system.

14. How would you design a tool registry for a production AI agent?

A production system should not expose every tool to every agent.

A tool registry could look like:

tools = {
    "database_search": {
        "version": "2.1",
        "permissions": ["analyst", "admin"],
        "input_schema": "...",
        "health_status": "healthy"
    }
}

The source recommends a standard schema containing tool identity and version information, dynamic permissions, context-based loading, and health monitoring for failing or timing-out tools.

Conceptually:

Agent
  ↓
Task Context
  ↓
Tool Registry
  ↓
Load Only Relevant Tools
  ↓
Permission Check
  ↓
Execute

This reduces:

• tool confusion
• prompt size
• attack surface
• unnecessary complexity

15. What failure modes can occur during tool execution?

A successful API call does not necessarily mean a successful agent action.

Common failures include:

Network failure

Timeout
Connection error
Service unavailable

Validation failure

Invalid date
Missing parameter
Incorrect JSON

Semantic failure

The API succeeds.

But returns:

[]

Or irrelevant information.

Partial failure

Suppose the agent processes 10,000 records.

7,000 completed
3,000 failed

Restarting everything may duplicate work.

The source explicitly distinguishes network/timeout failures, validation failures, semantically useless results, and partial failures requiring recovery rather than simple restart.

16. What is the ReAct pattern?

ReAct stands for:

Reasoning + Acting

The loop is conceptually:

Think
  ↓
Act
  ↓
Observe
  ↓
Think Again
  ↓
Act Again

For example:

Question:
What was the company's revenue last quarter?
Action:
Query financial database
Observation:
Revenue = $24.3M
Next Action:
Retrieve previous quarter
Observation:
Previous = $20.1M
Final Response:
Revenue increased by approximately 21%.

The source describes the ReAct cycle as Thought → Action → Observation, emphasizing that observations ground subsequent steps and make actions easier to validate.

The major advantage is that the agent can update its next action based on actual tool output rather than relying only on its initial assumptions.

17. Why can uncontrolled reasoning loops become dangerous?

Consider:

Tool fails
   ↓
Retry
   ↓
Fails
   ↓
Retry
   ↓
Fails
   ↓
Retry forever...

Now multiply this across thousands of users.

Your API bill becomes interesting.

Other failure modes include:

• circular reasoning
• repeated research
• goal ambiguity
• endless retries

The source specifically recommends hard maximum iteration limits because uncontrolled loops can repeatedly execute the same actions and consume significant resources.

A production agent needs:

MAX_ITERATIONS = 10
MAX_TOOL_RETRIES = 3
MAX_EXECUTION_TIME = 60

Stopping is a feature.

18. How would you detect that an agent is stuck?

One simple approach is tracking repeated actions.

tool_calls = [
    ("search", "Tesla revenue"),
    ("search", "Tesla revenue"),
    ("search", "Tesla revenue")
]

That’s suspicious.

Other approaches include:

State hashing

If the state isn’t changing:

State A
State A
State A
State A

the agent may not be making progress.

Similarity detection

Compare consecutive outputs.

If the agent repeatedly produces nearly identical decisions, it may be looping.

Explicit stuck state

Allow the agent to return:

STATUS = "STUCK"

The source suggests state hashing, repeated tool-call detection, similarity analysis across consecutive reasoning steps, and explicit exit mechanisms.

19. What are the limitations of long chain-of-thought style workflows?

More reasoning is not always better.

Potential problems include:

Error propagation

One incorrect assumption can influence every later step.

Token growth

Long reasoning workflows consume context.

Plan rigidity

The system may continue following an outdated plan even after receiving new evidence.

The source frames these issues as error propagation, token exhaustion, and models persisting with bad initial plans, while suggesting more structured planner-executor approaches for complex workflows.

A better architecture can be:

Planner
   ↓
Plan
   ↓
Executor
   ↓
Results
   ↓
Evaluator
   ↓
Continue / Replan / Stop

This separates responsibilities.

20. How do you prevent an agent from hallucinating tool outputs?

This is a critical production problem.

The LLM should never be trusted to “simulate” a tool response.

Wrong:

LLM:
"The database returned 10,000 customers."

But the database was never called.

Better:

LLM
 ↓
Tool Call Request
 ↓
Actual Tool Execution
 ↓
Real Output
 ↓
Structured Result
 ↓
LLM

The source emphasizes strict separation between LLM reasoning and actual tool execution, structured representation of real outputs, and validation before continuing.

The core principle

The LLM can propose an action.
The application executes the action.

Those are two separate responsibilities.

The Bigger Lesson

After going through these questions, one thing becomes clear.

AI agents are not primarily about writing a better prompt.

They’re about designing a reliable system around an unreliable decision-making component.

A production agent needs:

                 ┌──────────────┐
                 │ User Goal    │
                 └──────┬───────┘
                        ↓
                 ┌──────────────┐
                 │ LLM / Router │
                 └──────┬───────┘
                        ↓
                ┌───────▼────────┐
                │ Planner/Policy │
                └───────┬────────┘
                        ↓
                 ┌──────────────┐
                 │Tool Selection│
                 └──────┬───────┘
                        ↓
                ┌───────▼────────┐
                │ Validation     │
                │ Authorization  │
                └───────┬────────┘
                        ↓
                 ┌──────────────┐
                 │ Tool Execution│
                 └──────┬───────┘
                        ↓
                 ┌──────────────┐
                 │ State Update  │
                 └──────┬───────┘
                        ↓
                  Continue?
                   /      \
                 Yes       No
                  ↓         ↓
                Loop      Answer

The LLM is only one component.

The actual engineering challenge is everything around it.

Key Takeaways

If you’re preparing for an AI Engineer or GenAI Engineer interview, make sure you understand these concepts beyond the framework level.

You should be able to explain:

• The difference between a chatbot and an agent
• When agents make systems worse
• Reactive vs. deliberative architectures
• Adaptive reasoning depth
• State management
• Tool selection
• Tool authorization
• Tool failure handling
• ReAct loops
• Infinite loop detection
• Stopping conditions
• Structured tool outputs
• Production guardrails

The strongest candidates don’t just know how to write:

agent.invoke()

They understand what can go wrong after that line executes.

And in production AI systems, that is usually where the interesting problems begin.

About the Author

Ved Prakash is a Data Scientist and AI Engineer working with Machine Learning, Generative AI, LLMs, RAG, AI Agents, and production AI systems. He writes practical tutorials and interview-focused content on GeekyCodes, covering AI Engineering, Data Engineering, Machine Learning, and Generative AI.