My Evolution Towards Becoming an Applied AI Engineer

After building my first AI application, I thought I had the gist of it — I just needed to refine and improve to build good AI applications.

Then yesterday's class exposed how much I was still missing.

Even after applying layers of defense to prevent abuse of my LLM API, there is so much more to building a good, production-ready AI application.

But I have taken the first step in becoming an applied AI engineer.

What Makes an AI Application Production-Ready?

The other components needed for a production-ready AI application are:

1. Guardrails

   • Input and output guardrails to protect the system and prevent leaking sensitive information to the user.

2. Permission or approval-based tool calling

   • Also known as human-in-the-loop.
   • Done to prevent the AI from taking destructive actions.

3. Context window and context window management

4. Memory

5. Structured response and output

   • SDKs provide a mechanism to get structured output from an LLM.

We will be studying all of these in depth in the coming classes of the Gen AI cohort.

Role-Based Tool Access

We also learned that based on the user's role, we select different system prompts with different lists of tools.

So an attacker cannot exploit the system even after getting a tool call request from the LLM. Since tools are loaded based on role, it reduces the blast radius.

Authorization-based prompt flow — user request passes through authentication and authorization checks, system prompt is selected based on role, then LLM APIs are called and the response is returned to the user

REST API vs SDK vs Agent SDK

We learned the difference between an SDK and an Agent SDK.

LLM providers expose REST APIs so we can call them directly in our application — opening HTTP connections, passing parameters, handling response codes, parsing JSON, and processing output ourselves.

SDKs provide abstraction over this by giving us functions to call that internally handle all of these things, making a developer's life easier and providing convenience.

When we need to build an AI agent using an SDK by providing tools to the LLM, we write the loops, tool calling, human-in-the-loop, and orchestration code ourselves.

But the Agent SDK provides abstraction over the SDK — it handles tool calling, loops, guardrails, and retries internally, offering even more convenience.

Diagram comparing REST API, SDK, and Agent SDK abstraction layers — from HTTP handling and JSON parsing, to model input/output, to prompts, tools, guardrails, and retry loops

Understanding these decisions helps us make better architectural choices when building AI applications.