AI Application Security

An AI application combines a model with ordinary software, data sources, prompts, retrieval systems, tools, identities, and infrastructure. AI application security protects that whole system. Securing only the model leaves the most important trust boundaries unexamined.

The useful mental model is a probabilistic component inside a deterministic security envelope. The model can interpret language and generate useful responses. It can also misunderstand instructions, follow hostile content, disclose context, or produce unsafe output. Code around the model must enforce the decisions that require certainty.

users and external content
           ↓
authentication → application policy → context builder
                                      ↓
data stores → retrieval controls → model → output controls
                                      ↓
                              tool authorization
                                      ↓
                              downstream systems

Every arrow crosses a boundary. Ask who controls the input, what authority moves across the boundary, and what happens if the receiving component treats data as instructions.

Why the security model changes

Conventional application security still applies. You need secure authentication, authorization, dependency management, secrets handling, isolation, patching, logging, and incident response. AI adds failure modes because model behavior depends on data expressed in the same language as instructions.

A web page retrieved for summarization is data to the application. To the model, a sentence inside that page can look like a new instruction. A generated string is output from the model. To a browser, shell, database, or tool, that string may become executable input. This blurring of code, instruction, and data creates new paths through otherwise familiar systems.

Models are also nondeterministic. One successful test does not prove that a nearby prompt, a longer conversation, a different document, or a model update will behave the same way. Security testing therefore needs scenario sets, repeated trials where useful, measurable outcomes, and regression baselines.

Start with the application, not the prompt

An application threat model should include at least these assets and boundaries:

user prompts, conversation history, and uploaded files;
system prompts, templates, policies, and model configuration;
training, fine-tuning, evaluation, and retrieval data;
model files, adapters, embeddings, and software dependencies;
model-provider and retrieval APIs;
vector stores, databases, caches, logs, and feedback pipelines;
tools that read data or cause side effects;
identities, credentials, approval paths, and downstream permissions;
generated text, code, queries, media, and structured data;
budgets, quotas, latency targets, and recovery mechanisms.

AI Application Security

Why the security model changes

Start with the application, not the prompt

Relevant careers