Skip to main content
Mapagam

Securing AI Agent Usage

1. Preventing Prompt Injection Attacks

Attack Type Description Mitigation Strategy
Prompt Injection Malicious input designed to manipulate AI agent behavior by injecting unwanted commands or data into prompts Input sanitization, strict prompt formatting, context validation

Example: Vulnerable Prompt Handling

// Vulnerable code
const userInput = "... malicious input ...";
const prompt = `You are a helpful assistant. User says: ${userInput}";

Example: Secure Prompt Handling

// Secure code with sanitization
const sanitizeInput = (input) => {
  return input.replace(/(\${{.*?}})/g, "");
};
const userInput = sanitizeInput("... malicious input ...");
const prompt = `You are a helpful assistant. User says: ${userInput}";
Note: Always validate and sanitize user inputs before including them in prompts to prevent injection attacks.

Key Mitigation Techniques

  • Input validation and sanitization
  • Template-based prompt construction
  • Context-aware input filtering
  • Rate limiting and monitoring for suspicious patterns

2. Avoiding Secrets Exposure in Prompts

Note: AI agents often process user inputs as prompts, which can inadvertently expose sensitive data like API keys, passwords, or personal information.
Exposure Risk Description Mitigation Strategy
API Keys in Prompts Hardcoded or dynamically injected API keys in prompts can be logged or exposed. Use environment variables or secure secret management tools (e.g., AWS Secrets Manager, HashiCorp Vault) to inject secrets at runtime.
User Input Handling Unfiltered user input may contain sensitive data that gets processed in prompts. Implement input sanitization and filtering to remove or mask sensitive patterns (e.g., regex for PII).
Prompt Logging Logs or audit trails may capture full prompts including secrets. Ensure logging systems strip or mask sensitive fields before storing or transmitting logs.

Example: Secure Prompt Construction

const apiKey = process.env.API_KEY;
const userInput = "...";

// Securely inject key without exposing it in prompt
const prompt = `You are an assistant. User input: ${userInput}. API key is hidden.`;

// Log only sanitized version
console.log("Prompt sent to AI: " + prompt.replace(apiKey, "[HIDDEN]"));

Best Practices for Prompt Security

  • Never embed secrets directly in prompts or user-facing text.
  • Use secret management systems to inject sensitive values at runtime.
  • Sanitize user input before including it in prompts or logs.
  • Log only sanitized or anonymized versions of prompts for debugging.

3. Restricting Agent File System Access

Permission Description Implementation
read Allows reading files and directories { "read": [ "/safe/dir1", "/safe/dir2" ] }
write Allows creating and modifying files { "write": [ "/safe/output" ] }
execute Allows executing scripts or binaries { "execute": [ "/safe/scripts" ] }

Example: Configuring File System Permissions

{
  "aiAgent": {
    "fileSystem": {
      "read": [ "/project/src", "/project/config" ],
      "write": [ "/project/logs" ],
      "execute": [ "/project/scripts" ]
    }
  }
}
Note: Ensure all paths are absolute and restricted to necessary directories to prevent unauthorized access.
Warning: Granting write or execute permissions to sensitive directories can lead to data corruption or security breaches.
Feature Status Notes
Path Sandboxing NEW Restricts file system access to a defined sandboxed environment
File Access Logs DEPRECATED v2.0 Logs all file access attempts (replaced by audit trails)

4. Reviewing Agent-Executed Commands

Example: Command Execution Log

{
  "timestamp": "2026-01-15T10:30:00Z",
  "agentId": "ai-agent-456",
  "command": "npm install express",
  "executedBy": "user123",
  "status": "success",
  "output": "added 156 packages, removed 23 packages"
}
Field Description Notes
timestamp ISO 8601 formatted time of execution Used for audit trails
agentId Unique identifier for the AI agent Helps in tracking origin
command Shell or CLI command executed Must be logged exactly as run
executedBy User or system that initiated the command Can be a human or automated process
status Execution result (success, failed, etc.) Used for monitoring and alerts
output Standard output or error logs from command Optional but recommended for debugging
Note: All commands should be reviewed before execution in high-security environments.
Warning: Commands that modify system state or access sensitive data require additional approval steps.

Best Practices for Command Review

  • Log all commands with timestamps and user context
  • Implement command whitelisting for critical systems
  • Use sandboxed environments for AI-agent commands when possible
  • Set up alerts for unusual or high-risk command patterns

5. Sandboxing Agent Tool Execution

Tool Description Sandboxing Method Status
Node.js Child Processes Executes external tools in isolated subprocesses child_process.spawn() with restricted environment NEW
Docker Containers Runs tools in isolated container environments Docker API with resource limits and network isolation DEPRECATED
Web Workers Executes code in isolated browser threads Worker API with sandboxed context BETA

Example: Node.js Sandboxing

const { spawn } = require('child_process');

const sandboxedTool = spawn('python', ['script.py'], {
  stdio: 'pipe',
  env: { ...process.env, NODE_ENV: 'production' },
  timeout: 5000,
});

sandboxedTool.stdout.on('data', (data) => {
  console.log(`Output: ${data}`);
});

sandboxedTool.on('close', (code) => {
  console.log(`Process exited with code ${code}`);
});
Note: Sandboxing must prevent access to sensitive system resources and enforce strict input validation.
Warning: Inadequate sandboxing can lead to privilege escalation or data leakage.

6. Auditing Agent Action Logs

Action Description Log Fields Security Impact
API Call Agent initiates API request to external service timestamp, requestId, endpoint Exposes data flow to external systems
Data Retrieval Agent fetches data from database or storage source, dataId, accessLevel Potential for unauthorized data access
Code Generation Agent generates or modifies code codeSnippet, language, modifiedFiles Risk of malicious or insecure code injection
User Interaction Agent responds to user input or command userInput, response, sessionID Can be exploited for phishing or manipulation

Example: Log Format for API Call

{
  "timestamp": "2026-04-05T14:30:00Z",
  "requestId": "req-123456789",
  "endpoint": "/api/data/v1/retrieve",
  "source": "agent-xyz",
  "user": "user-987654321",
  "status": "success"
}
Note: Logs should be stored securely and retained for at least 90 days to support forensic analysis.
Warning: Ensure sensitive fields like userInput or codeSnippet are redacted or encrypted in logs to prevent data leakage.

Key Audit Points

  • Verify all external API calls are logged with full context
  • Validate that data access logs include source and target identifiers
  • Check code generation logs for potential injection risks
  • Confirm user interaction logs capture input and response content

7. Handling Untrusted Tool Output Safely

Example: Sanitizing Tool Output

// Example of sanitizing untrusted tool output before use
const sanitizer = require('sanitize-html');

const rawOutput = '<script>alert("XSS")</script><p>Safe content</p>';
const sanitized = sanitizer(rawOutput, {
  allowedTags: ['p', 'br'],
  allowedAttributes: {}
});

console.log(sanitized); // <p>Safe content</p>
Approach Description Use Case
Input Validation Validate and sanitize all inputs before processing Preventing injection attacks from tool responses
Output Filtering Filter or escape output to prevent code execution Rendering tool results in UI safely
Sandboxing Execute untrusted code in isolated environments Running tool-generated scripts safely
Note: Always treat tool outputs as untrusted, even if they come from a known source.
Warning: Never execute untrusted tool output directly in the main application context.

Best Practices for Tool Output Safety

  • Sanitize all tool-generated content before rendering
  • Use sandboxed environments for executing generated code
  • Implement strict input validation on tool parameters
  • Log and monitor all tool interactions for anomalies

8. Validating Agent-Generated Dependencies

Dependency Type Validation Method Tools/Approaches Notes
NPM/Yarn Packages Static Analysis npm audit, yarn audit Checks for known vulnerabilities in dependencies
Python Packages Static Analysis pip-audit, bandit Scans for security issues in Python packages
Docker Images Image Scanning Trivy, Snyk Scans container images for vulnerabilities and misconfigurations
Java/JAR Libraries Static Analysis OWASP Dependency-Check, SpotBugs Detects known vulnerabilities in Java libraries
Go Modules Static Analysis gosec, govulncheck Scans Go modules for security issues and vulnerabilities

Example: Validating NPM Dependencies

npm audit

Example: Scanning Docker Images for Vulnerabilities

trivy image node:18-alpine
Note: Regular validation should be part of CI/CD pipelines to ensure that agent-generated dependencies do not introduce security risks.

Was this article helpful?

Related Cheat Sheets