Building an AI Feature into Flask

The architecture

Browser ──fetch──▶ Flask route ──SDK──▶ Model API
   ▲                   │
   └──── JSON ─────────┘   (key, limits and logging stay on the server)

The endpoint

import anthropic
from flask import Blueprint, jsonify, request, current_app
from flask_login import login_required, current_user

ai_bp = Blueprint("ai", __name__, url_prefix="/api/ai")
client = anthropic.Anthropic()            # create once, reuse (connection pooling)

SYSTEM = ("You are PandaDev's learning assistant. Help beginners with Python, "
          "SQL and web development. Use short explanations with small code examples.")
MAX_INPUT_CHARS = 4000

@ai_bp.post("/ask")
@login_required
@limiter.limit("20 per hour")
def ask():
    question = (request.get_json(silent=True) or {}).get("question", "").strip()
    if not question or len(question) > MAX_INPUT_CHARS:
        return jsonify(error="Question must be 1–4,000 characters."), 400
    try:
        response = client.messages.create(
            model=current_app.config["AI_MODEL"],       # e.g. "claude-opus-5-5"
            max_tokens=16000,
            output_config={"effort": "low"},          # quick answers for chat
            system=SYSTEM,
            messages=[{"role": "user", "content": question}],
        )
    except anthropic.RateLimitError:
        return jsonify(error="The assistant is busy. Try again in a minute."), 429
    except anthropic.APIConnectionError:
        return jsonify(error="Could not reach the AI service."), 503
    except anthropic.APIStatusError as exc:
        current_app.logger.error("AI error %s: %s", exc.status_code, exc.message)
        return jsonify(error="The assistant failed. Please try again."), 502

    if response.stop_reason == "refusal":
        return jsonify(answer="I can’t help with that request."), 200
    answer = "".join(b.text for b in response.content if b.type == "text")
    log_usage(current_user.id, response.usage)            # track cost per user
    return jsonify(answer=answer)

Key points:

• Authentication and rate limits stop strangers and bots from spending your budget.
• Input limits cap cost and abuse.
• Specific exception handling: retryable errors (429, 5xx, network) vs your own bugs (400). The SDK already retries transient errors twice by default.
• stop_reason tells you why generation ended: end_turn (finished), max_tokens (cut off), tool_use (wants a tool), refusal (declined for safety). Always check it before using the content.
• Effort (low → max) trades depth of reasoning for speed and cost; chat often works well at low, coding and analysis at higher levels.

The frontend

const csrf = document.querySelector('meta[name="csrf-token"]').content;
async function ask(question) {
  const res = await fetch("/api/ai/ask", {
    method: "POST",
    headers: { "Content-Type": "application/json", "X-CSRFToken": csrf },
    body: JSON.stringify({ question }),
  });
  const data = await res.json();
  output.textContent = res.ok ? data.answer : data.error;   // textContent: never innerHTML
}

Model output is untrusted text. Render it with textContent, or convert Markdown to HTML with an escaping renderer, never by injecting raw HTML.

Exercise

Add the /api/ai/ask endpoint to a Flask app with login, a rate limit, input validation and a small page that sends questions and displays answers. Log tokens per request.

Check yourself

• Why rate-limit AI endpoints per user?
• Which errors are worth retrying?
• Why render model output with textContent or an escaping Markdown renderer?