FastAPI: Deployment & graceful shutdown

A deployment replaces running processes that are in the middle of serving customers. Done carelessly, every release causes a burst of 502s, lost requests and duplicate payments. Done well, nobody notices. This part covers the container image, what the proxy in front of you changes (client IPs, HTTPS, path prefixes), and the shutdown sequence, including the timing details most teams learn from an incident.

1. The container image

Dockerfile (a pattern; Docker was not run for this part)

# syntax=docker/dockerfile:1
FROM python:3.14-slim AS build
ENV PIP_NO_CACHE_DIR=1 PIP_DISABLE_PIP_VERSION_CHECK=1
COPY requirements.txt /tmp/
RUN python -m venv /venv && /venv/bin/pip install -r /tmp/requirements.txt   # pinned versions

FROM python:3.14-slim
ENV PATH="/venv/bin:$PATH" PYTHONDONTWRITEBYTECODE=1 PYTHONUNBUFFERED=1
RUN useradd --create-home --uid 10001 app                                  # never run as root
COPY --from=build /venv /venv
WORKDIR /srv
COPY --chown=app:app app ./app
USER app
EXPOSE 8000
# exec form: uvicorn is PID 1 and receives SIGTERM directly (shell form would swallow it)
CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8000", \
     "--forwarded-allow-ips", "10.0.0.0/8", "--timeout-keep-alive", "75", \
     "--timeout-graceful-shutdown", "20", "--no-server-header"]

2. Behind a proxy: client IPs and HTTPS

In production a load balancer or ingress terminates TLS and forwards plain HTTP. Your app then sees the proxy’s IP and scheme http, unless it trusts the proxy’s X-Forwarded-For and X-Forwarded-Proto headers. Uvicorn does this with ProxyHeadersMiddleware (enabled by default, but trusting only 127.0.0.1; widen it with --forwarded-allow-ips). The same middleware runs here directly:

who is the client? It depends on whom you trust

import asyncio
import httpx
from fastapi import FastAPI, Request
from uvicorn.middleware.proxy_headers import ProxyHeadersMiddleware

app = FastAPI()

@app.get("/whoami", name="whoami")
async def whoami(request: Request):
    return {"client_ip": request.client.host, "scheme": request.url.scheme,
            "self_link": str(request.url_for("whoami"))}

LB = "10.0.0.2"                          # the load balancer's address
FORWARDED = {"X-Forwarded-For": "49.36.12.7", "X-Forwarded-Proto": "https", "Host": "api.brew.example"}

async def ask(label, asgi_app, peer, headers):
    transport = httpx.ASGITransport(app=asgi_app, client=(peer, 50000))
    async with httpx.AsyncClient(transport=transport, base_url="http://api.brew.example") as c:
        print(f"{label:<44}", (await c.get("/whoami", headers=headers)).json())

async def main():
    await ask("no proxy handling, via the LB", app, LB, FORWARDED)
    trusted = ProxyHeadersMiddleware(app, trusted_hosts=[LB])
    await ask("trust the LB, via the LB", trusted, LB, FORWARDED)
    await ask("trust the LB, attacker direct + spoofed XFF", trusted, "198.51.100.66",
              {"X-Forwarded-For": "127.0.0.1", "X-Forwarded-Proto": "https"})
    everyone = ProxyHeadersMiddleware(app, trusted_hosts="*")
    await ask("trust '*', attacker direct + spoofed XFF", everyone, "198.51.100.66",
              {"X-Forwarded-For": "127.0.0.1", "X-Forwarded-Proto": "https"})
asyncio.run(main())

Output

no proxy handling, via the LB                {'client_ip': '10.0.0.2', 'scheme': 'http', 'self_link': 'http://api.brew.example/whoami'}
trust the LB, via the LB                     {'client_ip': '49.36.12.7', 'scheme': 'https', 'self_link': 'https://api.brew.example/whoami'}
trust the LB, attacker direct + spoofed XFF  {'client_ip': '198.51.100.66', 'scheme': 'http', 'self_link': 'http://api.brew.example/whoami'}
trust '*', attacker direct + spoofed XFF     {'client_ip': '127.0.0.1', 'scheme': 'https', 'self_link': 'https://api.brew.example/whoami'}

3. Behind a path prefix: root_path

When the proxy serves your API under a prefix (https://brew.example/orders-api/…) and strips it before forwarding, the app must still know the public prefix. Otherwise /docs fetches the wrong openapi.json and every generated link is wrong. Tell it with --root-path, here on a real uvicorn server:

--root-path on a real server

import os
import re
import signal
import socket
import subprocess
import sys
import time
from pathlib import Path
import httpx

Path("rp_app.py").write_text('''
from fastapi import FastAPI, Request
app = FastAPI()

@app.get("/whoami", name="whoami")
async def whoami(request: Request):
    return {"path": request.url.path, "self_link": str(request.url_for("whoami"))}
''', encoding="utf-8")

for extra in ([], ["--root-path", "/orders-api"]):
    with socket.socket() as s:
        s.bind(("127.0.0.1", 0))
        port = s.getsockname()[1]
    flags = subprocess.CREATE_NEW_PROCESS_GROUP if os.name == "nt" else 0
    server = subprocess.Popen([sys.executable, "-m", "uvicorn", "rp_app:app", "--port", str(port),
                               "--log-level", "warning", *extra], creationflags=flags)
    base = f"http://127.0.0.1:{port}"
    for _ in range(100):
        try:
            httpx.get(base + "/whoami")
            break
        except httpx.TransportError:
            time.sleep(0.2)
    # the proxy receives /orders-api/whoami, strips the prefix, and forwards /whoami:
    print("uvicorn", " ".join(extra) or "(no root path)")
    print("  request     ", httpx.get(base + "/whoami").json())
    print("  openapi     ", httpx.get(base + "/openapi.json").json().get("servers"))
    print("  /docs loads ", re.search(r"url: '([^']+)'", httpx.get(base + "/docs").text).group(1))
    server.send_signal(signal.CTRL_BREAK_EVENT if os.name == "nt" else signal.SIGTERM)
    server.wait(timeout=30)

Output

uvicorn (no root path)
  request      {'path': '/whoami', 'self_link': 'http://127.0.0.1:12173/whoami'}
  openapi      None
  /docs loads  /openapi.json
uvicorn --root-path /orders-api
  request      {'path': '/orders-api/whoami', 'self_link': 'http://127.0.0.1:12180/orders-api/whoami'}
  openapi      [{'url': '/orders-api'}]
  /docs loads  /orders-api/openapi.json

4. Graceful shutdown

On SIGTERM (from docker stop or Kubernetes), uvicorn stops accepting connections, lets in-flight requests finish (up to --timeout-graceful-shutdown), then runs the lifespan shutdown, and only then exits. Here is the whole sequence on a real server, with a slow request in flight:

SIGTERM with a request in flight

import os
import signal
import socket
import subprocess
import sys
import threading
import time
from pathlib import Path
import httpx

Path("main.py").write_text('''
import asyncio, logging
from contextlib import asynccontextmanager
from fastapi import FastAPI
log = logging.getLogger("uvicorn.error")

@asynccontextmanager
async def lifespan(app):
    log.info("startup: pools opened")
    yield
    log.info("shutdown: flushing the outbox, closing pools")

app = FastAPI(lifespan=lifespan)

@app.get("/checkout")
async def checkout(seconds: float = 3.0):          # a slow in-flight request
    await asyncio.sleep(seconds)
    return {"order": "paid"}
''', encoding="utf-8")

with socket.socket() as s:
    s.bind(("127.0.0.1", 0))
    port = s.getsockname()[1]
flags = subprocess.CREATE_NEW_PROCESS_GROUP if os.name == "nt" else 0
server = subprocess.Popen([sys.executable, "-m", "uvicorn", "main:app", "--port", str(port),
                           "--timeout-graceful-shutdown", "20", "--no-access-log"],
                          stdout=subprocess.PIPE, stderr=subprocess.STDOUT, text=True, creationflags=flags)
base = f"http://127.0.0.1:{port}"
for _ in range(100):
    try:
        httpx.get(f"{base}/checkout?seconds=0")
        break
    except httpx.TransportError:
        time.sleep(0.2)

t0 = time.perf_counter()
events = []
def in_flight():
    r = httpx.get(f"{base}/checkout?seconds=3", timeout=30)
    events.append(f"t={time.perf_counter() - t0:3.1f}s  in-flight request finished: {r.status_code} {r.json()}")
worker = threading.Thread(target=in_flight)
worker.start()

time.sleep(0.5)
server.send_signal(signal.CTRL_BREAK_EVENT if os.name == "nt" else signal.SIGTERM)   # what `docker stop` sends
events.append(f"t={time.perf_counter() - t0:3.1f}s  SIGTERM sent (CTRL_BREAK on Windows)")
time.sleep(0.3)
try:
    httpx.get(f"{base}/checkout?seconds=0", timeout=1)
    events.append("new request accepted?!")
except httpx.TransportError:
    events.append(f"t={time.perf_counter() - t0:3.1f}s  new connection not accepted: the listener is closed")
worker.join()
server.wait(timeout=30)
events.append(f"t={time.perf_counter() - t0:3.1f}s  process exited")
print("\n".join(events))
print("\nserver log:")
print("\n".join("  " + line for line in server.stdout.read().splitlines() if "running on" not in line.lower()))

Output

t=0.5s  SIGTERM sent (CTRL_BREAK on Windows)
t=1.8s  new connection not accepted: the listener is closed
t=3.0s  in-flight request finished: 200 {'order': 'paid'}
t=3.2s  process exited

server log:
  INFO:     Started server process [37744]
  INFO:     Waiting for application startup.
  INFO:     startup: pools opened
  INFO:     Application startup complete.
  INFO:     Shutting down
  INFO:     Waiting for connections to close. (CTRL+C to force quit)
  INFO:     Waiting for application shutdown.
  INFO:     shutdown: flushing the outbox, closing pools
  INFO:     Application shutdown complete.
  INFO:     Finished server process [37744]

5. Zero-downtime releases on Kubernetes

The subtle part: when a pod is deleted, Kubernetes sends SIGTERM and removes the pod from the Service endpoints at the same time, and load balancers take a few seconds to notice. Without the preStop sleep, uvicorn closes its listener while traffic is still arriving, which shows up as connection errors on every deploy.

deployment.yaml (a pattern; not applied to a cluster here)

apiVersion: apps/v1
kind: Deployment
metadata: {name: orders-api}
spec:
  replicas: 3
  strategy: {type: RollingUpdate, rollingUpdate: {maxUnavailable: 0, maxSurge: 1}}
  selector: {matchLabels: {app: orders-api}}
  template:
    metadata: {labels: {app: orders-api}}
    spec:
      terminationGracePeriodSeconds: 45
      containers:
        - name: api
          image: registry.example/orders-api:1.4.0        # immutable tag
          ports: [{containerPort: 8000}]
          envFrom: [{secretRef: {name: orders-api}}]
          resources:
            requests: {cpu: 500m, memory: 256Mi}
            limits: {memory: 512Mi}                       # memory limit, but no CPU limit (avoids throttling)
          startupProbe:   {httpGet: {path: /livez, port: 8000}, periodSeconds: 2, failureThreshold: 30}
          livenessProbe:  {httpGet: {path: /livez, port: 8000}, periodSeconds: 10, timeoutSeconds: 2}
          readinessProbe: {httpGet: {path: /readyz, port: 8000}, periodSeconds: 5, timeoutSeconds: 2}
          lifecycle:
            preStop: {sleep: {seconds: 10}}               # sleep action: Kubernetes 1.30+

6. The details that cause incidents

Recap