BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//pretalx//programme.europython.eu//europython-2026//talk//E3GVVR
BEGIN:VTIMEZONE
TZID:CET
BEGIN:STANDARD
DTSTART:20001029T040000
RRULE:FREQ=YEARLY;BYDAY=-1SU;BYMONTH=10
TZNAME:CET
TZOFFSETFROM:+0200
TZOFFSETTO:+0100
END:STANDARD
BEGIN:DAYLIGHT
DTSTART:20000326T030000
RRULE:FREQ=YEARLY;BYDAY=-1SU;BYMONTH=3
TZNAME:CEST
TZOFFSETFROM:+0100
TZOFFSETTO:+0200
END:DAYLIGHT
END:VTIMEZONE
BEGIN:VEVENT
UID:pretalx-europython-2026-E3GVVR@programme.europython.eu
DTSTART;TZID=CET:20260716T124500
DTEND;TZID=CET:20260716T131500
DESCRIPTION:Your team ships an AI feature and users love it. Then one viral
  post turns "normal load" into hundreds of LLM requests per second.\n\nLLM
  calls don't behave like traditional API requests. They're slow (sometimes
  seconds)\, expensive\, rate-limited by providers\, and a single provider 
 outage can take your entire feature down. You can't just "add more servers
 ." You need a routing layer that knows where to send traffic\, when to bac
 k off\, and how to fail without taking everything with it.\n\nIn this talk
 \, we'll walk through the LLM traffic routing architecture we built in Pyt
 hon at Manychat\, where we serve AI-powered automation to thousands of Ins
 tagram and messaging accounts. Everything we'll show is running in product
 ion.\n\nWe'll cover the core gateway patterns for multi-provider LLM traff
 ic\, implemented using LiteLLM Router as a reference design.\n\nBy the end
 \, you'll walk away with:\n\n- A weighted routing blueprint you can adapt 
 to your own provider mix\n- Fallback and cooldown rules designed to surviv
 e real outages\n- Practical rate limiting (requests and tokens) with retry
  backoff\n- The monitoring baseline (latency\, tokens\, errors by provider
 \, weight drift) to catch issues before they cascade\n- A checklist for ro
 lling this out safely\, incrementally
DTSTAMP:20260524T130552Z
LOCATION:Chamber Hall B (S3B)
SUMMARY:Surviving LLM Traffic Spikes: Routing\, Rate Limits\, and Failover 
 in Python - Sergi Porta
URL:https://programme.europython.eu/europython-2026/talk/E3GVVR/
END:VEVENT
END:VCALENDAR