Inference cost vs UX: why cheaper AI models can quietly reduce engagement (Amplitude)

Original article (source): Amplitude - “How to Balance Inference Cost and User Experience for Agents” (Jun 17, 2026)

• https://amplitude.com/blog/agent-analytics-beta

What it says (in plain English)

Amplitude describes a very normal 2026 problem: you ship an in-product agent, the bill is painful, and you ask, “Can we switch to a cheaper model without breaking the experience?”

They ran a controlled experiment swapping their production agent from a higher-cost model to a cheaper one. The headline results were mixed:

• Costs improved: per active chat user session cost dropped (they cite ~$4.88 → ~$2.33).
• A conversion proxy stayed flat: they used a “valuable action same-day” definition (save/copy/view or click a surfaced CTA).
• But UX got worse in a way users felt: response time roughly doubled (they cite ~64s → ~120s) and users sent ~10% fewer messages on the cheaper model.

The key idea is practical: if latency rises, users self-censor. They stop asking “one more thing”, even if the answers are fine.

The useful takeaways

• Model choice is a product decision, not just a finance decision. Your unit economics only matter if engagement does not quietly slip.
• Offline evals miss the lived experience. Tool-call orchestration, retries, and real-world request mix can dominate latency and cost.
• Measure “friction”, not only “outcomes”. For agents, friction signals include response latency, tool-call count, error rate, and “messages per user/session”.

What to do next (tiny wins)

• Add two guardrails to any model swap test: p95 latency and messages/session (or “turns per session”).
• Segment by user type: new users, power users, and “came for one answer” users behave differently, so an average can lie.
• Write a rollback rule before you ship: e.g., “If latency increases by >30% and turns/session drop by >5%, we revert even if conversion is flat.”

Read the original: https://amplitude.com/blog/agent-analytics-beta