Engineering
How we grew 100 times without growing our AI bill
TL;DRCost per AI response fell 80% while users grew 100x. Send less up front, keep the prompt cache intact, and remember that a shorter request can still cost more.
Engineering
1 post
1 post
Engineering
TL;DRCost per AI response fell 80% while users grew 100x. Send less up front, keep the prompt cache intact, and remember that a shorter request can still cost more.