Blog
What I learn, written down.
Articles on AI in production, search and products. When an idea is easier to grasp by playing with it, the article includes a demo.
Streaming LLM answers: the proxy that held the words back
Streaming does not make a model faster, it makes it readable sooner. The words still have to arrive: buffering, compression, dropped connections, and what the interface has to get right. With a simulation of the trip.