LLM
What makes an LLM call cacheable is not temperature
Caching an LLM call looked trivial until I checked why it was safe. The reason I reached for first was wrong, and partial hits corrupted the totals.
Search posts
LLM
Caching an LLM call looked trivial until I checked why it was safe. The reason I reached for first was wrong, and partial hits corrupted the totals.