New KV cache compaction technique cuts LLM memory 50x without accuracy loss

MIT researchers developed Attention Matching, a KV cache compaction technique that compresses LLM memory by 50x in seconds — ...
InfoQ

Google BigQuery Previews Cross-Region SQL Queries for Distributed Data

Google Cloud has recently announced the preview of a global queries feature for BigQuery. The new option lets developers run ...

Databricks built a RAG agent it says can handle every kind of enterprise search

Databricks' KARL agent uses reinforcement learning to generalize across six enterprise search behaviors — the problem that breaks most RAG pipelines.
EDN

Last-level cache has become a critical SoC design element

LLC, positioned between external memory and internal subsystems, stores frequently accessed data close to compute resources.

The Real-Time Data Problem with LLMs

In many ways, generative AI has made finding information on the Internet a lot easier. But, because LLMs are trained on past ...