Tech Xplore on MSN

Adaptive drafter model uses downtime to double LLM training speed

Reasoning large language models (LLMs) are designed to solve complex problems by breaking them down into a series of smaller ...
NextBigFuture

Defeating Nondeterminism in LLM Inference by Thinking Machines

A research article by Horace He and the Thinking Machines Lab (X-OpenAI CTO Mira Murati founded) addresses a long-standing issue in large language models (LLMs). Even with greedy decoding bu setting ...
InfoWorld

Multi-token prediction technique triples LLM inference speed without auxiliary draft models

With reported 3x speed gains and limited degradation in output quality, the method targets one of the biggest pain points in production AI systems: latency at scale.
Semiconductor Engineering

Detailed Study of Performance Modeling For LLM Implementations At Scale (imec)

A new technical paper titled “System-performance and cost modeling of Large Language Model training and inference” was published by researchers at imec. “Large language models (LLMs), based on ...
TweakTown

Dell PowerEdge XE9712: NVIDIA GB200 NVL72-based AI GPU cluster for LLM training, inference

Dell has just unleashed its new PowerEdge XE9712 with NVIDIA GB200 NVL72 AI servers, with 30x faster real-time LLM performance over the H100 AI GPU. Dell Technologies' new AI Factory with NVIDIA sees ...
Digi Times

Pegatron unveils new server at OCP summit, targeting AI and LLM inference training

Pegatron is making significant strides in the server business, recently showcasing its latest AI solutions at the 2024 Open Compute Project (OCP) Global Summit. This includes six models of servers ...
VentureBeat

ServiceNow open sources Fast-LLM in a bid to help enterprises train AI models 20% quicker

Want smarter insights in your inbox? Sign up for our weekly newsletters to get only what matters to enterprise AI, data, and security leaders. Subscribe Now Training a large language model (LLM) is ...