📚 Josh Beckman's Highlights

Join 📚 Josh Beckman's Highlights

A batch of the best highlights from what Josh's read, .

When adding jitter to scheduled work, we do not select the jitter on each host randomly. Instead, we use a consistent method that produces the same number every time on the same host. This way, if there is a service being overloaded, or a race condition, it happens the same way in a pattern. We humans are good at identifying patterns, and we're more likely to determine the root cause. Using a random method ensures that if a resource is being overwhelmed, it only happens - well, at random. This makes troubleshooting much more difficult.

Timeouts, Retries, and Backoff With Jitter

Amazon Web Services, Inc.

Use a small model to generate a 'draft' output, then use a larger and smarter model to score the 'draft', then use a rejection sampling scheme to accept the tokens which are agreed by the small and large models. In tests, they find that a draft model can give them speedups ranging between 1.92X (on a summarization benchmark called XSum) and 2.46X on a code generation task called HumanEval.

Fwd: Import AI 317: DeepMind Speeds Up Language Model Sampling; Voice Cloning Tech Gets Abused; More Scaling Laws for RL

Josh Beckman

Crichton coined the term ***Gell-Mann amnesia effect*** to describe the phenomenon of experts reading articles within their fields of expertise and finding them to be error-ridden and full of misunderstanding, but seemingly forgetting those experiences when reading articles in the same publications written on topics outside of their fields of expertise, which they believe to be credible.

Michael Crichton - Wikipedia

wikipedia.org

...catch up on these, and many more highlights

Readwise.

Join 📚 Josh Beckman's Highlights

A batch of the best highlights from what Josh's read, .

LEARN

EXPLORE

CONTACT