Singleflight

When many apprentices clamour for the same scroll, send the familiar but once.

grimoire fragment · main.go

package main

import (
	"fmt"
	"sync"
	"time"

	"golang.org/x/sync/singleflight"
)

var grp singleflight.Group

func main() {
	var wg sync.WaitGroup

	for idx := 0; idx < 4; idx++ {
		wg.Add(1)

		go func(apprentice int) {
			defer wg.Done()
      
			scroll, err := lookup("the-prophecy")
			if err != nil {
			    fmt.Printf("apprentice %d error, %v\n", apprentice, err)

			    return
			}

			fmt.Println(scroll)
		}(idx)
	}

	wg.Wait()
}

func lookup(key string) (string, error) {
	// one in flight call per key.
	val, err, shared := grp.Do(key, func() (any, error) {
		time.Sleep(2000 * time.Millisecond)

		return "scroll of " + key, nil
	})
	if err != nil {
		return "", err
	}

  fmt.Println("shared?", shared)

	return val.(string), nil
}

The summoning circle

Dials & sigils callers 4 Number of goroutines shouting identical requests at the same instant. latency 2000 How long the upstream takes to answer singleflight on When off, every caller bothers the upstream. When on, all are bound to one casting.

What you are witnessing

A singleflight group is a small, watchful door warden. When multiple goroutines call g.Do(key, fn) with the same key at the same time, the warden lets exactly one familiar perform fn. The rest wait politely by the door for the familiar's return, upon which each receive an identical copy of the result.

Why a wizard cares

The classic use is avoidance of cache stampede. If a popular value expires when a thousand requests arrive in the same instant, then your db will thrash. With singleflight, it is asked once so that the other 999 sip from the same cauldron.

Try it! Crank callers up and turn singleflight off. Watch each apprentice hurridly reach for the scroll. Now turn it back on and their reaching collapse into a single ceremonious request. The reply perfectly fans out to all.

Pitfalls

Failure is shared. If the in flight call errors then every waiter receives that same error.
Slow callers are punished by the slowest. All waiters block until the call returns. Pair with a context if latency matters.
Keys must be canonical. "User:42" and "user:42" are separate doors.
It is not a cache. Once the call completes, the next caller starts a fresh casting. Combine with a real cache if you need memoisation.

Reagents required: go get golang.org/x/sync/singleflight.