guide.md

Using objpool

Back to the README · API reference · Executable examples

Building and resetting objects

Both constructors accept the same callbacks:

newFn   func() *T
resetFn func(*T) error

newFn runs whenever no reusable object is available, on the goroutine calling Get. It must return a fully initialized, non-nil object. Required setup belongs here because fresh objects do not pass through resetFn.

resetFn runs before Get returns a recycled object. Reset only what needs resetting and keep useful capacity: for a bytes.Buffer, call Reset; for a slice, use s.buf = s.buf[:0]. Replacing the backing storage defeats reuse. Pass nil when there is nothing to reset.

If resetFn returns an error, Get discards the object and calls newFn for a replacement. The caller does not receive the reset error. If failures need to be observable, record them inside the callback using concurrency-safe logging or metrics.

Reset happens in Get so a failed reset can be replaced immediately. Put does not clear contents or call a cleanup hook. Objects that need explicit cleanup, such as open files and connections, need a separate lifecycle.

Both callbacks can execute concurrently for different objects. Protect any shared state they capture.

Ownership and concurrency

Create a pool once and share it across operations. A successful Get gives the caller exclusive ownership, provided the application follows these rules:

• Return each checked-out object at most once, to the pool it came from.
• Do not use the object after Put, including slices or pointers into its storage. Copy output that must outlive the checkout before returning it.
• Do not share a checked-out object between goroutines without synchronizing access yourself.
• Use New or NewFreeList; the pool zero values are not usable. Do not copy a pool after first use.

Put(nil) is ignored. A nil newFn panics at construction, and a newFn that returns nil panics in Get.

For a buffer, returning buf.Bytes() and then deferring Put(buf) is a common ownership mistake: the returned slice still aliases the pooled buffer. Copy it with append([]byte(nil), buf.Bytes()...) if the caller must keep the bytes.

Resetting a buffer changes its length, not the bytes in its backing storage. If immediate erasure is required, clear the relevant storage before Put, or avoid pooling that data.

Choosing retention behavior

Pool

Pool[T] wraps sync.Pool. The runtime can discard stored objects at any time, and a Get need not return an object previously offered with Put. Use it for temporary objects that can always be rebuilt.

Do not depend on a particular number of garbage collections before eviction: the public sync.Pool contract makes no retention guarantee.

FreeList

FreeList[T] holds strong references to idle objects, so garbage collection does not evict them while the list remains reachable. Get takes the most recently returned object, or builds one if none is idle. A failed reset still causes replacement.

With balanced checkouts and returns of objects created by the list, idle object count stays at or below peak concurrent checkouts. This assumes callers do not insert additional objects or return the same checkout multiple times.

maxIdle adds an explicit bound on idle count. A Put past that bound discards its object; maxIdle <= 0 disables the bound. Choose a limit that accommodates the concurrency you expect to reuse. A smaller limit trades retained memory for more rebuilding after busy periods.

The limit never blocks Get, limits active callers, or bounds each object's capacity. Use a semaphore or worker pool separately if you need a concurrency limit.

Controlling retained memory

A buffer can retain the capacity of the largest request it has seen. Use one RatchetTrim per pooled object to decide when to discard that object:

type scratch struct {
	buf  []byte
	trim objpool.RatchetTrim
}

scratches := objpool.NewFreeList(64,
	func() *scratch {
		return &scratch{trim: objpool.DefaultRatchetTrim()}
	},
	func(s *scratch) error { s.buf = s.buf[:0]; return nil },
)

release := func(s *scratch) {
	if s.trim.Due(cap(s.buf), len(s.buf)) {
		return // omit Put so the object can be garbage collected
	}
	scratches.Put(s)
}

Call release once per checkout, after the work is complete and before clearing the usage information. For a buffer that is drained during processing, track the cycle's peak usage separately: its final length may be zero even though it did substantial work.

A cycle is wasteful when retained bytes are at least Floor and strictly greater than Factor × used. Strikes consecutive wasteful cycles make Due return true. A non-wasteful cycle resets the count.

Setting	Default	Meaning
Floor	16 << 20 (16 MiB)	Smaller objects are exempt.
Factor	4	Retaining more than four times the used bytes is wasteful.
Strikes	8	Drop after eight consecutive wasteful use cycles.

The zero value never trims (Strikes <= 0 disables the policy). A non-positive Factor counts every cycle at or above the floor as wasteful. Pass nonnegative byte counts.

RatchetTrim is a decision helper; it does not free memory itself or run in the background. An object sitting idle in a free list is never examined. If Due returns true, omit Put and release other references. Each policy has one holder at a time and is not safe for concurrent use.

Reset arguments known only to the caller

A compression writer needs a new output destination for each checkout. Construct the pool with a nil reset callback, then call Reset after Get:

writers := objpool.NewFreeList(8,
	func() *gzip.Writer { return gzip.NewWriter(io.Discard) },
	nil,
)

compress := func(dst io.Writer, data []byte) error {
	w := writers.Get()
	w.Reset(dst)
	if _, err := w.Write(data); err != nil {
		// Discard on failure; do not return an unfinished writer.
		return err
	}
	if err := w.Close(); err != nil {
		return err
	}
	w.Reset(io.Discard) // release the reference to the caller's destination
	writers.Put(w)
	return nil
}

Close completes the compressed stream before reuse. Resetting to io.Discard before Put avoids keeping a caller's destination alive while the writer is idle. The gzip example includes a complete round trip.