October 23, 2016

Inspecting arbitrary memory in Go

A coworker’s question the other day motivated me inspect the layout of certain Go objects in memory. Being a managed language, dumping arbitrary swaths of memory isn’t something that Go generally encourages. Doing so requires the unsafe package, whose name should inspire you with the appropriate degree of fear.

Note: This post presumes general familiarity with the way pointers operate. It’s easy to get confused when dealing with pointers, so you may want to pause and revisit pointers if you get lost.

Why would I want to do this?

In an actual production program, hopefully you wouldn’t. However it’s a very effective academic exercise to learn about the fundamentals of a Go program. A correct understanding of low-level details may explain the reasons for seemingly-odd language behavior or help you think more effectively about design decisions.

Still, this point bears repeating: if you find yourself doing this in a real program, stop immediately and seek help. You are doing something wrong. You’ve been warned.

Using unsafe.Pointer to reference arbitrary memory

The unsafe package provides a small but potent set of primitives. It defines only one type—unsafe.Pointer—which has some useful properties for inspecting memory. Specifically:

- A pointer value of any type can be converted to a Pointer.
- A Pointer can be converted to a pointer value of any type.

The first property allows one to obtain the memory address of any variable:

var x uint8 = 25
addr := unsafe.Pointer(&x)

Conversely, the second allows for doing the same operation in reverse:

var y *uint8
y = (*uint8)(addr)

Since an unsafe.Pointer carries no type information, there’s nothing to stop us from pretending that this address contains data of some other type:

var x uint8 = 25
// x = uint8(25)

addr := unsafe.Pointer(&x)
// addr = 0x1040e0f6 (some memory address)

var y *byte
y = (*byte)(addr)
// *y = byte(25)

Extracting the underlying memory of a variable

Using unsafe.Pointer and a little casting means we can read any byte from memory given its address. One way to read larger amounts of memory is to simply do this repeatedly. This Extract function reads size bytes, starting from the unsafe.Pointer address ptr:

// Extract reads arbitrary memory.
func Extract(ptr unsafe.Pointer, size uintptr) []byte {
	out := make([]byte, size)
	for i := range out {
		out[i] = *((*byte)(unsafe.Pointer(uintptr(ptr) + uintptr(i))))
	}
	return out
}

A uint16 is two bytes wide, so we can use that to experiment.

var x uint16 = 2016
mem := Extract(unsafe.Pointer(&x), 2)
// mem = [e0 07]

What if we’re not sure how many bytes long x is? The unsafe package conveniently provides the unsafe.Sizeof function for this reason, so we can invoke Extract on any variable whether we know its size or not:

var x int = 2016
mem := Extract(unsafe.Pointer(&x), unsafe.Sizeof(x))
// mem = [e0 07 00 00 00 00 00 00]

var sm sync.Mutex
mem := Extract(unsafe.Pointer(&sm), unsafe.Sizeof(sm))
// mem = [00 00 00 00 00 00 00 00]

Remember that on a little-endian machine, memory is arranged such that bytes will appear to be in reverse order: the hex value for 2016 is 0x07e0.

Worth noting:

  • This stuff is implemented in the github.com/tylerchr/memory package.
  • The way we’re converting unsafe.Pointer(uintptr(0xfoo)) is not valid; it may escape the garbage collector and leak memory in longer-running programs.

© Tyler Christensen 2016