Understanding Go's `for` loop with closures

In this post we’ll ponder the Go peculiarities when working with for range loops and closures.

Problem

On of the Go gotchas when coming from other languages is how for range loop operates with closures.

Here’s a simple example:

package main

import "os"

func main() {
	for _, arg := range os.Args {
		defer func() { println(arg) }()
	}
}

Running the example:

$ go run for.go hello world

one would expect the command line arguments printed in some order.

What happens instead is that the last argument gets printed n times(where n := len(os.Args):

world
world
world

Why is this happening? Good question!

What you see is what you get, not!

We know the Go’s rule of thumb:

everything in Go is passed by value

But for some reason the rule doesn’t apply in the example above. If every closure created in the loop body received the copy of the value then it would have printed the copy. Except, yes, except when the value is a shared pointer and it’s shared rather than copied.

This is getting interesting.

Reconstruction

Let’s try to recreate the behaviour manually, using the shared pointer.

package main

import "os"

func main() {
	var arg *string
	for i := 0; i < len(os.Args); i += 1 {
		arg = &os.Args[i]
		defer func() { println(*arg) }()
	}
}

Works exactly the same as the initial example.

Note var arg *string declared “outside” of for statement’s scope as it’s shared for all the closures.

What you see is what you get, not! ^2

Ok, now let’s peek a level below the Go syntax into a generated assembly:

go tool compile -S for.go > for.s

Generates disassembly(see Main disassembly below for the full main dump)

What’s interesting in the dump are lines like this:

	0x0087 00135 (for.go:6)	MOVQ	"".&arg+48(SP), BX

where symbol "".&arg looks like a use of pointers. Boom!

What is also interesting the reconstructed code’s disassembly is similar to what Go generates for the initial example. Which means that the reconstructed code is close to what actually going on under the hood.

Note: I’m still to master Go’s asm so take my findings with skepticism.

Conclusion

It’d be nice to actually understand the reasons behind that behaviour. I think i saw explanation somewhere but I can’t recall where :/

Please let me know if you’ve spotted a mistake or know where the explanation is.

Thank you!

References

Reading

Main disassembly

"".main t=1 size=288 value=0 args=0x0 locals=0x60
	0x0000 00000 (for.go:5)	TEXT	"".main(SB), $96-0
	0x0000 00000 (for.go:5)	MOVQ	(TLS), CX
	0x0009 00009 (for.go:5)	CMPQ	SP, 16(CX)
	0x000d 00013 (for.go:5)	JLS	277
	0x0013 00019 (for.go:5)	SUBQ	$96, SP
	0x0017 00023 (for.go:5)	FUNCDATA	$0, gclocals·69c1753bd5f81501d95132d08af04464(SB)
	0x0017 00023 (for.go:5)	FUNCDATA	$1, gclocals·cf89d5c81323c78771a60eb7aec9de00(SB)
	0x0017 00023 (for.go:6)	LEAQ	type.string(SB), BX
	0x001e 00030 (for.go:6)	MOVQ	BX, (SP)
	0x0022 00034 (for.go:6)	PCDATA	$0, $0
	0x0022 00034 (for.go:6)	CALL	runtime.newobject(SB)
	0x0027 00039 (for.go:6)	MOVQ	8(SP), BX
	0x002c 00044 (for.go:6)	MOVQ	BX, "".&arg+48(SP)
	0x0031 00049 (for.go:6)	MOVQ	os.Args(SB), BP
	0x0038 00056 (for.go:6)	MOVQ	os.Args+8(SB), CX
	0x003f 00063 (for.go:6)	MOVQ	os.Args+16(SB), BX
	0x0046 00070 (for.go:6)	MOVQ	BX, "".autotmp_0001+88(SP)
	0x004b 00075 (for.go:6)	MOVQ	$0, DX
	0x004d 00077 (for.go:6)	MOVQ	CX, "".autotmp_0001+80(SP)
	0x0052 00082 (for.go:6)	MOVQ	CX, "".autotmp_0003+24(SP)
	0x0057 00087 (for.go:6)	MOVQ	BP, "".autotmp_0001+72(SP)
	0x005c 00092 (for.go:6)	MOVQ	BP, CX
	0x005f 00095 (for.go:6)	MOVQ	"".autotmp_0003+24(SP), BP
	0x0064 00100 (for.go:6)	CMPQ	DX, BP
	0x0067 00103 (for.go:6)	JGE	$0, 232
	0x0069 00105 (for.go:6)	MOVQ	CX, BX
	0x006c 00108 (for.go:6)	MOVQ	CX, "".autotmp_0004+40(SP)
	0x0071 00113 (for.go:6)	CMPQ	CX, $0
	0x0075 00117 (for.go:6)	JEQ	$1, 270
	0x007b 00123 (for.go:6)	MOVQ	(CX), CX
	0x007e 00126 (for.go:6)	MOVQ	8(BX), AX
	0x0082 00130 (for.go:6)	MOVQ	DX, "".autotmp_0002+32(SP)
	0x0087 00135 (for.go:6)	MOVQ	"".&arg+48(SP), BX
	0x008c 00140 (for.go:6)	MOVQ	AX, "".autotmp_0005+64(SP)
	0x0091 00145 (for.go:6)	MOVQ	AX, 8(BX)
	0x0095 00149 (for.go:6)	MOVQ	CX, "".autotmp_0005+56(SP)
	0x009a 00154 (for.go:6)	CMPB	runtime.writeBarrier(SB), $0
	0x00a1 00161 (for.go:6)	JNE	$0, 254
	0x00a3 00163 (for.go:6)	MOVQ	CX, (BX)
	0x00a6 00166 (for.go:7)	MOVQ	"".&arg+48(SP), BX
	0x00ab 00171 (for.go:7)	MOVQ	BX, 16(SP)
	0x00b0 00176 (for.go:7)	MOVL	$8, (SP)
	0x00b7 00183 (for.go:7)	LEAQ	"".main.func1·f(SB), AX
	0x00be 00190 (for.go:7)	MOVQ	AX, 8(SP)
	0x00c3 00195 (for.go:7)	PCDATA	$0, $1
	0x00c3 00195 (for.go:7)	CALL	runtime.deferproc(SB)
	0x00c8 00200 (for.go:7)	CMPL	AX, $0
	0x00cb 00203 (for.go:7)	JNE	$1, 243
	0x00cd 00205 (for.go:6)	MOVQ	"".autotmp_0004+40(SP), CX
	0x00d2 00210 (for.go:6)	MOVQ	"".autotmp_0002+32(SP), DX
	0x00d7 00215 (for.go:6)	ADDQ	$16, CX
	0x00db 00219 (for.go:6)	INCQ	DX
	0x00de 00222 (for.go:6)	MOVQ	"".autotmp_0003+24(SP), BP
	0x00e3 00227 (for.go:6)	CMPQ	DX, BP
	0x00e6 00230 (for.go:6)	JLT	$0, 105
	0x00e8 00232 (for.go:9)	PCDATA	$0, $0
	0x00e8 00232 (for.go:9)	XCHGL	AX, AX
	0x00e9 00233 (for.go:9)	CALL	runtime.deferreturn(SB)
	0x00ee 00238 (for.go:9)	ADDQ	$96, SP
	0x00f2 00242 (for.go:9)	RET
	0x00f3 00243 (for.go:7)	PCDATA	$0, $0
	0x00f3 00243 (for.go:7)	XCHGL	AX, AX
	0x00f4 00244 (for.go:7)	CALL	runtime.deferreturn(SB)
	0x00f9 00249 (for.go:7)	ADDQ	$96, SP
	0x00fd 00253 (for.go:7)	RET
	0x00fe 00254 (for.go:6)	MOVQ	BX, (SP)
	0x0102 00258 (for.go:6)	MOVQ	CX, 8(SP)
	0x0107 00263 (for.go:6)	PCDATA	$0, $1
	0x0107 00263 (for.go:6)	CALL	runtime.writebarrierptr(SB)
	0x010c 00268 (for.go:7)	JMP	166
	0x010e 00270 (for.go:6)	MOVL	AX, (CX)
	0x0110 00272 (for.go:6)	JMP	123
	0x0115 00277 (for.go:6)	NOP
	0x0115 00277 (for.go:5)	CALL	runtime.morestack_noctxt(SB)
	0x011a 00282 (for.go:5)	JMP	0

Comments