In this post we’ll ponder the Go peculiarities when working with for range loops and closures.
Problem
On of the Go gotchas when coming from other languages is how for range loop operates with closures.
Here’s a simple example:
package main
import "os"
func main() {
for _, arg := range os.Args {
defer func() { println(arg) }()
}
}
Running the example:
$ go run for.go hello world
one would expect the command line arguments printed in some order.
What happens instead is that the last argument gets printed n times(where n := len(os.Args):
world
world
world
Why is this happening? Good question!
What you see is what you get, not!
We know the Go’s rule of thumb:
everything in Go is passed by value
But for some reason the rule doesn’t apply in the example above. If every closure created in the loop body received the copy of the value then it would have printed the copy. Except, yes, except when the value is a shared pointer and it’s shared rather than copied.
This is getting interesting.
Reconstruction
Let’s try to recreate the behaviour manually, using the shared pointer.
package main
import "os"
func main() {
var arg *string
for i := 0; i < len(os.Args); i += 1 {
arg = &os.Args[i]
defer func() { println(*arg) }()
}
}
Works exactly the same as the initial example.
Note var arg *string declared “outside” of for statement’s scope as it’s shared for all the closures.
What you see is what you get, not! ^2
Ok, now let’s peek a level below the Go syntax into a generated assembly:
go tool compile -S for.go > for.s
Generates disassembly(see Main disassembly below for the full main dump)
What’s interesting in the dump are lines like this:
0x0087 00135 (for.go:6) MOVQ "".&arg+48(SP), BX
where symbol "".&arg looks like a use of pointers. Boom!
What is also interesting the reconstructed code’s disassembly is similar to what Go generates for the initial example. Which means that the reconstructed code is close to what actually going on under the hood.
Note: I’m still to master Go’s asm so take my findings with skepticism.
Conclusion
- default behaviour of using closures with
forrangeloops is somewhat unexpected in Golang - generated code uses shared pointer under the hood rather than copying value one would expect based on the code
- to get the expected behaviour one must copy the value manually either into local variable or as the closure argument
- it’s not exactly
forloop specific behaviour rather it’s how closures capture values in Go - just a language idiom to be careful about
It’d be nice to actually understand the reasons behind that behaviour. I think i saw explanation somewhere but I can’t recall where :/
Please let me know if you’ve spotted a mistake or know where the explanation is.
Thank you!
References
Reading
Main disassembly
"".main t=1 size=288 value=0 args=0x0 locals=0x60
0x0000 00000 (for.go:5) TEXT "".main(SB), $96-0
0x0000 00000 (for.go:5) MOVQ (TLS), CX
0x0009 00009 (for.go:5) CMPQ SP, 16(CX)
0x000d 00013 (for.go:5) JLS 277
0x0013 00019 (for.go:5) SUBQ $96, SP
0x0017 00023 (for.go:5) FUNCDATA $0, gclocals·69c1753bd5f81501d95132d08af04464(SB)
0x0017 00023 (for.go:5) FUNCDATA $1, gclocals·cf89d5c81323c78771a60eb7aec9de00(SB)
0x0017 00023 (for.go:6) LEAQ type.string(SB), BX
0x001e 00030 (for.go:6) MOVQ BX, (SP)
0x0022 00034 (for.go:6) PCDATA $0, $0
0x0022 00034 (for.go:6) CALL runtime.newobject(SB)
0x0027 00039 (for.go:6) MOVQ 8(SP), BX
0x002c 00044 (for.go:6) MOVQ BX, "".&arg+48(SP)
0x0031 00049 (for.go:6) MOVQ os.Args(SB), BP
0x0038 00056 (for.go:6) MOVQ os.Args+8(SB), CX
0x003f 00063 (for.go:6) MOVQ os.Args+16(SB), BX
0x0046 00070 (for.go:6) MOVQ BX, "".autotmp_0001+88(SP)
0x004b 00075 (for.go:6) MOVQ $0, DX
0x004d 00077 (for.go:6) MOVQ CX, "".autotmp_0001+80(SP)
0x0052 00082 (for.go:6) MOVQ CX, "".autotmp_0003+24(SP)
0x0057 00087 (for.go:6) MOVQ BP, "".autotmp_0001+72(SP)
0x005c 00092 (for.go:6) MOVQ BP, CX
0x005f 00095 (for.go:6) MOVQ "".autotmp_0003+24(SP), BP
0x0064 00100 (for.go:6) CMPQ DX, BP
0x0067 00103 (for.go:6) JGE $0, 232
0x0069 00105 (for.go:6) MOVQ CX, BX
0x006c 00108 (for.go:6) MOVQ CX, "".autotmp_0004+40(SP)
0x0071 00113 (for.go:6) CMPQ CX, $0
0x0075 00117 (for.go:6) JEQ $1, 270
0x007b 00123 (for.go:6) MOVQ (CX), CX
0x007e 00126 (for.go:6) MOVQ 8(BX), AX
0x0082 00130 (for.go:6) MOVQ DX, "".autotmp_0002+32(SP)
0x0087 00135 (for.go:6) MOVQ "".&arg+48(SP), BX
0x008c 00140 (for.go:6) MOVQ AX, "".autotmp_0005+64(SP)
0x0091 00145 (for.go:6) MOVQ AX, 8(BX)
0x0095 00149 (for.go:6) MOVQ CX, "".autotmp_0005+56(SP)
0x009a 00154 (for.go:6) CMPB runtime.writeBarrier(SB), $0
0x00a1 00161 (for.go:6) JNE $0, 254
0x00a3 00163 (for.go:6) MOVQ CX, (BX)
0x00a6 00166 (for.go:7) MOVQ "".&arg+48(SP), BX
0x00ab 00171 (for.go:7) MOVQ BX, 16(SP)
0x00b0 00176 (for.go:7) MOVL $8, (SP)
0x00b7 00183 (for.go:7) LEAQ "".main.func1·f(SB), AX
0x00be 00190 (for.go:7) MOVQ AX, 8(SP)
0x00c3 00195 (for.go:7) PCDATA $0, $1
0x00c3 00195 (for.go:7) CALL runtime.deferproc(SB)
0x00c8 00200 (for.go:7) CMPL AX, $0
0x00cb 00203 (for.go:7) JNE $1, 243
0x00cd 00205 (for.go:6) MOVQ "".autotmp_0004+40(SP), CX
0x00d2 00210 (for.go:6) MOVQ "".autotmp_0002+32(SP), DX
0x00d7 00215 (for.go:6) ADDQ $16, CX
0x00db 00219 (for.go:6) INCQ DX
0x00de 00222 (for.go:6) MOVQ "".autotmp_0003+24(SP), BP
0x00e3 00227 (for.go:6) CMPQ DX, BP
0x00e6 00230 (for.go:6) JLT $0, 105
0x00e8 00232 (for.go:9) PCDATA $0, $0
0x00e8 00232 (for.go:9) XCHGL AX, AX
0x00e9 00233 (for.go:9) CALL runtime.deferreturn(SB)
0x00ee 00238 (for.go:9) ADDQ $96, SP
0x00f2 00242 (for.go:9) RET
0x00f3 00243 (for.go:7) PCDATA $0, $0
0x00f3 00243 (for.go:7) XCHGL AX, AX
0x00f4 00244 (for.go:7) CALL runtime.deferreturn(SB)
0x00f9 00249 (for.go:7) ADDQ $96, SP
0x00fd 00253 (for.go:7) RET
0x00fe 00254 (for.go:6) MOVQ BX, (SP)
0x0102 00258 (for.go:6) MOVQ CX, 8(SP)
0x0107 00263 (for.go:6) PCDATA $0, $1
0x0107 00263 (for.go:6) CALL runtime.writebarrierptr(SB)
0x010c 00268 (for.go:7) JMP 166
0x010e 00270 (for.go:6) MOVL AX, (CX)
0x0110 00272 (for.go:6) JMP 123
0x0115 00277 (for.go:6) NOP
0x0115 00277 (for.go:5) CALL runtime.morestack_noctxt(SB)
0x011a 00282 (for.go:5) JMP 0