It’s a closure oddity. The closure captures a reference to the variable, not the value of the variable at the time of its creation. Given that, the results are entirely predictable.
If we're really going to argue, I'd argue for a 'for' oddity. The for statement could've been defined with a fresh binding per iteration instead of a mutation.
How does that work in practice? Python stores local variables in the stack frame with a compiler-calculated index. Do you have a growable stack frame? Add stack frames for every loop iteration?
I don't think the Python compiler is to know if a loop variable is closed over and then adjust the code generation. So this change would likely introduce a lot of complexity, or be a pessimisation in the normal case.
I think the simple, although less helpful, semantics are clearly better here. FWIW I don't think this comes up in practice very much. Maybe because callbacks in Python APIs aren't common (and its not due to this).
Not really -- in practice that would imply that ``for`` needs to freeze the value of ``locals`` during every iteration, which doesn't make sense to me.
This is definitely a python gotcha. In fact, it's literally one of the canonical ones [1]. But it's purely the interaction between the way that names/variables work in python, and late-binding closures.
More explicitly: it's best to think of "variables" in python as "names", because in code, you refer to names, but the underlying variable might change. This is also why python is best described neither as call-by-reference nor call-by-value, but rather "call-by-assignment" or "call-by-object": you're passing a reference to a variable bound to a particular name. Unless you're writing C extensions, you never actually have a reference to the variable itself -- instead, to the name.
Meanwhile, closures in python are always late-binding: they are themselves "live", mutable objects, in the sense that they're just a reference to the namespace they're closing, and *not* a copy. So if you mutate that namespace between establishing the closure and making use of it, you'll see the mutated version.
It’s a closure oddity. The closure captures a reference to the variable, not the value of the variable at the time of its creation. Given that, the results are entirely predictable.
If we're really going to argue, I'd argue for a 'for' oddity. The for statement could've been defined with a fresh binding per iteration instead of a mutation.
How does that work in practice? Python stores local variables in the stack frame with a compiler-calculated index. Do you have a growable stack frame? Add stack frames for every loop iteration?
I don't think the Python compiler is to know if a loop variable is closed over and then adjust the code generation. So this change would likely introduce a lot of complexity, or be a pessimisation in the normal case.
I think the simple, although less helpful, semantics are clearly better here. FWIW I don't think this comes up in practice very much. Maybe because callbacks in Python APIs aren't common (and its not due to this).
2 replies →
It's unrelated to the for loop, the for only changes the variable, there are other ways you can do that and you'll run into the same "oddity"
2 replies →
That would make it work differently than the entire rest of the language. Python is function-scoped, not block-scoped.
1 reply →
Not really -- in practice that would imply that ``for`` needs to freeze the value of ``locals`` during every iteration, which doesn't make sense to me.
This is definitely a python gotcha. In fact, it's literally one of the canonical ones [1]. But it's purely the interaction between the way that names/variables work in python, and late-binding closures.
More explicitly: it's best to think of "variables" in python as "names", because in code, you refer to names, but the underlying variable might change. This is also why python is best described neither as call-by-reference nor call-by-value, but rather "call-by-assignment" or "call-by-object": you're passing a reference to a variable bound to a particular name. Unless you're writing C extensions, you never actually have a reference to the variable itself -- instead, to the name.
Meanwhile, closures in python are always late-binding: they are themselves "live", mutable objects, in the sense that they're just a reference to the namespace they're closing, and *not* a copy. So if you mutate that namespace between establishing the closure and making use of it, you'll see the mutated version.
[1] https://docs.python-guide.org/writing/gotchas/#late-binding-...
1 reply →