Comment by hanglong10
12 days ago
Further proof that Python is not fit for functional programming and that Python's namespaces and default lookup by reference are confusing.
In safe languages like ML you would have to create an explicit reference to get that effect, and then it is obvious to the reader.
Python is not built for complex abstractions, but unfortunately it is (ab)used for that, particularly in machine learning.
No. The code is completely consistent: the closures are different instances of a function that returns the value of 'number' at the moment it gets called. Not the value of 'number' when it gets created. And it gets called differently when a list comprehension calls it each step of the loop or when the list function loops completely through and then the list comprehension calls the instances. Nothing to see here.
> No. The code is completely consistent: the closures are different instances of a function that returns the value of 'number' at the moment it gets called. Not the value of 'number' when it gets created.
Isn't that exactly what GP said? And gave an example how to create that effect explicitly in functional languages, using refs?
Yes. But I found the explanations of the GP a little misleading, especially the eager/lazy part.
Naively I would expect that each iteration of a for loop creates a new loop variable (all with the same name, but effectively each in their own scope) and so each closure holds a reference to a variable named number whose value never changes, and there is a distinct one of these number variables per closure .
That is not how variables in Python work. Variable assignment is just name binding. This is clearly mentioned in every Python beginner tutorial I know of. Python code is a script. Follow the script and you know what Python does (with only a few exceptions, see e.g. below).
About the scope: Python knows in principal only 2 scopes: local and global. If you define a function (or a class) Python creates a new emtpy/filled with the function arguments local scope (system table). That gets filled successively by local variable assignments etc. There happens a little bit of magic: if you use a variable of the next higher scope, it gets copied (!) to the local scope. You can follow that easily by print(locals()). (You can also use the global scope, but that is another topic).
Edit: had to clarify some points
It's consistent, but unexpected. Especially if you don't know python well enough. And it leads to ugly effect sometimes, even for experienced pythonistas.
Yes, this bit me last week. I stumble over it about once a year. Recently encountered it in an open source project and filed an issue. I don't know where I fall on the "experienced" spectrum, but I've written a lot of python in the last 10 years.
No.
If you want to "close over" the value in the closure then there's a simple trick you can use:
Pass a keyword argument with the default value equal to the value you want to close over.
Voila!
Functions.partial can also supply captures in that way, and is slightly more obvious (and less likely to generate linter complaints if the kwarg is mutable).
er, functools.partial, too late to edit.
all one needs to understand is that there are names that refer to objects in Python. Nothing is passed by value.
The closure returns whatever object the name refers at the moment. That is all.
`j` refers to the same object (int) `i` refers to at the time of the lambda creation (the desired result). `j` is local to the lambda (each lambda has its own local namespace).
Just `lambda: i` code would result in an object that `i` refers to at the time of calling the lambda (`i` name is not local to the lambda).
https://docs.python.org/3/faq/programming.html#why-do-lambda...