Behaviour of modules depends on how they where imported

[mythsmith]

BPO	21553
Nosy	@brettcannon, @ncoghlan, @ericsnowcurrently
Superseder	bpo-13475: Add '--mainpath'/'--nomainpath' command line options to override sys.path[0] initialisation
Files	imptest.zip: imptest package exposing import inconsistencies

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2014-05-22.17:41:38.608>
created_at = <Date 2014-05-22.15:37:33.493>
labels = ['interpreter-core', 'type-bug']
title = 'Behaviour of modules depends on how they where imported'
updated_at = <Date 2014-05-23.00:29:13.421>
user = 'https://bugs.python.org/mythsmith'

bugs.python.org fields:

activity = <Date 2014-05-23.00:29:13.421>
actor = 'berker.peksag'
assignee = 'none'
closed = True
closed_date = <Date 2014-05-22.17:41:38.608>
closer = 'eric.snow'
components = ['Interpreter Core']
creation = <Date 2014-05-22.15:37:33.493>
creator = 'mythsmith'
dependencies = []
files = ['35314']
hgrepos = []
issue_num = 21553
keywords = []
message_count = 3.0
messages = ['218901', '218903', '218905']
nosy_count = 5.0
nosy_names = ['brett.cannon', 'ncoghlan', 'peter.otten', 'eric.snow', 'mythsmith']
pr_nums = []
priority = 'normal'
resolution = 'duplicate'
stage = 'resolved'
status = 'closed'
superseder = '13475'
type = 'behavior'
url = 'https://bugs.python.org/issue21553'
versions = ['Python 2.7', 'Python 3.2']

[mythsmith]

I found a condition where different behaviour could be observed depending on how a module is imported.

It seems to be different to write:

import module
# against:
from package import module

In the attachment you find a minimal package (imptest) with this organization:

imptest
|-> __init__.py (empty)
|-> m.py (module which initializes a variable "foo=0")
|- sub (package)
|-> __init__.py (empty)
|-> subm.py (module which, upon import, changes "m.foo=1")

And two scripts which can be directly executed:
|-> run0.py (using "import m")
|-> run1.py (using "from imptest import m")

Contents of the module m:
#########################
foo=0
def do():
global foo
foo=1
print('doing foo=',foo)
print("imported foo=",foo)

Contents of module subm:
#######################
from imptest import m
from imptest import m
print("imported sub, foo=",m.foo)

Both run0.py and run1.py imports module m and calls the do() function, thus theoretically changing foo to 1.
Both later import the subm module, which in turn imports again the m module. What I would expect is that,
since m is already in memory, it is not really imported again: so foo remains equal to 1 also after subm import.

I found that this actually depends on how I imported m in the script.

Contents of run0.py:
####################
import m
m.do()
print("importing subm")
from imptest.sub import subm

Result:
imported m; foo= 0
doing foo= 1
importing subm
imported m; foo= 0
imported sub, foo= 0

As you can see from printout "importing subm", the m module is imported again and thus foo is reset to 0. In run1.py,
I changed the line "import m" to "from imptest import m", and got the expected behaviour:

Contents of run1.py:
####################
from imptest import m
m.do()
print("importing subm")
from imptest.sub import subm

Result:
imported m; foo= 0
doing foo= 1
importing subm
imported sub, foo= 1

I know that directly running a module in the first level of a package may seem strange or not correct, but could someone explain why this is happening?
I would expect a module to be loaded in memory at the first import and then referred in any way I later or elsewhere in the program choose to import it.

[peterotten]

Here's a simpler demo for what I believe you are experiencing:

$ mkdir package
$ cd package/
$ touch __ini__.py module.py
$ export PYTHONPATH=..
$ python3
Python 3.3.2+ (default, Feb 28 2014, 00:52:16) 
[GCC 4.8.1] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import module, package.module
>>> module is package.module
False

Even though module and package.module correspond to the same file
Python does not recognize that you intend the former to be part of a package. Your run0.py script goes through its sys.path entries and unfortunately the first entry points into a package so that all modules in that package become also visible as toplevel modules.

A good way to avoid this trap is to use relative imports

>>> from . import module
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
SystemError: Parent module '' not loaded, cannot perform relative import

[ericsnowcurrently]

This is a consequence of a script's directory getting prepended to sys.path. See issue bpo-13475.