32-bit Python on 64-bit Windows reports incorrect architecture

[briancurtin]

BPO	7860
Nosy	@malemburg, @pitrou, @bitdancer, @briancurtin
Files	arch_misrepresented.diff: patch against trunk uname_WOW64_test.patch uname_WOW64_test2.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/bitdancer'
closed_at = <Date 2010-03-23.01:56:12.464>
created_at = <Date 2010-02-05.18:07:49.071>
labels = ['type-bug', 'library', 'OS-windows']
title = '32-bit Python on 64-bit Windows reports incorrect architecture'
updated_at = <Date 2011-06-13.19:53:31.347>
user = 'https://github.com/briancurtin'

bugs.python.org fields:

activity = <Date 2011-06-13.19:53:31.347>
actor = 'santoso.wijaya'
assignee = 'r.david.murray'
closed = True
closed_date = <Date 2010-03-23.01:56:12.464>
closer = 'r.david.murray'
components = ['Library (Lib)', 'Windows']
creation = <Date 2010-02-05.18:07:49.071>
creator = 'brian.curtin'
dependencies = []
files = ['16143', '16592', '16622']
hgrepos = []
issue_num = 7860
keywords = ['patch']
message_count = 11.0
messages = ['98890', '98896', '98897', '98902', '98922', '98926', '101331', '101352', '101509', '101524', '101528']
nosy_count = 5.0
nosy_names = ['lemburg', 'pitrou', 'r.david.murray', 'brian.curtin', 'santoso.wijaya']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue7860'
versions = ['Python 2.7', 'Python 3.2']

[briancurtin]

When running 32-bit Python on a 64-bit version of Windows, therefore running the process in WOW64 mode, platform.machine returns a misleading value. When running in WOW64, the processor architecture is masked to appear as "x86" when the machine is actually "AMD64" (which you see on a 64-bit app on 64-bit OS).

The change involves looking at an environment variable only set on WOW64 processes to see the native architecture. See http://blogs.msdn.com/david.wang/archive/2006/03/26/HOWTO-Detect-Process-Bitness.aspx

The patch does not include any test, as I'm not really sure how you could test this without using the change itself to figure out when it should be tested. Suggestions would be appreciated.

[pitrou]

It's also inconsistent with Linux behaviour:

>>> import sys 
>>> sys.maxsize
2147483647
>>> import platform
>>> platform.machine()
'x86_64'
>>> platform.architecture()
('32bit', 'ELF')

(on a Python compiled with "-m32" on a 64-bit Linux)

[pitrou]

The test could simply check that platform.machine() never returns "x86" when sys.maxsize is greater than 2**32.
(and it would even be platform-agnostic)

[malemburg]

The patch looks ok. Since we are emulating the Unix uname() function here, we might as well mimic its inconsistencies.

Note that platform.machine() and platform.processor() are not really very reliable ways of determining the machine type or processor.

[malemburg]

Antoine Pitrou wrote:

Antoine Pitrou <pitrou@free.fr> added the comment:

The test could simply check that platform.machine() never returns "x86" when sys.maxsize is greater than 2**32.
(and it would even be platform-agnostic)

I'm not sure what such a test would prove, e.g. on 64-bit Mac OS X,
platform.machine() return 'i386'. That could be mapped to 'x86'...

Note that 'x86_64' is just a 'x86' compatible processor with
the AMD 64-bit extensions, so 'x86' is a superset of 'x86_64'.

[pitrou]

I'm not sure what such a test would prove, e.g. on 64-bit Mac OS X,
platform.machine() return 'i386'. That could be mapped to 'x86'...

You're right. There doesn't seem to be much consistency accross
platforms.

Note that 'x86_64' is just a 'x86' compatible processor with
the AMD 64-bit extensions, so 'x86' is a superset of 'x86_64'.

Oops, indeed.

[briancurtin]

bpo-7347 depends on this for proper testing, and arch_misrepresented.diff seems to have been labeled as acceptable. Would anyone be willing to check it in?

[bitdancer]

I think it is actually pretty straightforward to write a *unit* test for this. We just need to check that the logic works correctly given the expected presence or absence of the environment variables. That doesn't test whether or not the right thing happens in the environment when you actually run a WOW64, but I don't think it is Python's responsibility to test that. If Microsoft changes the API, platform will break and the tests won't notice, but I don't think there's anything we can do about that, since as you say the API is the only way to find out what to expect for results.

Test patch attached. Brian, if you can confirm that this test fails before your patch and succeeds afterward, I will commit both patches.

[briancurtin]

Ah, that's simple enough :)

Minor changes to the test patch: uname caches it's results, so I added a few lines to clear the cache before the uname calls. In order to not affect other tests, the whole thing is in a try/finally so we don't leave anything behind in cache.

The test passes on both a 32 and 64 bit Python.

[bitdancer]

Patches applied to trunk in r79294, py3k in r79298. Should this be backported?

[malemburg]

R. David Murray wrote:

R. David Murray <rdmurray@bitdance.com> added the comment:

Patches applied to trunk in r79294, py3k in r79298. Should this be backported?

I don't think so: applications relying on the previous behavior
would need to be updated and requiring this for a patch level
release of Python is not a good idea.