unicode in email.MIMEText and email/Charset.py

[gdamjan]

BPO	1368247
Nosy	@loewis, @warsaw, @orsenthil, @vstinner, @bitdancer
Files	Charset.patch mimetext-unicode.patch: unicode mimetext support mimetext_unicode_input.patch mimetext_unicode_input.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/bitdancer'
closed_at = <Date 2010-06-03.02:06:49.885>
created_at = <Date 2005-11-28.14:15:40.000>
labels = ['type-bug', 'library']
title = 'unicode in email.MIMEText and email/Charset.py'
updated_at = <Date 2010-06-03.02:06:49.884>
user = 'https://bugs.python.org/gdamjan'

bugs.python.org fields:

activity = <Date 2010-06-03.02:06:49.884>
actor = 'r.david.murray'
assignee = 'r.david.murray'
closed = True
closed_date = <Date 2010-06-03.02:06:49.885>
closer = 'r.david.murray'
components = ['Library (Lib)']
creation = <Date 2005-11-28.14:15:40.000>
creator = 'gdamjan'
dependencies = []
files = ['6898', '12190', '17513', '17514']
hgrepos = []
issue_num = 1368247
keywords = ['patch']
message_count = 13.0
messages = ['49137', '49138', '76737', '76740', '76741', '87253', '87292', '103008', '104416', '106835', '106837', '106924', '106930']
nosy_count = 9.0
nosy_names = ['loewis', 'barry', 'orsenthil', 'vstinner', 'gdamjan', 'maxua', 'r.david.murray', 'bgamari', 'l0nwlf']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue1368247'
versions = ['Python 3.1', 'Python 3.2']

[gdamjan]

This is the test case that fails in python 2.4.1:
from email.MIMEText import MIMEText
msg =
MIMEText(u'\u043a\u0438\u0440\u0438\u043b\u0438\u0446\u0430')
msg.set_charset('utf-8')
msg.as_string()

And attached is a patch to correct it.

[loewis]

Your proposed patch doesn't seem to work in Python 2.5, or the trunk (i.e. it won't prevent an exception from occuring). Can you please revise it?

[maxua]

How about this version?

[vstinner]

It was proposed to rewrite MIMEText in Python 3.1 (and 2.7?) to use
unicode characters in the internals and reconvert to bytes to send it
to a socket (or a file).

[gdamjan]

The patch by maxua works fine with 2.6 too and solves the problem.
I'd suggest it be applied to the 2.6 branch, even if email is rewriten
for 2.7/3.x.

[bgamari]

What is the status of this?

[bitdancer]

It looks to me like MIMEText doesn't actually support unicode input. .

One way to get the example to work is to do this:

MIMEText(u'\u043a\u0438\u0440\u0438\u043b\u0438\u0446\u0430'.encode('utf-8'), 'plain', 'utf-8')

The above call produces valid output from as_string:

'Content-Type: text/plain; charset="utf-8"\nMIME-Version:
1.0\nContent-Transfer-Encoding: base64\n\n0LrQuNGA0LjQu9C40YbQsA==\n'

How you'd get it to use 8bit, I have no idea. Still, I'm inclined to
close this as invalid unless Barry tells me my analysis is wrong.

(CF: http://mg.pov.lt/blog/unicode-emails-in-python for a good example
of handling unicode using the email package, which I found after
figuring out the above.)

Clearly, the documentation of this could be better, but I suspect the
developers would rather spend their time fixing the email module in py3.
A doc patch would certainly be accepted. (Maybe someone could ask the
above blogger if we could borrow his example for the docs.)