Reading and Writing Devanagri Hindi characters in text file

Question

Having trouble reading and writing Hindi Devanagari characters in text files using Python.

1. Reading :
   Python is unable to read Devanagari characters in my text file when following code is implemented.

   Code:

    f=open(r"C:\Users\Dell\Desktop\abc1.txt","w") print(f.read()) f.close()

   O/P: म (instead of म)

   While the same code produces correct output for '&' symbols in my file as follows

   O/P: &

2. Writing :
   Following implementation throws an error message.
   Unicode for म being 092e
   Code:

    f=open(r"C:\Users\Dell\Desktop\abc1.txt","w") f.write(u"म") f.close()

   Error Message:

    Exception has occurred: UnicodeEncodeError 'charmap' codec can't encode character 'म' in position 0: character maps to <undefined> File "C:\Users\Dell\Desktop\Python\gg.py", line 2, in <module> f.write(u"म")

While the character writes successfully on standard output as follows:

Code:

print(u"\u092e")

O/P: म

Kindly explain why so? How can I read and write Devanagari characters on text file? Any alternatives?

Answer 1

Have you tried opening the file with UTF8 or UTF16? It depends on how your source file is saved (which encoding is used).

For example, try:

with open(r'C:\Users\Dell\Desktop\abc1.txt','r', encoding='utf-16') as f:
    print(f.read())