reading special characters text from .ini file in python

Question

I am running a script which takes a text "rAh%19u^l\\&G" ie which contains special characters as seen.

When i pass this text in my script as a argument it runs fine without any error.

example - : ./abc.py <username><pwd>

The above text is basically a password.

Now, when i place my values in a config file and read the above text, the script fails .

*******abc.ini *******

[DEFAULT]
username = rahul
pwd =  rAh%19u^l\&G

it says

/bin/sh:M command not found.

Reading the above values with help of config parser

******Below is the program abc.py ******

#! /usr/bin/python

parser = configparser.ConfigParser()
parser.read('abc.ini')
username = parser.get('DEFAULT','username')
pwd = parser.get('DEFAULT','pwd')


p = subprocess.Popen(
    "abc.py {0} {1}" .format(username, pwd), 
    shell=True, 
    stdout=subprocess.PIPE
)

out, err = p.communicate()

print(out)

I tried searching a lot but found nothing concrete.

So the question is how to read a text that contains special characters in a .ini file.

Answer 1

Looks like the % character is the problem here. It has special meaning if you are using ConfigParser . If you are not using interpolation, then use just RawConfigParser instead, otherwise you must escape the % by doubling it.

When I try the example file with ConfigParser it will blow with the following exception:

InterpolationSyntaxError: '%' must be followed by '%' or '(', found: '%19u^l\\&G"'

If I replace ConfigParser with RawConfigParser everything is fine.

The error you posted has nothing to do with it. We can't even tell if it is a python exception or a shell error message. Please update your question with the full error message. You may also want to check the sh module , a higher level wrapper around subprocess .

Answer 2

Adding up on Paulo Scardine's comment.

if you have special characters that need to be handled, you can set the ConfigParser 's interpolation argument to None and you won't have the error anymore. ConfigParser has interpolation set to BasicInterpolation() by default.

You can read more about this here: https://docs.python.org/3.6/library/configparser.html#interpolation-of-values

Further, as per the docs RawConfigParser is a Legacy variant of the ConfigParser with interpolation disabled by default and unsafe add_section and set methods.

Here's a snippet from there:

Example:

[Paths]
home_dir: /Users
my_dir: %(home_dir)s/lumberjack
my_pictures: %(my_dir)s/Pictures

In the example above, ConfigParser with interpolation set to BasicInterpolation() would resolve %(home_dir)s to the value of home_dir (/Users in this case). %(my_dir)s in effect would resolve to /Users/lumberjack . [....]

With interpolation set to None , the parser would simply return %(my_dir)s/Pictures as the value of my_pictures and %(home_dir)s/lumberjack as the value of my_dir .

Answer 3

you could escape the special char with same char. for example, to be able to read % you have to write it %%

Answer 4

I had came across this same issue. Understandably the answer has been posted. Using the comments from everyone on this post I had resolved this issue with the following:

My password was Sgf%ts54hhGtrf&yhgf

alembic ini file I was using: sqlalchemy.url = driver://myuser:Sgf%ts54hhGtrf&yhgf@localhost/my_application

Error received: configparser.InterpolationSyntaxError: '%' must be followed by '%' or '(', which informs me the special character is '%'

Resolved the issue by just wrapping the password in a single quote and doubled the % symbol as recommended from comments in this post to end up with:

sqlalchemy.url = driver://myuser:'Sgf%%ts54hhGtrf&yhgf'@localhost/my_application

No need to worry about changing the ConfigParser as long as you stick to the rule of wrapping the password containing special characters with a single quote and any special characters found in the error you receive to double it eg if the error is related to a % , add another and end up with %%