Using PHP to call Virtualenv’ed Python Script

Question

Last night I spent 5.5 hours trying make PHP execute and receive the output of Virtualenv'ed Python script. Nothing worked; except for scripts that were not Virtualenv'ed.

What I am trying to do:

I am trying to make PHP call a virtualenv'd install of the Newspaper lib output text when I call it.

What I have now:

PHP: (updated)

<?php
$output = exec('newspaper2/bin/python3 /var/www/html/components/python/test.py 2>&1', $output2);
print_r(error_get_last());
echo $output2;
echo $output;

…this works when using a non-virtualenv script

Python: (updated)

from newspaper import Article
url = 'http://example.com/'
article = Article(url)
article.download()
article.html
article.parse()
article.authors
article.publish_date
string = article.text
print(string)

What the issue is:

I can run the script that PHP is running from the command line and it outputs just fine.

What I have tried:

With PHP, (I have tried all the “exec” calls for PHP) it cannot seem to open the virtual environment and returns nothing.

Before the script I have called “python3” and a few other things to no avail.

Yes, I have chmoded it to be executable…

I feel like this should be so simple.

I have tried suggestions on other posts and all over the web to no avail.

Questions:

• Did I set up the virtualenv wrong?
• At the top of the Python script, instead of the “#!/usr/bin/env python3” should I call something else?
• If so, where do I find it? Should I start from scratch and will that help?

Thank you for your help;

PS: I am running Ubuntu16, PHP7 and I need to use Python3

Answer 1

In the virtualenv'ed scripts (ie installed via the setuptools' entry-points), you should not touch the shebang ( #!... first line). It is populated by the virtualenv & setuptools & related tools.

If you specify your own shebang, then it is not virtualenv'ed script. In that case, call python directly:

exec('/path/to/venv/bin/python3 /var/www/html/components/python/testing.py');

Alternatively, you can put the absolute path to the virtualenv's python binary to the py-script, but this does not look a good idea.

Also, remember that virtualenvs are non-relocatable. So they should stay in the path where they were created.

Also note that exec() returns only the last line of the output. You probably want shell_exec() or exec('...', $output) to get the whole output.

Also, it is unclear what happens with your script, and what is being printed on stderr. Try this command to see what is the error:

exec('/path/to/script 2>&1', $output)
#OR:
exec('/path/to/venv/bin/python3 /path/to/script 2>&1', $output)

Answer 2

OK, I finally figured it out and learned a lot in the process. The newspaper lib that I am using by default tries to write to the base of the users home directory. In this case, it was attempting to write to www-data , /var/www .

To fix this:

1. Go to the settings.py file in the newspaper library.
2. Edit the variable DATA_DIRECTORY = '.newspaper_scraper' and change it to DATA_DIRECTORY = '.path/to/writable/directory'
3. Save the file and you should be good to go.

I have no idea why it was not returning the errors that would have explained this sooner.

Hope this helps anyone else.

Thank you so much Sergey Vasilyev for your help. I appreciate it greatly.