Print Python output by PHP Code

Question

I have a scraper which scrape one site (Written in python). While scraping the site, that print lines which are about to write in CSV. Scraper has been written in Python and now I want to execute it via PHP code. My question is

how can I print each line which is getting printed by python code.

I have used exec function but it is none of my use and gives output after executing all the program. So;

Is it possible to get python output printed while it is getting executed via PHP.

Answer 1

If i understand it well, your python scraper output to a file and you want to "live" display the output via php. What about doing and loop in php in which you use filemtime to know whether or not the file has been updated? You might add a little sleep in order not to overload your server.

If your are using a web page, you may use AJAX to reload only the concerned part of the page at a regular interval.

Hoping this helps you.

Answer 2

Simple case

Assuming execution of scraper is limited to php runtime, run it via popen: http://php.net/manual/en/function.popen.php

More involved case

If you want scraper to run on background and only connect to it vis php from time to time, you can either use some pub/sub toolkit or implement a small web server in the scraper that you can fetch result updates with fopen("https://localhost:port/...") or curl. Many other rpc mechanisms are possible, domain sockets, watching a file, sysv rpc...

Answer 3

I'd communicate using stdout instead of a file. Meaning the python script writes to stdout and the php script reads that.

Using proc_open you can control the python process from php and also read it's output.

Answer 4

Instead of using exec you could use passthru , that will output the data directly to the browser. http://php.net/manual/en/function.passthru.php

That should be enough to get the println from your script.

Answer 5

I think I have a fair idea of what you are saying put I am not too sure what you mean.

If you mean to say that everytime the python script does a print, you want the php code to output what was print?

If that is the case you could pass it as a POST DATA via HTTP. That is instead of printing in Python, you could send it to the PHP Script, which on receiving the data would print it.

I am not too sure if this is what you want though.

Answer 6

For proper communication you need to setup any medium, so you can use fifo, through it you can write string in python and read it with php.

For PHP fifo http://php.net/manual/en/function.posix-mkfifo.php

For Python http://www.doughellmann.com/PyMOTW/Queue/

Answer 7

Simply use system() instead of exec() . exec() saves all lines of stdout output of the external program into an array, but system() flushes stdout output "as it happens".