fetch multiple urls data using PHP Curl, Jquery In Loop

Question

I found this script from ( http://w3lessons.info/2012/01/03/facebook-like-fetch-url-data-using-php-curl-jquery-and-ajax/ ) Problem is that i want to do in loop with multiple urls.

<link rel="stylesheet" type="text/css" href="style.css" />
    <!--[if lt IE 7]>
        <link rel="stylesheet" type="text/css" href="style-ie.css" />
    <![endif]-->
<script type="text/javascript" src="http://code.jquery.com/jquery-1.6.1.min.js"></script>
<script type="text/javascript" src="js/jquery.livequery.js"></script>
<script type="text/javascript" src="js/jquery.watermarkinput.js"></script>
<script type="text/javascript">
    $(document).ready(function(){
        // delete event
        $('#attach').livequery("click", function(){

            if(!isValidURL($('#url').val()))
            {
                alert('Please enter a valid url.');
                return false;
            }
            else
            {
                $('#load').show();
                $.post("curl_fetch.php?url="+$('#url').val(), {
                }, function(response){
                    $('#loader').html($(response).fadeIn('slow'));
                    $('.images img').hide();
                    $('#load').hide();
                    $('img#1').fadeIn();
                    $('#cur_image').val(1);
                });
            }
        });
        // next image
        $('#next').livequery("click", function(){

            var firstimage = $('#cur_image').val();
            $('#cur_image').val(1);
            $('img#'+firstimage).hide();
            if(firstimage <= $('#total_images').val())
            {
                firstimage = parseInt(firstimage)+parseInt(1);
                $('#cur_image').val(firstimage);
                $('img#'+firstimage).show();
            }
        });
        // prev image
        $('#prev').livequery("click", function(){

            var firstimage = $('#cur_image').val();

            $('img#'+firstimage).hide();
            if(firstimage>0)
            {
                firstimage = parseInt(firstimage)-parseInt(1);
                $('#cur_image').val(firstimage);
                $('img#'+firstimage).show();
            }

        });
        // watermark input fields
        jQuery(function($){

           $("#url").Watermark("http://");
        });
        jQuery(function($){

            $("#url").Watermark("watermark","#369");

        });
        function UseData(){
           $.Watermark.HideAll();
           $.Watermark.ShowAll();
        }
    });

    function isValidURL(url){
        var RegExp = /(ftp|http|https):\/\/(\w+:{0,1}\w*@)?(\S+)(:[0-9]+)?(\/|\/([\w#!:.?+=&%@!\-\/]))?/;

        if(RegExp.test(url)){
            return true;
        }else{
            return false;
        }
    }
</script>
<input type="hidden" name="cur_image" id="cur_image" />
<div class="wrap" align="center">
    <div class="box" align="left">
        <input type="text" name="url" size="64" id="url" />&nbsp;&nbsp;
        <input type="button" name="attach" value="Attach" id="attach" />
        <div id="loader">
        <div align="center" id="load" style="display:none"><img src="load.gif" /></div>
        </div>
</div></div>

Is there any i can do loop and load the different different url same time? Please help me!

Answer 1

In JavaScript who can't create thread.
So you can't fetch all those "different url at same time".

But you can "almost" achive the same thing using the event loop to request them quickly one by one without waiting for the HTTP response. Who end up being very quick !

Let's say for exemple you want to featch 3 url:

• www.mysite.com/myurl1
• www.mysite.com/myurl2
• www.mysite.com/myurl3

You can write something like that using jQuery:

$.get('http://www.mysite.com/myurl1', function(data) {
  alert('html for site 1:' +data);
});

$.get('http://www.mysite.com/myurl2', function(data) {
  alert('html for site 2:' +data);
});

$.get('http://www.mysite.com/myurl3', function(data) {
  alert('html for site 3:' +data);
});

It will request the 3 page "almost" in the same time. The first HTTP request will call the "alert('html for site x:...');"
but you don't know witch one will arrived first.

Anyway you probably need something more flexible.
Let's say you want to request 50,000 pages requesting them in "almost" the same time using 200 simultaneous request.
You can write something like that in JavaScript:

function getNextUrl(){

    urlIndex ++;

    if(urlIndex >= stopAtIndex){
        //nothing to do anymore in the event loop
        return;
    }

    $.get('http://www.mysite.com/myurl'+urlIndex, function(data) {
        // html receivend here
        getNextUrl();
    });

}

/* program start here */
int urlIndex = 0;
int stopAtIndex = 50000;
int simultaneousRequest = 200;

for( var i = 0; i < simultaneousRequest; i++ ) {
    getNextUrl();
}