PHP / MySQL IP到“国家/地区”统计页面

Question

I am currently converting a script for someone else and in their old code they had a CSV file listing all IPs and their countries. The file was in this format:

16777216    17367039    AU  AUS AUSTRALIA
17367040    17432575    MY  MYS MALAYSIA
17432576    17498111    AU  AUS AUSTRALIA
17498112    17563647    KR  KOR REPUBLIC OF KOREA
17563648    17825791    CN  CHN CHINA

Fine, that's what they did, I imported it to the new MySQL DB. But problem is they have 111K records here!

To do the comparison and find IP to country it takes around 40 seconds. Currently they select all records and compare to less than and more than the next field.

$datafile = file("iplist.csv");
    $countries = array();
    foreach($datafile as $data) {
        $data = explode(",", $data);
        foreach($iplist as $ipan => $ip) {
            if($ip > $data[0] && $ip < $data[1]) {
                $countries[$ip] = array($data[2], $data[4]);
            }
        }
    }

Of course I know this is a HORRIBLE way to do this, horrible! It's even bad when I do it in MySQL, I mean 111K records just sitting there.

I've been wracking my brain all day and can't think of anyway better to do this. IPs are stored in the DB in standard format. I thought I would just do a select from MySQL and in the where statement do a more than and less than. But for the life of me I can think of how. Considering this is how they make their IP list:

foreach($log as $row) {
        $iplist[$row[2]] = $row[2];

        if($last != "") {
            $dif = $last-$row[5];
            $avgtime += $dif;
        }
        $last = $row[5];
    }

I thought of doing one call per record, but if there are a normal of 120 records that come out on each stat page then that's out since that would be 120 calls to mysql minimum.

Can anyone give me a better and more efficient way to get this comparison?

I was thinking of using an outside service, but the requests would be so much I think the site would get blacklisted.

Answer 1

WHy not try this:

SELECT * FROM table WHERE ip_min >= $your_value and ip_max <= $your_value

Where ip_min would be that first column of data and ip_max is the 2nd column. $your_value would obviously be the ip value that is known to you.

Also with regards to geoIP location, why not look to one of the standard databases like MaxMind. I personally like to handle the country ip geoLocation on the web server using the mod_geoip module with MaxMind database. You can then just update this database on a monthly basis and not have to worry about trying to keep a current IP Database in your application. mod_geoip simply exposes a few additional value into the $_SERVER superglobal that you can read to get country code or country name (or even down to city).

Answer 2

You should use join in your mysql query:

SELECT s.*, g.country_code, g.country_name FROM stats s LEFT JOIN geoip g ON ( g.ip_min <= s.ip AND g.ip_max >= s.ip ) 

Then you'll have it in one query.

Faster would be use INNER JOIN instead of LEFT JOIN but then it won't select record with ip out of range

Anyway I think it would be much better to find that location before inserting to stats table and assign ID of country with stats record - you'll also have to join it, but it will work much faster