How do I Geocode 20 addresses without receiving an OVER_QUERY_LIMIT response?

Using the Google Geocoder v3, if I try to geocode 20 addresses, I get an OVER_QUERY_LIMIT unless I time them to be ~1 second apart, but then it takes 20 seconds before my markers are all placed.

Is there any other way to do it, other than storing the coordinates in advance?

Answers:

Answer

No, there is not really any other way : if you have many locations and want to display them on a map, the best solution is to :

  • fetch the latitude+longitude, using the geocoder, when a location is created
  • store those in your database, alongside the address
  • and use those stored latitude+longitude when you want to display the map.

This is, of course, considering that you have a lot less creation/modification of locations than you have consultations of locations.


Yes, it means you'll have to do a bit more work when saving the locations -- but it also means :

  • You'll be able to search by geographical coordinates
    • i.e. "I want a list of points that are near where I'm now"
  • Displaying the map will be a lot faster
    • Even with more than 20 locations on it
  • Oh, and, also (last but not least) : this will work ;-)
    • You will less likely hit the limit of X geocoder calls in N seconds.
    • And you will less likely hit the limit of Y geocoder calls per day.
Answer

You actually do not have to wait a full second for each request. I found that if I wait 200 miliseconds between each request I am able to avoid the OVER_QUERY_LIMIT response and the user experience is passable. With this solution you can load 20 items in 4 seconds.

$(items).each(function(i, item){

  setTimeout(function(){

    geoLocate("my address", function(myLatlng){
      ...
    });

  }, 200 * i);

}
Answer

I'm facing the same problem trying to geocode 140 addresses.

My workaround was adding usleep(100000) for each loop of next geocoding request. If status of the request is OVER_QUERY_LIMIT, the usleep is increased by 50000 and request is repeated, and so on.

And of cause all received data (lat/long) are stored in XML file not to run request every time the page is loading.

Answer

EDIT:

Forgot to say that this solution is in pure js, the only thing you need is a browser that supports promises https://developer.mozilla.org/it/docs/Web/JavaScript/Reference/Global_Objects/Promise


For those who still needs to accomplish such, I've written my own solution that combines promises with timeouts.

Code:

/*
    class: Geolocalizer
        - Handles location triangulation and calculations.
        -- Returns various prototypes to fetch position from strings or coords or dragons or whatever.
*/

var Geolocalizer = function () {
    this.queue          = [];     // queue handler..
    this.resolved       = [];
    this.geolocalizer = new google.maps.Geocoder();  
};

Geolocalizer.prototype = {
    /*
        @fn: Localize
        @scope: resolve single or multiple queued requests.
        @params: <array> needles
        @returns: <deferred> object
    */
    Localize: function ( needles ) {
        var that = this;
        // Enqueue the needles.
        for ( var i = 0; i < needles.length; i++ ) {
            this.queue.push(needles[i]);
        }
        // return a promise and resolve it after every element have been fetched (either with success or failure), then reset the queue.
        return new Promise (
            function (resolve, reject) {
                that.resolveQueueElements().then(function(resolved){
                  resolve(resolved);
                  that.queue    = [];
                  that.resolved = [];
                });
            }
        );
    },

    /*
        @fn: resolveQueueElements
        @scope: resolve queue elements.
        @returns: <deferred> object (promise)
    */

    resolveQueueElements: function (callback) {
        var that = this;
        return new Promise(
            function(resolve, reject) {
                // Loop the queue and resolve each element.
                // Prevent QUERY_LIMIT by delaying actions by one second.
                (function loopWithDelay(such, queue, i){
                    console.log("Attempting the resolution of " +queue[i-1]);
                    setTimeout(function(){
                        such.find(queue[i-1], function(res){
                           such.resolved.push(res); 
                        });
                        if (--i) {
                            loopWithDelay(such,queue,i);
                        }
                    }, 1000);
                })(that, that.queue, that.queue.length);

                // Check every second if the queue has been cleared.
                var it = setInterval(function(){
                    if (that.queue.length == that.resolved.length) {
                        resolve(that.resolved);
                        clearInterval(it);
                    }
                }, 1000);
            }
        );
    },

    /*
        @fn: find
        @scope: resolve an address from string
        @params: <string> s, <fn> Callback
    */
    find: function (s, callback) {
        this.geolocalizer.geocode({
            "address": s
        }, function(res, status){
           if (status == google.maps.GeocoderStatus.OK) {
               var r = {
                   originalString:  s,
                   lat: res[0].geometry.location.lat(),
                   lng: res[0].geometry.location.lng()
               };
               callback(r);
           }
            else {
                callback(undefined);
                console.log(status);
                console.log("could not locate " + s);
            }
        });
    }
};

Please note that it's just a part of a bigger library I wrote to handle google maps stuff, hence comments may be confusing.

Usage is quite simple, the approach, however, is slightly different: instead of looping and resolving one address at a time, you will need to pass an array of addresses to the class and it will handle the search by itself, returning a promise which, when resolved, returns an array containing all the resolved (and unresolved) address.

Example:

var myAmazingGeo = new Geolocalizer();
var locations = ["Italy","California","Dragons are thugs...","China","Georgia"];
myAmazingGeo.Localize(locations).then(function(res){ 
   console.log(res); 
});

Console output:

Attempting the resolution of Georgia
Attempting the resolution of China
Attempting the resolution of Dragons are thugs...
Attempting the resolution of California
ZERO_RESULTS
could not locate Dragons are thugs...
Attempting the resolution of Italy

Object returned:

enter image description here

The whole magic happens here:

(function loopWithDelay(such, queue, i){
                    console.log("Attempting the resolution of " +queue[i-1]);
                    setTimeout(function(){
                        such.find(queue[i-1], function(res){
                           such.resolved.push(res); 
                        });
                        if (--i) {
                            loopWithDelay(such,queue,i);
                    }
                }, 750);
            })(that, that.queue, that.queue.length);

Basically, it loops every item with a delay of 750 milliseconds between each of them, hence every 750 milliseconds an address is controlled.

I've made some further testings and I've found out that even at 700 milliseconds I was sometimes getting the QUERY_LIMIT error, while with 750 I haven't had any issue at all.

In any case, feel free to edit the 750 above if you feel you are safe by handling a lower delay.

Hope this helps someone in the near future ;)

Answer

I have just tested Google Geocoder and got the same problem as you have. I noticed I only get the OVER_QUERY_LIMIT status once every 12 requests So I wait for 1 second (that's the minimum delay to wait) It slows down the application but less than waiting 1 second every request

info = getInfos(getLatLng(code)); //In here I call Google API
record(code, info);
generated++; 
if(generated%interval == 0) {
holdOn(delay); // Every x requests, I sleep for 1 second
}

With the basic holdOn method :

private void holdOn(long delay) {
        try {
            Thread.sleep(delay);
        } catch (InterruptedException ex) {
            // ignore
        }
    }

Hope it helps

Answer

Unfortunately this is a restriction of the Google maps service.

I am currently working on an application using the geocoding feature, and I'm saving each unique address on a per-user basis. I generate the address information (city, street, state, etc) based on the information returned by Google maps, and then save the lat/long information in the database as well. This prevents you from having to re-code things, and gives you nicely formatted addresses.

Another reason you want to do this is because there is a daily limit on the number of addresses that can be geocoded from a particular IP address. You don't want your application to fail for a person for that reason.

Tags

Recent Questions

Top Questions

Home Tags Terms of Service Privacy Policy DMCA Contact Us Javascript

©2020 All rights reserved.