Author: John O'Brien

Updated! Pin Differential is no more!

Previous to Version 6 of the API, I implemented a pin differential algorithm that looped over the current data and added only the new pins, removing only the pins now out of view. In the last few days I've been work on the next generation of the solution. As part of this I have been testing the performance of the new bulk "addshape" method and determined its performance renders this older logic obsolete. Download the new JavaScript code below.

This article is part four in a series that explores the next generation clustering techniques for Virtual Earth. In part one Clustering a million points on Virtual Earth using AJAX and .Net we explored the general concept and made it work with ajax and vb.net. In part two Clustering Virtual Earth with MS AJAX and C# we refined the server side code with a complete rewrite in C# and we introduced ASP.NET AJAX web services. Part three looked at upgrading to Virtual Earth 5 and using the latest Object Orientated approach to Javascript from ASP.NET AJAX and also introduce a shape differential to only add and remove shapes as required to the map. In this article we upgrade to Virtual Earth 6, making use of the bulk addition of shapes, introduce some throttling logic to improve performance and consider PNG icons.

*Note: The server side code, and as such the core clustering logic, has not changed. The changes are purely client side in the javascript files.

In the very first article we learnt why we need clustering for large numbers of points and also for smaller numbers of point in the close proximity.

(Impossible to select an individual pin, large data size to client)

(Pins represent all pins under them, small data size to client)

Virtual Earth Version 6

Version 6 builds on the strong improvement made in V5 while making no breaking changes, simple by changing the URL of the API the application will run without change.

<script type="text/javascript" 
    src="http://dev.virtualearth.net/mapcontrol/mapcontrol.ashx?v=6"></script>    
    

Several key improvements from V6 we will utilise today including the bulk addition of shapes, bounds of birdseye scenes and css layout. The performance benefits and the support for Safari are great reasons to upgrade your application.

As usual the snippets I show are from the full source code that can be downloaded from the link at the end.

Bulk Shapes Addition

Shapes of type Pushpin can now be added in bulk, the change to the API in minimal, AddShape now accepts an array of shapes.

this._layer.AddShape([item1, item2, item3, ....., itemN]);

So we can modify our function that determines what needs to be added to the map and add to an array instead of directly to map. Then with a single call add all the shapes. I measured in the order of 5x the performance utilising this feature under normal conditions. Most interesting was measuring the time to add extremely large data examples to map, where adding 1500 shapes used to take over 2 min it now happens in a few seconds. The code changes: 

_OnMapDataSucceeded: function(results) {
    /// <summary>
    ///   Receive data for map.
    /// </summary>  
    /// <param name="result">The webservice result object - Optomised CSV string</param>  
        
    //decode pins
    var result=results.split(",")
    var locs = Utility.decodeLine(result[0]);
    var newShapes = new Array();
    
    //clear existing pins
    this._layer.DeleteAllShapes();
           
    //add new pins
    for(x = 0; x < locs.length; x++) {
        var loc = locs[x];
        var bounds = result[x+1];

        var newShape = new VEShape(VEShapeType.Pushpin, loc); 
        newShape.Bounds = bounds;         
        //set custom png pin, IE6 will require a PNG fix.
        if(Sys.Browser.agent==Sys.Browser.InternetExplorer&&Sys.Browser.version==6){
            newShape.SetCustomIcon("<div class='myPushpinIE6'></div>");
        }else {  
            newShape.SetCustomIcon("<div class='myPushpin'></div>");
        }
        
        newShapes.push(newShape);

    }       
    this._layer.AddShape(newShapes);           
},

Birdseye Bounds

When we upgraded to version 5 of VE we came across a problem, it was no longer possible to get the latitude and longitude bounds of the current scene in order to determine what data we needed from the server for the current view. The issue was one of license agreements concerning accurate reverse lookups of latitude and longitude on the birdseye scene. Although an encoded version is accessible for drawing and the like we had to resort to saving the centre of the view and approximating the area. Version 6 now gives us a new method that returns the approximate bounds, oversized to include at least the current scene. This gives us exactly what we need under the imposed license restrictions.

var rect = be.GetBoundingRectangle();

This means we can remove our previous hack and get our bounds when in birdseye mode like so:

if (this._map.GetMapStyle() == VEMapStyle.Birdseye) {    
    //set zoomlevel      
    zoom = 19;
    var be = this._map.GetBirdseyeScene();
    var rect = be.GetBoundingRectangle();
    points.push(rect.TopLeftLatLong);
    points.push(rect.BottomRightLatLong);

}else {

Throttling the web service calls

Using this code every time you move the map it calls the webservice to get the data for the current view. This is very powerful and allows us to stream just the information the user requires on demand. The issue is if you move the map many times in a short period of time a request is made for every move and this data must be processed. By default only two requests can be executed against the same domain so we must wait. The effect is the map data seems to lag, mimicking your previous movements as they load.

The solution is to only make, and process, one request at a time. Utilising the throttling concept from Omar Al Zabir we override the web service call and build a buffer of requests only allowing the most recent map data request to be made. If the server request for map data took one second to return and the map was moved 10 times within that one second, where previously we would see a staggered 10 sets of data be rendered over the 5-10 seconds instead only 2 calls are ever made, the initial move and the most recent, the others are discarded. The user sees the current views data 1 sec after they stop moving the map, the most ideal result.

var GlobalCallQueue = {
    _callQueue : [],    // Maintains the list of webmethods to call
    _callInProgressNames : [],    // Maintains webmethods names in progress by browser
    _callInProgress : 0,    // Number of calls currently in progress by browser
    _maxConcurrentCall : 2, // Max number of calls to execute at a time
    call : function(servicePath, methodName, useGet, 
        params, onSuccess, onFailure, userContext, timeout)
    {
        
        var queuedCall = new QueuedCall(servicePath, methodName, useGet, 
            params, onSuccess, onFailure, userContext, timeout);


        //if method name is already waiting then remove from queue then add the new call
        for (var x=0;x<GlobalCallQueue._callQueue.length;x++)
        {
            if (GlobalCallQueue._callQueue[x]._methodName==queuedCall._methodName)
            {
                Array.removeAt( GlobalCallQueue._callQueue, x);
                break;
            }
        }   

        Array.add(GlobalCallQueue._callQueue,queuedCall);
        GlobalCallQueue.run();
    },
    run : function()
    {
        /// Execute a call from the call queue
        if (GlobalCallQueue._callInProgress < GlobalCallQueue._maxConcurrentCall)
        {
            if( 0 == GlobalCallQueue._callQueue.length ) 
            {
                return;
            } 
                       
            GlobalCallQueue._callInProgress ++;
        
            //get first index that is not already in progess
            var runIndex = -1;
            for (var x=0;x<GlobalCallQueue._callQueue.length;x++)
            {
                var found=false;
                for (var y=0;y<GlobalCallQueue._callInProgressNames.length;y++)
                {
    if (GlobalCallQueue._callQueue[x]._methodName==GlobalCallQueue._callInProgressNames[y])
                    {
                        found=true;
                        break;
                    }
                }  
                if (!found)
                {
                    runIndex = x;
                    break;
                }
            }
            if (runIndex!=-1)
            {
            

                var queuedCall = GlobalCallQueue._callQueue[runIndex];
                Array.removeAt( GlobalCallQueue._callQueue, runIndex );             
                
                //set name in progress
                Array.add(GlobalCallQueue._callInProgressNames,queuedCall._methodName);
                
                // Call the web method
                queuedCall.execute();
            }else
            {
                GlobalCallQueue._callInProgress --;
            }
        }          
    },
    callComplete : function(methodName)
    {
        GlobalCallQueue._callInProgress --;
        //remove name in progress
        for (var x=0;x<GlobalCallQueue._callInProgressNames.length;x++)
        {
            if (GlobalCallQueue._callInProgressNames[x]==methodName)
            {
                Array.removeAt( GlobalCallQueue._callInProgressNames, x );
                break;
            }
        }
        GlobalCallQueue.run();
    }
};

QueuedCall = function( servicePath, methodName, useGet, params, 
    onSuccess, onFailure, userContext, timeout )
{
    this._servicePath = servicePath;
    this._methodName = methodName;
    this._useGet = useGet;
    this._params = params;
    
    this._onSuccess = onSuccess;
    this._onFailure = onFailure;
    this._userContext = userContext;
    this._timeout = timeout;
}

QueuedCall.prototype = 
{
    execute : function()
    {
        Sys.Net.WebServiceProxy.original_invoke( 
            this._servicePath, this._methodName, this._useGet, this._params,  
            Function.createDelegate(this, this.onSuccess), // Handle call complete
            Function.createDelegate(this, this.onFailure), // Handle call complete
            this._userContext, this._timeout );
    },
    onSuccess : function(result, userContext, methodName)
    {
        this._onSuccess(result, userContext, methodName);
        GlobalCallQueue.callComplete(methodName);            
    },        
    onFailure : function(result, userContext, methodName)
    {
        this._onFailure(result, userContext, methodName);
        GlobalCallQueue.callComplete(methodName);            
    }        
};

//override invoke
Sys.Net.WebServiceProxy.original_invoke = Sys.Net.WebServiceProxy.invoke;
Sys.Net.WebServiceProxy.invoke = 
    function Sys$Net$WebServiceProxy$invoke(servicePath, methodName, 
        useGet, params, onSuccess, onFailure, userContext, timeout)
{   
    GlobalCallQueue.call(servicePath, methodName, useGet, params, 
        onSuccess, onFailure, userContext, timeout);
}

Using PNG icons

Shapes have an icon, usually an image. As the shapes are simply HTML elements the type of image that can be used is limited to the browser. The PNG format supports a full alpha channel, that is 256 levels of transparency for every pixel. This differs from the GIF format that only support fully transparent pixels. For map icons the format allows us to use drop shadows, smooth edges and effectively more eye-popping icons that stand out from the map.

The issue we face is that IE6 does not support the PNG transparency format without some help. The most effective solution is to use an IE filter in your CSS to load the PNG image. Now the trick is you need to detect IE6 and only perform this trick for that browser and version. The AJAX library has just the functions we need, in this code I use a <div> for my icon and set a slightly different css class based on the browser detection.

if(Sys.Browser.agent==Sys.Browser.InternetExplorer&&Sys.Browser.version==6){
    newShape.SetCustomIcon("<div class='myPushpinIE6'></div>");
}else {  
    newShape.SetCustomIcon("<div class='myPushpin'></div>");
}

CSS

.myPushpin {
    margin-left: 4px;
    margin-top: -24px; 
    background:url(pin.png); 
    height:40px; 
    width:35px;}
.myPushpinIE6 {
filter:progid:DXImageTransform.Microsoft.AlphaImageLoader(src='pin.png', sizingMethod='image'); 
    height:40px; 
    width:35px;}

Sample Source Code Link Below

You must download and install MS ASP.NET AJAX to run this code.

The sample code for this article gets it data from an XML file of Australian postcodes. I have cached this generated data in memory for performance, for small sets of static data this maybe an option for you. For larger sets of data I recommend storing the data in a database allowing you to query based on the map bounds.

Conclusion:

Version 6 of Virtual Earth is very much welcomed by this code, we improve the performance and remove a "hack" required previously. Additionally I shared with you the concept of throttling data calls to increase the effective response time of the system and the use of PNG graphics.

Spatial support in SQL Server 2008 is just around the corner. The ability to move much of the server-side cluster logic to the data tier promises to increase the performance of this concept significantly. I look forward to this database technology providing many improvements for working with large sets of spatial data with Virtual Earth. 

Have a comment or used this code on your site? Why don't you tell me about it at my blog