Mashing up the Enterprise

by Paul Ramsey

Introduction

Popular culture has a history of using and abusing language, and the wheel of change seems to spin faster and faster lately. It took years for a sofa to become a chesterfield and a tissue to become a kleenex, but now hardly a day goes by without a new piece of trendy terminology springing up. Rip (as in CDs), burn (also for CDs), tivo (as a verb), blog (noun and verb), google (verb), podcasting or podslurping (not as yucky as it sounds), and now, mash-up.

The term mash-up was invented in the land of black-clad hipsters to describe a form of music that digitally combines the vocals of one song with the backing instrumentals of another. A classic example of a mash-up is the "Grey Album", which fuses the vocals of rapper Jay-Z's Black Album with sampled instrumentals from the Beatles' White Album.

Nothing stays cool forever, and mash-up has recently been dragged kicking and screaming into the geeky Internet world. In a technology context, it refers to a Web site created through the combination of information from two or more other Web sites. Most frequently, mash-ups are made by combining non-spatial information from one site with map information from Google Maps, adding a new level of spatial context to the original information.

The Early Days

When Google Maps was released to beta in February 2005, it was an immediate sensation, even though the core consumer-mapping concept had been around for years. The appealing speed and interactivity of the interface combined with some friendlier cartography conventions to make Internet mapping seem new again.

The speed and interactivity of the interface was not an accident. It was derived from a relatively new form of Web interface technology. In fact, the interface was entirely client-side Javascript and HTML Document Object Model, so it was (and still is) possible to download and read the source code. In addition, the graphical map tiles were accessible as static URLs, so it was theoretically possible for anyone to reuse Google data in their own applications.

The original Internet mapping sites, such as Mapquest and Yahoo! Maps, were designed to prevent developers from repackaging map data or services through URL obfuscation and careful code containment on the server side. In contrast, the modular, readable, client-side Google design practically screams, "Reuse me!"

Predictably, the elegance and speed of the Google interface served as its own undoing. Developers just had to know how it worked, and within days, analyses of the Google code were popping up all over the Internet. The analyses showed that by using the Google Web services, developers could gain access to a free base map, a free geocoder, and a free routing engine. A couple of weeks later, the first Google Maps mash-ups started to appear, directly using the Google Javascript code to extend the interface and add custom data to the maps.

Some early mash-ups took popular sites and recontextualized them with a spatial interface. Housingmaps.com, for example, combined the apartment listings service CraigsList.org with Google Maps to present a real-time map of available apartments for rent in different cities (see Figure 1). Other mash-ups took obscure sources of data and became more popular than the original sites. Chicagocrime.org took public information on recent crime reports from the Chicago Police Department and mapped them into a spatial view of crime in the city (see Figure 2). That site has since won an award for innovative journalism and has been referenced in numerous newspaper and magazine articles.

The appeal of a mash-up such as housingmaps.com is one any unreconstructed geography nut can relate to -- it makes explicit the spatial context of a body of information. Geography nuts can ramble on for hours about the underlying spatial nature of most databases, but mash-ups expose that nature in an immediate, visual, and useful way.

Exposing the Ingredients

The pent-up motivation to create mash-ups has been around for a lot longer than Google Maps, so the current explosion of mash-ups is a bit of a conundrum. Why now? After all, Google did not invent the motivation to map the unmapped; they just set it free to roam.

The answer lies in the components of a mash-up and the relative effort involved in creating them. Most mashups make use of a base map, a geocoder, and a Web interface. Before Google Maps, it was possible to get access to all three from such service sellers as ESRI (ArcWeb Services) and Microsoft (MapPoint), but at a price that was unappealing to the amateur enthusiast.

Free alternatives existed in the form of TIGER data, such opensource software as Mapserver and Geocoder.US, and Web technologies, but putting it together into a complete application required a huge amount of time spent grunting on technology, not enjoyably mashing data.

For Web developers, the magic of Google Maps is the low technical barrier to entry. Developing a mash-up site using open-source technology and free data requires knowledge of databases, UNIX, software compilation, GIS data formats, map projections, HTML, and Javascript. With Google Maps, only knowledge of HTML and JavaScript is required, two of the basic tools of any Web developer.

The Fine Print

The (thus far) unwritten story of Google Maps involves the reaction at Google headquarters as thousands of geography nuts madly reverse-engineered the system after the beta release. Eventually, Google Maps beta transitioned into Google Local, a nifty mashup of Google Maps and Google's own search index. That was probably part of the plan. An explosion of thousands of unofficial mash-ups using the internal Google code base was not.

In any event, by June 2005, Google had an answer: the Google Maps application program interface (API). Henceforth, mash-ups would no longer be a grey-area activity of questionable legality; they would be sanctioned with an official programmer's API. Using the API would require a license key, and getting a key would require acceptance of Terms of Use.

The Terms of Use have both common-sense clauses and interesting clauses that might give users pause. The common-sense ones prevent users from simply taking raw data and moving on to commit various crimes or abuses, misusing the Google trademark, and so on. The interesting clauses include:

  • Giving Google the right to add advertising to their maps at a future date and requiring the user to display them without modification; and,
  • Requiring the user to only use the API for services that are "generally accessible to consumers without charge".

Once the API and Terms of Use were released, the legal space existed for a new category of mash-up, otherwise known as the "enterprise mash-up".

Mashing for the Public Good

Portlandmaps.com is the public face of the City of Portland's corporate GIS department. Rather than stick to serving internal customers, Portlandmaps decided to bring their data to the general public.

"I think most cities are scared that if they put all their data on the Web, they will just have that much more responsibility to deal with," said Phillip Holmstrand, one of the Portlandmaps developers. "We shared that concern, but we didn't want it to stop us."

Armed with a standard disclaimer about using public data for reference purposes only, the Portlandmaps team started putting the data on the Web. Like most sophisticated GIS departments, the Portland team had access to a suite of ESRI software (including ArcServer) that they used as the basis for much of their online spatial querying and analytical map generation.

When Google Maps was introduced, the Portlandmaps team knew they wanted to make use of the interface in their application. "Google Maps is an amazing achievement in both map delivery and Web application development," said Holmstrand. "We were simply blown away when we first saw it."

The new Advanced Search feature of Portlandmaps (see Figure 3) embeds Google Maps as the default mapping component on top of a query and transformation process that uses the city's ArcServer and database backends to provide fine-grained access to Portland property data.

Mashing for Profit

Public agencies are not the only ones who recognized the vast potential of Google Maps as a way of making data more accessible. In the Fortune 500, every move by the search behemoth seems to generate a fresh tidal wave of reaction and comment.

Justin Lokitz of Oracle Corporation writes the OraGIS blog, where he recently published an article about integrating Oracle Spatial data with Google Maps. Lokitz sees Google Maps as raising the profile of spatial capabilities with decision makers. "Within the enterprise, I believe Google Maps is really just a great 'how-to'. We (in the enterprise software space) are already starting to hear a tremendous rumbling from both the public and private sector surrounding the use of pure Web-based maps for business intelligence."

Oracle itself has started offering a service to enterprise spatial customers that looks like the building blocks of Google Maps: eLocation has base maps as a Web service, geocoding as a Web service, and routing as a Web service.

Lokitz says he sees Google Maps as an extension of a larger trend to "de-GIS-ify" spatial data handling. "Given that location information is also ubiquitous and can be found in probably 99 percent of all databases, I believe it was just a matter of time before people really started to use it," he said. "And by use it I don't mean ship the location data off to your GIS department where geeks like me hunch over workstations all day, which is what we've been doing for 35 years. I mean really use the location data as simply another data type to describe something of value."

A Mashing Example

Whether built by a hobbyist or a corporate IT engineer, a mash-up consists of the same basic set of components: a source of input data, an engine to make that data spatial, a mapping component (in our case, Google Maps), and some user interface components for controlling the input data.

In an enterprise mash-up, the source of input data is often a corporate database system. The corporate database usually contains street or postal addresses that are passed through a geocoder, which then turns the address descriptions into explicit spatial coordinates that are ready to place on a map. Finally, the points are added to the mapping component. However, before you can work with Google Maps, you will need to get an API key from Google (http://www.google.com/apis/maps/signup.html). Trying the examples in this article without inserting your own key in the code will not work.

To get a feel for why developers were so excited about the ease of using Google Maps, take a look at the code sample in Figure 5, which creates a basic map on an otherwise empty page. Only three lines of Javascript are used, and the last two are not even strictly necessary! You can see the page in action at sample1.html.

For an example mash-up, we will take the real-time earthquakes information published by U.S. Geological Survey (USGS) and place it on a world map using the Google Maps API. You can see the mash-up at sample2.html.

First, we need the earthquake data, our input datastream. Fortunately, USGS publishes the data in an XML format (see Figure 6), which is easy to handle in Javascript. The latest XML file is always available at http://earthquake.usgs.gov/earthquakes/feed/v1.0/summary/4.5_week.quakeml. Note how each earthquake is stored as an <item>, with subelements for <title>, <geo:lat>, <geo:long> and so on. This is enough information to plot them on the map!

Because the Javascript security model doesn't allow Web pages to fetch information from sites different from the originating page, we use a small script (see Figure 7) on our server to pull the XML file from its home and redirect it. This is to make the USGS real-time file seem to reside on our server.

Now the only thing needed is the Javascript to glue the XML file into the Google Map (see Figure 8). The first three lines of Javascript create the map window, as in our simple example above. The next three download the XML data file, via our small redirection script, "sample2.php". The next block of 20 lines is the magic part. The code loops through each <item> in the XML data file, extracting the <title>, <geo:lat>, <geo:long>, and other information. Each <item> is used to build a GMarker (the little red bats in Google Maps), which is added to the map.

That's it! Each time the page is loaded, a fresh copy of the XML data file is fetched. this means that the page is always up-todate with the latest earthquake information that is available (see Figure 9).

Beyond GIS

More than the technology itself, the huge sweep of Google's Internet mind-share is changing the way ordinary people think about maps. In fact, mash-up is the term that Internet developers are using almost universally to describe data presented on a map -- not GIS.

Oracle's Lokitz said that this is just the leading edge of a larger trend. "My personal belief is that most end users of maps (or mapping applications) neither need nor want anything to do with GIS," he said. And now with Google Maps and mash-ups, they do not have to.


Figures


Figure 1


Figure 2


Figure 3


Figure 9