Valery Hronusov discovers that Dropbox, a new free (2GB) or premium (50GB for $10/month) file syncing and sharing service, works great for hosting KML and KMZ files, including network links. What’s truly innovative about Dropbox is that it in addition to syncing specific folders across the internet to other computers, it also automatically syncs a public folder on your hard drive with an internet-accessible directory — just like Apple’s MobileMe, except that Dropbox also provides unique URLs for the files, making them easy to link to.
This means, for example, that if you keep and update KML files in a designated public folder on your computer at home, these will automatically update on the web in a fully scalable fashion.
The upshot: If you have desktop software that automatically produces, say, KML files of your georeferenced photos (as Houdahgeo for Mac does) you should be able to create some nice automated workflows that lets ou publish directly to the web. [Update: iPhotoToGoogleEarth would work great too.]
There’s more: As Valery points out, Dropbox seamlessly handles network links (after all, they are just files), and of course they can be made to reference other Dropbox URLs. So: If you were to keep one fixed network link whose URL you make public and don’t change, you could have it reference other KML files via their Dropbox URLs which you do update. To illustrate this point, I just converted my recent Egypt trip KML file from last month to Dropbox, which required nothing more than updating two URLs. Here it is. And from now on, all I need to do to change their content on your Google Earth is to update them on my laptop. No FTP needed.
(The only thing that hasn’t been properly tested yet: What is the bandwidth limit for DropBox: Can it handle lots of simultaneous uploads, like Google Earth Community can? Feedback on speed appreciated.)
Perhaps, over the holidays, you read about the unspoiled forest “discovered” using Google Earth by researchers from Kew Garden, London, who subsequently found a range of new species when they travelled there. Given that the press speaking points (and hence the news articles) put such an emphasis on the role of Google Earth in the narrative, you’d expect that the media would perhaps show the forest on a Google Map alongside the web versions of their articles. No such luck.
Geonames continues to amaze me with how it always has a location for any place name you care to throw at it. One of my most used tools in Google Earth is in fact the network link to Google Earth that Geonames provides after you search its database. The network link does a location-based search of its database for your field of view every time you stop moving — and as such it is great for browsing remote places. Just save that link in My Places and turn it on anytime you want to know the name of that strange feature you’ve found at in the middle of the Sahara, Siberia, or north-central Mozambique.
A slightly more whimsical thought: Notice how the newly discovered forest is defined by a dearth of named features in Geonames’s database? Perhaps it might be possible to devise an algorithm to find other such “holes” and then seeing which of those are unspoiled biomes (as opposed to lakes or mountain ranges. (Panoramio, Google Earth Community, Wikipedia and every other base layer in Google Earth all come up empty-handed for this particular spot — yet another hint.)
[Update January 7m 2009: An automatically updated version of this map can now be found here. (Accompanying post).]
The Middle East Strategy at Harvard blog made me aware of a just-released UNOSAT map detailing the attacks inside and around the Gaza Strip from Dec 25 and up to Dec 31, 2008 (yesterday). The map is available for download from the UNOSAT site as a PDF. I downloaded it, rasterized it and geopositioned it in Google Earth as an image overlay. Download the KMZ file for an overview of this latest war on Google Earth.
As usual, play with the transparency slider for the overlay to switch between the map view and the underlying view of Google Earth.
UNOSAT continues to update the map, so do go and get yourself the latest one if you don’t read this post today.
In the rush to publish the previous post about my Middle Egypt KML trip report (I was also packing for a day just spent travelling) I forgot to mention several further points I had on my mental to-do list:
Google Earth + iPhone = awesome ruin navigation tool: While the iPhone isn’t any good for making tracks, it was a most excellent mobile Google Earth — Especially in Egypt, where the base imagery is 2.5m Spot Image instead of 15m used elsewhere, and where most archaeological sites have glorious high-resolution DigitalGlobe imagery, the result of many special requests by archaeological institutes over the years.
My iPhone happens to be locked to a Swedish contract, which implies ridiculously expensive data rates abroad, so I don’t do mobile surfing with it while in Egypt. (When-oh-when will EU regulators call this practice what it is — cartel pricing — and make it illegal for European carriers?). I do, however, surf on hotel wifi (provided free even in the cheap ones) so during my trip, I spent some time each evening visiting the next day’s sites on my iPhone.
Since Google Earth for iPhone keeps its imagery in a huge cache, I don’t need to connect to the internet later to use Google Earth while in the field — all the imagery is right there in the cache from the night before.
On several occasions while traipsing around Akhetaten (aka Amarna) we all ended up huddling around the iPhone to better see the view from above in Google Earth, zooming and rotating until we could identify the feature right in front of us.
I also ended up using a trick that makes flicking between favorite views a lot faster: The night before, I’d set up good views of a site and then take snapshots of the screen. (While keeping the Home button pressed down, hit the on/off button). After a while, I’d have a folder of Google Earth views ready for browsing on the iPhone without the startup lag or processing cycles of Google Earth (which helps preserve batteries). For example:
(One more thing (so much minutiae:-) — GPS on the iPhone without an internet assist (The A in A-GPS) makes i as slow as a dog, much like the early days of the Nokia N95. So if you turn off roaming data while abroad to save money, you can forget about getting a fix in anything near a reasonable time, if at all. Another win for a dedicated GPS unit.)
Google Earth panorama FOV bug: If you fly into a 360-degree panorama via the current version of Google Earth, you’ll notice that you can’t really zoom out all that much to get a wide-angle field of view. That turns out to be a known bug, as 360Cities’ Jeffrey Martin pointed out to me.
To see the difference, view the same panorama in 360Cities’ web viewer (click on the link at the bottom of the popup window.)
Imagery positioning in Google Earth: I’ve been faced with this dilemma before: In Cairo, the road layer in Google Earth is positioned about 125m northwest of the imagery layer, so which should I use as a reference when contributing placemarks to the neogeo commons? I’ve been using the imagery layer, as it appears that the road layer was published to an older datum before being released to the public domain — and all mapping services have the same problem.
But now, after using a Garmin Oregon 300, one of the new generation of GPS devices with accuracy that routinely surpasses 5m, I’ve had a chance to compare that imagery layer in Google Earth to some real world locations, only to find that you can’t assume the the accuracy of the positioning of the imagery matches the imagery’s resolution.
I should have known this, of course, but until now I have been blissfully placing placemarks with great care, to within the meter if the resolution of the imagery allowed it. Such confidence I now realize is misplaced, though for a while I wondered if the disparity wasn’t due to some kind of systemic drift in the GPS signal, or improperly orthorectified images — it looked like the higher I climbed the cliffs of the Nile valley, the more divergent the track became. But then the obvious answer was to be found at the seams of the imagery mosaic:
The above seam is between two DigitalGlobe imagery tiles taken on the same day and part of the same strip, just south of El Minya. There is more where that came from:
It turns out some tiles are just better positioned than others.
As a result, the KML trip report’s georeferenced photos are now faithful not to the base imagery but to the GPS track, even if they look misplaced — because one day soon an imagery update will produce more accurately positioned imagery, which means in turn that placemarks previously placed using the imagery as a reference will suddenly be off-target.
Of course, Google Earth doesn’t come with any guarantees for accuracy, and accurately positioning a globe’s worth of imagery is a gargantuan task, but it is good to keep in mind that Google Earth’s imagery is not the gold standard when it comes to goereferencing a location. GPS units are.
Earlier this month I spent a week travelling to archaeological sites in Middle Egypt with a group of Egyptology students from the universities of Leiden and Leuven. The trip was a rare chance to visit places that are off-limits to non-Egyptologists in a part of the country not often visited by foreigners; but it was also a great opportunity to produce a georeferenced addendum to the trip report, with a view to giving it some archival value, as well as making these places come alive for others.
Oxen with scarves, on a wall in the tomb of Djefai-Hapy, Asyut
Here’s the end result that I will write about in the rest of this post — a KMZ network link that is best opened in Google Earth, as it currently has the best satellite imagery of the region.
The returned KML contains a folder with georeferenced photos uploaded to Flickr, georeferenced 360-degree panorama photos uploaded to 360Cities, destination placemarks, daily tracks, and — turned off by default — every single track point collected during the trip, with popups detailing our movement. The panoramas and photos have captions, while the destination popups contain links to further information on the web.
The making of the KML:
For the duration of our trip, I had a GPS unit in my rucksack, gathering data. I used a Garmin Oregon 300, which proved to be very robust and doggedly determined to stay locked onto its satellites — often even well into a rock tomb, with just a small opening to the sky. It would eat a pair of AA batteries per day, and every evening I also made sure to save the day’s readings into a track. Because I had a laptop with me, I also backed up the data to my Mac daily using the free LoadMyTracks, saving both GPX and KML versions of the data. The Garmin can keep multiple days of readings in its memory before filling up, but I wanted a redundant set of data.
At the start of the trip I made sure to calibrate the time on my GPS with the time on my camera, and then took pictures and 360-degree panoramas at will. (I’ve already written up how I make my panoramas.)
Once home, I georeferenced my photos using the free GPSPhotoLinker for the Mac. I gave it all the GPX files I had from the trip — GPSPhotoLinker seamlessly collated them, and then batch-processed the photos to a time-weighted average of the closest track points. After editing them in Aperture for Mac, I uploaded a subset to Flickr using the FlickrExport plugin. Flickr can read the EXIF coordinate metadata, so all the photos in the trip’s Flickr set are georeferenced. I also wrote detailed captions for each uploaded photo.
I then used Adam Franco’s still unique (to my knowledge) Flickr Photo Set to KML web app to create a KML file that references the Flickr photos in the set, and also shows their captions. The app works well but hasn’t been updated in a while, so the resulting KML needed some tweaking, for example to make sure it references the newer default Google Earth icons.
Turning to the 360-degree panoramas: Once I was happy with them, I exported a 100% quality full-size JPEG of each panorama — weighing in at 32MB and around 12,000×6,000 pixels per file — and uploaded these to my account at 360Cities. For each photo, 360Cities lets you select its location on a map, either by dropping a pin on a Google Map or by entering the exact coordinate data. As you can zoom in a lot closer on the imagery in Google Earth than on Google Maps, I made accurate new placemarks for them in Google Earth and then copied their coordinates into 360Cities.
360Cities also has a cool tool that lets you align your panorama with the compass. As a result, inside each panorama hosted at 360Cities, there is the option to navigate to nearby panoramas by clicking on hotspots in the appropriate direction. To accurately align my panoramas, I often found myself comparing landmarks in Google Earth to the view in the panorama. With Google Earth aligned to north, it is quite easy to get the orientation right to within a few degrees. And for each panorama in 360Cities, here too I added a detailed caption.
360Cities provides a downloadable KML file for all the photos by a specific photographer, including captions. I used this file as the starting point for my panoramas of Middle Egypt — 360Cities does not yet support sets, so I deleted the panoramas I took of Sweden this past summer from the resulting file.
(And 360Cities is also about to launch an embedding function for panoramas. As soon as it goes live, I will replace this text with an embedded panorama.)
The tracks were very easy to make: I just used the files produces by the import-as-KML function of LoadMyTracks.
For the individual track points, I needed some heavy artillery, so I turned to Adam Schneider’s excellent GPSVisualizer web app. It can extract track data from GPX files, analyse them and then export them as time-stamped KML placemarks with popups that contain precise speed and direction data. This is not interesting to most people, but it does create an easy way to verify exactly what we were up to when and where, and where we were going and how fast — if only for archival purposes.
Next: Bringing together these individual KML files into one big file, with simplified, coordinated styles. Here, unfortunately, it was just a matter of having to manually ogle the KML markup in a text editor (I use BBEdit for Mac) and adding/tweaking the required tags. The KML reference is handy for this. The collection of individual trackpoints was simply too huge, so I turned that into an optional download via a network link. I gave each day’s track a separate color (and hence style), and created separate styles for the photo-, panorama- and destination placemarks.
Because in the future I might change the contents of this file, I am distributing a small public network link with a permanent URL that automatically uploads the latest version of my file. Downloading the network link today doesn’t mean an outdated version of the content tomorrow.
And there you have it. There are many different ways to navigate the data — chronologically, geographically or by content type. One interesting use of the trackpoints is to turn them all on and then set up Google Earth’s timeline navigation tool so that it shows only about an hour’s worth of trackpoints at a time. Then press play and watch the trip unfold in a kind of balletic frenzy.
I learned some GPS-related things on my trip:
GPS-enabled mobile phones are no substitute for a good, dedicated GPS device. If I have my iPhone running a GPS tracking program, it drains its batteries in a few hours — and meanwhile, I can’t use my phone for anything else. That’s very suboptimal. After trying for almost two years with my Nokia N95 and now with my iPhone, the verdict is in: It’s can’t replace a serious GPS device. Sure, it’s good for saving you the trouble of telling it where you are right now when you want to know something about your environs, but anything more ambitious is not ideal on a mobile phone.
On-camera GPS units face a slightly different problem. Sure, it’s great to be able to have your photos georeferenced as soon as you take them, but since you are unlikely to do anything with them until you get home and edit them, adding coordinate metadata via an application like GPSPhotoLinker is a minute extra step, and it means you can buy a more versatile GPS device than a camera-specific unit. Furthermore, why make your camera any more bulky than it needs to be? A GPS unit need not be affixed to the camera, and can be tucked away safely in a nearby bag.
More 3D for GE:CyberCity 3D Blogannounces that a whole range of new 3D buildings content has been added to Google Earth — textured models for California, and untextured models for Paris.
African conflicts:The Humanitarian Information Unit The Harvard Humanitarian Initiative (HHI) and NiJeL make a good layer highlighting Africa’s current conflicts. (Via iRevolution)
Free our data, Google edition: At the AGU Fall meeting, Google’s Michael Jones outlines Google’s recommendation to the Obama administration to link increased funding for basic scientific research to an obligation to share the resulting data freely. Watch the video of the speech. Google’s also been planning to offer to host this data for free, though that project has now been put on the back burner, apparently a victim of the current financial crisis.
I’ve been playing with Microsoft Seadragon on my iPhone — a mobile client for quickly browsing and zooming in on gigapixel imagery. The technology, developed at Microsoft Live Labs, really does provide a very efficient means of surveying large amounts of data and focusing on the information you need. In addition to some fantastic deep space gigapixel shots, there is a great set of historical maps from the US Library of Congress:
Microsoft also includes 2D Virtual Earth map datatsets — as if to prove the point that in the future, this continuous approach to zooming maps will trump the discrete layered tiles approach that Google Maps and 2D Virtual Earth currently use.
It’s great that Microsoft is building mobile tools for Apple’s iPhone, just as Apple builds iTunes for Windows, Google builds Google Earth for iPhone, and Microsoft builds Office for Mac. But where is Virtual Earth 3D for Mac? The Virtual Earth 3D plugin for PC browsers has been out since 2006, and yet there still is not a Mac version in sight. Google’s browser plugin for Google Earth came out for PCs at the end of May, and six months later the Mac version was out.
It’s wonderful to be innovative with tools like like a virtual globe browser plugin, but as long as the plugin is tied to a specific operating system, the purpose of using a web browser is defeated — that purpose being to make the operating system irrelevant. As long as Virtual Earth 3D is Windows only, it can only lose out in the adoption race against Google’s multiplatform Google Earth browser plugin. How can that be a good business decision?
Notes on the political, social and scientific impact of networked digital maps and geospatial imagery, with a special focus on Google Earth.
Valery Hronusov discovers that Dropbox, a new free (2GB) or premium (50GB for $10/month) file syncing and sharing service, works great for hosting KML and KMZ files, including network links. What’s truly innovative about Dropbox is that it in addition to syncing specific folders across the internet to other computers, it also automatically syncs a public folder on your hard drive with an internet-accessible directory — just like Apple’s MobileMe, except that Dropbox also provides unique URLs for the files, making them easy to link to.
