Sunday, September 3, 2017

KML-KMZ Bugs: For Google Every File is a URL

Recently a friend was struggling with KML-KMZ so called "ground overlays" -- draping local maps and imagery over specific map regions in desktop & mobile application Google Earth.

Specifically, aerial data processing via DroneDeploy -- and other photogrammetry apps -- and exported via KML-KMZ data "containers" -- when viewed in Google Earth desktop -- was producing the dreaded BIG RED X -- like this:
Google Earth: Big Red "X" for broken URL links.
Why does this happen? Before we dive into specifics, perhaps it is wise to recognized this key software design premise "baked" into every Google application and product: There are NO "files" on the web -- every file is really a web link.

KML-KMZ RAPID DIAGNOSIS?  Generally two possibilities before we can mental "see" the necessary fix:

(1) Missing local file: The image map overlay is missing from the local computer / mobile device -- or is NOT in the same folder path from which the KML-KMZ was launched (or opened), or,

(2) Busted URL (Universal Resource Locator): The KML-KMZ file contains a "broken link" -- a link that FAILS to point to actual internet "cloud" location.  This broken link can be "busted" by just a single character -- and Google Earth will be clueless -- and paste the Big Red X in the image map overlay space -- signally that Google Earth "knows" some image map *should* be there -- but fetch of the URL failed (404 file not found).

NERD DETAILS:  Opening the "broken" KML file in a plain ASCII or UTF-8 text editor -- GVIM (or MS Notepad ) -- and examining the guts -- we note the key <href> line:
Screenshot: Plain text editor GVIM internal view of 
"busted" KML file and generating Big Red X
(click for bigger)
The "fix" is simple -- from inside a plain text editor -- force the HREF line to "point" to an appropriate image map overlay -- stored -- somewhere -- in the internet cloud.  

Somewhere? Where to store the Google Earth image map "ground overlay"?  Two places -- one the KML-KMZ user can control -- and another that is more "ownership" problematic over the long term (months to years):

(1) Private Web Server w/ Public Exposure: Store the image map overlay on your personal web server -- a web server that exposes a PUBLIC link the world wide web.  For this author: Upside? This is a preferred method -- as "ownership" and control is well defined.  Downside? Requires build or purchase of a web server (i.e. Blue Host, Hostgator, Linode or Tektonic or related).

(2) "Public" web server: Store the image map overlay "in the cloud" -- and with publicaly exposed "hard" link that "reaches" into Google Photos -- or Imgur -- or Amazon S3 -- or some other "photo sharing" website -- THAT CAN AND WILL EXPOSE a "permanent" public link with *NO* demands for "log in." (The big offender here is Facebook and others attempting to enforce viral marketing of their "walled garden").  

CRAZY and OBSCURE LINKS: "Cloud Storage" w/ commercial vendors like Google -- or Imgur -- or Amazon S3 -- is tricky -- because "permanent" and hard links are kinda wild and tricky to determine and extract for use in human edited-updated KML-KMZ files.  Example -- for this topo image map we examine how stored in Google Blogger:
1976 USGS Topo Map "image overlay" for 
Cibolo Nature Center near Boerne TX
(click for full resolution)
The public, "permanent" link looks like this wild mess :

https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi0cNC7K7VEpK-PRj8Z_vmULcTGb8X5TNMMt-502WhLDmjvyFP2Ww041Bm0cjnrSIkj2v5VenCSMMZLkxOplckG6nflSX9nmTBsFtJlGyDKBy6xX0HK500K-I03u73DldGL788CsakpE2A/s1600/cibolo_nature_usgs_topo_1976_83dd_v0.png

(makes sense to a web browser or mobile app -- bizzare to any human eye not attached to nerd programmers)

Why so ugly? Google Blogger is "storing" the image map overlay in some deep database index that only makes "sense" to your browser or web app -- and the "Google System."
  
MIXING THE MAGIC: How to FIX the BIG RED X?  And use Google "cloud storage" inside this KML?  Edit-update the KML-KMZ file -- so that the HREF line utilizes the big ugly Blogger image link of above (or a "simple" URL on your personal web server).  Specifically for this example:
Corrected KML file with wild, long and 
ugly "permanent" URL link into Google
Blogger "cloud" storage (click for bigger)
SUCCESS?  First a Google Earth screen shot:
1976 USGS Topo image map ground overlay
properly draped inside Google Earth and 
over Cibolo Nature Center near Boerne TX
(click for bigger)
And click (or tap) this link to view the "live" KML file inside Google Earth.

SUMMARY:  In order to MAX SHARE DroneDeploy and other "cloud processed" aerial imagery data products -- exported via KML-KMZ "containers" -- the key image map overlays must be stored "in the cloud" -- with "permanent" and hard public links.  Google KML-KMZ inherit the Google design premise that every file is really a web link or URL.

Good luck!  And happy value hunting via data explorations!