{"id":650,"date":"2012-07-29T00:12:30","date_gmt":"2012-07-29T07:12:30","guid":{"rendered":"http:\/\/blog.light42.com\/wordpress\/?p=650"},"modified":"2012-07-30T21:18:33","modified_gmt":"2012-07-31T04:18:33","slug":"calling-bluff","status":"publish","type":"post","link":"http:\/\/blog.light42.com\/wordpress\/?p=650","title":{"rendered":"Calling Bluff"},"content":{"rendered":"<p><script type=\"text\/javascript\">\/\/ < ![CDATA[\n\/\/ < ![CDATA[\njQuery(document).ready(function($) { $(\"#gallery a\").lightBox({fixedNavigation:true});});\n\/\/ ]]><\/script><\/p>\n<div id=\"gallery\"><a href=\"wp-content\/uploads\/2012\/07\/parcels_overview_06nov10-2.png\"><img decoding=\"async\" src=\"wp-content\/uploads\/2012\/07\/parcels_overview_06nov10-2-300x134.png\" alt=\"parcels\" align=\"right\" \/><\/a><\/div>\n<p><strong>Two<\/strong> years ago a hard disk arrived on the desk of a colleague, from persons claiming to have some kind of parcel data &#8220;for every state in the US.&#8221; Naturally, being very skeptical and at the same time, just a bit eager to show off open source tools on linux, I whipped up a script to make this visualization \u2014 the result was many gigabytes of shapefiles represented by one 135K png.<\/p>\n<p><em>What was the difference between loading potentially half of North America as parcels into a database, or simply gathering what is needed and moving on without loading anything?<\/em><\/p>\n<p>=> recursively find each shapefile; use <code><strong>ogrinfo<\/strong><\/code> to get just the shapefile layer&#8217;s BBOX; use a regular expression to emit an INSERT statement into a spatial postgis table defined for this purpose; use <code><strong>psycopg2<\/strong><\/code> to execute each INSERT, periodically COMMIT; COMMIT again at the end of the loop.<\/p>\n<p>On completion you have a table of BBOX, one for each shapefile layer. For the visualization, I used QGis. All shapefile BBOX&#8217;s are displayed with transparency to show the density and the metro areas background.<\/p>\n<p>Even given the excess inherent in using its BBOX to represent each shapefile, the graphic easily showed that the collection of data was not exactly as it was described.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Two years ago a hard disk arrived on the desk of a colleague, from persons claiming to have some kind of parcel data &#8220;for every state in the US.&#8221; Naturally, being very skeptical and at the same time, just a bit eager to show off open source tools on linux, I whipped up a script [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[5,4,6],"tags":[],"_links":{"self":[{"href":"http:\/\/blog.light42.com\/wordpress\/index.php?rest_route=\/wp\/v2\/posts\/650"}],"collection":[{"href":"http:\/\/blog.light42.com\/wordpress\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/blog.light42.com\/wordpress\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/blog.light42.com\/wordpress\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"http:\/\/blog.light42.com\/wordpress\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=650"}],"version-history":[{"count":54,"href":"http:\/\/blog.light42.com\/wordpress\/index.php?rest_route=\/wp\/v2\/posts\/650\/revisions"}],"predecessor-version":[{"id":727,"href":"http:\/\/blog.light42.com\/wordpress\/index.php?rest_route=\/wp\/v2\/posts\/650\/revisions\/727"}],"wp:attachment":[{"href":"http:\/\/blog.light42.com\/wordpress\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=650"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/blog.light42.com\/wordpress\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=650"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/blog.light42.com\/wordpress\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=650"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}