Berkeley ImageSegbis-2
python environment is virtualenv; found a recipe to run ipython
notebook from within
source activate ENVNAME pip install ipykernel python -m ipykernel install --user --name ENVNAME --display-name "Python (whatever you want to call it)"
ETL Openstreetmap Buildings Import
* libosmium tool scripted
input is California-latest.pbf filtered by tag “building”
Code: ECN_osm_import Github -LINK-
Report: Summary Counts -CSV-
This screenshot shows OSM California ogc_fid object 1, which is Hanger One at NASA-Ames Airflied, and other OSM Buildings around Sunnyvale, CA; imported using libosmium 2.11.
—
—
Comparison of libosmium import versus ogr import in San Luis Obisbo; libosmium layer uniques are bright red.
Not all data is as expected; a LINESTRING marked “building” -LINK-
----------------------------------------------------------------------- gid | 5 id | 13863133 tags | "name"=>"Gymnasium","building"=>"yes","created_by"=>"Potlatch 0.5c" version | 2 changeset | 600831 timestamp | 2007-11-23T17:48:58Z uid | 2376 user | Bman geom | 0102000020E6100000050000009EFB613A63465DC02D1D18E53E05414033EF82D261465DC0ACF188C047054140E1AB675254465DC04411F7FD400541404BB846BA55465DC0A21EEC5E380541409EFB613A63465DC02D1D18E53E054140 -[ RECORD 6 ]-------------------------------------------------------------------------------
Some interactive manipulation of post-import stats: -LINK-
Jupyter Notebook Tech Available: -LINK-
California Air Resources Board — 2030 Scoping Plan UPDATE -LINK-
ref. page 95 Section IV Key Sectors
… The Governor’s Green Buildings Executive Order B-18-12 for State buildings and the California Green Building Standards (CALGreen) Code 130 are key state initiatives supporting emission reductions associated with buildings, and some local governments are taking action by adopting “beyond code” green building standards…
Section 3: Efforts to Reduce Greenhouse Gases
…
Ongoing and Proposed Measures – Electricity
ref. page 103
Per AB 802, adopt the forthcoming CEC regulations governing building energy
use data access, benchmarking, and public disclosure.
18 Jan 17
Automated Ingestion Setup — California AB 802 Support
========================================================
Backup
* 50GB BluRay Dual-Layer disk capacity
burn time ~120 minutes for one disk
ATAPI: PIONEER BD-RW BDR-207D, 1.21
Ridata BD-R DL Media $3/disk for 25 at Frys
* Disk-to-Disk
75GB/hour for USB connection
300GB/hour for eSATA
geodatamine_f2 compressed backup 13GB
NAIP compressed backup 2000GB
TIGER compressed backup 20GB
Hardware
Desktop: Core i7-2720QM @ 2.20GHz
Server: Core i7-960 @ 3.20GHz
Software Toolchain
Berkeley Image Seg BIS
Openstreetmap Toolchain
Openstreetmap Buildings Coverage
SF Bay Area example
18 Jan 2017
Task 2 Workflow Sketch
18 Jan 2017
11 Jan 17
Automated Ingestion Setup — California AB 802 Support
========================================================
WA 18 / Task 2
WA18 Objectives
“… develop data collection processes to compile or create existing building footprints using GIS and assign unique identification numbers to these footprints”
New Buildings Acquisition Software Stack:
Evaluation copy of Berkeley Image Seg
– Is this an acceptable component of the delivery ?
– what software environment will be delivered ?
– what documentation of the environment will be delivered ?
Example Python Worksheet from LA Buildings Import -LINK-
Example Data Browser — Berkeley Geo Library -LINK-
Example Crowd Source Project — MissingMaps -LINK-
Buildings Data Life Cycle
- acquisition
- storage
- review stages
- feedback from stakeholders
Review Stages:
Bldg Needs Review
Bldg Reviewed – OK
Bldg Reviewed – Flag Serious Errors
Bldg Reviewed – Flag needs improvement (re-review)
Coverage Assesment:
CoStar coincident
CoStar no match count
Team
Ten Weeks Left — Making the Most of the Time
* What are the blockers to getting Garlynn started ?
– NORESCO agreement / invoicing
– confidentiality
– other?
Server i7d
– postgresql / postgis network config
– geoserver config
– NAIP mapserver config
– storage ! 2x1TB | 2x3TB | 2x4TB | RAID1 boot
dbb@i7d: Filesystem Size Used Avail Use% Mounted on /dev/md0p1 218G 122G 85G 60% / /dev/sdg1 2.7T 1.2T 1.5T 45% /mnt /dev/sdf1 917G 167G 705G 20% /mnt /dev/sdd 2.7T 73M 2.6T 1% /mnt /dev/sdb1 3.6T 2.7T 765G 79% /mnt /dev/sdc1 3.6T 942G 2.5T 28% /mnt /dev/sdh1 917G 647G 224G 75% /mnt
01 Jan 17
Data Integration pre-Setup — California AB 802 Support
========================================================
* Rebuilt i7d Linux Server
– PostgreSQL 9.6
– PostGIS 2.3
– GEOS 3.6
– osm2pgsql 0.91(p)
new drives to replace four year old disks… lots of file management.. deferred librarianship
* First experiments with Berkeley ImageSEG
21 Dec 16
Data Integration pre-Setup — California AB 802 Support
========================================================
* Berkeley ImageSEG
– email license terms with James
* Similarity Measures JTS Interface
– -LINK-
* Example Work Context
– EPA POINT in Davis, California -LINK-