CitizenMediaCourse

1. Planned Outline
2. Data

I'll be teaching a session on mashups to a class of the Citizen Journalism course on Thursday, March 6, 2007. The course is taught by Dan Gillmor and Bill Gannon.

Planned Outline

• outline for talk

• Introduction

• Who am I? What I do? Where I come from?

• Class Introduction: Who are you? What do I want to learn?

• The problem: what exactly is it?

• Let's talk about the problem.

• What I did in what I perceived to have been the problem: making sense of Katrina as a lay-person (with geek tendencies) -- brief presentation

• A prompt: the SF Chronicle article on the census

• Claremont-Elmwood / Homogeneity in Berkeley? Well, yeah

• A model: Robert Darnton

• The New York Review of Books: The New Age of the Book

• what I did -- do we want to elaborate on this work?

• I looked at the students' maps

• Steven Leckart

• //re:digg\\

• http://platial.com/stevenleckart/map/15097?title=Go_East,_Young_Man:____Obama's_Education

• Eric Zassenhaus

• http://platial.com/zassenhaus/places

• Nick Guroff

• Nick Guroff

• http://www.platial.com/nickguroff/map/15131?title=Island_on_the_Edge

• car chase

• http://www.platial.com/baaguilar/map/15020?title=Car_Chase_in_Oakland

• Megan Finn

• http://www.platial.com/map/15098

• Omar

• http://platial.com/omar

• I watched the Katrina videos on PBS

• NOVA | Storm That Drowned a City | PBS

• FRONTLINE: the storm | PBS

• data list to focus on -- that's what I've been following

• data of interest

• GIS data

• for shape->KML: MapGis + Shape2Earth

• Damage assessment is helpful: http://bbs.keyhole.com/katrina/Katrina-damage-outages-14Sep-1000.kmz

• Google Earth - Hurricane Katrina Imagery

• EO News: NASA Post-Hurricane Katrina Images Available On Google Earth - February 3, 2006

• Brian Flood : Google Earth Time Sample - Part 2 -- timelines

• points to http://bbs.keyhole.com/ubb/download.php?Number=607833

• NASA - NASA Post-Hurricane Katrina Images Available On Google Earth

• http://inst.wff.nasa.gov/eaarl/

• http://inst.wff.nasa.gov/eaarl/index.php?module=pagemaster&PAGE_user_op=view_page&PAGE_id=5

• visualization of quantitative data

• the data set I'm working on

• http://quickfacts.census.gov/qfd/states/22000.html which points to http://quickfacts.census.gov/cgi-bin/qfd/extract-xls?22000

• swivel

• Tasty Data Goodies - Swivel -- more interesting visualization techniques -- might be interesting to show the J-School students.

• Swivel-Junkie: Swivel Hacks

• many eyes

• http://services.alphaworks.ibm.com/manyeyes/data/Sh3S9FsOtha6B3kdjfh9F2-

• platial API

• Google Earth: data sets and shapefile -> KML

• Yahoo Pipes -- RSS

• Flickr

• before Aug 23, 2005

• http://flickr.com/map/?&fLat=29.954067&fLon=-90.004806&zl=7&max_taken_date=2005-07-24%2000:00:00

• My reactions

• I've been trying to make sense of what happened and is happening with Katrina

• overwhelming amounts of data

• I'd advocate a layering model

• More details -- lots of technical details

• dealing with shapefiles and seeing whether I can put them on an online map/KML, etc.

• I have two data sets that I'm immediately interested in

• the Berkeley campus

• Katrina-related data

• FEMA: Hurricane Katrina Flood Recovery Maps (Louisiana)

• try Orleans parish

• http://www.fema.gov/multimedia/hazard/flood/recoverydata/katrina_la_or_sil.zip

• Of course, there a worldful of data in ESRI shapefile -- or so I think.

• some basic references on ESRI shape files

• Shapefile - Wikipedia, the free encyclopedia

• A number of issues

• I need lat/lon. I need to be able to draw lines to simulate shapes

• There's the basic being able to read the data at all -- plain file format issue

• Then there's the projection issue

• Coordinate Systems Overview was very helpful for getting a basic idea of what the state plane projection is.

• Projections List points to the Lambert Conic Canonical....

• Acknowledgements

• kueda@ISchool.Berkeley.EDU Ken-ichi Ueda was very helpful

• Data sources

• campus buildings

• http://www.gisc.berkeley.edu/data/ucberkeley.html#Bldgs

• 3-d work on campus

• http://www-video.eecs.berkeley.edu/~frueh/3d/index.html

• sketchup of campus

• UC Berkeley by Lucid - Google 3D Warehouse

• Berkeley, CA, USA by Google - Google 3D Warehouse

• What I've done

• I downloaded a viewer and could see the shapes but didn't see any way to get data out.

• I tried Shape Viewer -- and this shows me the shapes but not the absolute location.

• I thought that an online system using Virtual Earth might do the trick but I couldn't get it working.

• the system is at http://www.blogthevote.net/veshapefilec.htm

• I uploaded the campus building shape file to http://raymondyee.dreamhosters.com/misc/buildings.shp

• Python wrapper for the shapelib C library

• shapelib

• Shapefile C Library V1.2

• Python wrapper

• Index of ftp://intevation.de/users/bh/

• Hobutools 1.0 — Hobu, Inc

• code I've tried

• import shapelib

• s = shapelib.shapelib.open(r'D:\Document\Docs\2007\02\shapefile\buildings.shp')

• s.info()

• How to do projections?

• if I want to stick with Python, py-Projection 0.1 — Hobu, Inc -- except the binary uses Python 2.3 -- I could muck with the source, etc.

• I've thought about downloading Mapserver -- and might still do that -- for various platforms -- but that seems overkill for just working with shape files!

• how hard is it to install MapServer on...

• Windows

• Mac

• Dreamhost

• EC2

• Ubuntu

• Ken-ichi mentioned the Python Cartography Library -- I might install that if I need to do more work -- especially do the centroid calculation

• HowTo - PCL - Trac

• Python

• There are commercial offerings -- but not a free pipeline yet

• Google Earth Blog: Shape2Earth - GIS Data to Google Earth

• Proximity Shp2Xml Conversion

• The authors of the Apress book led to me to something that seems to work if I understand the input parameters needed.

• Importing Shapefiles at Beginning Google Maps Applications

• shp2text

• shp2text - export and convert ESRI shapefile to text/xml/spreadsheet for Excel/SQL/Google Map

• I also don't know what it does about projections. Maybe I have to do that separately

• The C code: [http://www.obviously.com/gis/shp2text/shp2text.c ]

• The best solution so far -- ogr2ogr (from the FWTools: Open Source GIS/RS Binary Kit

• I went this way because of Shapefile to Lat/Lon List - Google Mapki

• D:\App\FWTools1.2.2\bin\ogr2ogr.exe -s_srs lcc -t_srs WGS84 -f "GML" out.gml buildings.shp

• now how did the projection happen?

• It came from using mapwindow gis which created a buildings.prj

• MapWindow GIS - Downloads

• PROJCSLambert_Conformal_Conic",GEOGCS["GCS_North_American_1983",DATUM["D_North_American_1983",SPHEROID["GRS_1980",6378137,298.257222101]],PRIMEM["Greenwich",0],UNIT["Degree",0.017453292519943295]],PROJECTION["Lambert_Conformal_Conic,PARAMETER["standard_parallel_1",37.06666666666667],PARAMETER["standard_parallel_2",38.43333333333333],PARAMETER["latitude_of_origin",36.5],PARAMETER["central_meridian",-120.5],PARAMETER["false_easting",6561666.666666667],PARAMETER["false_northing",1640416.666666667],UNIT["Foot_US",0.30480060960121924]]

• where do I get a list of ogr2ogr projections?

• part of the fwtools suite http://www.remotesensing.org/proj/

• I can get a list of projections (Manpage of PROJ)

• D:\App\FWTools1.2.2\bin\proj -l

• Lambert Conformal Conic: is lcc

• that didn't work....

• work on Katrina data

• FEMA: Hurricane Katrina Flood Recovery Maps (Louisiana)

• D:\Document\Docs\2007\03\Katrina

• I should first look at what others have done: Google Earth Blog: Katrina Time Animation in Google Earth

• Future work

• figure out the complexities of .prj

• very useful to have it shipped around -- it's what allowed ogr2ogr to do its work.

• There is a human readable .met file but .prj would be nice.

• I want to know more about it -- here's what I've learned so far

• [FWTools Dumping to wkt for a .prj file - meter vs. metre] -- to get a feel for the subtleties

• prj has something to do with wkt (PRJ - ArcView ESRI Coordinate System Definition (ESRI), PRJ Converters: The files use the GIS industry standard well-known text (WTK) format to describe the coordinate systems.)

• OGR: OGR Projections Tutorial

• looking for GML to KML converters

• attempts

• [http://members.home.nl/cybarber/geomatters/GML2KML.xslt ]

• run the point encoder to get into something optimized for GE

• [http://www.thrall.net/~mking/maps/points.py ] via Google Maps - GoogleMaps via Drawing arbitrary GIS data on Google Maps | Holovaty.com

• Adrian Holovaty on how he got zip boundaries on a Google map

• ProQuest Information and Learning - 0596101619 - Google Maps Hacks: Similarly, Chicagocrime.org lets you navigate crimes by city ward, and there's a "Find your ward" feature on the ward page: http://www.chicagocrime.org/wards/. For ward and ZIP Code pages, chicagocrime.org uses the Google Maps polyline-drawing API to draw the border for the given ward or ZIP Code on the map. I did this by obtaining the ward and ZIP Code boundaries in ESRI Shapefile format from the City of Chicago's GIS department at http://www.cityofchicago.org/gis/. I loaded the shapefiles into a PostgreSQL database and converted the data into longitude-latitude coordinates using the conversion functions in PostGIS. Finally, it was just a matter of feeding the points into the Google Maps polyline-drawing API, and voila: we have ward and ZIP Code borders.

• I've been thinking that I need to parse the .dbf file -- maybe not. If I do, there is a Python recipe that might help: http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/362715

• ASPN : Python Cookbook : DBF reader and writer

• Some more references

• http://www.google.com/url?sa=t&ct=res&cd=1&url=http%3A%2F%2Fdownload.oracle.com%2Fotndocs%2Fproducts%2Fspatial%2Fpdf%2Fosuc06_usarmy_coe_google.pdf&ei=--jmRbBzjuiBA-6ZqdgJ&usg=__BBJt9ve-ncjIQQbi7PLprYmEn8k=&sig2=L3HEtg6oZfi-7biH5I6XvA

Data

data of interest

I found that looking at a lot of raw data is hard; I found that the BBC overview of Hurricane Katrina very helpful:

• BBC NEWS | INDEPTH | HURRICANE KATRINA

• New Orleans map

FEMA: About Hurricane Katrina

The class has been using Platial.com. I will use the Platial API (javascript version).

Background links

• Citizen journalism - Wikipedia, the free encyclopedia

Some of the platial maps generated by the folks in the course:

• Zassenhaus's Places on Platial.com

Links related to the Katrina project

• Items that the class wants to be documented: Citmedia: List of features and Citmedia: Reduced list

Things that we might want to try:

• try some visualization techniques from Many Eyes and Many Eyes

MoinMoin Powered

Immutable page (last edited 2007-03-09 00:07:40 by RaymondYee)

Or try one of these actions: AttachFile, DeletePage, LikePages, LocalSiteMap, SpellCheck