PROJ, GDAL, and OGR – Oh, My!

Here are brief introductions and home-brew installation instructions for two libraries (and sets of command line tools) widely used by open-source GIS applications.

In most cases you probably won’t need to compile these yourself, as end-user programs are typically packaged with everything you need. Sometimes batteries aren’t included, though, or you might want to write a program that uses these tools directly. They’re like wheels you don’t need to reinvent to build construction vehicles for your novel GIS problems.

proj-4.6.1: PROJ.4 Cartographic Projections Library

From the documentation (OF90-284):

Program proj (release 3) is a standard Unix filter function which converts geographic longitude and latitude coordinates into cartesian coordinates, (λ, φ) → (x, y), by means of a wide variety of cartographic projection functions.

Cartographic projection is the process by which points on a sphere are mapped to points on a plane. There are many ways this can be accomplished, all of which are necessarily compromises. PROJ.4 provides a programming interface to perform such projections.

1. ./configure
2. make
3. make install

gdal-1.5.2: Geospatial Data Abstraction Library

GDAL provides a common programming interface to translate and access a variety of geospatial raster (i.e. bitmap) data formats. GDAL includes the OGR Simple Features Library, which performs a similar function for vector data. OGR uses PROJ.4.

1. ./configure
2. make
3. make install

GDAL’s configure script reports which optional features are available. Many have additional dependencies, such as the TIFF and PNG libraries built in a previous guide.

Posted on Monday, September 22nd, 2008.

American FactFinder: Geo within Geo

A great deal of census data is available through American FactFinder. The custom table interface allows you to retrieve a single table containing exactly the data you want. To obtain data for many locations within another location, use the easily-overlooked “geo within geo” selection method to avoid the tedium of retrieving and combining separate tables for a group of locations.

As an example, I’ll show you how to retrieve the year 2000 population by age and sex for each block in Broome County, NY.

First, select Decennial Census from the Data Sets menu at factfinder.census.gov:

Then select Custom Table from the Census 2000 Summary File 1 category:

Here’s the important step. Click the geo within geo selection method tab:

Choose Blocks from the Show me all menu (your table will contain data at this resolution):

By default, the table will contain all block groups within a county. Select New York and Broome County from the following menus to select which county:

Select All Blocks from the list of blocks in Broome County and click Add:

Now that you have defined the enumeration units and area of interest, click Next to choose your variables:

Select P12 Sex by Age (Total Population) from the list of tables in Summary File 1 and click Go:

Check which variables you want from table P12 – all of them. Click Add, then click Next to proceed:

Click Show Result to display the table (this may take a few moments):

Lastly, choose Download from the Print/Download menu to obtain the table in spreadsheet form:

How you use the data is up to you. Perhaps you’ll use dbfjoin to map it with an appropriate shapefile from the Census Bureau’s TIGER/Line repository. (Users of pro GIS tools don’t need dbfjoin, but I’m bringin’ it to the streets!) Alternatively, you might want to plot the age/sex composition of a few blocks with Population Analyst. Stay tuned for more bargain-bin demography and cartography.

Posted on Sunday, September 21st, 2008.

Compiling ChoroWare

ChoroWare is a suite of tools used to develop optimal data classifications for choropleth maps. I am surprised that I did not discover it until recently, because this is one of my favorite topics in cartography.

ChoroWare might prove to be particularly useful for ambitious users of free GIS applications like Quantum GIS or ArcExplorer. These programs do not offer very sophisticated classification schemes, but they do allow classes to be defined manually. In this way, I plan to try mapping classifications generated with ChoroWare.

Executable versions of ChoroWare are not available for Mac OS X (or any other platform), so you must build it from the source code. ChoroWare has two main dependencies: GTK+ and GSL. GTK+ itself has numerous prerequisites, which I have described in a separate article (again, it may be wise to use a package manager). Building GSL is relatively straightforward:

gsl-1.11: GNU Scientific Library

1. ./configure
2. make
3. make install

With all the prerequisites in place, building ChoroWare is as simple as:

1. make

I found that make install did not work, so I performed the equivalent action of copying the piping-hot new executables from their various subfolders to /usr/local/bin.

The included example files run without fault. ChoroWare on Mac OS X:

More details about the theory and implementation of ChoroWare can be found in the documentation (PDF) and associated papers (PDF).

Posted on Sunday, September 21st, 2008.

Building GTK+ the Hard Way

Update: Four days after posting this list, GTK+ OS X is available. I’m not sure how to use it build ChoroWare yet, but I’m sure it’s possible (and much easier than the process outlined below).

GTK+ is a user interface toolkit used by many open source software projects.

This is a record of how I managed to build GTK+ on my Mac, not a prescription for the best way to do it. The versions identified here may be out of date, for instance, and there may be additional dependencies I have failed to list. (Most of this work was done months ago, so these steps are based on old notes.)

This procedure yields an X11 version of GTK+. I understand that there is an effort to build a native version of GTK+ for Mac OS X, but this is not it.

I have provided a link to the web site or download area for each library, along with the commands used to build it. You should cd to the library’s directory before running each set of commands. The libraries are listed in the general order in which they were built.

This is probably trivial with a package manager like Fink.

libiconv-1.11: Libiconv (Unicode conversion)

1. ./configure
2. make
3. make check
4. make install

gettext-0.14.6: Gettext (internationalization)

1. ./congfigure
2. make
3. make check
4. make install

libtool-1.5.22: Libtool (shared libraries)

Needed by LibJPEG.

1. ./configure
2. make
3. make check
4. make install

jpeg-6b: LibJPEG

1. ./configure --enable-shared
2. (Edit line 39 of Makefile to point to path of GNU Libtool.)
3. make
4. make test
5. make install

tiff-3.8.2: LibTiff

1. ./configure --with-apple-opengl-framework
2. make
3. make check
4. make install

libpng-1.2.12: LibPNG

1. ./configure
2. make
3. make install

libxml2-2.6.26: LibXML 2

Needed by FontConfig.

1. ./configure
2. make
3. make check
4. make install

fontconfig-2.4.1: FontConfig

Needed by Cairo.

1. ./configure
2. make
3. make check
4. make install

cairo-1.2.4: Cairo (graphics renderer)

1. ./configure
2. make
3. make test
4. make install

pango-1.14.4: Pango (text renderer)

1. ./configure
2. make
3. make check
4. make install

atk-1.12.3: ATK (accessibility)

1. ./configure
2. make
3. make check
4. make install

glib-2.12.4: The GIMP Library

1. ./configure
2. make
3. make check
4. make install

Last, but not least:

gtk+-2.10.6: The GIMP Toolkit

1. ./configure
2. make
3. make check
4. make install

Anyone brave enough to try all this is encouraged to post the inevitable corrections.

Posted on Saturday, September 20th, 2008.

Rewriting the Population Analyst

In college I developed a web site called the Population Analyst. It is primarily a tool for demographic visualization with population pyramids. Here are Population Analyst representations of two communities in upstate New York:

These graphs are drawn with a common scale to facilitate accurate comparison. Clearly, a much greater percentage of Ithaca’s population is comprised of young adults. Plotting population pyramids of Binghamton and Ithaca on an absolute scale reveals that Binghamton has a larger population but similar numbers of young adults.

What do you know about these places that might explain the differences—or similarities—between their population structures? Alternatively, what can you predict about these towns or others with similar compositions?

I would like to rewrite the Population Analyst to make it more useful and easier to use. In addition to streamlining the interface and offering more control of diagram style, I would like to make these basic improvements:

• Provide direct integration with American FactFinder.
• Dramatically simplify input of data from other sources.
• Support arbitrary cohort intervals (instead of 5-year intervals from 0 to 85).
• Support arbitrary population segmentation (not just male and female).
• Offer vector output (PDF, PostScript, or SVG).

Asymmetric data sets (those without matching cohort intervals) should be supported as segments of the same graph as long as a common unit defines their cohort intervals.

The rewrite will not include a population projection component.

My dilemma is whether to implement the new version as a web app or a cross-platform desktop application. The original Population Analyst is a rudimentary web app, of course, but I am better prepared to deliver the desired data handling and interactive visualization features in a desktop environment.

Nevertheless, a program deployed as a web site has the great advantage of immediate accessibility. It would be a useful challenge to learn more contemporary techniques for doing nifty things in the browser. If I can identify good libraries and tool kits to help do some of the things I’ve planned for the new Population Analyst, it may yet remain on the web.

Posted on Thursday, September 18th, 2008.