dbfjoin
This program creates a DBF output table by joining records with matching keys from a data table to a main table. Every record from the main table is copied to the output table. If a record with the same key exists in the data table, that data record is appended to the corresponding output record. If the data table does not contain a match, null values are appended instead.
If the format of the main table is “MAINKEY, VAL” and the format of the data table is “DATAKEY, DATA”, the format of the output table will be “MAINKEY, VAL, DATA”.
If multiple data records have the same key, only the first record will be used. If multiple main records have the same key, each will appear in the output followed by the same values from the data table, if any.
Only string, integer, and double (floating-point number) field types are supported.
Example
To use dbfjoin, state the names of the key fields from the main and data tables, followed by the paths to the respective input tables and lastly the path to the output table.
dbfjoin MAINKEY DATAKEY main.dbf data.dbf output.dbf
Acknowledgements
- Shapelib provides an attribute table API to read and write DBF files.
- Christopher Clark’s C Hash Table code helps locate data table records.
- Dan Bernstein’s djb2 hash function powers the hash table.
Download
- Mac OS X universal binary 18.2 KB
- dbfjoin-1.0 source code 11.6 KB
To compile dbfjoin, you will need to edit the makefile to locate your copy of Shapelib.
Posted on Saturday, September 20th, 2008.
Rewriting the Population Analyst
In college I developed a web site called the Population Analyst. It is primarily a tool for demographic visualization with population pyramids. Here are Population Analyst representations of two communities in upstate New York:
These graphs are drawn with a common scale to facilitate accurate comparison. Clearly, a much greater percentage of Ithaca’s population is comprised of young adults. Plotting population pyramids of Binghamton and Ithaca on an absolute scale reveals that Binghamton has a larger population but similar numbers of young adults.
What do you know about these places that might explain the differences—or similarities—between their population structures? Alternatively, what can you predict about these towns or others with similar compositions?
I would like to rewrite the Population Analyst to make it more useful and easier to use. In addition to streamlining the interface and offering more control of diagram style, I would like to make these basic improvements:
- Provide direct integration with American FactFinder.
- Dramatically simplify input of data from other sources.
- Support arbitrary cohort intervals (instead of 5-year intervals from 0 to 85).
- Support arbitrary population segmentation (not just male and female).
- Offer vector output (PDF, PostScript, or SVG).
Asymmetric data sets (those without matching cohort intervals) should be supported as segments of the same graph as long as a common unit defines their cohort intervals.
The rewrite will not include a population projection component.
My dilemma is whether to implement the new version as a web app or a cross-platform desktop application. The original Population Analyst is a rudimentary web app, of course, but I am better prepared to deliver the desired data handling and interactive visualization features in a desktop environment.
Nevertheless, a program deployed as a web site has the great advantage of immediate accessibility. It would be a useful challenge to learn more contemporary techniques for doing nifty things in the browser. If I can identify good libraries and tool kits to help do some of the things I’ve planned for the new Population Analyst, it may yet remain on the web.
Posted on Thursday, September 18th, 2008.
Recent Things Dock Stacks
Hot on the heels of an alternative to opening things with the Dock comes this tip (via Mac OS X Hints) about a different way to open things with the Dock. Fire up Terminal and enter these lines to restart the Dock with a new “recent things” stack:
defaults write com.apple.dock persistent-others -array-add '{ "tile-data" = { "list-type" = 1; }; "tile-type" = "recents-tile"; }' killall Dock Right-click the stack to choose what sort of items to display:
Visit the Appearance system preferences to choose how many items are displayed for each of the “Recent” options. The “Favorite Volumes” option displays connected drives, and the “Favorite Items” option shows the “Places” listed in the Finder sidebar (which can include files and applications as well as folders, by the way).
Here’s an example of what you might see in a Recent Applications stack:
You can run the command more than once to create a couple instances of the stack – useful if you’d like to use more than one of the display options.
Posted on Saturday, September 13th, 2008.
FastScripts as Apple Menu
Back in the day, the System Folder contained an Apple Menu Items folder, the contents of which appeared in the Apple menu. Choosing an item from the menu opened the item from the folder. It was easy to customize the Apple menu by adding folders and aliases to the Apple Menu Items folder.
Here’s what it looked like by default in Mac OS 8.5:
In Mac OS X, of course, the Apple menu is no longer customizable. It still contains some useful items, but it’s not quite the familiar place it once was. The Dock has taken its place as a quick way to access favorite files and folders.
So, if you want to access a bunch of applications from a tidy little menu, and you don’t find it useful to throw your whole Applications folder in the Dock, you turn to the world of third party launchers. Here’s an unlikely solution.
FastScripts is a program I recommend for running scripts with keyboard shortcuts. It’s a power user thing. However, a little-advertised fact about FastScripts is that it will open anything in your Scripts folder, not just scripts.
Choose Open Scripts Folder from the FastScripts menu, and put whatever you’d like—including subfolders—in the folder that appears. Now you can launch those files from the FastScripts menu:
The script items in the menu reside in a subfolder of the Scripts/Applications folder. The subfolders are named after specific applications, and appear in the menu only when that application is frontmost.
Posted on Saturday, September 13th, 2008.
Drawn Wrong Handed
I’ve started a Flickr group called Drawn Wrong Handed. Here are some of my initial contributions:
I’ve spent this evening searching Flickr for people to invite, and I’ve already got a new member. Ever done any wrong handed mark making? Upload the evidence!
Posted on Wednesday, July 9th, 2008.
Bricksmith Custom Categories
Here is a hack to add new categories to the Bricksmith part browser. The part catalog is normally organized into categories based on the type of each part (such as brick, plate, or tile). However, if you’re building a model based on a real set, it’s just as useful to present the parts from that set in one category.
Adding a category from a Peeron inventory
To create a category consisting of the parts from set 4891, enter 4891 and click Peeron Inventory:
The script will retrieve the corresponding list of parts and populate a new category containing them:
Adding a category from a file
To create a category based on an arbitrary group of parts, save a list of part file names, like this:
3005.dat
3004.dat
3003.dat
3001.dat
Then enter a name for your category and click From File. You’ll be prompted to select the list you just created.
The script will add the listed parts to the named category:
Limitations
The script is awfully slow. AppleScript’s property list commands are not well suited for making many changes.
Parts listed in the file or inventory which are not already present in your part library will not be included in the category.
New categories are not visible until you restart Bricksmith. (I recommend quitting Bricksmith before running the script anyway.) Custom categories are lost when you reload the part list from Bricksmith’s Parts preferences.
No feedback is provided while the script is running.
Errors are not handled particularly well.
Mac OS X 10.5 is required.
It’s really slow.
Download
Bricksmith Catalog Custom Categories 26.4 KB
Posted on Saturday, June 21st, 2008.
Mis Bicicletas
I bought an old road bike for $9 last week. I’ve enjoyed tuning it up and thinking of new missions for it and my other bike.
The basket on the red bike is neat because it can be unclipped and carried around like a regular shopping basket. Quite practical! I’ll upload more pictures to my bikes set as the fleet evolves. I expect a rack and further repairs may be in store for the blue bike.
Posted on Thursday, June 19th, 2008.
A New Study Suggests That…
My friend Dan has authored a study which has received considerable press coverage this week. It is about the circumstantial advantages of ADHD. Articles about his paper have appeared at Scientific American, Slate, The Economist, New Scientist, The Daily Telegraph, and elsewhere.
It’s all in a day’s work for Dan, but it’s nice to see such widespread recognition for his research. He’s given me many insights into the evolutionary elegance of the world around us, and now the world has a chance to consider his insight.
Posted on Friday, June 13th, 2008.
Unnatural Selection
As you may know, the runner-up at this year’s Kentucky Derby, Eight Belles, was euthanized shortly after the race due to ankle injuries she sustained just after crossing the finish line.
Interestingly, Eight Belles and all of her Kentucky Derby competitors were in part descended from Native Dancer, an accomplished racehorse whose career was eventually curtailed by a “recurring foot injury” and whose bloodline has a reputation for “producing fragile horses”.
Selective breeding leads to the inheritance of desired traits like being fast and light. When taken to extremes (as with the inbred pure-breeding of some Thoroughbreds, arguably), it’s not too surprising that narrow emphasis on a few traits breeds animals that may otherwise be unfit to withstand aspects of the stress they’re expected to endure.
But then again, Kentucky is home to the Creation Museum. Maybe if they ask nicely someone upstairs will just intelligently design a stronger racehorse.
Posted on Thursday, June 5th, 2008.