Tag Archive: “code”

An arcencode example

Arcencode is my Tcl implementation of the compression algorithm used by both MapQuest and the Google Maps API to encode a list of coordinates as a printable string. It can be used, for example, to obtain a relatively compact representation of a route that can be passed as a single parameter value to a web map.

path-700

Here is a GPX file exported from a map created with MapMyRun. It contains a track representing a route through a nearby park, pictured above. The track consists of 790 points, listed here in plain text form. If the contents of that plain text coordinate list are assigned to a variable named coordinates, the encoded form can be obtained with:

set encoded [arcencode $coordinates]

With line breaks added, the contents of encoded are:

q}g`GrlfnMPDND\DJ?L?T?P?N?N?N?P?VAPANANAPANAPANATGNELGXENANANCNANA^BNDPDNBNDNDND
NDPDNDNBNDNDPDNDNDNDVHNFPFNFNFNFPFNFNFNFPFNFNFNFPFNDNFNFPFNFNFNFVFNBPBNBNBNBVFND
PDNDPDNFPDNDPDNDPDNDPDNDPDNDPDNFPDNDPDNFPFNDPFNFPFNDPFNFPFNDZLHFJDNFPHNFNHPFNH\F
LAJ?V?F?PFNFNDNFNHZNLHNHLHLHLHNJLHLHT?N?PAN?PAPAVDPFNDNDPDNFNDPDNDNFPDNDPDNFNDPD
ND\JLDNBNDLDNDNBTFNDLBLDVDN@PBN@PBN@PBNBP@XHNHLFNHNFNFLHNFNFPVBNPVJFHHRLHFPANCPC
PAPAR?L@R@P@P?N@P@P@P@P@VFLDLFRHNFNFPFNFNFNFNFNFNFTLHFLPLNJLRTJHPJJDPBTAJCPGPGNE
PGNGPGNGNEPGNGPGNEPGNGNGPGNEPGZIJAJAVJLDLDRAPENCPCNCPENCPCNEPCXGNCLCNCNCPDPHTTJJ
HLHJLNJLLLRLLBPFRNJLJJJLJLTHL@T@L?N@THLHLHLHNLNJLLLJNJ\LL@N@N@L?N@ZALENCLCNENCRI
NGNELGZKNGNGNGNGNGNGNENGNGNGNGNGNGNGTUFQHOHOFQN[HOFMPYHKHINQJKLLHRHPHPHRJPHPHRHP
J^BPDRDPDVJVHNHPPNNJLHLHLJRFPFNDNDPFNDNDPFXHLFLDLDRFNFNDNDRBNBN@RENCPEPCNEXCH@P@
VJJHHHFBEWCSEUGSIUO[KOIMIMUWKMMKMMKMMMMKKMMMMMSMOGMIOIOGOIYQMIMIMGMIKIUMMKOIOKMI
OIOKMIOIOKOISKOEMGOEUCOCQAOAQCQAYCMAMCSEOEOEQCOEWCM@WBMBOBODOBUJMFMHMHMLOJMLWPMF
MFMFWFMBOBODYBO@O?O@O@O@SCOAOCQEOEQEQEOGQEOGWGMAU?M@O@QDODQBWFQBOBO@QBOBOBQBOBSB
O@QB[DOBOBODOBOBOBOBUDQDOBOBQDOBOBW?KEQGMGQ@QBQFOD_@NODOFODOFOFODOFODOFODOFOFODO
FODYJK@Q?SGKCOMKKQQKMKOKMOKUMOGOGOGOGOGOGOGQGOGWKIEOGWEQ?OAQAOAQ?OAQAOAQ?W?SGIIK
GKOKQMOOMMISCUHGFQGOGOIOGOIOGOIQIOGUGQAOCQAOCQAOCQAOC[GOEOEQEOCOEOEOEOEOEOESEOGQ
EOEQGOEQEOEQGUGOEQGOEOEOEQGOEU?Q?O@Q?O@O@YIOIMKMIOIMIOIMIMKQGQGOGOGOGYEM?M?Q@[GM
GOGMGMGMGUIMGOGWMKEMCQIQEOGOGQEOGOGOGQE[KQEOEOEQEOEOEQEOEOEQEOEOEQEOEOEQEOGOEQEO
EWEOCQCOEOCOC_@KOGOGOGOEOGOGQGOGOGOGOGOGOGOEOGOGOGOGOGOGOGWGOEQEOEQEOEQEOEQEOEQE
OEQEOEQEOE[AK@M@Q@Q@OBQ@QFOFOFWBO@Q@O@Q@O@Q@Q@U?O?Q?Q?O?Q?Q?O?]CKEMCME

Succinct, considering it encodes detailed geometry. A coordinate list of 790 corresponding points can be recovered from this block of text with the complementary arcdecode procedure:

set decoded [arcdecode $encoded]

The decoded coordinate list can be examined here.

An important proviso: this encoding scheme is lossy. Specifically, coordinate values are rounded to five decimal places. Compare the input coordinates to the decoded output for an example. Any precision can optionally be specified, but greater precision compromises the amount of compression. (Note that the Google Maps API is only compatible with five-digit precision).

How does this algorithm work? Google offers a technical step-by-step explanation. Essentially, compression is achieved by storing only the difference between each coordinate value and the previous value in the sequence; this requires fewer digits than storing each value in full, especially since precision is limited to a fixed number of digits. The values are packed into a printable Base64 representation in a way that eliminates the need for delimiters between coordinate values: values are bit-shifted left before output, and the ones place is used to mark whether the subsequent character represents a new coordinate or a continuation of the same value.

I updated my WheresThatSat site to use JavaScript and (on the Twitter bot side) Ruby implementations of this algorithm. It’s an appropriate application – previously, satellite ground tracks were encoded as a repetitive sequence of separate point parameters. In addition to yielding somewhat shorter URLs, this compression method ensures the correct point sequence is preserved even if the URL parameters are re-ordered.

Posted on Sunday, March 3rd, 2013.

Pinboard bookmarks recipe for Calibre

Here is a script for Calibre which which retrieves your unread bookmarks from Pinboard and compiles them into an ebook. If you use Pinboard to save long articles to read later, and if you like to read long articles with an ereader instead of with your computer’s web browser, this may be the recipe for you. The script is called Pinboard.recipe and you can get it here: github.com/anoved/Pinboard-Recipe (see the Setup section to get started).

This is similar to the Safari Reading List recipe I wrote earlier this year.

Posted on Monday, December 10th, 2012.

Making Ground Track Generator

I wrote a little program to make shapefiles (GIS map layers) of satellite ground tracks. Here’s the story of its development, recounted from my comments on Twitter (the internet’s water cooler).

Posted on Saturday, March 31st, 2012.

Save Selection and Compare Selected Files scripts for BBEdit

Here are a pair of handy scripts for BBEdit (or TextWrangler). Paste them in to Applescript Editor, save them in your BBEdit Scripts folder, and assign shortcut keys in BBEdit > Preferences > Menus & Shortcuts.

Save Selection as New File

Select some text and invoke this script to create and save a new file containing the selection. (Add close doc to the end of the block if you don’t need to edit the file.)

tell application "BBEdit"
	set txt to contents of selection
	set doc to make new document with properties {contents:txt}
	save doc
end tell

This is pretty similar to choosing File > New > Text Document (with selection) and then choosing Save, but it combines the two into one step, which is helpful if you need to do this repeatedly.

Compare Selected Project Files

Use this script to compare two files selected in the files list of a disk browser or project window.

tell application "BBEdit"
	set selectedProjectItems to selected items of project window 1
	if (count of selectedProjectItems) is not 2 then return
	set comparisonResults to compare first item of selectedProjectItems against second item of selectedProjectItems
	if not differences found of comparisonResults then
		display alert "No differences found:" message reason for no differences of comparisonResults
	end if
end tell

You can accomplish the same thing with Search > Find Differences, but if you’ve already selected some files this is quicker.

Posted on Thursday, March 29th, 2012.

Send Reading List to Instapaper

I made a tool called Send Reading List to Instapaper. It does just what it says: it sends the unread articles from your Safari Reading List to Instapaper, a multi-platform service that provides a superset of Reading List’s features. Download and more details at GitHub.

Screenshot

The script is based on readinglistlib, an offspring of my previous experiments with Reading List. It also uses instapaperlib by Daniel Schauenberg.

Articles are not automatically removed from your Reading List when sent to Instapaper, which may limit its utility as a way to “sync” the two. However, it may be useful if you’ve accumulated a large number of Reading List bookmarks which you’d rather read with Instapaper.

Posted on Thursday, March 8th, 2012.

Reading List integration for Vienna

Vienna is a desktop newsreader for OS X. Here is an AppleScript to add the article currently selected in Vienna to Safari’s Reading List:

Paste it in Applescript Editor, save it in ~/Library/Scripts/Applications/Vienna, and run it with your script runner to read some articles.

Note: As of the Vienna 3 beta, something like this is now built in to the app: see the “Add to Safari reading list” item in the Article menu.

Posted on Thursday, February 16th, 2012.

Reading List Reader

I’ve written a standalone script called readinglistreader.py derived from my ebook recipe for Safari Reading List. This script simply lists the contents of your Safari Reading List. It has many options to allow detailed control of the output list. By default, output is formatted as a CSV table, but it can also be formatted as a bookmarks file suitable for importing into other programs or services. In fact, since there seems to be no other easy way to grab the contents of Reading List (besides Safari’s interface), this script is intended is to help export your Reading List bookmarks to other services, presumably as part of some other script you write.

Get it at Github.

Posted on Monday, February 13th, 2012.

Ebook Recipe for Safari Reading List

Reading List is a Safari browser feature that helps you bookmark articles you want to read later. Calibre is an ebook utility program. I wrote a script for Calibre that generates an ebook of the articles in your Reading List, so you can read them at your leisure on the device of your choice.

Click here for a more detailed introduction and instructions…

Posted on Thursday, February 9th, 2012.

Strange Horizons ebook issue generator

Attentive readers may notice I’ve been on a bit of an ebook kick recently. Here’s a tool I made to generate ebook issues of Strange Horizons, an SF magazine:

Strange Horizons is a weekly magazine of and about speculative fiction. Calibre is a free and open source ebook library management application.

Calibre has an extensible system “for downloading news from the Internet and converting it into an ebook.” The scripts Calibre uses to retrieve and format news are known as recipes. Recipes can be configured as simple RSS readers or as custom Python scripts using Calibre’s recipe API.

Strange Horizons is published online as a web site. This Calibre recipe retrieves the current issue of Strange Horizons and outputs an ebook suitable for reading on a Kindle or other ereader device.

The script and other details, including installation and usage notes, are available on GitHub. It is included with Calibre since Calibre version 0.8.38.

Update, March 4, 2012: This script has been acknowledged on the Strange Horizons blog.

Posted on Tuesday, January 31st, 2012.

McReadability updated

Readability was a bookmarklet developed by Arc90 Lab that would conceal distractions and reformat article text in a consistent manner. I created a modified stylesheet (“McReadability”) that presented the article text in multiple columns, like a newspaper. Readability has since been relaunched and rebranded as a paid service. It still performs the same basic function – it makes web pages nicer to read – but now also offers financial support for article authors and publishers. This is a commendable alternative to the annoying advertising-driven segmentation of articles into multiple pages that is so common on the web. (Web advertising is often paid per page view – thus there is incentive to spread “content” across as many pages as possible. That’s why you see those “Page 1 of 4” links at the bottom of every online newspaper article. It’s not like web pages run out of paper at the bottom.)

Anyway, my multi-column version of Readability relied on some regular Readability code hosted at Arc90. This dependency broke shortly after the introduction of the revamped service. Fortunately, the original Readability code is open source, so I have updated the McReadability bookmarklet maker to be self-contained. The McReadability bookmarklets now use a copy of the article-cleansing code hosted here at anoved.net. If your McReadability bookmarklet recently stopped working, I encourage you to update it.

Besides this fix, McReadability remains unchanged – although now that is independent of Readability proper, I may customize it more in the future.

Posted on Sunday, March 6th, 2011.