Dive Into Python

Mark Pilgrim's Dive Into Python is now available in book form . Reading it in print reminded me how good it was, and it has grown a few chapters since I first read it online.

If an aversion to reading online has kept you back, go buy it now. It is a very impressive piece of work - it manages to explain the Python language, and, more importantly the idioms and customs of Python programming, through a series of well-chosen and interesting examples. The chapter on test-driven development shows why this makes sense, how Python supports it, and how it leads to better code.

Mark writes as he does - look at his excellent Universal Feed Parser, with its thousands of test cases for a concrete example of the power of test-driven development.

Python is my favorite programming language. Mark does a great job of explaining why. Try it, you'll like it.

Call Off the Search

Explaining Technorati and blogging to people has been tricky. Once I explain we keep track of what people are saying and linking to in weblogs, their first reaction is to say 'Oh you're a search engine' (silent subtext: Google will eat you for breakfast).
This is where it gets tricky, because search is a task, not a goal.
Jeff Bezos and John Battelle help explain this better:

[Bezos uses] "discovery" as an umbrella term which incorporates search. I think in the end when I use the word "search" I really mean "discovery" as Jeff uses it. What's discovery? Well, much more in the book, but in the end, it's search plus what happens when the network finds things for *you* - based on what it knows of you, your actions, and your inferred intent.

Last week I watched Steve Jobs explain Technorati's advantage over Google - he was talking about Safari's RSS search, but Technorati searches millions of blogs for you within minutes of them updating, not just the RSS feeds you have already subscribed to.

But again, searching for keywords is missing the point.

The great thing about weblogs is when you discover someone. Someone who makes sense to you, or someone who surprises you with a viewpoint you hadn't thought of. Once you have found them you can subscribe to their feeds and see how they can keep inspiring or surprising you.
You can even start a blog, link to them, and join the conversation,
The continuity of viewpoint within a blog is key - you can see more about them than just the one comment, and you can keep discovering and growing with them. Conversely, being aware that what you are writing is 'on your permanent record' means that you write more carefully for a blog than for an email.

Blogcritics sent me a CD to review - Call Off the Search by Katie Melua. Rosie loved it, but the title song sums up what I'm getting at here: "Now that I've found you I'll call off the search."

Blogging is about what you discover, not about what you search for.
How you can follow the conversations and make new discoveries is what I'm working on. [updated 2014 - original Steve Jobs link was broken 5 times over by Apple: it linked to homepage.mac.com (Which they killed) hosting a Quicktime reference movie (which they killed) to a streaming Quicktime movie (which they killed) of a Steve Jobs keynote (which is now offline) explaining Safari RSS search (Which they killed).

Millions counted

Yesterday, Technorati hit 3 million blogs, and Apple hit 97 million iTunes songs.
So, if every blogger buys one song, the 100 million prize can be won.

No prize here, but the 3 millionth Technorati blog was Mi eterna ciclotimia

Citation and deep linking

Dorothea points out some problems around piecemeal citation:

[...] there’s no automated way to add callouts to one individual paragraph without adding callouts to all of them.

A more subtle explication of the problem: I could, if I chose, add individual id attributes to paragraphs on CavLec I thought especially worthy of notice. But who’s to say that my idea of noteworthy paragraphs meshes with any other blogger’s? Nobody, that’s who. (Not least because it’s an open question whether any paragraphs on CavLec are noteworthy.) The only way to ensure that anyone who wants to link to noteworthy paragraphs can do so is to assume that all paragraphs are potentially noteworthy.

Worse, even if I do add id attributes, there’s no way for a would-be linker to get at them for linking purposes except by inspecting my HTML code. Green hash marks may be crufty, but they address a genuine issue, one we might call “identifier invisibility.”

The way around this is to do what I just did - copy in the piece you are citing and link to the whole. It's a little cumbersome, but it has the benefit of resilience (the original might vanish or be re-edited). A way to take this technique further is to use QuickTopic Document Review, as I did for AKMA's speech for example. This both adds the paragraph citation links, enables inline comments, and archives a copy of the cited source elsewhere, protecting against it changing or vanishing and thus invalidating the citation link.

This is the same issue as discussed by Jon Udell last month for MP3's.
If you want to cite an MP3 in a stable way, you can do it by copying a fragment and saving it locally, and linking back to the original source file. We don't try to dynamically insert chunks of text from other people's servers into the middle of our prose; why do it for media?
What is missing here is the rich media equivalent of QuickTopic Document Review, which mirrors media and adds annotation. Building something to enable this would be a fine project for the Internet Archive.

Dashboard is nice, but how DO I make Safari sidebars?

Dave Hyatt said something interesting about the new dashboard widgets:

Just to prove a point that there are many ways to think about this new feature, here's another take on what Dashboard is. From a browser geek's perspective, the Dashboard is a collection of HTML sidebar panels liberated from the browser window and placed anywhere on your screen. The "Web pages as widgets" concept is really just a logical extension of the Web sidebar panel metaphor fused with Exposé.[...]
However the sidebar metaphor suffers from usability problems, such as the inability to scale up to many panels as well as being constrained by the browser's window width. It's also hard to view multiple panels at once. The panels are also tied to a particular application (the browser) despite frequently having no connection to the application itself.

Which makes sense for sidebars that are decoupled from the current page context. However, what is missing here (and as far as I could tell at WWDC last week, is not available at all in Safari) is a way to put a sidebar in the browser that is informed by the page context therein and can interact with it. As Hyatt says, this is possible in IE on Windows, and in Mozilla derivatives. Can we do this within Safari, using Dashboard or AppleScript, or anything other then hacking the nibs inside Safari itself?

DropDV: Convert MPEG to DV

More than 10 years ago, Apple added MPEG playback to QuickTime. In order to work around QuickTime not having support for bi-directional difference frames, and some issues with MPEG's format being a bitstream rather than a byte stream, this was done my making it a special media type rather than showing up as separate video and audio tracks. As a side effect of this, exporting audio from MPEG movies didn't work. This was one of those features that was annoying to few enough people and just enough work that it has been continually deferred from release to release (hey, I spent 6 years there and never got to it either). There have been MPEG stream manipulation tools from Sparkle onwards, but they have always needed a deep understanding of the format, a degree in computer science and lots of patience to use.
Now that there are video cameras and digital cameras that record MPEG directly, this is suddenly a lot more annoying, but fortunately there is an answer:
DropDV: Convert MPEG to DV: DropDV is a Mac OS X droplet which converts MPEG video into DV video streams. This allows the video to be edited in iMovie, Premiere, Final Cut, or any other DV video editing system.
Features
Handles both video and audio
Uses high quality bicubic scaling for the best video image
Decodes in YUV color space, other tools use RGB.
Supports both NTSC and PAL output
A Simple, drag-and-drop interface

At least pick a socialist who can write

A clunky new version of some classic prose via the C of E::

The more usual 'Give us this day our daily bread', from the Lord's Prayer, becomes: 'You are giving us our daily bread when we manage to get back our lands or get a fairer wage'.

'Yea, though I walk through the valley of the shadow of death, I shall fear no evil.' has been removed from the 23rd Psalm in favour of 'Even if a full-scale violent confrontation breaks out I will not be afraid, Lord'.

George Orwell, in Politics and the English Language warned about this sort of thing:

Now that I have made this catalogue of swindles and perversions, let me give another example of the kind of writing that they lead to. This time it must of its nature be an imaginary one. I am going to translate a passage of good English into modern English of the worst sort. Here is a well-known verse from Ecclesiastes:

I returned and saw under the sun, that the race is not to the swift, nor the battle to the strong, neither yet bread to the wise, nor yet riches to men of understanding, nor yet favour to men of skill; but time and chance happeneth to them all.

Here it is in modern English:

Objective considerations of contemporary phenomena compel the conclusion that success or failure in competitive activities exhibits no tendency to be commensurate with innate capacity, but that a considerable element of the unpredictable must invariably be taken into account.

This is a parody, but not a very gross one. [...] It will be seen that I have not made a full translation. The beginning and ending of the sentence follow the original meaning fairly closely, but in the middle the concrete illustrations -- race, battle, bread -- dissolve into the vague phrases "success or failure in competitive activities." This had to be so, because no modern writer of the kind I am discussing -- no one capable of using phrases like "objective considerations of contemporary phenomena" -- would ever tabulate his thoughts in that precise and detailed way. The whole tendency of modern prose is away from concreteness.

Social software arbitrage

Jonas:

As seen on eBay - we've gots 'em. Nine brand new, shiny, gmail invites are sitting in my inbox waiting to further dilute the value some people are apparently willing to assign to this...

It's not free, however. If you're interested in one, comment here and let me know what you're willing to do for it. Not to me (though I am more than ready to trade for a few good massages), but to someone else. A random act of kindness, maybe? Work in a soup kitchen? Help out at a needle exchange? Or maybe you're doing that already - you'd be the ideal recipient.

Something for Nothing

Something for Nothing: The Free Culture AudioBook Project is a great essay on Copyright as seen through the lens of AKMA's project ot make an audio book from Lessig's Free Culture.

Technorati Developers Salon

May 19th from 6pm to 8pm was the Technorati Developers Salon.
see what people have said about it.

Free Culture readathon

I joined in AKMA’s "free culture readathon":

Anyone feel like recording a chapter of Lawrence Lessig’s new book?

The license pretty clearly indicates that, so long as we’re not making a commercial venture of it, we can make a recording of (“perform”) the text. There are a Preface, Introduction, fifteen chapters, a conclusion and an afterword.

I recorded the Preface of Free Culture which has me referring to Code and Other Laws of Cyberspace as 'my first book', which is a kind of lèse majesté, or Lessig majesté.

I used the Audacity Open Source Audio editor for this, which works very nicely, and reminds me of SoundEdit from long ago.

Here's a QuickTime version for people on slow modems.

[updated 2014 to fix dead links thanks to Apple being crap] Here's the previously unpublished dance remix I made then:

Life imitates code?

At Technorati, I've been writing 'spiders' - little bits of code that scuttle over the web indexing pages when roused. We normally have hundreds running around at once. So I was interested to see these spider hatchlings in the garden on Thursday, swarming over a web, trying to make sense of it.

Big picture of baby spider cluster

How prior would you like that art?

Jeneane's Phonecon made me smile, but the early days of telephony were full of inventive people trying out new services and business models.

With the Patent Office moving toward rejecting the Eolas plug-in patent on prior art grounds, maybe it's time for them to reconsider the Acacia and SightSound patents that take an obvious idea and add the word 'digital', and then go around shaking down anyone doing rich media online.

Alexander Graham Bell invested heavily in a company to send opera over the telephone for a fee in the 19th Century.

Live music over the phone was happening in 1877, and remote playback by telephone in 1888 and even remote paid playback of recordings over the phone on demand, from 1909.

Black, White, Grey and mediAgora

I know I've been quiet on the mediAgora front recently, but the Grey Album case is in many ways a perfect example of the mediAgora principle of rewarding Creators of both derivative and original works.

Imagine, if you will, a parallel universe where The Beatles 'White Album' and Jay-Z's 'Black Album' had been released under mediAgora licences.

Along comes Danger Mouse, and mixes the two to make the 'Grey Album', and releases it for sale under a mediAgora license too.

So what happens?

He list both monochromatic albums as 'source works'. Everyone who buys 'The Grey Album' has to own a copy of the two source albums too. If they already do, they just pay Danger Mouse; if they own The Beatles but not Jay-Z they pay him and Danger Mouse.

As DM is generating incremental sales in this way, he gets promotion fees from the other two.

And all those bloggers pointing to the Grey Album? They get promotion fees from Danger Mouse, insofar as they generate sales (and have bought a copy themselves).

End result - every Customer has 3 great albums, and all Creators involved get paid the price they set.

And even Glenn Miller and the orchestras George Martin and the Beatles sampled could be rewarded too.

Technorati, Xanadu and other dreams

Tim Oren says nice things about Technorati:

As we all know, Ted Nelson meant hypertexts to have bidirectional links. But due to a laboratory accident in Switzerland, we ended up with this lame thing. Mechanisms such as Google link search and Technorati are just hacks, ways to leverage Moore's Law to ameliorate a fundamental flaw in our hypertext data architecture, crawling the Web faster and faster to aggregate all of our trackbacks.

Yesterday, David Sifry convinced me that's just wrong. What Nelson missed, with his focus on 'literary' architectures, is that networked hypertexts are inhabited by people. Links are not just citations. They are gestures in a social space, parts of conversations or other interactions. There's an inherent value in looking at the dynamics of the record as it is created.

Obviously I agree with the broad thrust of this or wouldn't be working at Technorati. However, I think the one-way nature of links was necessary for the Web to achieve what it did. The globally connected nature of the web as a small world network is built on a scale-free distribution of linkage. If all links are required to be two-way, this rapidly becomes unwieldy and cumbersome - imagine if the front page of Apple.com showed all the inbound links to it. The unidirectionality created the permission-free linking culture the web depends on, and reversing those links in a useful way is an interesting problem we're having fun solving - the hot products page is an example of this.

In fact, that page addresses another issue, if obliquely. Shelley asked how being a community member affects your writing:

[W]eblogging [is] different than Big Media, because it puts publishing in the hands of the people. I have to presume they think this is a good thing because webloggers can write what they want, and aren't censored. Unlike Big Media, we aren't accountable to an editor, or big companies, or important politicians.

But I guess we're accountable to each other, and that's the most dangerous censorship of all -- it's the censorship of the commons.

Indeed. I think this is a good thing. The fact that when blogging we are accountable for our writings and their public history acts, in general, in a good way - it makes us stop to think about our reactions before they 'end upon our permanent record'. Shelley's own campaign against comment spammers that violate community norms in this way is an example. David Weinberger in 'Small Pieces Loosely Joined' put it this way:

A human being raised in isolation would not be identifiably human in anything except DNA. Sociality grants a mute herd of brutes their souls and selves.

The example of what happens when anonymity is allowed in Amazon reviews, leading to all kinds of dubious behaviour was revealed in the NYT , and picked up by auctorial bloggers like Cory and Neil Gaiman. The bloggers' comments show up in context with the rest of their writings, so you can gather whether you are likely to agree with them generally too.

For example, when Lago attacks Vote Links for reinforcing hegemony I can see that he is the same person who threatened to offer Joi a reading list, but then didn't, so I can offer him one instead: