I’m also a big tech geek and do my best to keep up, and maybe even develop, things that help our society make the next great leap forward. I helped develop the original Netscape browser which helped the internet explode and ripped control of computer networks from Microsoft, handing it to the masses. I’m working on doing the same with open, public data right now at Freebase. I do believe that science and technology are bettering our society as a whole and that the risks and drawbacks far outweigh the rewards. I think they are making the world a more equitable place and giving more choices to more individuals than ever before, and I think this is a good thing.

So when it comes to GMO food, I’m a little confused. On the one hand, the notion of actually modifying the genetics of an organization at a cellular level seems like some kind of creepy science. On the other hand, this is just science improving the quality of life, driving down the costs of basic human sustenance. It’s just a logical extension of breeding crops for various traits, right? Some years ago I read one or two random articles (I think one was in Harpers, can’t remember what else I read) that had me thinking that on the whole, GMO food is bad. The science behind it can’t begin to address the massive complexity of our ecosystem. Further, the politics and policy behind patents on organisms, the limits that Big Agra puts on farmers for seed retention, and the notion of GMO as a way to reduce genetic diversity are really bad.

But with all the hubub recently about GMO + Organic and the Obama administration’s interest in the food system has given me a chance to at least try to reevaluate my position. The problem comes when I watch video’s like this Bill Nye video (In three parts: one, two, three) that I found via this Civil Eats Article on GMO food. I love Bill Nye. I think he makes science really cool and fascinating and I can’t wait until my kids are old enough to watch him. But this video is incredibly biased against GMO while trying to appear like he’s showing both sides. The worst part is that most of the anti-GMO bits are either morally heavy, substance free (”But isn’t genetic modification just creepy? Should we really be messing with organisms like this?”) or just fear mongering (dramatic enactments of monster food killing people, theoretical implications that haven’t actually happened, etc)

One argument I’ve heard (that got Agriculture Secretary Tom Vilsack booed) is that GMOs can be used for good – to feed the world! The counter argument I’ve heard to that is that basically we have enough food, that it’s really a distribution problem – that the GMO-to-feed-the-world is a lot of bunk. What I wonder though, is if it’s really a “distribution problem” why can’t we find ways to grow food near the people that need it? The bay area has lots of self-proclaimed locavores who aspire to eat food grown within 50-150 miles from them, but why then do we need to ship food from one side of africa to the other? What if one solution to that is GMO crops that crow in climates that currently don’t support human-food agriculture? What if it would take 500 years to breed the equivalent crop?

So I don’t know. I think next up I’m going to watch a bunch of Long Now talks: Organically Grown and Genetically Engineered, Rethinking Green, and Michael Pollan’s Deep Agriculture to see if I can gain any more insight.

When I first started hearing about these carts, it was really all about word-of-twitter for finding these folks – there were few online resources and you had to just know. Personally that kind of thing drives me nuts – mostly because I wasn’t in-the-know. While there’s something exciting about knowing these little out-of-the-way places, it feels a little like a high school clique. On top of that, I have a friend who has been wanting to do what I consider a fairly original independent food thing for a while – he started an LLC, rented some kitchen space, but didn’t really get beyond the stages of trying recipes. He lost steam because he couldn’t figure out a good market for his food.

..and he wasn’t even aware of the whole food cart scene! So in the interest of promoting openness and transparency in the foodie scene, I present my list of favorite food carts, most of which I haven’t visited because I don’t live in the mission.

• cremebruleecart – one of the originals, he gets the top of my list because Heather and I tried and failed to find him once on a friday night (our timing was off) and Heather’s favorite dessert is Creme Brulee. Mostly around Dolores Park, but goes to lots of special events
• carte415 – Looks like fancy organic sandwiches and salads. My cheapo go-to for sandwiches is the Toaster Oven – their sandwiches are actually pretty good but I feel a little guilty going there because it’s a chain – so any chance of getting reasonably cheap organic sandwiches has got me excited. Going there today.
• chowdermobile – Seems like this guy has been trying to get into SF forever. Not sure what the holdup is but I really, really want some good clam chowder… he seems to go up and down the penninsula.
• littleskillett – ok, this isn’t a cart, and is only this low on the list because I have been there, a few times even. They make some pretty amazing fried chicken but the best thing I had was this crazy pile of pulled-pork and other fixins on top of grits. Plus, they have Blue Bottle coffee to prevent post-chicken food coma. The reason they fit into this list is because they serve out of a counter in an alley and you eat on the loading dock. So it’s still street food because you’re sitting on the street, quite literally.

That’s all I have for now, there are a few more I’m curious about like kitchenettesf, chezspencergo, and SexySoupCart, but they’ll have to wait until I’ve exhausted the above list.

(If you’re looking for more, you can look at the folks I’m following on twitter.)

Now all these projects seem to be on hold. New businesses have sprung up in the latter lots using the existing buildings. I must admit I’m pretty disappointed with how the whole thing turned out.

I am actually a bit in favor of some actual development along this corridor because the vacant lots and failing businesses were of questionable value to the neighborhood. (The used care dealership was just full of broken down cars, I always wondered who actually went there to buy a car, because the cars never seemed to change) But with this housing downturn we may be heading back where it was before…

And now I just found this: San Pablo Condos Top ZAB’s Agenda – looks like some nearby projects are still going forward. Huh.

I just want to rave about “tramp mode” in emacs. If you haven’t yet seen this, it allows you to load up files from a machine that you have ssh access to. Accessing it is super-easy. Rather than C-x C-f to load a local file path, just enter the file path as ssh://userid@host:/path/

After that everything you save will be saved over ssh/scp. Brilliant.

Reason number one: Probably the reason Heather used to call organic strawberries “armpit fruit”:


Yes, that is a dead worm in my artichoke. Yes, I had to eat this far to discover it.

Reason number two: Sure they don’t use pesticides, but I don’t want babies working the fields any more than I want 12 year olds making my shirts.

I have a theory that many more people would be against the Iraq war and more critical of the Whitehouse administration if they simply understood the implications for this country. For example, I wonder how many people know that our budget this year for defense is $336 billion, yet our educational budget is $61 billion? I wonder how many people would support the simplest proposal of say, cutting $30 billion from the defense budget in order to increase the education budget by a whopping 50%?

And so I can’t begin to express how pleased I am that someone like Andy Rooney, who is typically viewed as fairly harmless, suddenly has become so vocally critical of the war. I think the mainstream media finally got some backbone with their outrage over the handling of Katrina, but I’m going to predict that Andy Rooney’s segment yesterday is a turning point for public criticism of the war and this administration. I think this changes the face of opposition. I think for many people it all sounds like the just the rantings of some that crazy mom Cindy Sheehan, or some crazy Californians who are too disconnected from the real world to have a legitimate voice, or some vocal celebrities jumping on the bandwagon of rebelliousness.

In my graph, nodes are represented by Python tuples that refer to the metadata culled from the song list. For example, there is a node for (’Artist’, ‘U2′) and another for (’Genre’, ‘Rock’). I keep track of the relationship between these nodes with a weight that comes from the number of songs that have both of these pieces of metadata.

So for example there is a line between (’Artist’, ‘U2′) and (’Genre’, ‘Rock’) which has a weight of 15, because their new album is categorized as ‘Rock’ – though songs from the album October are categorized as ‘Rock/Pop’

When I combine all the different pieces of metadata in my collection I get a whopping 1589 different facets, represented by nodes in my graph. But whats more interesting is that about 1500 of these nodes are connected, and the other 90 or so are divided into about 30 different individual chunks of 3-4 facets each. I tried to visualize this with GraphViz but the data was just too big.

But this got me thinking more about how to chunk the graph. It was really surprising that so many of the nodes were connected, but really what matters to me is knowing which nodes are the most connected. This means that I could start dropping lines (connections) between nodes where the weight is just 1… or 2, or whatever number yields an appropriately chunked graph. Hopefully that will break up the large cluster of facets into smaller, more usable clusters.

In the presentation, these virtual collections were presented with colored regions, and the items themselves were little dots that exist in multiple regions. I think what could be potentially interesting is the way that these regions overlap because of the songs that link them.

So I am developing a graph based model, a very simple one really, where each vertex is a particular value, or facet, such as “Artist=U2″ and each line between two verticies represents the songs that exist in both collections. So along each line is a set of actual songs, and the verticies themselves exist only in the virtual sense. The lines can be assigned a particular weight based on the songs that it represents. A simple weight would simply be the number of songs on the line itself.

By connecting all this information I believe what we’ll come up with is a fairly well connected graph, but with a great varition in line weights. This variation isn’t random and the patterns that develop will correlate with clusters in the graph.

It may be obvious by now that this is really just a graph-based representation of the correlation matrix. I’m making a wild-ass assumption that Factor Analysis doesn’t deal well with large numbers of factors, but perhaps some graph-walking algorithms can at least reduce the graph cluster at a time? Time to dust off my old Algorithms text book from college…

Last year I read (most of) The Mismeasure of Man by Stephen J Gould. Aside from being a fantastic book, its last chapter on Factor Analysis has been floating around in my head for quite some time. I think this could be one way to extract the kind of chunks I am looking for.

Factor Analysis, as I understand it

Briefly, Factor Analysis is a way of taking long lists of data, usually multiple datapoints of multiple items, and trying to figure out how many factors are really at play. As Gould says, “factor analysis simplifies large sets of data by reducing dimensionality and trading some loss of information for the recognition of ordered structure in fewer dimensions.”

Here’s an example of how this might work in a biological study. Measurements of 10 different bones in 50 different members of a species are taken. You generate a correleation matrix of each of the 10 measurements, so you have a 10×10 matrix. Each measurement is perfectly correlated with itself of course so the diagonal is 1.0. Then you can actually try to factor the matrix. What this does is reduce the number of dimensions (factors) that can predict all of the other measurements with a reasonable degree of accuracy.

In a simple case, you might find that all bones are consistently about the same proportional length in each creature. The femur is consistenly about 10% longer than the tibia, so the tibia could be used as a ruler for all other bones. Thus, there might be only one factor, growth, which is determining the size of all 10 bones. That factor, which is a number, can be used to predict the length of all 10 bones with some high degree of accuracy.

In a more complex case, you might find that there are really 2 factors at play, and that the measurement of those 10 bones is a function of those two factors. The femur length might be 1.1 x tibia length times 1.03 x fibula. This means there are independent factors contributing to the length of the tibia and the fibula, but given these two measures, you can predict other bone lengths.

In both of these cases, if you don’t care about absolute accuracy, you no longer have to keep the 10 measurements of all the bones when describing a creature. You could just refer to its size as it relates to the tibia, or the tibia and the fibula.

So how does this apply in the world of metadata?

If you imagine that instead of 50 creatures, we have 10,000 songs. Each of those songs has some amount of metadata, or properites, associated with it.

Now most of this metadata is not numeric, so its hard to compare the value of one Artist (”U2″) to another (”Suzanne Vega”). Whats more important is to determine a value that can be used for correlation between any two bits of metadata. For instance, if U2 and Suzanne Vega appear on an album together, then they are pretty closely correlated. If U2 and Coldplay are in the same genre, they may also be closely correlated. There are lots of possibilities – if two albums came out in the same year, if two artists both covered the same song, if two genres have songs by the same artist, and so forth.

So really what you end up with is a correlation matrix between all combinations of metadata. i.e. “Album: Unforgettable Fire” and “Genre: Hip Hop” are just two “values” or columns in the correlation matrix.

Looking at my iTunes collection, I see that I have 77 Genres, 623 artists, and 743 albums. All told that’s a correlation matrix 1443×1443. Wow, that’s a big matrix. Lets hope Factor Analysis can be used on such huge datasets!

So what does it mean to factor such a matrix? If you imagine that your data is not evenly distributed within each of the metadata categories (i.e. you might have more u2 than anyone) then what you have to imagine is that each of these clusters have a few primary themes running through them. As I understand Factor Analysis, what we should end up with is the sort of ‘hubs’ within clusters.

Factor Analysis is typically used to find a “principle component” – a primary dimension that can often determine much of the rest of the dataset. This primary component can be measured by checking how many of the vectors in the matrix project well onto this primary component. So for many biological systems, you might find that the principle component describes some large portion of the information recorded, and thus its not necessary to find other components.

In the case of information stored in iTunes, I’m guessing that the principle component will only weakly describe a set of the data. Instead of describing some 90%, or even 50% of the correlations in the database, I’ll bet the “principle component” is describes less than 10% of correlations well. So if your principle component doesn’t describe much, you want a secondary component. In factor analysis, all components are perpendicular to each other. What I’m hoping in the case of iTunes is that this means that if my principle component is say “Artist: U2″, then my secondary component might be something totally unrelated like “Genre: Hip Hop” (And part of me secretly wonders if all the components are going to boil down to Genres, which might be sad)

So I think I have the tools to generate a correlation matrix, but the question is whether I have the tools to turn that matrix into a set of useful factors?