Alec's thoughts

The ultimate IDE

I've been working at Macromedia for a month or two now, and I've learned to appreciate the coolness of Flash and ActionScript. People may whine and complain that its not open, that its proprietary, that it duplicates some web functionality, but when it comes down to it, its just damn fun. I'm hoping to put together some interacitve flash tutorials on Mozilla architecture, but that's a subject for another day.

What I've been thinking about lately is the way that Macromedia has put together this slick authoring environment to allow people to develop for Flash. The latest version, "Flash MX 2004" is an IDE that combines UI/animation design (similar to the old Visual Basic tool) with a programming environment. What's especially interesting is that they've integrated the compiler, the editor, and the help system all into one space.

Given that this is a commercial product, its probably worth me explaining what the environment is like. I'll say right now, that I know the ActionScript editor is too small, and weak in the editing department, but that doesn't mean its without its merits!

The authoring environment consists of a stage, where you place objects, an editor, where you write ActionScript, and a bunch of helpful panels with color swatches, and UI components, debugging tools, and more.

But whats especially nice is the way the help system integrates into the editor. There are a few basic features that are really nice:

Autocomplete of methods/member variables   When I type the name of an object instance, and then type ".", a popup menu appears that auto-completes the possible attributes on this object. You can still type whatever you want, but as it narrows down the possibilities, you can see the correct spelling of a method/member variable or actually just select it with the arrow key.

How does this work? Well in the old Flash MX world, they used code hinting to try to match instance names with types. For instance, if you were referring to a text field, it was best if you named it foo_txt, and the editor would match _txt to a text field and give you possible matches.

In the new world of ActionScript 2.0, the compiler can actually know what symbol you're typing in basic on the syntax and type declarations preceding it. The editor is actually using the parsing engine to determine the context of what you're typing. Very cool. I don't think they do this yet, but this could allow even non-system classes to autocomplete methods and member variables, because the editor can figure out what variable you're referring to.

Code hinting of parameters   When you type an open-parentheses as part of a function call, this type-parsing goes and looks up the signature of the function you're starting to call. It then displays the list of parameters in a popup, and even some help to go along with it. The help is usually just a little sentence describing the particular parameter you're typing in. It recognizes commas, so it can handle multiple parameters.

Smart hyper-linking to help   If the cursor is in a class name, method name, or variable name, the help system can use its special knowledge about the symbol to get an exact help entry for your code. For example, if I have 3 classes all with a .GetName() method, and I hit F1, it can look at the function I'm typing, and figure out exactly which GetName() to show me help on.

Targeting objects   This one is a little more complex, and a little more flash specific. Flash has its own sort of DOM for flash movies which in some ways is similar to Microsoft's document.all. Basically when you have objects on the "Stage" (the visual screen where draw your movies) you can give them instance names, which correspond directly to variable names in a tree hierarchy. _root is the root of the entire document, and the _parent member variable is always a reference to the owner of the particular object.

In any case, this means that if I want to refer to a particular object on the stage, and I'm currently in an event handler for another object, I can just click the "Target" button to view a hierarchy of the DOM for the objects on the stage. I can choose an object, and the editor will insert the correct path given the current context. For instance, if I'm a button on the stage, and I want to set the text field that resides in a box that is a peer of the button, I can use the targeting feature to automatically insert "_parent.boxname.textname" into my code.

So this is all very neat. I'm actually working on the technical documentation and its interesting to see how this is all put together. I'm not revealing any real secrets here when I explain that most of this code hinting, etc, comes from a whole slew of XML files that contain these little help fragments and possible class members. The XML to connect syntax with help pages is generated off the help that I write in FrameMaker. All the other stuff (the method parameters, etc) must be hand-coded in some XML. (You can find all this stuff on a Windows box in C:\Documents and Settings\\Local Settings\Application Data\Macromedia\Flash MX 2004\en\Configuration)

So why don't we have an open-source version of this? I think the reason is that there are lots of very smart people who only care about their one tool - i.e. the compiler, the editor, or the documentation for those systems. I've heard rumors that Eclipse does some of this and I need to explore it to find out its capabilities. I think it would inspire people to write more documentation if it were more closely connected to the code you were writing.

Update @ 5:25pm: It looks like Eclipse really does want to do some of what I'm talking about. See the Eclipse C/C++ Development Tools (CDT) page. It talks a little bit about code-completion using Control-Space.

DocBook

So since I've been techwriting for a few months, and hating life with FrameMaker, I decided to see what open source alternatives were available out there.

The other day I happened upon DocBook. I had heard mention of it many times before but didn't really get what it is. Here's what it seems to be: a well defined XML schema for writing books and such. The idea behind it is to define your document semantically rather than worrying about layout and presentation. You define chapters and books and sections and paragraphs, and DocBook tools create any kind of output you want. You can include graphics and notes and warnings, footnotes, cross references and everything. Its all very exciting.

What I particularly like about it is that you can use so many existing tools. One thing I was thinking for the mozilla world was IDL / DocBook mapping. In its simplist form, you could make an IDL file generate a DocBook API reference skeleton. If you were feeling daring, you could take the JavaDoc comments out of the IDL, and dump them into the skeleton, initializing the whole skeleton.

But if the IDL changes, is there a way to keep things in sync? The problem is that after generating a skeleton, you'd want to be able to modify the XML to your heart's content. But as long as the XML file retains a well understood structure (which DocBook has!) then you should be able to read in the IDL and the DocBook and alert the developer/writer if the docs don't match the source. Neat.

But for now I'm going to explore some DocBook tools and see what I can make of them.

OpenSP and CygWin

So in my DocBook exploration, I've discovered the beauty of sgml mode on Xemacs. If only I could get it to do validation. For that, I needed nsgmls, which is a part of James Clark's SP package. But it turns out he's all but abandoned it and the OpenJade group has taken up the cause and made OpenSP, currently at version 1.5.1.

The problem I ran into there is that the freakin' thing doesn't compile on cygwin. What I kept running into were all sorts of link errors:

undefined reference to
 `__static_initialization_and_destruction_0(int, int)'

After hunting around quite a bit, I discovered that this is a common cygwin error, and to fix it you need to comment out all the references to #pragma interface and #pragma implementation.

Of course! That's what I thought it was all along. Well at least there are enough keywords here for google to index this highly, and help someone else.

Now back to mozilla doc hacking.

Mozilla "contributors"

This morning I got a private e-mail in response to bug 178809. It went something like this...

Disclaimer: I do not speak for mozilla.org. I do not even claim to represent the views of any other mozilla developer other than myself.

I'm sorry, but this was just too much. This message epitomizes what is wrong with most of the people who use bugzilla. I responded privately to him, and I am going to use some of the same language that I used with him. I am also going to be a bit harsh, harsher than I was with him privately.

Now to address each comment individually:

This demonstrates how little this guy understands the development process, or anything about the mozilla codebase. Anyone using bugzilla should be open-minded enough to learn a little about how this works, what a bug is, and how people can help. In the bug I outlined the steps that any developer would take to break down this problem. So many of our bugs (especially the bloat and performance bugs) have literally thousands of possible causes. To narrow those causes down to 2 or 3 possibilities is a huge help. Too often people in bugzilla say "Well, that's a developer's job, I'm doing testing, I'm not supposed spend my time narrowing down the problem."

Well guess what folks, that's bullshit. If you want a bug fixed, you're going to have to roll up your sleeves and actually help out. What, you say you don't have time? Also bullshit. If you have time to sign up for a bugzilla account and post "I see this problem too! Someone needs to fix it." then you have time to try to solve the problem as well. If you have time to whine on mozillazine forums that nobody is fixing your bugs, then you have time to narrow down the cause of those bugs.

And for the supposition that a developer would find the problem in 1/100th the time? Also bullshit. Lets say, for a bug like this, that there is a single cause for this wierd allocation. The steps involved would go something like this:

1. Narrow down the cause of the problem.
2. Analyze the code
3. Fix and test the problem
4. Get code reviewed and check it in

Ok, now the issue is with #1 above. Now lets just say that a developer could narrow down the cause (using the steps I outline in the bug!) in 1 hour. The next few steps could take another 3 hours of his time. That's 4 total hours of developer time spent working on this bug.

Now, suppose that instead one of the people reading this bug (we'll call him "the tester") spent 2 hours narrowing down the problem. He's spent 2 hours of his time, sure. But now the developer only needs to spend 3 hours analyzing the code, fixing the problem and checking it in. That's one more hour the developer can spend working on another bug. And sure, the total time spent on this is greater, and the tester's time is not invaluble, but now that tester has a greater knowledge of how to narrow down problems and create test cases. In the future, he might be able to tackle a similar problem in only one hour.

Sadly, there is not a room somewhere full of an infinite number of developers on an infinite number of computers eventually typing out the bug fixes to every bug ever in bugzilla. And sadly, not everyone agrees on which bugs are the most important. And not quite as sadly, those mozilla developers are not at the beck and call of anyone who signs up for a bugzilla account.

There are many factors that determine if a bug is going to be fixed, not the least of which is the basic bang for the bug mentality. If you're going to whine and bitch that your pet bug isn't fixed, you've got to at least make the bug attractive for a developer to work on. And guess what, whining and bitching is not attractive. Ever.

Lets say I am a developer and their is this one bug that everyone is bitching about and posting "Me too! I hate this problem! Someone needs to fix it. This prevents me from deploying Mozilla to my mom." Now lets even say for a moment that those people aren't just whiny 14 year olds with nothing better to do than react to testosterone build-up, and that the bug is actually kind of a big issue.

Now lets suppose that it would take a developer who really knows his shit about 40 hours of his time to narrow down a problem, fix it, and check it in. Now lets say he's got 20 other fairly important bugs on his plate, each of which take about 2 hours of time total. If you're that developer, which are you going to work on? Probably the 2 hour bugs. (yeah yeah, there's plenty of rewarding hours involved in really nailing a nasty 40 hour bug, but lets face it, most (all?) mozilla developers are men and 38 hours of foreplay is not our strength)

This is a little like Suze Orman's advice to pay down your Morgage before you contribute to your 401k: Because in the near term, you want to have equity that you can use. If I have only 30 hours to fix bugs, and it was a 40 hour bug, guess what I still haven't fixed any bugs. But I would have fixed 15 of those 2 hour bugs had I worked on them.

So if you want a bug fixed, make it more attractive! Take those 40 hours, figure out if you and a few other folks on #mozillazine can break down the issue and make it a 20 hour bug. Testing isn't about finding problems. That's only half the job. The other half is getting a best guess on what is causing those problems.

It sure is easy for people to make judgements about "Mozilla developers" and how they spend their time. But unless you're actually helping, it doesn't matter what you think.

Mozilla docs

I've been doing some contract work at Macromedia and learned a few things about the documentation process. I've also discovered a fantastic tool that Macromedia created, LiveDocs.

I'll write more about some of the techwriting process that I think Mozilla could adapt, but I wanted to post about LiveDocs because it is so simple, but so brilliant.

The two pieces of the application are:

1. A documentation set generated off of some FrameMaker files, and a nice UI on top of it. The UI is similar to developer documentation that you can find on MSDN. I won't describe it here beyond explaining that it has a nice, hierarchical table of contents that is easily navigated.
2. An "Add Comment" button on the bottom of each page. This allows readers to add feedback to any page of the documentation. The brilliance of this is that authors can maintain ownership of style and content of the document, while easily being able to recieve and process feedback on the document itself.

This is definitely something that open source projects would benefit from. The fact that the files from from FrameMaker is irrelevant, but there does need to be some kind of structured means to develop hierarchicial, indexed documentation.

As an aside, a friend of mine recently was telling me about JavaDoc and how it has grown into a standard system for storing API documentation, allowing any IDE (I think Eclipse is the one he mentioned) to import documentation and integrate it into their editors or help systems. I think this is something that could be investigated as an import format for the system described above.

Song Fight

Last night I saw a truly unique show - one that was more "indie" than any other indierock show that I had attended. The show was a sort of showcase of artists from songfight.org. I didn't actually visit the site until just hours before the show but I absolutely love the concept.

It basically works like this: Each week, the folks at songfight.org post some song titles. As an artist who frequents the site, you have exactly 7 days to write a song with that title, record it, and submit your MP3 to the site. Song titles this week include "There Are So Many Possibilities" "Twelve Monkeys" and "Hold this for me" After the songs are due, other visitors to the site get to rate the songs. At the end of the week, winners are announced. Ingenious.

At the show, the more frequent winners of these "Song Fights" came up and played some of their greatest hits. The venue was a little coffee shop, Belladonna. The artists were talented, if a bit nervous at times. The audience, like me, was a little geeky. But its a small web site where you have to be able to record your own MP3.. what geek who could do that wouldn't be nervous?

When we arrived there was a curly haired white guy on stage in a faded yellow shirt rapping. I wasn't sure if he was the MC, but I caught some lyrics. I gotta say he was a pretty witty guy. The actual MC later referred to him later as "farmer rap" which described his clothes and style well. He was like an indie rock guy pretending to be a redneck, freestyling.

The rapping farmer left the stage soon after we arrived and another band (note to self: track them down) came up for some quality rocking. The lead singer kept the humor level up and his own band in check, but switching back and forth between eels-style melodies to classic indie folk rocking. After my early forays into listening to local bar bands in San Francisco ended in disappointment, I was reasonably impressed that a local, unsigned band playing their own music didn't suck. But I guess that's where SongFight comes in. Even if 100 bands submit their music, at least there is SOME filtering by the masses. And since the crowd was a lot of mid-twenties geeky white guys, it seemed only naturual that I might share some tastes in music with them.

The night kept getting better. During the last band, I had seen a guy in the crowd who looked remarkably like a guy I used to work with. Well sure enough this guy takes the stage and starts playing. While he appeared slightly awkward as he walked around the stage trying to figure out where to plug in his guitar, he was remarkably talented. His name was John Benjamin, leader of the John Benjamin Band. Strangely, the John Benjamin Band appeared to be the band's namesake plus the members of the previous band (need to fill this in!) In the end it didn't matter who this guy's bad was, he was the star of the show. His folk rock and very smart lyrics made for some catchy stuff.

After John Benjamin a tall guy in a white buttondown shirt, bow tie, and thick rimmed glasses took the stage and began screaming into the mike. MC Frontalot does not take his performance lightly. This was one man serious about his nerd.core agenda. Like most quality gangstas, explaining to the world why they should fear him was his primary modus operandi. (though according to MC Frontalot, he is a gangster) With the dancing talents of a 9th grade dungeons and dragons player, and the lyrical creativity of Outkast, MC Frontalot showed us why most of the folks in the coffee shop were there just for him. As his set went on, more and more people started filing into the cafe and Frontalot's energy kept building. Never letting down his tough-as-nails attitude, Front rhymed the sad story of his electronic interception of a request for help from a certain Nigerian businessman. Appearently he wasn't able to help the man smuggle millions of dollars out of his country, but he got a good rap out of it.

All in all it was a surprisingly good show. I was really expecting something different but mediocre, but between John Benjamin and MC Frontalot, I got my live music bug itch scratched for the week and discovered some new talent as well.

Now to go download some MP3s...

One more gallery trick

oh yeah, I forgot that I had also added another thing for gallery integration.

I found this neat hack which includes a php file that outputs javascript to display a link. I tweaked the script to look like this:

..
echo "document.write('<a href=\"" . 
    makeAlbumUrl($gallery->album->fields["name"]) . "\">" . 
    $gallery->album->getThumbnailTag($gallery->album->getHighlight() ) . 
    "</a>');";
..

getThumbnailTag is cool because it its the complete <img> tag to display the particular thumbnail. It wouldn't be hard to make that a random number to display a random thumbnail.

so now I can also say thumbnail=yes and it will display a thumbnail from the gallery specified by the gallery keyword.

Adding gallery support

For another secret website I'm making for a certain future trip, I needed a way to easily link to a photo gallery (created with the awesome Gallery Project software) in a blog post.

That was pretty easy. I just added
<MTKeyValues source="[MTEntryKeywords]"> <MTIfKeyExists key="gallery"> <div class="blog-entry-gallery"> <a href="/gallery/<$MTKeyValue key="gallery"$>">gallery</a> </div> </MTIfKeyExists> </MTKeyValues>

And now in my keywords field, I can just say "gallery=foo" and it will automatically add a link in the blog entry.

Creating a private category

Today I hacked on MovableType to accomplish two things:

• A private category that I could post without worrying that my stuff goes public
• Sane URLs instead of those annoying date fields

Ok, really I was just trying to make sane URLs. But then these ended up overlapping and the act of making sane URLs allowed me to make private categories.

For my own version of sane URLs, I didn't want to dirify every blog entry like some folks do, but I didn't want to limit my entries to just their date like "http://..../2004/04/30/13.45.12/"

I realized that what I really wanted was to allow certain blog entries that have important information to have special URL suffixes like "how_to_fix_your_car" and have most other entries just appear with the date.

The way I've been doing conditional meta information is with Brad Choate's excellent Key Values plugin. I decided that I would add a keyword, shortname to the Keywords field, which would designate the shortened url name. So for instance, you could put
shortname=creating_a_private_category

in the Keyword field, and that would become the end of the magic url. But then the other problem became, where to put this entry? I decided that the primary category of the entry made the most sense, so that this post ends up as /archives/projects/creating_a_private_category

So to do that, I edited the Individual Archive File template name in Weblog Config->Archiving. To start, I made it:

<MTKeyValues source="[MTEntryKeywords]"><MTIfKeyExists key="shortname"><MTEntryCategory>/<MTKeyValue key="shortname"></MTIfKeyExists><MTIfNoKeyExists key="shortname"><$MTEntryDate format="%Y/%m/%d/%H.%M.%S"$></MTIfNoKeyExists></MTKeyValues>/index.html


This allows the entry to go into /archives/<category>/<shortname> if there is a shortname, and into /archives/<date> if not.

So this was pretty huge, and it accomplished my goal of making sane URLs. It did strike me that this means some of my entries are kind of scattered about in the archive structure. However, this inspired me to password protect any "private" category directories. Since a bunch of individual entries were now going into special category-specific directories, I wondered if perhaps there was a way to make all the date-based archives for those categories go there as well. Sure enough, there is.

I got my hands on another Brad Choate specialty: the Supplemental Category Tags plugin. I needed this for . My plan was that any category that had "private" in the name would automatically go into a private category. For instance, I could say:
<MTIfNotCategory pattern="private">.. </MTIfNotCategory

And I could use that in my Individual Archive Template url:


<MTKeyValues source="[MTEntryKeywords]"><MTIfKeyExists key="shortname"><MTEntryCategory>/</MTIfKeyExists><MTIfCategory pattern="private"><MTEntryCategory>/</MTIfCategory><$MTEntryDate format="%Y/%m/%d/%H.%M.%S"$></MTIfNoKeyExists></MTKeyValues>/index.html


The problem I ran into here is that in MySQL, that field is a VARCHAR(255) and thus couldn't hold the complete URL template. And VARCHAR can't get any bigger than 255 characters. Well, it turns out that it is totally safe to just change that field to a TEXT. Now I have that giant Individual Template field, and everything works. anything private without the shortname keyword is forced into the private, password protected directory.

Now that the individual archives are stored in the right place, the last thing to do was to make sure that the normal listings (i.e. main index, archives, etc) didn't show these entries. The trick here was to use the <MTIfNotCategory> tag again, and just wrap it inside every place where I saw .

I'm not incredibly psyched about this solution because it does wierd things. For instance, if I say <MTEntries lastn=10><MTIfNotCategory pattern="private">... and 5 of the last 10 entries are private, then only the other 5 appear. I wish that the <MTEntries> tag was simply smart enough to be able to use NOT for the category parameter.

Oh well. Maybe now that I've got public and private blog entries, I can actually start making some public ones. :)