Section Numbers

Another feature that’s probably been asked for, that I’d like myself, is auto-numbering of sections. In the style which I prefer, top-level sections are numbered “1.”, “2.”, “3.”, subsections are numbered “1.1”, “1.2”, “1.3”, and subsubsections are numbered “1.1.1”, “1.1.2”, “1.1.3”.

In Notebook markup, sections are created by adding first, second, and third-level headers, i.e.,

  = Main Section =

  Random verbiage...

  == First Subsection ==

  blah, blah, blah...

  == Second Subsection ==

  yadda, yadda, yadda...

In classic Notebook, the section headers are just the strings that appear here: “Main Section”, “First Subsection”, and so forth. But now you can have them automatically numbered, i.e., “1. Main Section”, “1.1 First Subsection”, etc. All you need to do is add a “meta parameter” somewhere to the text of your page, like this:

  #meta sectionNumbers

Meta parameters were added toward the end of Notebook 2.1 development, and were never really used for anything. This is the first documented meta parameter; its presence simply turns on section numbering.

Of course, this just whets my appetite for additional features. For example:

  • How about a floating table-of-contents at the top right of the page? Click on a section and jump there!
  • And then, clicking on a section header should probably take you back to the top of the page, where the table-of-contents is.
  • And there should probably be a preferences item that turns on auto-section-numbering for all “nbm3” pages.
  • And probably you should be able to choose your preferred style of section numbering

I don’t say that all of these are going to go in immediately, mind you…though the table-of-contents would be really slick.

At long last: Numbered Lists

One of things Notebook users have been asking for almost from the beginning is numbered list, like this:

  1. First
  2. Second
  3. Third
    1. Third, Part 1
    2. Third, Part 2

My latest development build supports this, using MediaWiki-like syntax:

  # First
  # Second
  # Third
  ## Third, Part 1
  ## Third, Part 2

It isn’t quite ready for prime time; there’s a bug in the HTML widget I use for rendering “nbm3” markup that’s preventing me from assigning the numbers properly, and working around it would require significant changes in how I render a page. The Tkhtml3 developer, Dan Kennedy, has fixed the problem; I just don’t have that version of the widget yet.

I might decide to revamp my rendering scheme anyway, as it would produce more typical HTML than my current scheme. But the bottom line is that the next release of Notebook will support numbered lists.

Notebook Markup: Headers and Whitespace

In Notebook 2, you can add section headers into your pages. This, for example, gets turned into a first-level header:

    = All About Penguins =

Or, rather, it sometimes gets turned into a first-level header. There pretty much has to be a blank line before it; if you enter something like the following, your header will become part of the previous paragraph:

    ...end of previous sentence.
    = All About Penguins =

Also, in Notebook 2 you have to remember the space characters; this won’t work:

    =All About Penguins=

I’ve fixed all of these things in the new parser, which is a good thing. However, there are still some issues.

In normal HTML, text marked as a header will typically get separated from the preceding and following paragraphs by a certain amount of whitespace. In Notebook 2, by contrast, you get blank lines where you put blank lines in the text of your page. That means you always get at least one blank line before a header (since, as I noted above, it isn’t a header without one) and you get a blank line after a header only if you want one. That is, this gives you a blank line:

    = All About Penguins =

    Penguins are fierce creatures who roam the Antarctic veldt.

And this doesn’t:

    = All About Penguins =
    Penguins are fierce creatures who roam the Antarctic veldt.

I can retain this behavior, or I can change it so that blank lines before and after the header line in the input are ignored, and the header always gets whitespace before and after.

Any one have any thoughts one way or the other?

Notebook 3 Markup Design

I’m beginning to contemplate extending and changing Notebook’s preferred markup for Notebook 3. The plan is as follows:

  • Notebook 2 markup (nbm2) will continue to be supported indefinitely. In Notebook 3, you can choose the kind of markup you want to use, page by page.
  • Notebook 3 markup (nbm3) will be based on nbm2, but won’t be identical. It will be extended (i.e., with support for tables) and may have other changes designed to make it easier to use, easier to parse, more powerful, or all three.
  • If nbm3 turns out not to simply be a superset of nbm2, I expect to provide a translator between nbm2 and nbm3 so that you can easily update your old pages if you would like to.

In short, while I’m not setting out to define something entirely different, I’m open to making significant changes as well.

So…if there’s something about the existing notebook markup that you dislike, or something that you’d like to have, please get in touch! (Note that there are already quite a few requests in the bug tracker.)

I’ll give an example: In nbm3, enclosing a word or phrase in stars *really* ought to make it boldface, as that’s a convention most of use in e-mail and other plain text anyway. So nbm3 might handle boldface like this:

  • Star-notation, i.e., *this is bold*, might replace apostrophe notation, i.e., '''this is bold'''; or we might support both.
  • The older <b>this is bold</b> notation will be retained, as it can work in cases where one of the other forms would be ambiguous.

If you’ve got opinions, let them be heard!

New Orleans

So as I say, I spent this past week at the 14th Tcl/Tk Conference, which was held at the Bourbon Orleans Hotel in New Orleans, pretty much smack dab in the middle of the French Quarter. The 2nd Tcl/Tk Conference (which I did not attend) was also held there, as was the 11th Tcl/Tk Conference (which I did).

Thus, I saw the Quarter about a year before Katrina, and now I’ve seen it about a year after. So what’s changed? The answer is, I’m not sure. That is to say, I noticed changes; but whether the differences I noticed represent real changes in the nature of the French Quarter I can’t say. Anyway, here’s what I noticed.

Overall, things seemed more or less the same. There was little flooding in the Quarter, as I understand it; apparently the first settlers built on the high ground.
Last time, there was an aura of (I suspect carefully cultivated) seediness about the French Quarter, especially the residential areas. This time I noticed much more fresh paint, much more repair-work-in-progress, combined with more real disrepair.

It seems to me—I can’t say for sure, and it’s entirely possible that I’m mis-remembering—that the T-shirts on sale in the tourist shops, as well as the general tenor of Bourbon Street, are several degrees cruder and ruder than they were a few years ago. I dunno.

So much for the bad and the icky, now it’s time for the good. The high-points of the trip, tourist-wise, were a visit to Preservation Hall to hear the Preservation Hall Jazz Band, something I’d gladly do again even though Preservation Hall is rather a pit, and café-au-lait and beignets for breakfast at the Café du Monde.

And then, of course, there was the conference, which as always was fabulous. Thanks much to Gerald Lester and Ron Fox, the committee chairs, as well as (in no particular order) Steve Landers, Steve Redler, Donal Fellows, Michael Cleverly, Clif Flynt, Joe English, Sean Woods, and the rest of the gang (practically speaking, I can’t list everybody). It was a small conference this year, but we had attendees from Canada, England, Australia, and Germany—four from Germany alone! I’m still digesting everything I learned, and I’m in my typical post-conference state: really excited and inspired, and too tired to do anything about it. Time for a few quiet evenings at home.

A Long, Hot Weekend

It’s been a long, hot weekend: up to 100 degrees fahrenheit by 9 or 10 every morning, and no less than 90 at bedtime. I don’t think it’s been below 82 even once. So we’ve mostly been living indoors all weekend (with a few hurried excursions for this and that, and two blessed sessions in my brother’s pool).

On the other hand, I have gotten a lot done on Notebook; as I hoped, Notebook can now (in principle) an arbitrary number of text markups within a single notebook, and choose the appropriate plugin to render and manipulate each. There’s much more to be done, but I’ve been working on the infrastructure for this bit since this spring, and it feels good to finally have it all working.

Each To His Own Taste

I’ve been quiet for the last week for several reasons. First, I had a slew of all-day meetings this week (what fun!) which generally left me feeling like nothing much at the end of the day. Productive, but…. Second, Metroid Prime 3 came out this week, and so that’s been taking some of what brain cells are left.

But mostly I’ve been focussing on Notebook this week. In late Spring I got involved in revamping how Notebook handles User Preferences, which are trickier than they might seem. Just allowing the user to edit the preferences (apart from making the preference settings actually do anything) is time-consuming; and I didn’t like the way it looked in Notebook v2.1.3, so I had to redo most of that.

But that was, in most ways, the easy part. The tricky thing in Notebook v3 is that it has a plugin architecture. As delivered it will support two or three different flavors of markup, and it will be possible to add support for new kinds of markup just by dropping in new plugins. Even the stock markup types are supported by plugins; they just happen to be plugins delivered with the software.

The tricky thing about plugins is that they aren’t part of the application code; yet they have preferences of their own. I needed to provide a way for the plugins to save their preferences in the application’s preferences file, and also I needed to provide a way for the user to edit plugin preferences.

I’ve been working on this in a desultory fashion for about the last month; and finally, after all this time, I have a set-up I’m reasonably pleased with. The application is finally using my new renderer plugin for standard Notebook markup to render pages; and I’m poised to make Notebook’s pageviewer check each page’s markup type and pick an appropriate renderer. This will be Way Cool, and will open up broad new frontiers. Among other things, once this is in place I can begin to experiment with new kinds of markup and new ways of rendering pages–i.e., I can begin to think about supporting tables and other significant markup changes without breaking existing pages.

Woo-hoo!

Dogfood Day

I’ve not blogged much this year; at first I was just tired and uninterested, but for the last month-and-a-half I’ve been working on Notebook, an application I first wrote in 2000 and have been developing on and off ever since. Although I’ve been using it daily, the last release (representing the last serious work) was in 2005. Why the gap? Well….

When I started work on Notebook, it was the first serious Tcl/Tk application I’d attempted. I learned an awful lot about writing Tcl/Tk applications from it, and those early experiments also led to my developing Snit, a Tcl/Tk object framework that’s been increasingly popular over the years. Now, up until May of 2005 my day job mostly involved writing software in C. I wrote a few tools in Tcl/Tk, but most of my Tcl/Tk work was at home, on my own time. In May of 2005 I started a new project, one written almost entirely in Tcl/Tk. My experience writing Notebook stood me in good stead. The project’s schedule was ambitious, and using Tcl/Tk allowed to work very fast. In short, I’ve spent the last two years of my working life coding my brains out in Tcl/Tk. This has had two major effects. First, I’ve learned an awful lot more about writing large applications in Tcl/Tk; and second, I’ve had next to no energy to do any amount of programming at home on my own time. And so Notebook has languished.

About a month-and-a-half ago, though I got motivated to start working on it again. And this time I wanted to Do The Job Right–to make use of everything I’ve learned over the last two years, and to build an application that I’ll have an easier time coming back to after time spent working on other things. In particular, there were some serious infrastructure changes I wanted to make. And so I started building all new infrastructure from the ground up, writing thorough documentation and test suites as I went along, as well as a variety of new tools.

And today, finally, I was able to hack a copy of the old application to use much of the new infrastructure. For the first time I am now able to edit Notebook files using the code I’ve been working on for the last month-and-a-half. Woo-hoo.

So what about “Dogfood Day”? Since the very earliest days I’ve always kept all of my Notebook development notes in a Notebook file, and used the development version of Notebook to browse and edit them. During this recent effort I’d not been able to do that…until today.

There’s a practice in software development called “Eating Your Own Dogfood”. It means that you actually use the software you’re writing, rather than somebody else’s. If a Microsoft developer goes home from work and surfs the Web using Firefox or Safari rather than Internet Explorer, he’s not eating his own dogfood.

Consequently, “Dogfood Day” is the day that a new product is sufficiently mature that the developers can start using the development version in their work. So today is Dogfood Day at our house; and Gosh! it tastes good.

iGloo: A Problem with Delayed Creation

Last week I looked into the problem of creating megawidgets using TIP 257‘s oo:: framework, and concluded that it was possible to hijack the Tk widget command in the class’s create method.

It turns out that there is a serious problem with this technique: you can’t easily create subclasses of your new megawidget class. (To follow the rest of this discussion you should go back and read the earlier post now, if you haven’t already.)

In the example, I create a simple read-only text widget called rotext. The first step in the technique I described was to redefine the new megawidget class’s create method to do the hijacking step. As shown, we create a text widget, rename it, and then call next to get the default create method, which actually creates the object and calls its constructor:

    self.method create {win args} {
        # FIRST, create a text widget with the desired name.
        # (We could also hijack a widget that had already been
        # created.)
        text $win

        # NEXT, rename it to something else.
        set hull ::_$win
        rename $win $hull

        # NEXT, create a new object with the original name,
        # passing the new name to the constructor.  Note that
        # we add the "::" to make sure the command is created in
        # the global scope.
        next ::$win $hull {expand}$args

        # NEXT, return the bare window name
        return $win
    }

Then, in the constructor we take the renamed text widget command, and save it in an instance variable called hull so we can use it. We do this by passing the renamed command name to the constructor as an argument, because that’s really the only way the class has to communicate this name to the instance.

    constructor {theHull args} {
        my variable hull

        # FIRST, save the hull
        set hull $theHull

        # NEXT, pass the options to the hull
        $hull configure {expand}$args
    }

So what happens if we try to subclass our new rotext type? Here’s a simple subclass:

    oo::class create subrotext {
        superclass rotext

        # ....
    }

Now, watch happens when we try to create an instance of subrotext:

    % subrotext create .text -width 40 -height 10
    Error in startup script: invalid command name "-width"
        while executing
    "$hull configure {expand}$args"
        (class "::rotext" constructor line 8)
        invoked from within
    "subrotext create .text -width 40 ..."
       ....

What happened here? What happened is that subrotext inherited the constructor from rotext, but not the rotext class method create. Thus, we got the default create method, which does no hijacking, and doesn’t pass the new hull command name to the constructor.

In short, every subclass of rotext would need to explicitly redefine its create class method to do the hijacking. So much for reuseable code.

This is rather a poser, and I’ve not figured out my way around it yet. Should class methods be inherited by subclasses? If not, why not?

Update 11/10/2006: Aha! Figured it out. My mistake, above, was using oo::class to create the subclass of rotext. I should have defined a new metaclass, called igloo::widgetadaptor or something of the sort, that guarantees the right create semantics, and created the subrotext as follows:

    igloo::widgetadaptor create subrotext {
        superclass rotext
        # ....
    }

Here, grasshopper, is a deep and abiding truth: a class gets its instance methods from itself and its superclasses and mixins, but it gets its class methods from itself and from the metaclass used to create it. Write that on your forehead in permanent ink.

iGloo: A Dirty Trick

Recently I suggested that it would be interesting if TIP 257 defined a myclass command which an object could use to access its class’s unexported methods and also its class variables. I’ve figured out a remarkably tacky way to do so–it’s so tacky that I’m almost ashamed to describe it.

First, remember that myclass would simply be the class’s own my command aliased into the instance’s namespace. If the instance knew the class’s namespace it could define myclass quite simply using interp alias:

    interp alias {} [self namespace]::myclass {} ${classns}::my

Given this, the instance could then declare and use its class’s variables; this is often useful for lookup tables which are static and can be shared by all of the class’s instances:

    myclass variable lookupTable

    set value $lookupTable($key)

The question is, how can the instance acquire the class’s namespace? One way would be to define an exported class method called namespace, but that affects the class’s public interface, which we don’t want to do. So here’s a dirty trick: we temporarily export the class’s eval method, and use it to query the class’s namespace. We can do this any place in the instance code:

    # FIRST, get the class's name
    set class [info object class [self]]

    # NEXT, export its eval method, making it public.
    oo::define $class self.export eval

    # NEXT, get the namespace
    set classns [$class eval {namespace current}]

    # NEXT, unexport its eval method, making it private
    oo::define $class self.unexport eval

    # NEXT, alias in myclass
    interp alias {} [self namespace]::myclass {} ${classns}::my

As I say, I’m almost ashamed of this trick. It’s a very roundabout way to get information the class doesn’t really want me to have; and then, also, it’s rather fragile (at least when applied to classes found in the wild). I’m making a couple of unwarranted assumptions: first, that the class hasn’t redefined eval for its own purposes, so that it no longer does what I think it does, and second, that the class wants eval to be private. Both are just reasonable enough that the corner-cases will catch you unawheres.

In a controlled environment, though, this trick might be worth using. For example, an instance of an igloo::type might conceivable use this trick during its construction…if I can’t figure out an easier way to get the job done.