Last week I looked into the problem of creating megawidgets using TIP 257‘s oo:: framework, and concluded that it was possible to hijack the Tk widget command in the class’s create method.

It turns out that there is a serious problem with this technique: you can’t easily create subclasses of your new megawidget class. (To follow the rest of this discussion you should go back and read the earlier post now, if you haven’t already.)

In the example, I create a simple read-only text widget called rotext. The first step in the technique I described was to redefine the new megawidget class’s create method to do the hijacking step. As shown, we create a text widget, rename it, and then call next to get the default create method, which actually creates the object and calls its constructor:

    self.method create {win args} {
        # FIRST, create a text widget with the desired name.
        # (We could also hijack a widget that had already been
        # created.)
        text $win

        # NEXT, rename it to something else.
        set hull ::_$win
        rename $win $hull

        # NEXT, create a new object with the original name,
        # passing the new name to the constructor.  Note that
        # we add the "::" to make sure the command is created in
        # the global scope.
        next ::$win $hull {expand}$args

        # NEXT, return the bare window name
        return $win
    }

Then, in the constructor we take the renamed text widget command, and save it in an instance variable called hull so we can use it. We do this by passing the renamed command name to the constructor as an argument, because that’s really the only way the class has to communicate this name to the instance.

    constructor {theHull args} {
        my variable hull

        # FIRST, save the hull
        set hull $theHull

        # NEXT, pass the options to the hull
        $hull configure {expand}$args
    }

So what happens if we try to subclass our new rotext type? Here’s a simple subclass:

    oo::class create subrotext {
        superclass rotext

        # ....
    }

Now, watch happens when we try to create an instance of subrotext:

    % subrotext create .text -width 40 -height 10
    Error in startup script: invalid command name "-width"
        while executing
    "$hull configure {expand}$args"
        (class "::rotext" constructor line 8)
        invoked from within
    "subrotext create .text -width 40 ..."
       ....

What happened here? What happened is that subrotext inherited the constructor from rotext, but not the rotext class method create. Thus, we got the default create method, which does no hijacking, and doesn’t pass the new hull command name to the constructor.

In short, every subclass of rotext would need to explicitly redefine its create class method to do the hijacking. So much for reuseable code.

This is rather a poser, and I’ve not figured out my way around it yet. Should class methods be inherited by subclasses? If not, why not?

Update 11/10/2006: Aha! Figured it out. My mistake, above, was using oo::class to create the subclass of rotext. I should have defined a new metaclass, called igloo::widgetadaptor or something of the sort, that guarantees the right create semantics, and created the subrotext as follows:

    igloo::widgetadaptor create subrotext {
        superclass rotext
        # ....
    }

Here, grasshopper, is a deep and abiding truth: a class gets its instance methods from itself and its superclasses and mixins, but it gets its class methods from itself and from the metaclass used to create it. Write that on your forehead in permanent ink.

Recently I suggested that it would be interesting if TIP 257 defined a myclass command which an object could use to access its class’s unexported methods and also its class variables. I’ve figured out a remarkably tacky way to do so–it’s so tacky that I’m almost ashamed to describe it.

First, remember that myclass would simply be the class’s own my command aliased into the instance’s namespace. If the instance knew the class’s namespace it could define myclass quite simply using interp alias:

    interp alias {} [self namespace]::myclass {} ${classns}::my

Given this, the instance could then declare and use its class’s variables; this is often useful for lookup tables which are static and can be shared by all of the class’s instances:

    myclass variable lookupTable

    set value $lookupTable($key)

The question is, how can the instance acquire the class’s namespace? One way would be to define an exported class method called namespace, but that affects the class’s public interface, which we don’t want to do. So here’s a dirty trick: we temporarily export the class’s eval method, and use it to query the class’s namespace. We can do this any place in the instance code:

    # FIRST, get the class's name
    set class [info object class [self]]

    # NEXT, export its eval method, making it public.
    oo::define $class self.export eval

    # NEXT, get the namespace
    set classns [$class eval {namespace current}]

    # NEXT, unexport its eval method, making it private
    oo::define $class self.unexport eval

    # NEXT, alias in myclass
    interp alias {} [self namespace]::myclass {} ${classns}::my

As I say, I’m almost ashamed of this trick. It’s a very roundabout way to get information the class doesn’t really want me to have; and then, also, it’s rather fragile (at least when applied to classes found in the wild). I’m making a couple of unwarranted assumptions: first, that the class hasn’t redefined eval for its own purposes, so that it no longer does what I think it does, and second, that the class wants eval to be private. Both are just reasonable enough that the corner-cases will catch you unawheres.

In a controlled environment, though, this trick might be worth using. For example, an instance of an igloo::type might conceivable use this trick during its construction…if I can’t figure out an easier way to get the job done.

Snit allows you to delegate unknown methods to some specific component. If you’re adapting a widget, for example, you can redefine just those methods you care about, and delegate all of the remaining methods to the widget you’re adapting by adding the following to your widget definition:

    delegate method * to hull

This feature obviously requires a way to detect and handle unknown methods. Now, TIP 257 provides such a mechanism: the unknown method. Consider the following code:

    oo::class create myclass {
        method unknown {methodName args} {
            puts "Unknown Method: $methodName $args"
        }
    }

An Observation: As I’ve commented elsewhere, methods in TIP 257 can be exported or unexported. If they are exported, they can be called by users of the object; if unexported, they can only be called using the object’s my command. Now, methods whose names begin with a lower case letter are exported by default–which means that the unknown method is exported by default. For my part, I can’t think of any reason why I’d want unknown to be exported…which means that I need to remember to write it this way instead:

    oo::class create myclass {
        method unknown {methodName args} {
            puts "Unknown Method: $methodName $args"
        }
        
        # Remember to unexport it!
        unexport unknown
    }

This is a nuisance. It would be much nicer if unknown were called either Unknown or _unknown instead.

But this is by the way; my real concern here is how to do delegation of unknown methods. Currently available versions of Snit use two different methods: Snit 2.1 uses namespace ensemble‘s unknown handler, and Snit 1.2 does it the hard way.

Unknown Methods and Namespace Ensemble: When you create an ensemble command using namespace ensemble you have the option of specifying an -unknown handler. The operation of the handler is somewhat surprising: it does not, in fact, execute the method itself.

Remember that a namespace ensemble is defined by a dictionary of method names and fully-qualified commands. The -unknown handler is called whenever a method is called that doesn’t appear in the dictionary. It’s the -unknown handler’s job to come up with the fully-qualified command that corresponds to the previously unknown method. Now, here’s the trick: the -unknown handler doesn’t call this fully-qualified command; instead, it simply installs it back into the ensemble’s dictionary. The ensemble itself then makes one more attempt to call the hitherto unknown method.

This is, quite simply, brilliant, though you likely won’t see why unless you’ve tried to implement a similar scheme yourself–that is, when you’ve done it the hard way.

Method Dispatch The Hard Way: If you don’t use namespace ensemble, then dealing with unknown methods in pure-Tcl code is both easier and harder. It’s easier, because dispatching to a previously unknown method is no more difficult than dispatching to a known method–you’ve got to look up the method name anyway, to see how to dispatch to it, so unknown methods are just a special kind of lookup. It’s harder, because dispatching to any method is a royal pain in pure-Tcl, at least if you handle all of the corner cases.

Ideally, writing an object method should be just like writing a proc. A method should be able to do all of the things a proc can do, and in the same way. In particular,

• A method should be able take any argument list supported by proc, including the use of optional arguments and args.
• A method should be able to use upvar to access variables in its caller’s scope.
• A method should be able to use uplevel 1 to execute code in its caller’s scope.
• A method should be able to return -code to return unusual error codes, especially break–this is often needed when writing Tk event handlers.

If you write your own method dispatcher in pure Tcl without using namespace ensemble, you have to handle all of this yourself. Suppose, for example, I want to implement an ensemble command myobject. The methods will be implemented as procs called myobject_name. A pure-Tcl dispatcher will look something like this:

proc myobject {method args} {
    # FIRST, determine the actual command.  For this example we'll
    # skip the error checking, and simply assume that the proc that
    # implements a method is myobject_$method.

    set cmd [linsert $args 0 myobject_$method]

    # NEXT, call the command using uplevel 1; this effectively removes
    # [myobject] from the call stack, so that upvar and uplevel
    # will work properly in the method body.

    set retval [catch {uplevel 1 $cmd} result]

    # NEXT, if the return code is anything but "ok" we need to rethrow
    # it--this time, we're removing [myobject] from the call stack on
    # the return trip.
    #
    # Note that error returns need a little extra help.

    if {$retval} {
        if {$retval == 1} {
            return \
                -code error             \
                -errorinfo $::errorInfo \
                -errorcode $::errorCode \
                $result
        }
        return -code $retval $result
    }

    return $result
}

Here’s a sample method; it takes a variable name and increments the variable.

proc myobject_incr {varname} {
    upvar 1 $varname theVar
    incr theVar
}

Here’s how it looks in practice:

% set a 1
1
% myobject incr a
2
% set a
2
%

Note that it took me about an hour to write myobject and prove to myself it really handles all of the corner cases; the code and the test cases are available at http://www.wjduquette.com/igloo as rethrow.tcl. It isn’t quite perfect, either; the stack trace you get in the case of an error thrown in a method body is way ugly, and reveals too much information about how the ensemble is implemented. (This is one of Snit 1.2’s real warts.)

Back to Namespace Ensemble: The beauty of namespace ensemble is that it handles all of this for you, and it does it in C code so there’s little run-time penalty. And because of the brilliant design of the -unknown handler, it works for unknown methods too.

Unknown Methods in TIP 257: So, how does the TIP 257 unknown method stack up against namespace ensemble? The answer, unfortunately, is “badly”. You’re essentially back in the bad old days: the unknown method has to do all of the work myobject does in the example shown above. The resulting code works, but it’s ugly and slow. Since automatic delegation of unknown methods is one of Snit’s strong points, and since one of the motivations for building Snit on TIP 257 is speed, this is a serious concern.

Here’s some example TIP 257 code that shows, in classic Snit style, how to make a dog wag its tail. For simplicity, I’ve omitted code that exercises the corner cases. The full code is available as unknown1.tcl.

::oo::class create tail {
    method wag {speed} {
        if {$speed eq "fast"} {
            return "Wag, wag!"
        } elseif {$speed eq "slow"} {
            return "Wag."
        } else {
            return "Whine!"
        }
    }
}

::oo::class create dog {
    # A dog has a mytail component
    constructor {} {
        my variable mytail

        set mytail [tail new]
    }

    # Let's delegate unknown methods to mytail.
    method unknown {methodName args} {
        my variable mytail

        # FIRST, determine the actual command.
        set cmd [linsert $args 0 $mytail $methodName]

        # NEXT, call the command using uplevel 1; this effectively 
        # removes [[self] unknown] from the call stack, so that upvar 
        # and uplevel will work properly in the method body.

        set retval [catch {uplevel 1 $cmd} result]

        # NEXT, if the return code is anything but "ok" we need to 
        # rethrow it--this time, we're removing [myobject] from the 
        # call stack on the return trip.

        if {$retval} {
            if {$retval == 1} {
                return \
                    -code error             \
                    -errorinfo $::errorInfo \
                    -errorcode $::errorCode \
                    $result
            }
            return -code $retval $result
        }

        return $result
    }

    # Make unknown invisible to uses
    unexport unknown
}

Can we do better than this? Possibly. TIP 257 supports a minimal delegation mechanism that allows any method to be forwarded to any arbitrary command. The semantics are similar to those of interp alias or namespace ensemble, which is just what we’re looking for. The problem is, in order to forward an unknown method we have to know the method name–and we don’t know the method name until after unknown is called. This suggests a hybrid scheme:

• When an unknown method is called, dispatch it “the hard way”.
• If the call succeeds, the method is no longer unknown; forward it explicitly, so that subsequent calls do it “the easy way”.

We can do this by adding a single line of code after the [catch] statement in the unknown method shown above: (See unknown2.tcl)

    if {$retval != 1} {
        oo::define [self] self.forward $methodName  $mytail $methodName
    }

We now do things the slow way the first time only; after that, we get namespace ensemble-like goodness and speed. In my benchmarks, the hybrid version is just over twice as fast on average.

Some Warts: Both methods of delegating unknown methods have some warts, especially in the area of error returns. The wag method shown above requires one argument. Let’s look at some error messages when we implement delegation the naive way:

% source unknown1.tcl
% spot wag fast slow
wrong # args: should be "::oo::Obj6 wag speed"
% spot wag
wrong # args: should be "::oo::Obj6 wag speed"
%

Here the error message is written in terms of the tail component, ::oo::Obj6, which is seriously ugly. If we use the hybrid approach we see things like this:

% source unknown2.tcl
% spot wag fast slow
wrong # args: should be "::oo::Obj6 wag speed"
% spot wag fast
Wag, wag!
% spot wag fast slow
wrong # args: should be "spot wag"
% spot wag
wrong # args: should be "spot wag"
%

First, we get the “naive”-style error messages until the first time the method call succeeds. This is annoying. Then, once the method had been forwarded we get an error message that gets the object right but omits the argument names. One doesn’t know whether to laugh or cry. I choose to look at this as a bug in TIP 257.

Conclusions: The hybrid approach is definitely the preferred way to delegate unknown methods, given the current implementation, although the automatically generated error messages are unfortunate.

It would be spiffy if the unknown handler could do this:

    # Let's delegate unknown methods to mytail.
    method unknown {methodName args} {
        my variable mytail

        # FIRST, determine the actual command.
        set cmd [linsert $args 0 $mytail $methodName]

        # NEXT, forward it.
        oo::define [self] self.forward $methodName  $mytail $methodName

        # NEXT, ask the object to retry the call.
        return -code retry
    }

The new return code retry would tell the caller to retry the call–that is, to call it exactly one more time. This would give us similar semantics to namespace ensemble‘s unknown handler.