iGloo: Facing the Unknown

Snit allows you to delegate unknown methods to some specific component. If you’re adapting a widget, for example, you can redefine just those methods you care about, and delegate all of the remaining methods to the widget you’re adapting by adding the following to your widget definition:

    delegate method * to hull

This feature obviously requires a way to detect and handle unknown methods. Now, TIP 257 provides such a mechanism: the unknown method. Consider the following code:

    oo::class create myclass {
        method unknown {methodName args} {
            puts "Unknown Method: $methodName $args"
        }
    }

An Observation: As I’ve commented elsewhere, methods in TIP 257 can be exported or unexported. If they are exported, they can be called by users of the object; if unexported, they can only be called using the object’s my command. Now, methods whose names begin with a lower case letter are exported by default–which means that the unknown method is exported by default. For my part, I can’t think of any reason why I’d want unknown to be exported…which means that I need to remember to write it this way instead:

    oo::class create myclass {
        method unknown {methodName args} {
            puts "Unknown Method: $methodName $args"
        }
        
        # Remember to unexport it!
        unexport unknown
    }

This is a nuisance. It would be much nicer if unknown were called either Unknown or _unknown instead.

But this is by the way; my real concern here is how to do delegation of unknown methods. Currently available versions of Snit use two different methods: Snit 2.1 uses namespace ensemble‘s unknown handler, and Snit 1.2 does it the hard way.

Unknown Methods and Namespace Ensemble: When you create an ensemble command using namespace ensemble you have the option of specifying an -unknown handler. The operation of the handler is somewhat surprising: it does not, in fact, execute the method itself.

Remember that a namespace ensemble is defined by a dictionary of method names and fully-qualified commands. The -unknown handler is called whenever a method is called that doesn’t appear in the dictionary. It’s the -unknown handler’s job to come up with the fully-qualified command that corresponds to the previously unknown method. Now, here’s the trick: the -unknown handler doesn’t call this fully-qualified command; instead, it simply installs it back into the ensemble’s dictionary. The ensemble itself then makes one more attempt to call the hitherto unknown method.

This is, quite simply, brilliant, though you likely won’t see why unless you’ve tried to implement a similar scheme yourself–that is, when you’ve done it the hard way.

Method Dispatch The Hard Way: If you don’t use namespace ensemble, then dealing with unknown methods in pure-Tcl code is both easier and harder. It’s easier, because dispatching to a previously unknown method is no more difficult than dispatching to a known method–you’ve got to look up the method name anyway, to see how to dispatch to it, so unknown methods are just a special kind of lookup. It’s harder, because dispatching to any method is a royal pain in pure-Tcl, at least if you handle all of the corner cases.

Ideally, writing an object method should be just like writing a proc. A method should be able to do all of the things a proc can do, and in the same way. In particular,

  • A method should be able take any argument list supported by proc, including the use of optional arguments and args.
  • A method should be able to use upvar to access variables in its caller’s scope.
  • A method should be able to use uplevel 1 to execute code in its caller’s scope.
  • A method should be able to return -code to return unusual error codes, especially break–this is often needed when writing Tk event handlers.

If you write your own method dispatcher in pure Tcl without using namespace ensemble, you have to handle all of this yourself. Suppose, for example, I want to implement an ensemble command myobject. The methods will be implemented as procs called myobject_name. A pure-Tcl dispatcher will look something like this:

proc myobject {method args} {
    # FIRST, determine the actual command.  For this example we'll
    # skip the error checking, and simply assume that the proc that
    # implements a method is myobject_$method.

    set cmd [linsert $args 0 myobject_$method]

    # NEXT, call the command using uplevel 1; this effectively removes
    # [myobject] from the call stack, so that upvar and uplevel
    # will work properly in the method body.

    set retval [catch {uplevel 1 $cmd} result]

    # NEXT, if the return code is anything but "ok" we need to rethrow
    # it--this time, we're removing [myobject] from the call stack on
    # the return trip.
    #
    # Note that error returns need a little extra help.

    if {$retval} {
        if {$retval == 1} {
            return \
                -code error             \
                -errorinfo $::errorInfo \
                -errorcode $::errorCode \
                $result
        }
        return -code $retval $result
    }

    return $result
}

Here’s a sample method; it takes a variable name and increments the variable.

proc myobject_incr {varname} {
    upvar 1 $varname theVar
    incr theVar
}

Here’s how it looks in practice:

% set a 1
1
% myobject incr a
2
% set a
2
%

Note that it took me about an hour to write myobject and prove to myself it really handles all of the corner cases; the code and the test cases are available at http://www.wjduquette.com/igloo as rethrow.tcl. It isn’t quite perfect, either; the stack trace you get in the case of an error thrown in a method body is way ugly, and reveals too much information about how the ensemble is implemented. (This is one of Snit 1.2’s real warts.)

Back to Namespace Ensemble: The beauty of namespace ensemble is that it handles all of this for you, and it does it in C code so there’s little run-time penalty. And because of the brilliant design of the -unknown handler, it works for unknown methods too.

Unknown Methods in TIP 257: So, how does the TIP 257 unknown method stack up against namespace ensemble? The answer, unfortunately, is “badly”. You’re essentially back in the bad old days: the unknown method has to do all of the work myobject does in the example shown above. The resulting code works, but it’s ugly and slow. Since automatic delegation of unknown methods is one of Snit’s strong points, and since one of the motivations for building Snit on TIP 257 is speed, this is a serious concern.

Here’s some example TIP 257 code that shows, in classic Snit style, how to make a dog wag its tail. For simplicity, I’ve omitted code that exercises the corner cases. The full code is available as unknown1.tcl.

::oo::class create tail {
    method wag {speed} {
        if {$speed eq "fast"} {
            return "Wag, wag!"
        } elseif {$speed eq "slow"} {
            return "Wag."
        } else {
            return "Whine!"
        }
    }
}

::oo::class create dog {
    # A dog has a mytail component
    constructor {} {
        my variable mytail

        set mytail [tail new]
    }

    # Let's delegate unknown methods to mytail.
    method unknown {methodName args} {
        my variable mytail

        # FIRST, determine the actual command.
        set cmd [linsert $args 0 $mytail $methodName]

        # NEXT, call the command using uplevel 1; this effectively 
        # removes [[self] unknown] from the call stack, so that upvar 
        # and uplevel will work properly in the method body.

        set retval [catch {uplevel 1 $cmd} result]

        # NEXT, if the return code is anything but "ok" we need to 
        # rethrow it--this time, we're removing [myobject] from the 
        # call stack on the return trip.

        if {$retval} {
            if {$retval == 1} {
                return \
                    -code error             \
                    -errorinfo $::errorInfo \
                    -errorcode $::errorCode \
                    $result
            }
            return -code $retval $result
        }

        return $result
    }

    # Make unknown invisible to uses
    unexport unknown
}

Can we do better than this? Possibly. TIP 257 supports a minimal delegation mechanism that allows any method to be forwarded to any arbitrary command. The semantics are similar to those of interp alias or namespace ensemble, which is just what we’re looking for. The problem is, in order to forward an unknown method we have to know the method name–and we don’t know the method name until after unknown is called. This suggests a hybrid scheme:

  • When an unknown method is called, dispatch it “the hard way”.
  • If the call succeeds, the method is no longer unknown; forward it explicitly, so that subsequent calls do it “the easy way”.

We can do this by adding a single line of code after the [catch] statement in the unknown method shown above: (See unknown2.tcl)

    if {$retval != 1} {
        oo::define [self] self.forward $methodName  $mytail $methodName
    }

We now do things the slow way the first time only; after that, we get namespace ensemble-like goodness and speed. In my benchmarks, the hybrid version is just over twice as fast on average.

Some Warts: Both methods of delegating unknown methods have some warts, especially in the area of error returns. The wag method shown above requires one argument. Let’s look at some error messages when we implement delegation the naive way:

% source unknown1.tcl
% spot wag fast slow
wrong # args: should be "::oo::Obj6 wag speed"
% spot wag
wrong # args: should be "::oo::Obj6 wag speed"
%

Here the error message is written in terms of the tail component, ::oo::Obj6, which is seriously ugly. If we use the hybrid approach we see things like this:

% source unknown2.tcl
% spot wag fast slow
wrong # args: should be "::oo::Obj6 wag speed"
% spot wag fast
Wag, wag!
% spot wag fast slow
wrong # args: should be "spot wag"
% spot wag
wrong # args: should be "spot wag"
%

First, we get the “naive”-style error messages until the first time the method call succeeds. This is annoying. Then, once the method had been forwarded we get an error message that gets the object right but omits the argument names. One doesn’t know whether to laugh or cry. I choose to look at this as a bug in TIP 257.

Conclusions: The hybrid approach is definitely the preferred way to delegate unknown methods, given the current implementation, although the automatically generated error messages are unfortunate.

It would be spiffy if the unknown handler could do this:

    # Let's delegate unknown methods to mytail.
    method unknown {methodName args} {
        my variable mytail

        # FIRST, determine the actual command.
        set cmd [linsert $args 0 $mytail $methodName]

        # NEXT, forward it.
        oo::define [self] self.forward $methodName  $mytail $methodName

        # NEXT, ask the object to retry the call.
        return -code retry
    }

The new return code retry would tell the caller to retry the call–that is, to call it exactly one more time. This would give us similar semantics to namespace ensemble‘s unknown handler.