Thursday, May 30, 2013

The Widgetmakers' Frankenwidgets

Partly out of necessity, partly out of interest and habit, and partly in the name of science, I've spent a considerable part of the past two or so years building widgets. Examples of applications containing a plethora of such widgets are shown below:

FileShell - An experimental, (attempted) cross-platform file browser, built as a test bed for a number of things I've been developing for my research work (and also to host a few features I've personally wanted to see in a file browser). Nearly every single widget shown here has had some amount of bashing applied to it. Toolkit: Qt / PyQt.


G6 Feedback - There are number of widgets on display here, notably the "EditableLabel" (when you hover over a textual label that is currently acting as a placeholder, a box will be drawn around it while you hover, and it will be replaced with a textbox where you can actually change the text when you click on it - e.g. "Click to set question"), and time-selector combobox (for either selecting a preset time, or manually specifying in terms of the units you care about). Plus, the fact that all of this was done in an antiquated toolkit meant that everything needed an extra level of polish on it.  Toolkit: Java Swing (Nimbus theme)



For a long time, I've affectionately referred to them as "Frakenwidgets", since I'd often start with some fairly pedestrian widgets (offered in practically every GUI toolkit worth its grains of salt) which were the closest equivalents (to the desired widget) I could find, and then proceed to conduct open-heart brain surgery on them to patch in a range of capabilities. These would range from simply getting them up to modern standards of behaviour (e.g. placeholder support, a plain old context menu, sane padding and/or alignment), to going all out and adding in additional widgets, timer-delayed effects and actions, special themeing, fancy tooltips (e.g. multiple tooltips at once, interactive tooltips, long-lasting tooltips, multi-line tooltips with strategically used bold/italic fonts to ease scannability, etc.), custom drawing functions + event handlers + data models.

Of course, the Object-Orientated nature of these toolkits quite naturally seem to be designed for and even encourage this type of development (after all, reusable, modular, and extensible code are all founding principles of the whole OO software engineering movement). But admittedly, every time I embarked on yet another one of these massive hack-fests, or perhaps more accurately, when I've been floundering up to my eyebrows for the past 4-5 hours on debugging some or other focus-change related issue which really "should" have taken less than 3 or so to fix (*), there has always been this nagging feeling that this kind of hacking I've been doing is some pretty nasty stuff; that these frankenwidgets are and the ways in which they've been implemented are really dirty, ugly kludges that we really shouldn't be unleashing on unsuspecting users.
(*) Argh! Focus change related issues are THE most annoying issue - followed closely by sizing/alignment issues - as you often start to find yourself having to either reinvent the focus grabbing cycle to prevent an obscure textbox halfway down a form grabbing focus from your all-important-super-widget-which-was-based-on-a-label-or-plain-widget, or things mysteriously don't gain focus or keep holding focus as long as you'd like them too. Occasionally too, you get a widget that has overstayed it's welcome - i.e. one of those stray tooltips that you may have encountered at some time or other that have somehow managed to hang around hours after the original spawner has been disappeared.

With this in mind, it was quite a (refreshing) surprise to read a post of a few months ago on Google+, reportedly from a guy who has worked as a core dev for the Chrome Browser, that said that the "omnibox" (i.e. that addressbar/search/suggestions widget thing running right across the top of that browser) was an example of a frankenwidget, in much the same tradition as my own humble creations (NOTE: in the post, he refers to this as being a "common-pattern" for C++ developers working on Windows "to use a various platform-native types and controls to implement your UI", which it seems in this case includes widgets within that UI).

Surprise is probably too gentle a term to describe my reaction to this (which was actually a bit closer to shock TBH), especially upon learning that 1) the omnibox is actually built on top of a "RichEdit" textbox control (at least on Windows), and 2) MsWord basically uses just a slightly embellished version of this. Why the shock you may ask? Well, perhaps it was just past expectations, but when you typically think of a RichEdit control, you're typically going to lean towards imagining Wordpad (or some crappy knockoff of it that you find in the examples section of any good book or tutorial covering certain toolkits). That is, a widget allowing multi-line text that can be formatted (usually shown by default in size 16, black, Times New Roman), roughly rectangular (320x240 or equivalent ratios), with really rough rendering, and hardly any polish/fanciness for anything else (such as page margins/borders, region indicators, etc.).

Hence, to think that a crucial component of a UI in wide use today across the world was effectively clobbered together on top of a widget like this, complete with a bunch of other patchy hacks that even the author acknowledges sound pretty nasty and "out there" (i.e. "hotlinking" in custom functionality from a foreign DLL file into the drawing and event handling internals of this widget), is both frightening and fascinating at the same time.

Like fellow travellers stumbling along offbeat rocky roads one after the other, there was a certain "knowing feeling" from reading through some of the hurdles they went through to implement these things. In particular, the following quote (which I've slightly modified and generalised) is a particularly telling one, which anyone who's ever attempted to do this sort of hacking will have encountered at some stage or other:
[Base Widget]was basically built to be used in [Pedestrian Use Case], and never really tested in [The Special Mode That You're Now Trying to Use] mode, which is what we used for the [Frankenwidget Name Thyself].  This led to some interesting bugs, the most annoying of which I called the "phantom newline".    -- Peter Kasting (with modifications by me highlighted)
Basically, whenever you start pushing toolkits beyond their original intended use cases, you start getting a bit of fightback. These often come in the form of "the phantom bug" that is nigh impossible to truly defeat as it is quite ingrained into the core of the implementation. These may be hardcoded colours that cannot be modified at all (*ahem* GTK!), randomly overflowing or unnaturally clipped labels combined with spacing that's too sparse or too tight (*ahem* Qt when manipulating the QListView widget into the form seen in the Fileshell screenshot), invalid focus order/chaining grabbing (*ahem* Java Swing), unwanted or missing padding (almost every toolkit at some or other point in time), selection and drawing glitches (*ahem* Java Swing), etc.

But perhaps the most painful of them all is usually when trying to recreate the "document tabs" pattern seen in all modern web browsers (and shown in both examples above). It turns out that it's an absolute nightmare to try to implement this in almost every toolkit, as they all have different glitches:
 - Java Swing:  It's impossible to get the "x" close buttons to stick to the RHS and everything else to the LHS (i.e. they all tend to clump together in the middle). This is at its worst when there are too many tabs to fit in one row, and all of a sudden, you get a few really wide tabs that take up the whole row (and contents all smooshed together in a clump in the middle of those tabs) while all the other tabs are all crowded together into a single line. Rearranging tabs also needs to be implemented from scratch, and the "add tab" button/tab can only be done using some hacks involving fiddling with the set of tabs to convert the current tab into a real tab when it is activated (and creating a new "new tab" for the purposes of still having an "add" button).

 - Qt: Unfortunately, despite a lot of other things being really nice in Qt, this is one of the areas it really drops the ball. Although it handles the close buttons well (and natively IIRC), it's only really possible to have the "Add tab" button as a fixed button assigned to either the top left or top right corners. That's perhaps not so bad in the long term, but more problematic is that the tab rearranging shows some glitches, especially when a tab is dragged past the region occupied by all current tabs, in which case it gets abruptly truncated and disappears into the ether, only to snap back into place when the mouse is released.


Another notable dilemma you'll encounter when doing this sort of work is always: "Do I really want to implement ALL of that from scratch?". As Peter says,
"Drawing my own ... would be extremely hairy, so that was out"
At some point, you're going to have to make this tradeoff between having full control (and hence, proactively fending off latent bugs in the widget you're piggybacking that would normally not be triggered as "no one in their right mind" (yeah right!) would think of using that combination of features on that widget) over the thing, allowing potentially unlimited creative freedom (err... well, at least as far as your own hacking skill will take you to figure out how to implement these things)  vs just having something out the door today vs in a month's time, and still be able to pivot easily if it turns out that you need to modify the mechanics/presentation of the thing by embellishing it.

This very situation played out earlier today, when I was working on a new feature which would have involving popping up a little popup list. At first, I contemplated implementing my own custom list widget, with its own simplified drawing so that it could draw out a tree-list but without all the icons and expanders. However, upon a bit more pondering, the appeal of this approach waned when I started to think about how to position the labels, the custom theming work that'd need to be done on those to get them to respond and be laid out sensibly. So, once again, I resorted to abusing some of the standard list widgets (QListWidget to be precise). TBH, I'm often far too lazy to be bothered using the ones which allow for proper model-view separation - i.e. a separate QAbstractItemModel subclass for my datamodel + QListView or QTreeView to render that - as those take a lot of work, and usually end up a bit sluggish too! This situation turned out to be a blessing in disguise, as after a few discussions, we decided to change direction with the display of this popup, and start including a bit more highlighting, indentation, centered drawing, and so forth. Because I'd used a standard list widget earlier, I was able to leverage its existing capabilities for this sort of thing, allowing me to quickly hack up these things (extra tidbit - I was extra lazy here, and decided against a proper/full-blown tree, so to preserve the tree-like appearance of the list, I simply appended spaces to the names of items to get them to have the desired appearance). And most importantly, it ended up looking pretty darned flash within minutes. Woot!

At this point, I should also point out the importance of choosing a good base toolkit to base all your work in. A good/nice toolkit will ideally do/have the following:
  1. Have a decent set of widgets - including all standard ones you're likely to need, and then some - that look and behave as you'd expect such widgets in modern GUI environments to behave. By default. With no or almost no intervention on your part to ensure that a whole bunch of settings allow them to have acceptable modern behaviour.
  2. A consistent and flexible API - Consistency helps you to learn how to use it, but also to predict how to use unfamiliar parts before you've gone there before. Flexibility here really refers to how easily you can stub in your own extensions, and have these act as part of the widget (essential if you want/need to do Frakenhacking). See The Little Manual of API Design for more details.
  3. Clear documentation and an active community of developers using/working with/on it. These will help you immensely if/when you get stuck, especially when starting out. If these are in place, you'll be able to school yourself up on a few techniques which work in this space, and gradually you'll be making up your own based on the leg-up the extensive examples you've seen already have given you.
  4. Is a joy to use. Really, if the thing sucks before you get started bashing extending it (i.e. in the default state, when you're using it "as intended"), then things can only really go downhill from there. There are notably exceptions (i.e. comparing Java vs Python for toy examples as opposed to full blown applications are two different matters), though a painful experience in the best of times will typically lead only to immense pain when trying to find a way out of a tricky situation when extending things.
What fits the bill then?
I'm currently biased towards Qt (as expected) for real production work. However, as I've pondered over the contents of this post for a few days now, I've come to realise that there might still be value in cutting your teeth on these techniques with Java Swing, as that old dinosaur literally forces you to customise the crap out of it to do anything sane, often with many different dedicated component, which holds you in good stead for when you need to do this sort of thing in future (if you need to at all). It also helps you truly appreciate a good toolkit when you see one. 

1 comment: