DWI is an experimental platform for exploring ways of making application
development simpler. It is still hard to write large, complex,
multi-user, data-driven (SQL-backed) applications. It doesn't matter
if your development platform is the web and Enterprise Java Beans,
C# and .net or Mono, or whether its the Gnome/GTK or KDE widget set
and the Linux desktop; its still just plain hard. DWI is an effort
to change this situation. DWI currently offers a simple way of
developing data-driven (that is, SQL-backed) Gnome applications
(designed with the Glade GUI designer). It does this by avoiding
"programming" (or at least, "traditional programming" in C. C#, perl,
python or any other "traditional" language), substituting instead
a configuration-file like format that defines how various GUI elements
should be hooked up to various objects (such as GLib GObjects)
or SQL fields and tables. The current primary effort with DWI is
to provide a number of well-documented, easy-to-understand, working
examples that show how to use DWI. These examples currently
include a stand-alone bug-tracker-like application, examples of
integrating with existing GTK applications, and an example of
hooking up a Glade-designed interface to a GLib GObject with almost
no C programming at all (assuming you have a GLib GObject already
handy :-).
What is a Data-Driven System? To better illustrate the idea of a "data driven" system, think of an application for a credit card (or possibly a purchase of an online airplane ticket). You have to fill out your name, address and other info: this is the "form". If its on the web, then you click a button and the values of these fields are collected and whisked into a database on the web server. Then, based on your responses, and some calculations, and some other data in the database, a new web page is computed, and the resulting "report" is displayed to you (this report may in fact be a form asking for additional info). The bouncing between forms and reports can be quite complex: to purchase an airplane ticket, you start by filling out a form with the dates you want to travel, and the place you want to go. The first "report" is a listing of flight numbers, departure/arrival times, and ticket prices. This "report" is also a "form": you can click on a checkbox to select a flight, and then proceed onwards to make a purchase.
In fact, there's a huge class of applications that are "data driven" in this same way: these range from "bug trackers" such as Bugzilla, parts and inventory trackers, customer trackers, on-line shopping systems, systems for processing government paperwork (such as getting a new water meter installed, or getting the building permit), financial accounting, purchase requisitions, and the like. You fill out a form, there is little or no computation done, except to save th data, and the result of filling out the form is another form that appears on the bosses desk, or on the shipping clerks desk, or on the loan officers desk, etc. This large class of applications share one common trait: they can be described in a mostly declarative fashion. Pressing this button causes data to be copied from here to there, with little in the way of hard computation.
There are two ways that such applications are typically written: hard-coded and arbitrarily flexible. Bugzilla is a great example of the "hard-coded" style: you can customize the list of projects that it presents, and a (large) number of other aspects, but, in the end, Bugzilla can never be more than a bug tracker. Now, if you could get Bugzilla to store price information, then you could use Bugzilla as a purchase requisition system: instead of opening a bug, you ask for a new chair; instead of assigned the bug to a programmer, you assign the chair request to the boss, who will accept or decline, etc. MS Access represents the "arbitrarily flexible" approach: you can design any form and hook it to any database table; if you don't like your first shot at this, you can start all over again.
Why is it so hard to write applications? There have been many contributing technologies that have helped to make this kind of programming easier; but they all have their shortcomings. We'll try to review some of these here. MS Access made a lot possible, but it was fragile, wasn't really multi-user, was ugly, and didn't scale well. It couldn't talk to other systems. CRM/ERP systems, such as those from Siebel or PeopleSoft, get around these limitations, but are incredibly complicated and stunningly costly. The web (HTML and HTTP) revolutionized the creation of these kinds of applications, by allowing the programmer to run the database on the same physical box as the web server, and thus gave the web programmer multi-user capabilities "for free". However, the web did not provide any sort of high-level or declarative language tools: one coded in perl or PHP. Creating a web application is a lot like creating a desktop application; one still has a huge amount of mucking-around to do; its just that one gets "multi-user" almost for free on the web, whereas "multi-user" is still hard for a typical Gnome, KDE (or Microsoft) desktop app. For this reason, and for this reason alone (and really for no other reason), (and really that's the truth, we pay no lip service here to other bogus theories here), almost all large, complex applications are being developed for the web, and not for the desktop.
There are two other types of technologies that programmers look to when trying to code these types of applications. Desktop developers have traditionally cast their eyes upon CORBA, (Sun) RPC, and SOAP as a means for getting multi-user abilities. The idea is that a desktop application, when properly retro-fitted with CORBA, RPC or SOAP, could talk in some kind of language to some server, or even share data in some P2P type way. Indeed, SOAP is an important part of the Microsoft's .net framework; it is an important part precisely because Microsoft is a desktop application heavyweight.
The other technology that programmers look to is innovation in programming languages. When one programs large, complex object-oriented applications for a while, one eventually discovers the limitations of traditional languages like C and C++. What I refer to here, in particular, is the idea of "object introspection". With object introspection, you can ask an object to describe itself, thus making it much easier to ship that object over the net (marshal and unmarshal), store it in a file, and etc. It is no accident that both Java and C# both explicitly support introspection However, although these languages are quite innovative in this regard, they offer no panacea, none whatsoever, to the programmer of data-driven applications. This is because, at their root, they are still highly procedural general purpose programming languages, coming from the same stock as C, C++, perl or python. They don't really make it fundamentally easier to create data-driven applications.
A huge premise of DWI is the observation that data-driven applications can be developed using a declarative rather than a procedural language. Lets take a moment to explain what this means. For example, int i; is a declaration: it declares that i is an integer; its static; it doesn't cause any computation to be done in and of itself. for (i=0; i<100; i++) is a procedure: its a set of instructions to perform a certain activity. Declarations need not be boring like int i; C structs and C++ classes are declarations as well, as are function prototypes. Although in the above examples, declarations seem static, they need not be; one can use declarations to describe something dynamic. For example, a signal handler that has been set up is a kind of a declaration: until the signal arrives, nothing happens. It is merely a statement that if a signal were to arrive, then the signal handler should be invoked. DWI uses declarations in this kind of a way. The programmer makes a declaration that this-and-such widget is linked to that-and-such database table column. When a signal arrives, for example when the user presses a button, then, and only then, under the covers, the contents of the widget is copied to the database. In DWI, there are no variables, no assignments, no for-loops or iterations. There are, however, descriptions of linked chains of events, such that a simple signal can set off a rather complicated chain of events that eventually result in the GUI being updated. Although, in the end, declarative languages are not as powerful as procedural languages, they are still stunningly powerful and flexible and capable. For example, a purely declarative language can still cause a column of numbers to be added or graphed, and can be used to search for all employees whose first name is "Bob" and whose salary is more than $10,000. The premise of DWI is that a simpler, less complex declarative language is enough to code up most typical data-driven applications, and that a good interface to traditional procedural programming languages pretty much takes care of the rest.
Note that XML and HTML are essentially declarative languages, and so is "richtext" and the Abiword word-processor file format. Part of the power of a declarative language is that it is easy to build a WYSIWYG editor for a declarative language. By contrast, it is impossible to build a WYSIWYG interface to a procedural language. Take, for example, postscript, which is commonly used to do graphical layout for printing on paper. Postscript is a procedural language: it is not hard to write a postscript program that will compute the first million digits of Pi. One cannot create a WYSIWYG page layout program that uses postscript as its native language: if you feed it the postscript for computing Pi, it would have no idea what that means. You can't change the font, color or position of Pi; it is Lewis Carol nonsense to even think of it. The statement here is deep: if almost all data-driven applications can be described with a declarative language, then almost all such applications can be created with a WYSIWYG editor. You really don't need to be a programmer, or write a single line of C#, Java, Python or Scheme code create these apps. You just need that WYSIWYG editor. This may sound like magic, but its not: the MS Access program is a crude, primitive form of this kind of WYSIWYG editor. Glade is essentially in the same ballpark: Glade is WYSIWYG, and in the widget signals panel of glade, you can hook up signals to (a very limited) set of actions. In a certain crude sense, the goal of DWI is to vastly expand the set of pre-built signals that can be hooked up to a widget using Glade.
At this point, this system has enough features to be adequate for creating form-editing and reporting applications. Multiple SQL database vendors are supported through ODBC (http://www.iodbc.org) or libdbi (http://libdbi.sourceforge.net) drivers. There is very simple db-driver infrastructure so its easy to support for additional SQL API's. The system supports all of the basic Gtk widgets, and an additional half-dozen Gnome I/O widgets, such as GnomeDateEntry.
DWI is powered by an 'engine' that has some fairly generic procedures for mapping 'fields', such as SQL table columns or widget values, between each other, and also between other things, such as objects, hash tables and etc. This engine is currently being expanded so that it becomes possible to easily add support for all kinds of new object systems: i.e. for the engine to become a generic re-mapper between not just SQL and GTK but between many different types of object systems and data sources/sinks.
Built on top of this engine is a DWI application that parses an XML-based file, the "DWI file", that describes the connections between glade widgets and database tables. Currently, the only way to create DWI files is by hand. Unfortunately, this can be a fairly long and laborious process itself, especially when creating something a bit more sophisticated. In the future, we hope to have an extension to Glade, or possibly an extension to a database-browsing tool that will allow you to graphically make such connections. (Work has begun on such a tool, written in DWI itself).
The grim reality is that DWI won't ever become popular without a graphical designer. Although fairly complex apps can be readily created using DWI, it does have a non-trivial learning curve. When we say "can be created quickly", we mean "days" or "weeks", as opposed to "months" for traditional database application development cycles. Graphical RAD tools have a way of being brainlessly pleasant to use, and give the impression of an even faster development cycle, even though the learning curve is identical.
Note that the design of the XML format is sufficiently generic that it is not directly tied to Glade. It is envisioned that it could be used to create data-driven web pages. That is, Glade is currently the only GUI driver, but other drivers for other GUI's should be possible.
These screenshots show a very basic note-taking bug-tracking-like application. The screenshots are not of DWI itself, but of a simple application created with it. Actually, what is shown was created by Glade; you'll have to take my word that there's a database behind this. Note that if you have good glade skills, you can create prettier interfaces than this.
Here's a screenshot of DWI using the columned-tree widget. Again, not much point to this, other than to show that DWI does support displaying the results of SQL queries as columned trees (which can be tricky).
To create a DWI application, you need to know SQL, and you need to be able to write XML files. You do not need to program in C or any other language. The dwi-run interpreter will run the entire application based on the DWI file. The testbasic.dui file describes the basic note-taking application whose screenshots are shown above. This example file works as a tutorial; it contains almost complete documentation of the file format.
The Bicycle Ride Calorie Calculator directory contains the example source for a DWI-to-GObject usage example. This example creates a GObject using GOB and hooks it up to a Glade-designed GUI using DWI. The example sources shown there is DWI file, the GOB file and the C glue-code.
A graphical editor, to automate the creation of these XML files, is being developed; its currently in the pre-pre-pre-alpha stage. Its being written in DWI itself, and so should prove that DWI can handle 'real-world' apps (as well as simplifying the bootstrapping of the editor).
The HINTS file provides some HOWTO-type suggestions for implementing application features, and should give you a hint of what is possible in DWI.
The DWI distribution also includes an example of how to query for a username/password, for database login, and an example of how to use well over a dozen different kinds of widgets within DWI.
Downloads are here; but see also the DWI Sourceforge project site: it may have better bandwidth. The current version is 0.5.0. It works; we could call it 1.0, but more is planned before we call it that.
DWI is now located at SourceForge; The DWI Project Site is a good place to report bugs and make other postings and comments.
Note that the DWI design is fundamentally "declarative" in that it essentially is like a declarative language. For example, an "SQL Trigger" is a declaration: by itself, it doesn't do anything. However, it does describe what should happen in a particular situation. DWI is similar, in that it describes what should happen when a user presses a button. DWI is a way of describing the chain of events, how things get hooked up to things: a button press can cause the value of a widget to be used in an SQL statement, the results of which are used to fill in another table widget. This description or "declaration" of the chain of events makes DWI very very different than procedural languages such as C, FORTRAN, Python or perl. The point behind this is that its a lot easier to describe the chain of events, than to write programs in C or perl (or scheme for that matter). The 'ease of use' of DWI depends on it being declarative in nature.
The declaration of the chain of events is done using XML. (It might be interesting to use a non-XML way of describing the chain of events, but XML is adequate, if a bit klunky.) The "run-dwi" evaluator reads in this static declaration file, and sets up engine structures that correspond to the XML declaration. Then the evaluator goes into the main loop, and signals and events make everything happens according to the pre-determined definitions in the XML file. There is no run-time interpretation. There is no 'just-in-time' compilation. It just runs along the fixed paths, where each path had been pre-defined in the XML file, and each run-through is initiated by the user clicking or typing in the GUI.
A DWI "application" consists of one or more <database> sections, each of which consists of one or more <window> sections. The application windows themselves interact with the user as a <report>, displaying data, and/or as a <form>, collecting user input. Here is an example file.
A brief but important note about the words "report" and "form". These two words provide the key conceptual underpinning for the DWI XML file format. DWI distinguishes between 'data sources' and 'data sinks'. When one writes a DWI file, one declares the 'pipes' that run between GUI data sources/sinks and database sources/sinks. When a 'pipe' connects GUI data sources to an SQL statement, this 'pipe' is given the arbitrary name "form". The 'pipe' that connects a database record-set (query result) to GUI elements (sinks), this is a "report". For programming architectural design reasons, these pipes are always one-way. But, of course, at the user interface level, you want to have any given widget to possibly be both a source and/or a sink. So, for instance, I can have a GtkEntry act as a source for one (or more) "forms". That just means that the value of the GtkEntry is read, and used to formulate an SQL query. And I can have that same GtkEntry also be a 'sink' for a report: for example, a report that puts today's date into the GtkEntry. So any given widget can be a be a source or sink (or both, several times over, in a one-to-many relationship). However a "form" always inputs, and a "report" always outputs.
A DWI application contains a rudimentary mechanism for handling application state: a global set of key-value pairs (implemented as a hash table). You can create a key-value pair at any time, and later use the values in database queries or in reports. Most form-entry/reporting applications will typically not need to maintain state; more complex applications will typically use global state to map tree-like/hierarchical structures into the database. In addition to the global key-value storage, values can be set and retrieved on individual widgets (using the widget data field).
Whenever a value is obtained from a widget, with the intent of putting it into the database, or from the database, with the intent of putting it in a widget, it can be passed through a "filter". The filter transforms the value in some way, such as stripping out leading whitespace, checking for null, or putting it through a lookup table. Currently, only a very minimal set of filters have been defined; although more are envisioned. (Basically, what's there seems nearly enough, although a good date-format filter would be nice.)
Many Gtk and Gnome widgets can be used for user input and display, such as GtkEntry for string input/output, GtkLabel for string output, and GtkCheckButton displaying/getting boolean values. Most Gtk and Gnome widgets for which it "makes sense" to set or read a value are supported. An effort was made to set values that 'make sense': thus, setting the value for a GtkCheckButton sets the state of the button as checked or not checked, instead of setting the button label. However, setting the value for a GtkButton sets the button label, since it doesn't 'make sense' to set anything else. Note that some 'unusual' I/O widgets are supported: for instance, the window title can be set on GtkWindow, as well as the label on a GtkFrame.
Sometimes it is useful to set other attributes for widgets, other than the 'obvious' one. To do this, the GtkArg mechanism is supported, thus allowing any and every attribute on any and every widget to be set. Thus, for example, the height of a widget can be set by specifying arg="GtkWidget::height", and using a value pulled from a database. More useful is perhaps making a widget conditionally sensitive to input (grayed out or not grayed out), with arg="GtkWidget:sensitive". The 'testwidgets.dui' example shows a large variety of widgets, as well as the usage of args.
Note also that the GtkCList and GtkCTree widgets are supported for multi-row input/output. The GtkCList ("columned list") widget is a basic, simple table, and is 'automatically' filled by DWI. The GtkCTree widget is a hierarchical columned tree; its a bit harder to fill out, as each sub-tree requires a separate database query.
Currently, DWI ignores data types, and treats everything as a string. This may change in the near future. Since SQL database table definitions are typed, there is a natural pressure to put this kind of typing into schema designers and the higher level app (such as MS Access). So far, however, there seems to be no particular need for typing within DWI. So we've resisted it.
"Meta" operations, such as table browsing, are supported, but barely. Other operations, such as dynamic form generation, are currently not supported. These "meta" operations are needed if one wants to build a database browser (or a graphical DWI designer!). Almost all "normal" data-driven applications do not need these features: tables and forms are predefined, and are tailored for the application. However, because they are needed for a DWI designer, they are mentioned here. The current plan of attack is to convert all DWI internal structures to GObjects, so that they can be treated as if they were widgets themselves. This adds a fair bit of mind-bending abstraction to the design and use of DWI, and so we are proceeding slowly to minimize mistakes.