XForms Everywhere

Developers have known for a long time that unit testing can make a difference in software. In particular, solid unit tests help increase its robustness and make it easier to refactor source code. Extreme Programming (XP) in particular considers testing a core piece of its lightweight methodology (but many developers do unit tests without doing XP).

Orbeon Presentation Server (OPS) hasn’t always had many unit tests, but it was decided at some point that tests were necessary, and we started building a test framework for it. We hate the Not Invented Here (NIH) syndrome in software, and it is therefore slightly incorrect to say that we built a new framework. JUnit is essentially designed to run tests on Java code, not on XML, so we created a layer, leveraged by JUnit, that allows describing the tests in a way that is much more natural to OPS, and at the same time, we can still leverage JUnit and its front-ends, like the one integrated in IntelliJ IDEA.

One of the concepts at the center of OPS is that of XML processor. It is a software component, with the particularity that its interface to the outside world consists of incoming and outgoing XML documents. XML processors are typically used within XML pipelines described with XPL, the OPS XML pipeline language. So one common test scenario consists in feeding an XML processor with one or more XML documents, and checking that the XML processor returns one or more expected XML documents.

This idea is at the core of the ProcessorTest class, which extends the JUnit TestCase class. ProcessorTest reads a configuration file called tests.xml that describes all the XML-based unit tests. tests.xml XIncludes other files, so that tests can be nicely grouped by the type of functionality they are testing. One of the simplest tests you can imagine is this one:

<test description=\"Identity\" name=\"oxf:identity\">
    <input name=\"data\">
        <root>
            <a/>
            <b/>
        </root>
    </input>
    <output name=\"data\">
        <root>
            <a/>
            <b/>
        </root>
    </output>
</test>

It just makes sure the Identity processor produces on its output the same document it receives on its input, for a very simple input document consisting of just a few elements. From there, you can build much more complicated tests based on other OPS processors, including testing entire XML pipelines written with XPL.

Overview of XML Processor Testing

The story is not over, because there are other scenarios that need testing in OPS, for example caching, a tricky piece of functionality. You want to be able to tests such scenarios as “If this file hasn’t changed on disk, make sure the cached result is used, otherwise redo the processing.” To do so, a special Test Script processor was devised.

The Test Script processor is an XML processor like any other in OPS. Its particularity is that it is able to run specific test scripts operating on the guts of OPS itself, in particular the XPL engine.

While XML processors can perform operations in a vacuum (like for example doing a pure XSLT transformation of an input to an output based on a stylesheet), often XML processors depend on external factors. In OPS, this is represented by the ExternalContext interface, which abstracts requests and responses so that XML processors can work in Servlet, Portlet, command-line, and embedded contexts.

To test scenarios depending on such an external context, one could launch a Servlet container in a controlled environment, and run requests against it. This is not a bad way of proceeding, and it is in fact the ultimate way, in that it reproduces actual running conditions. The disadvantage is that it is cumbersome to setup.

So the Test Script processor uses a simpler method, which consists in building internally an ExternalContext instance from an XML document describing what that context looks like. This is actually quite cool, because with a simple XML document, you can describe an HTTP request as seen by OPS. Setting the request can also be scripted by the Test Script processor, so you can do things like testing the Request generator. For examples using the Test Script processor, check out the source code and look for tests-cache.xml.

The OPS test framework does not yet support producing an XML document representing the ExternalContext response, but this can be added fairly easily. In addition to that, some scripting facility that allows transforming a response into a new request will allow creating test scripts that simulate entire request / response cycles.

The number of OPS unit tests has rapidly grown over the last six months and today there are about 120 unit tests. This is a good start and has already caught bugs before developers check in their code, and more during the continuous integration builds. Because yes, you really want to have a system like Anthill continuously building your application, and making sure that a build that does not pass the automated unit tests suite is considered broken.

Of course, many more unit tests should be added to OPS over time. Some areas of functionality are not yet covered. But developers have now taken the habit of writing tests whenever they fix a bug or implement new functionality. We have much more confidence that the code we write does the right thing, and does not break existing functionality. This is simply invaluable.

A 3-Tier Architecture (3TA) is traditionally used when building web applications. It makes a logical separation between the presentation layer, the business logic layer, and the database layer. For instance, on the Java platform, the presentation layer could be implemented with JSP and JSF, the business layer with session EJB, and the data layer with entity EJB or Hibernate. It is important here to note that the 3TA introduces only a logical separation between layers. The 3 layers together make a given application, which gets deployed on a given server or cluster, and built by a given team. The 3TA can be seen as an implementation of the divide and conquer strategy for the use of the application architect, as it lets him think of his application in term of layers.

Most companies have been implementing applications following the 3TA for years and have now a collection of applications that all behave and look differently, and are hard to integrate. The 3TA introduces a welcome division between the presentation, business logic, and data layer, but is damageable in the sense that it leads architects to think in term of application silos.

3-Tier Architecture

The Service Oriented Architecture (SOA) builds on top of the 3TA and addresses its shortcomings. Instead of looking at the IT infrastructure as set of application silos, SOA looks at a set of services and applications. Services implement some kind of functionality and are used by applications and other services. Services communicate between each other and with applications by exchanging XML documents. Comparing SOA to 3TA, SOA applications correspond to the 3TA presentation layer, while SOA services correspond to the 3TA business logic and data layers.

Service Oriented Architecture

With SOA, all the complexity of the system is encapsulated in coarse grained services and applications are kept extremely simple. In fact, the only concern of applications is to display XML data they get from services and to send XML data to services based on user input.

SOA does not compete with 3TA, but introduces three additional requirements:

• Completely separating the presentation layer from the other layers.
• Breaking down the other layers by coarse grained functionality to make services.
• Having the services communicate with each other and with applications by exchanging XML documents (typically using web services).

The Mozilla project recently announced the first beta version of their XForms support for the Mozilla and Firefox browsers.

As the W3C puts it, XForms is the next generation of web forms. It aims at replacing HTML forms within XHTML, as well as providing other languages such as SVG the ability to handle forms.

XForms was largely intended by its authors to run on the client, that is, within a web browser, as HTML forms do today. It offers benefits such as:

• Clear separation between the data captured and the presentation (separation of concerns).
• Flexible, structured XML-based data model. Define your data model as an XML document, then create forms accessing the data.
• Built-in facilities for validation and data integrity based on XML Schema and/or assertions.
• Requires much less, or no scripting at all for many common scenarios that do require scripting with HTML forms. In other words, XForms’s approach is very declarative.
• Ease of internationalization.
• Support for a variety of devices.

As we know, after Netscape succumbed to Internet Explorer back in the 90s, client-side web browser technologies have largely stalled. The theory is that Microsoft, with a market share of over 90% of the installed web browsers base, does not see any point in making any improvement at all. After IE 5, Internet Explorer hasn’t had any serious upgrade. With that spirit, Microsoft has said it wouldn’t support XForms. In fact, Microsoft has also its own competing proprietary technology to push: InfoPath.

The good news is that while all this has been going on, the Mozilla project has not rested on its laurels and has produced an ever more solid browser, culminating with the release of Firefox 1.0 just a few months ago, a milestone on the path to a much bigger market share.

The Mozilla XForms beta is important, because it announces the day where Firefox will be fully XForms 1.0 compliant. Developers can play with a beta and get their hands on the technology. You can look forward to the day where XForms-based applications can at last be freely deployed on the intranet, where the browser environment is controlled.

But there is still a major issue, which is that if Microsoft doesn’t come back on its decision not to implement XForms, it still won’t be possible to deploy XForms on the web - that is, unless you don’t care about users being able to access your site or web application. As much as we like to see Firefox’s market share grow, it has yet to reach 10%.

This is why it is important today to closely look at server-side XForms implementations, one of which is provided by the open source Orbeon PresentationServer today.

The idea behind a server-side XForms implementation is quite simple: instead of requiring the web browser to interpret the XForms tags, software running on a server does it. To do so, it typically looks at XHTML and XForms documents, and transforms them into good old HTML and JavaScript. When the web browser submits an HTML form, the server-side software, again, transforms the submission to make it look to the application like it was submitted by an XForms engine.

The huge benefit of this approach is that almost every browser currently deployed can now render XForms, while the application developer does in fact write XHTML and XForms. It also opens the possibility for XForms-enabled web browsers to fully benefit from XForms, while legacy browsers use the server-side implementation.

To be fair, there are some drawbacks. For example, a promise of XForms is to reduce the number of roundtrips to the server. With a server-side implementation, this may not be easily achieved. Also, some features of XForms may take time to be implemented by such server-side implementations because they are harder to implement than on the client-side. PresentationServer aims at implementing eventually all of the XForms specification. The list of implemented features is available here.

In spites of those drawbacks, this approach is in some people’s opinion the only way XForms will be able to be widely deployed on the web in the fairly short-term future. So go ahead and give PresentationServer a try!