XForms Everywhere

Now that Orbeon Forms 3.7 beta 1 is out, we have committed a series of changes that intend to improve the performance of the XForms engine. These particular changes initially focused on <xforms:insert> and <xforms:delete>, but they ended up requesting a fairly major refactoring of the XForms controls code on the server. Some (probably obscure) developer information is available on our wiki.

In short, we used to duplicate the tree of controls on the server at every possible turn. Now we:

• never do it during page initialization

• do it at most once per Ajax request

Instead we update the tree of controls incrementally upon insert/delete and upon refresh.

The result is that the performance has in fact improved quite a bit for large forms, especially those that use <xforms:insert> and <xforms:delete> heavily. In particular, one of the main goals of this work was to make Form Builder faster. There is still work to do there, but performance has already improved quite a bit!

This is a good step forward: both the code and the processing model for the controls are now much clearer. We are also more XForms 1.1 compliant (in particular regarding repeat index updates). This also fixed along the way a few bugs related to insert/delete/repeat and UI events dispatching.

What’s even better, this lays down a healthy foundation for further improvements, including at some point implementing a dependency mechanism for the controls which should yield even better performance, especially for large forms.

We have just made Orbeon Forms 3.7 beta 1 available for download. This release contains about 200 issues addressed since Orbeon Forms 3.6.

There are still a few blockers to address before we can release 3.7 final. Here are useful links to see what’s left to do:

• Issues by highest priority

• Recently open issues

• Recently closed issues

Enjoy!

In the last few months, we have been regularly posting updates to Twitter so we have had less of a need to write full blog entries. But now we feel that owe the followers of this blog a higher-level update:

• Form Builder and Form Runner. A big priority is our continued work on Form Builder and Form Runner. We are getting closer from a 1.0 version, and we have a great incentive as we have customers sponsoring features and planning to deploy very soon.

• Oracle XML support in Form Runner. Initially, Form Runner only supported the built-in eXist database. However, we designed from the get-go a REST-based API for handling persistence. We have now implemented a connector for storing form types and form data into Oracle.

• Noscript mode. The “noscript mode” in Orbeon Forms allows you to use a single XHTML+XForms description of a page and to target browser that have JavaScript disabled. This is now integrated into Form Runner, which means you don’t need JavaScript on the client to fill-out forms created by Form Builder. This is also the basis for the accessibility features we are working on (including WCAG AA compliance).

• Offline mode. We have worked quite a bit on an offline mode for Orbeon Forms, based on Google Gears. This is very exciting, but it is still a bit too early for us to integrate that into Form Runner.

• Components. An important aspect of any UI platform is the ability to reuse code. XForms should not be different. So we have started implementing a component system in the XForms engine, based on XBL 2. Simple components already work, and we are adding locally-scoped XForms models.

• Date and time controls. We have completely overhauled the date picker. You can now type in values and the date picker is by default the YUI date picker. We have also added controls for xs:time and xs:dateTime types, which was long overdue.

• Access request and session directly from XForms. In the past, accessing request and session information from XForms required using the page flow controller or possibly a page model written in XPL. Now, you can use XPath functions to do so.

• New YUI. We have upgraded to YUI 2.5.2. It’s good to be up to date and to benefit from the latest features and bug-fixes.

• Control the HTML title from XForms. Orbeon Forms now supports xforms:output within xhtml:title. This is very convenient to change the HTML page’s title directly from XForms.

• More flexible placement of AVTs and XForms models. AVTs are supported outside the HTML body. This allows for example controlling the lang attribute on the root HTML element. XForms models can alse now be placed outside the HTML head (e.g. within dialogs).

• Refactoring. A lot of refactoring of the Java code (and some of the JavaScript code) has taken place.

• Loads of bug-fixes. See the list of changes since Orbeon Forms 3.6. We hope that version 3.7 is not too far.

• New wiki. We now use Google Sites to host our development documentation.

This does sound like a lot, doesn’t it? And while all this is a lot of fun to work on, we need to mention that these new features and developments are driven by projects for our great customers.

Now, on to polishing Form Builder and getting 3.7 out!

It is with great pleasure that we are announcing Orbeon Form Builder. Form Builder allows you to build forms right from your browser and aims at taking the pain out of building forms. It is 100% open source, standards-based, and built on top of the Orbeon Forms XForms implementation, which we have been refining for years.

An alpha version of Orbeon Forms Builder is currently available to developers for testing and feedback. You can download it as part of the Orbeon Forms nightly builds.

More information is available on the Orbeon Form Builder product page, in the Orbeon Forms flyer (PDF), and in the following press release which we wrote with OW2 (PDF also available):

OW2 and Orbeon make web forms easy to build and deploy

San Francisco, May 6 2008 – Orbeon, Inc. and the OW2 Consortium today announced Orbeon Form Builder, a web-based, standards-based open source WYSIWYG web forms editor. Unlike most form editors, Orbeon Form Builder does not require any client-side software install and runs directly off a web browser by leveraging Ajax and Web 2.0 technologies.

Orbeon Form Builder includes a visual grid editor, attachments, validation rules, printable views, PDF output, full internationalisation, Web Services integration, XML-based persistence, an offline mode, as well as an “expert” mode allowing advanced form authors to provide their own XForms markup.

“For years now our customers have asked us for a visual form editor that leverages the power of XForms,” said Alessandro Vernet, CTO of Orbeon. “We believe that with Orbeon Form Builder, we are answering that call and we are making it dramatically easier for less technical users to build full-fledged web forms, while at the same time building on the technical merits of XForms.”

Orbeon Form Builder also provides the Orbeon Form Runner runtime environment, which manages deployed forms, presents searchable form data, handles validation as you type and takes care of the plumbing necessary to capture, save, import and export form data.

Orbeon Form Builder is built on top of the open source Orbeon Forms platform, which delivers the W3C XForms standard to web browsers without the need for plugins or additional client-side software. “The Orbeon Forms XForms engine is the technology that underlies Form Builder” said Erik Bruchez, principal architect of Orbeon Forms. “Without it, we wouldn’t have been able to progress as fast as we have over the past few months, and we are very proud of the fact that it is entirely open source.”

About Orbeon

Founded in 1999, Orbeon delivers and supports Orbeon Forms, an open source, standard-based forms solution. Orbeon Forms is delivered to web browsers thanks to XForms and Ajax technology, with no need for client-side software or plugins. Companies and public sector agencies that rely on Orbeon Forms include CSC, Teleflex, SunGard, Esprit Limited, CRIX International, the State of Oregon, and Amnesty International. Visit: www.orbeon.com

About OW2

Founded in January 2007 OW2 is an independent industry community dedicated to developing open source code middleware and to fostering a vibrant community and business ecosystem. The OW2 Consortium hosts some one hundred technology Projects, including Acceleo, ASM, Bonita, eXo Platorm, Funambol, JOnAS, Lomboz, Orbeon Forms, SpagoBI and Xwiki. Visit www.ow2.org

Orbeon, Inc.
Communications Department
Tel: +1 650 938 3120
Fax: +1 650 618 2600
info@orbeon.com
www.orbeon.com

OW2
Management Office
Tel: +32-24 26 85 23
Fax: +32-27 919 219
mo@ow2.org
www.ow2.org

That’s it, JavaOne 2008 is over! We were on the OW2 booth for three days last week, busy demoing the latest in Orbeon Forms.

We got a few good contacts during the conference, which also provided us with the opportunity of knowing the rest of OW2 guys better. Go check out the web site, you will find some pretty cool software, including eXo, XWiki, Spago, and more.

The photo shows our demo station and our brand new poster and brochure, which we will make available online soon.

Orbeon is now on Twitter, under the username orbeon. We have started providing regular updates about Orbeon Forms there.

If you don’t know what Twitter is, or if you are thinking “Why would I join another social network?”, think again: Twitter is different. Maybe this blog entry by Mark McGuinness will help you understand what it’s about, although the best way to get it is to give it a try!

We are looking forward to seeing you on Twitter!

With XForms 1.0, xforms:output was limited to displaying plain text. With the introduction of XForms 1.1, xforms:output adds powerful capabilities which include handling images, HTML content and more, depending on your XForms implementation.

To achieve this, XForms 1.1 introduces a new mediatype attribute. You use it to specify the type of the content to display, using the familiar Internet media type (or “MIME type”) syntax. This includes values such as image/* (any image content) or text/html (HTML content).

Orbeon Forms already supported the text/html mediatype, so we had to add support for image mediatypes. To achieve this, we made Orbeon Forms a little smarter.

• Consider that XForms allows you to refer to images not only as URLs (e.g. http://example.org/my/image.png), but as binary data embedded in an XForms instance. With Orbeon Forms, this data is handled on the server. So somehow, this information has to be brought to the client. In addition, browsers typically do not support providing image content as binary (although with some clients this is possible with the data: URI scheme)

• Even when using URLs, consider the following scenario: an image resides on your client computer, you upload it to the server through xforms:upload, at which point the image is available, in your XForms instance, as a temporary server-side URL. Now bind an xforms:output to that image URL: clearly, your browser cannot access that URL directly since it is server-side and therefore private.

Orbeon Forms solves these problems by creating proxy URLs for dynamically produced image content. Concretely, you see in the resulting HTML page paths of the form /xforms-server/dynamic/7FF0AD061DD9BAD27BCA1FF4674C75D6. Orbeon Forms takes care of handling these URLs as needed: you don’t need to worry a bit about this.

That was for the “smart” bit, so how does it look in the end to you, the form author? It’s as simple as this:

<xforms:upload ref="my-image">
  <xforms:label>Logo</xforms:label>
  <xforms:hint>Please upload an image</xforms:hint>
</xforms:upload>
<xforms:output mediatype="image/*" ref="my-image"/>

Yes that’s right: the few lines above handle uploading an image, attaching it to your data, and displaying the uploaded image to the user. Beat that.

We are glad to announce that we will be present at JavaOne 2008 on the OW2 Consortium stand. This year, JavaOne will take place May 6-9.

We have some great Orbeon Forms demos in the works so be sure to stop by!

Wow, it’s been a month since our last post in this blog, and one of the reasons for this is that we have been very busy enhancing Orbeon Forms! Here is a quick list of the most notable additions:

• XPath variables. XPath variables can be used within controls, models and actions, with the xxforms:variable (or exforms:variable) element. Variables are extremely useful, for example to avoid repeating long XPath expressions, or to give an XPath expression unambiguous access to data computed in enclosing xforms:group or xforms:repeat elements. If you are familiar with XSLT variables, you will feel right at home. And once you start using them, you won’t be able to go back!

• XML Schema validation improvements. Inline XML Schema support is now implemented. This allows you to embed XML Schema directly within the xforms:model element. This is particularly useful when you create a new schema which is just used within an XForms document, or to define a series of simple types used for validation. We have also added support for the xsi:type attribute in XForms instances. This is a convenient shortcut which allows you to avoid using binds just to specify types.

• Attribute Value Templates (AVTs) on XHTML elements. In Orbeon Forms, AVTs were already supported on some XForms elements, but you can now use AVTs on XHTML elements as well. This allows you to use XForms to influence aspects of XHTML such as class, style or colspan attributes, and all this without script.

• Action context enhancements. We added the exforms:iterate attribute, XForms 1.1 context() function, and a generalized context attribute on XForms elements (which may appear in XForms 1.2).

• Events enhancements. We now have full support for ev:observer and ev:target attributes on action handlers. We also added support for lists of event names, observers, and targets (an extension which we hope will be available in XML Events 2, currently in Working Draft at W3C). Finally, you can now pass context information to custom events with the extension xxforms:contex child element of actions such as xforms:dispatch or xforms:event.

• Image handling. xforms:output supports displaying images when used with xs:anyURI or xs:base64Binary datatypes and an image mediatype, as per XForms 1.1. It has never been easier to upload and display images from XForms!

• XPath extension functions. We added XPath functions to access headers and request parameters directly from XForms (xxforms:get-request-parameter() and xxforms:get-request-header()), as well as extension functions to create elements and attributes by name (xxforms:element() and xxforms:attribute()).

• Submissions. Submissions allow PUT and POST of binary content directly from XForms. This makes it very easy to upload a binary file using xforms:upload and then send it out to a service through xforms:submission.

For more information, check the temporary list of changes for the upcoming Orbeon Forms 3.7! We already have over one hundred enhancements implemented since version 3.6.

XForms 1.1’s small number of actions are surprisingly powerful. By combining them, you can do almost any operation you can think of within an XForms page. Yeah, XForms is Turing complete.

There are however a few use cases that remain difficult to achieve. For example, you can easily repeat an action while a condition is satisfied with the while attribute. However, you cannot yet directly iterate over a sequence of node and perform a group of actions for each iteration. As the XForms Working Group is working on fixing this, we have implemented in Orbeon Forms the exforms:iterate attribute (also available as xxforms:iterate attribute) on XForms action elements. This attribute was first proposed as part of the eXforms extensions.

So how does it work? Consider the following three XForms instances:

    <xforms:instance id="main-instance">
        <instance/>
    </xforms:instance>
	
    <xforms:instance id="source-instance">
        <instance>
            <title>Don Quixote de la Mancha</title>
            <author>Miguel de Cervantes Saavedra</author>
            <title>Jacques le fataliste et son maître</title>
            <author>Denis Diderot</author>
            <title>Childhood's End</title>
            <author>Arthur C. Clarke</author>
        </instance>
    </xforms:instance>
	
    <xforms:instance id="template-instance">
        <book>
            <title />
            <author />
        </book>
    </xforms:instance>

The goal is to take data from source-instance, and copy it over to main-instance but formatted differently. Here is a way to do it:

    <xforms:action xxforms:iterate="instance('source-instance')/title">
        <xforms:insert context="instance('main-instance')" nodeset="book" origin="instance('template-instance')"/>
        <xforms:setvalue ref="instance('main-instance')/book[last()]/title" value="context()"/>
        <xforms:setvalue ref="instance('main-instance')/book[last()]/author" value="context()/following-sibling::author"/>
    </xforms:action>

The outermost action iterates over the <title> elements of source-instance. For each of those, a new <book> element, copied from the template stored in template-instance, is inserted into main-instance. Then values from the <title> and <author> elements are copied over to the new structure. The XForms 1.1 context() function, also recently implemented in Orbeon Forms, provides access to each of the iterated nodes. The resulting main-instance is as follows:

    <instance>
        <book>
            <title>Don Quixote de la Mancha</title>
            <author>Miguel de Cervantes Saavedra</author>
        </book>
        <book>
            <title>Jacques le fataliste et son maître</title>
            <author>Denis Diderot</author>
        </book>
        <book>
            <title>Childhood's End</title>
            <author>Arthur C. Clarke</author>
        </book>
    </instance>

It looks easy this way, doesn’t it?