XForms Everywhere

In the last few months, we have been regularly posting updates to Twitter so we have had less of a need to write full blog entries. But now we feel that owe the followers of this blog a higher-level update:

• Form Builder and Form Runner. A big priority is our continued work on Form Builder and Form Runner. We are getting closer from a 1.0 version, and we have a great incentive as we have customers sponsoring features and planning to deploy very soon.

• Oracle XML support in Form Runner. Initially, Form Runner only supported the built-in eXist database. However, we designed from the get-go a REST-based API for handling persistence. We have now implemented a connector for storing form types and form data into Oracle.

• Noscript mode. The “noscript mode” in Orbeon Forms allows you to use a single XHTML+XForms description of a page and to target browser that have JavaScript disabled. This is now integrated into Form Runner, which means you don’t need JavaScript on the client to fill-out forms created by Form Builder. This is also the basis for the accessibility features we are working on (including WCAG AA compliance).

• Offline mode. We have worked quite a bit on an offline mode for Orbeon Forms, based on Google Gears. This is very exciting, but it is still a bit too early for us to integrate that into Form Runner.

• Components. An important aspect of any UI platform is the ability to reuse code. XForms should not be different. So we have started implementing a component system in the XForms engine, based on XBL 2. Simple components already work, and we are adding locally-scoped XForms models.

• Date and time controls. We have completely overhauled the date picker. You can now type in values and the date picker is by default the YUI date picker. We have also added controls for xs:time and xs:dateTime types, which was long overdue.

• Access request and session directly from XForms. In the past, accessing request and session information from XForms required using the page flow controller or possibly a page model written in XPL. Now, you can use XPath functions to do so.

• New YUI. We have upgraded to YUI 2.5.2. It’s good to be up to date and to benefit from the latest features and bug-fixes.

• Control the HTML title from XForms. Orbeon Forms now supports xforms:output within xhtml:title. This is very convenient to change the HTML page’s title directly from XForms.

• More flexible placement of AVTs and XForms models. AVTs are supported outside the HTML body. This allows for example controlling the lang attribute on the root HTML element. XForms models can alse now be placed outside the HTML head (e.g. within dialogs).

• Refactoring. A lot of refactoring of the Java code (and some of the JavaScript code) has taken place.

• Loads of bug-fixes. See the list of changes since Orbeon Forms 3.6. We hope that version 3.7 is not too far.

• New wiki. We now use Google Sites to host our development documentation.

This does sound like a lot, doesn’t it? And while all this is a lot of fun to work on, we need to mention that these new features and developments are driven by projects for our great customers.

Now, on to polishing Form Builder and getting 3.7 out!

It is with great pleasure that we are announcing Orbeon Form Builder. Form Builder allows you to build forms right from your browser and aims at taking the pain out of building forms. It is 100% open source, standards-based, and built on top of the Orbeon Forms XForms implementation, which we have been refining for years.

An alpha version of Orbeon Forms Builder is currently available to developers for testing and feedback. You can download it as part of the Orbeon Forms nightly builds.

More information is available on the Orbeon Form Builder product page, in the Orbeon Forms flyer (PDF), and in the following press release which we wrote with OW2 (PDF also available):

OW2 and Orbeon make web forms easy to build and deploy

San Francisco, May 6 2008 – Orbeon, Inc. and the OW2 Consortium today announced Orbeon Form Builder, a web-based, standards-based open source WYSIWYG web forms editor. Unlike most form editors, Orbeon Form Builder does not require any client-side software install and runs directly off a web browser by leveraging Ajax and Web 2.0 technologies.

Orbeon Form Builder includes a visual grid editor, attachments, validation rules, printable views, PDF output, full internationalisation, Web Services integration, XML-based persistence, an offline mode, as well as an “expert” mode allowing advanced form authors to provide their own XForms markup.

“For years now our customers have asked us for a visual form editor that leverages the power of XForms,” said Alessandro Vernet, CTO of Orbeon. “We believe that with Orbeon Form Builder, we are answering that call and we are making it dramatically easier for less technical users to build full-fledged web forms, while at the same time building on the technical merits of XForms.”

Orbeon Form Builder also provides the Orbeon Form Runner runtime environment, which manages deployed forms, presents searchable form data, handles validation as you type and takes care of the plumbing necessary to capture, save, import and export form data.

Orbeon Form Builder is built on top of the open source Orbeon Forms platform, which delivers the W3C XForms standard to web browsers without the need for plugins or additional client-side software. “The Orbeon Forms XForms engine is the technology that underlies Form Builder” said Erik Bruchez, principal architect of Orbeon Forms. “Without it, we wouldn’t have been able to progress as fast as we have over the past few months, and we are very proud of the fact that it is entirely open source.”

About Orbeon

Founded in 1999, Orbeon delivers and supports Orbeon Forms, an open source, standard-based forms solution. Orbeon Forms is delivered to web browsers thanks to XForms and Ajax technology, with no need for client-side software or plugins. Companies and public sector agencies that rely on Orbeon Forms include CSC, Teleflex, SunGard, Esprit Limited, CRIX International, the State of Oregon, and Amnesty International. Visit: www.orbeon.com

About OW2

Founded in January 2007 OW2 is an independent industry community dedicated to developing open source code middleware and to fostering a vibrant community and business ecosystem. The OW2 Consortium hosts some one hundred technology Projects, including Acceleo, ASM, Bonita, eXo Platorm, Funambol, JOnAS, Lomboz, Orbeon Forms, SpagoBI and Xwiki. Visit www.ow2.org

Orbeon, Inc.
Communications Department
Tel: +1 650 938 3120
Fax: +1 650 618 2600
info@orbeon.com
www.orbeon.com

OW2
Management Office
Tel: +32-24 26 85 23
Fax: +32-27 919 219
mo@ow2.org
www.ow2.org

That’s it, JavaOne 2008 is over! We were on the OW2 booth for three days last week, busy demoing the latest in Orbeon Forms.

We got a few good contacts during the conference, which also provided us with the opportunity of knowing the rest of OW2 guys better. Go check out the web site, you will find some pretty cool software, including eXo, XWiki, Spago, and more.

The photo shows our demo station and our brand new poster and brochure, which we will make available online soon.

Orbeon is now on Twitter, under the username orbeon. We have started providing regular updates about Orbeon Forms there.

If you don’t know what Twitter is, or if you are thinking “Why would I join another social network?”, think again: Twitter is different. Maybe this blog entry by Mark McGuinness will help you understand what it’s about, although the best way to get it is to give it a try!

We are looking forward to seeing you on Twitter!

With XForms 1.0, xforms:output was limited to displaying plain text. With the introduction of XForms 1.1, xforms:output adds powerful capabilities which include handling images, HTML content and more, depending on your XForms implementation.

To achieve this, XForms 1.1 introduces a new mediatype attribute. You use it to specify the type of the content to display, using the familiar Internet media type (or “MIME type”) syntax. This includes values such as image/* (any image content) or text/html (HTML content).

Orbeon Forms already supported the text/html mediatype, so we had to add support for image mediatypes. To achieve this, we made Orbeon Forms a little smarter.

• Consider that XForms allows you to refer to images not only as URLs (e.g. http://example.org/my/image.png), but as binary data embedded in an XForms instance. With Orbeon Forms, this data is handled on the server. So somehow, this information has to be brought to the client. In addition, browsers typically do not support providing image content as binary (although with some clients this is possible with the data: URI scheme)

• Even when using URLs, consider the following scenario: an image resides on your client computer, you upload it to the server through xforms:upload, at which point the image is available, in your XForms instance, as a temporary server-side URL. Now bind an xforms:output to that image URL: clearly, your browser cannot access that URL directly since it is server-side and therefore private.

Orbeon Forms solves these problems by creating proxy URLs for dynamically produced image content. Concretely, you see in the resulting HTML page paths of the form /xforms-server/dynamic/7FF0AD061DD9BAD27BCA1FF4674C75D6. Orbeon Forms takes care of handling these URLs as needed: you don’t need to worry a bit about this.

That was for the “smart” bit, so how does it look in the end to you, the form author? It’s as simple as this:

<xforms:upload ref="my-image">
  <xforms:label>Logo</xforms:label>
  <xforms:hint>Please upload an image</xforms:hint>
</xforms:upload>
<xforms:output mediatype="image/*" ref="my-image"/>

Yes that’s right: the few lines above handle uploading an image, attaching it to your data, and displaying the uploaded image to the user. Beat that.

We are glad to announce that we will be present at JavaOne 2008 on the OW2 Consortium stand. This year, JavaOne will take place May 6-9.

We have some great Orbeon Forms demos in the works so be sure to stop by!

Wow, it’s been a month since our last post in this blog, and one of the reasons for this is that we have been very busy enhancing Orbeon Forms! Here is a quick list of the most notable additions:

• XPath variables. XPath variables can be used within controls, models and actions, with the xxforms:variable (or exforms:variable) element. Variables are extremely useful, for example to avoid repeating long XPath expressions, or to give an XPath expression unambiguous access to data computed in enclosing xforms:group or xforms:repeat elements. If you are familiar with XSLT variables, you will feel right at home. And once you start using them, you won’t be able to go back!

• XML Schema validation improvements. Inline XML Schema support is now implemented. This allows you to embed XML Schema directly within the xforms:model element. This is particularly useful when you create a new schema which is just used within an XForms document, or to define a series of simple types used for validation. We have also added support for the xsi:type attribute in XForms instances. This is a convenient shortcut which allows you to avoid using binds just to specify types.

• Attribute Value Templates (AVTs) on XHTML elements. In Orbeon Forms, AVTs were already supported on some XForms elements, but you can now use AVTs on XHTML elements as well. This allows you to use XForms to influence aspects of XHTML such as class, style or colspan attributes, and all this without script.

• Action context enhancements. We added the exforms:iterate attribute, XForms 1.1 context() function, and a generalized context attribute on XForms elements (which may appear in XForms 1.2).

• Events enhancements. We now have full support for ev:observer and ev:target attributes on action handlers. We also added support for lists of event names, observers, and targets (an extension which we hope will be available in XML Events 2, currently in Working Draft at W3C). Finally, you can now pass context information to custom events with the extension xxforms:contex child element of actions such as xforms:dispatch or xforms:event.

• Image handling. xforms:output supports displaying images when used with xs:anyURI or xs:base64Binary datatypes and an image mediatype, as per XForms 1.1. It has never been easier to upload and display images from XForms!

• XPath extension functions. We added XPath functions to access headers and request parameters directly from XForms (xxforms:get-request-parameter() and xxforms:get-request-header()), as well as extension functions to create elements and attributes by name (xxforms:element() and xxforms:attribute()).

• Submissions. Submissions allow PUT and POST of binary content directly from XForms. This makes it very easy to upload a binary file using xforms:upload and then send it out to a service through xforms:submission.

For more information, check the temporary list of changes for the upcoming Orbeon Forms 3.7! We already have over one hundred enhancements implemented since version 3.6.

XForms 1.1’s small number of actions are surprisingly powerful. By combining them, you can do almost any operation you can think of within an XForms page. Yeah, XForms is Turing complete.

There are however a few use cases that remain difficult to achieve. For example, you can easily repeat an action while a condition is satisfied with the while attribute. However, you cannot yet directly iterate over a sequence of node and perform a group of actions for each iteration. As the XForms Working Group is working on fixing this, we have implemented in Orbeon Forms the exforms:iterate attribute (also available as xxforms:iterate attribute) on XForms action elements. This attribute was first proposed as part of the eXforms extensions.

So how does it work? Consider the following three XForms instances:

    <xforms:instance id="main-instance">
        <instance/>
    </xforms:instance>
	
    <xforms:instance id="source-instance">
        <instance>
            <title>Don Quixote de la Mancha</title>
            <author>Miguel de Cervantes Saavedra</author>
            <title>Jacques le fataliste et son maître</title>
            <author>Denis Diderot</author>
            <title>Childhood's End</title>
            <author>Arthur C. Clarke</author>
        </instance>
    </xforms:instance>
	
    <xforms:instance id="template-instance">
        <book>
            <title />
            <author />
        </book>
    </xforms:instance>

The goal is to take data from source-instance, and copy it over to main-instance but formatted differently. Here is a way to do it:

    <xforms:action xxforms:iterate="instance('source-instance')/title">
        <xforms:insert context="instance('main-instance')" nodeset="book" origin="instance('template-instance')"/>
        <xforms:setvalue ref="instance('main-instance')/book[last()]/title" value="context()"/>
        <xforms:setvalue ref="instance('main-instance')/book[last()]/author" value="context()/following-sibling::author"/>
    </xforms:action>

The outermost action iterates over the <title> elements of source-instance. For each of those, a new <book> element, copied from the template stored in template-instance, is inserted into main-instance. Then values from the <title> and <author> elements are copied over to the new structure. The XForms 1.1 context() function, also recently implemented in Orbeon Forms, provides access to each of the iterated nodes. The resulting main-instance is as follows:

    <instance>
        <book>
            <title>Don Quixote de la Mancha</title>
            <author>Miguel de Cervantes Saavedra</author>
        </book>
        <book>
            <title>Jacques le fataliste et son maître</title>
            <author>Denis Diderot</author>
        </book>
        <book>
            <title>Childhood's End</title>
            <author>Arthur C. Clarke</author>
        </book>
    </instance>

It looks easy this way, doesn’t it?

A couple of weeks ago I attempted to order prints from Flickr. I added a few photos to my cart and was redirected to a service called QOOP. This has a neat interface that allows you organize photos for printing in a book format. I played a bit with the interface and then was caught by other tasks. When I came back to my browser window and attempted to preview the photo album, I got a wonderful error message about a session having expired. The result: my album was lost. I gave up.

My adventure with expired sessions is quite common and I can bet you have had this kind of issues in the past. The least that can be said about this is that it is not user-friendly!

To address this kind of issues, we recently introduced a new feature in Orbeon Forms called the “session heartbeat”. The idea is that if you happen to leave a browser window open on your computer, chances are that you will get back to that window and keep using the application. The last thing you want to happen when you come back is lose your session and therefore your data, as happened to me.

This is not always a correct guess of course: you may just happen to leave a window or tab open without planning to use it again. Conversely you may have a page which is not actually visible, for example in your browser history, yet you will come back to it. This approach wouldn’t be good for banking applications either. Still, in many situations, such as filling-out large forms, it sounds like a good idea to keep your session alive for open pages.

To achieve this goal you could make all server sessions longer. However this is harder to configure for the application developer, and this won’t discriminate between pages that are actually open on a client and the ones that are not. And while it may be ideal to have infinitely long sessions, unfortunately many applications are not ready for this kind of approach.

So we turned to the idea of implementing a “session heartbeat”. Here is how this works:

• When this feature is enabled, an open XForms page regularly pings the server through Ajax to keep the current session alive.

• The ping delay is automatically computed based on the server’s session timeout. The client pings the server at 80% of the session expiration time after the last interaction with the server.

• We are careful not to hit the XForms engine too much, in fact we do a very limited amount of work on the server for each ping, so they should run fast.

• XForms state information for pages hit with the heartbeat just migrates to the disk store over time if RAM is used by other pages, so keeping even large numbers of pages open should not have any negative impact on server RAM.

• When a user gets back to using the page, state information migrates back from disk to RAM, and the page will be live again.

• Sessions do eventually expire as nobody keeps a browser open forever.

Note that whenever an application keeps sessions alive for a long time, it is a good idea to keep as little data as possible in the session. The Orbeon Forms XForms engine itself uses a global state store and does not use session objects for storage, but do remember to keep your sessions small!

Hopefully, this will help prevent many occurrences of the infamous “session expired” error message. The beauty of it is that as an Orbeon Forms application developer you don’t have to worry about anything: the session heartbeat is enabled by default (but it can be turned off globally in properties.xml or individually for each page).

I recently booked a flight on the Continental web site. I knew approximately the flight I wanted to book, so dug right away into the search to select it. I encountered the following issues:

• When the results came, the prices were in Swiss francs. Since I was in Switzerland but using a US credit card, I decided to switch my location using the quite nicely done language and location menu available on the result page.

  I first noticed a funny glitch: for Switzerland, the language selection was English and German. English was selected by default, which suited me, but English was the second radio button in the list. When I switched to the United States location, the second radio button remained selected. However, since for that location the language selection was English first, then Spanish, the result was that the Spanish language was selected. Oops.

• This was just a little bug though. The bigger problem was that as the language was switched, somehow the search page kept my flight dates, but lost the selected flights. So I had to go through them again.

• When doing the final checkout, I omitted a digit in the credit card number. It took the web site almost one minute, after I pressed the submit button, to tell me that there was a problem with that number.

• When that was fixed, I also happened to have forgotten to enter the billing address. The same thing happened: I had to wait for quite a long time after submitting the form to get a message telling me that that was wrong.

Now let’s say the developers of the Continental web site decide to switch to Orbeon Forms and XForms for the next version of the web site:

• With XForms, they easily set constraints, such as saying that the value of the radio button should be the currently selected value (say, ‘en’) if available, or then the default value if unavailable.

• Localized user interface resources are stored in a separate XForms instance (which is an XML document). When switching the resources document, the entire user interface updates without losing the other data captured by the user.

• XForms has several mechanisms to check input data: you can use XML Schema types or XPath constraints (XForms 1.1 even supports a credit card number type) to make sure that a Visa or Mastercard number is at least approximately correct in length and that the redundancy digit applies correctly.

• XForms also support ways to mark fields as required so the user doesn’t forget to enter some information.

And this is all interactive too, meaning that the user doesn’t need to wait one minute to know that some fields are missing or incorrect.