XForms Everywhere

Orbeon is now on Twitter, under the username orbeon. We have started providing regular updates about Orbeon Forms there.

If you don’t know what Twitter is, or if you are thinking “Why would I join another social network?”, think again: Twitter is different. Maybe this blog entry by Mark McGuinness will help you understand what it’s about, although the best way to get it is to give it a try!

We are looking forward to seeing you on Twitter!

With XForms 1.0, xforms:output was limited to displaying plain text. With the introduction of XForms 1.1, xforms:output adds powerful capabilities which include handling images, HTML content and more, depending on your XForms implementation.

To achieve this, XForms 1.1 introduces a new mediatype attribute. You use it to specify the type of the content to display, using the familiar Internet media type (or “MIME type”) syntax. This includes values such as image/* (any image content) or text/html (HTML content).

Orbeon Forms already supported the text/html mediatype, so we had to add support for image mediatypes. To achieve this, we made Orbeon Forms a little smarter.

• Consider that XForms allows you to refer to images not only as URLs (e.g. http://example.org/my/image.png), but as binary data embedded in an XForms instance. With Orbeon Forms, this data is handled on the server. So somehow, this information has to be brought to the client. In addition, browsers typically do not support providing image content as binary (although with some clients this is possible with the data: URI scheme)

• Even when using URLs, consider the following scenario: an image resides on your client computer, you upload it to the server through xforms:upload, at which point the image is available, in your XForms instance, as a temporary server-side URL. Now bind an xforms:output to that image URL: clearly, your browser cannot access that URL directly since it is server-side and therefore private.

Orbeon Forms solves these problems by creating proxy URLs for dynamically produced image content. Concretely, you see in the resulting HTML page paths of the form /xforms-server/dynamic/7FF0AD061DD9BAD27BCA1FF4674C75D6. Orbeon Forms takes care of handling these URLs as needed: you don’t need to worry a bit about this.

That was for the “smart” bit, so how does it look in the end to you, the form author? It’s as simple as this:

<xforms:upload ref="my-image">
  <xforms:label>Logo</xforms:label>
  <xforms:hint>Please upload an image</xforms:hint>
</xforms:upload>
<xforms:output mediatype="image/*" ref="my-image"/>

Yes that’s right: the few lines above handle uploading an image, attaching it to your data, and displaying the uploaded image to the user. Beat that.

We are glad to announce that we will be present at JavaOne 2008 on the OW2 Consortium stand. This year, JavaOne will take place May 6-9.

We have some great Orbeon Forms demos in the works so be sure to stop by!

Wow, it’s been a month since our last post in this blog, and one of the reasons for this is that we have been very busy enhancing Orbeon Forms! Here is a quick list of the most notable additions:

• XPath variables. XPath variables can be used within controls, models and actions, with the xxforms:variable (or exforms:variable) element. Variables are extremely useful, for example to avoid repeating long XPath expressions, or to give an XPath expression unambiguous access to data computed in enclosing xforms:group or xforms:repeat elements. If you are familiar with XSLT variables, you will feel right at home. And once you start using them, you won’t be able to go back!

• XML Schema validation improvements. Inline XML Schema support is now implemented. This allows you to embed XML Schema directly within the xforms:model element. This is particularly useful when you create a new schema which is just used within an XForms document, or to define a series of simple types used for validation. We have also added support for the xsi:type attribute in XForms instances. This is a convenient shortcut which allows you to avoid using binds just to specify types.

• Attribute Value Templates (AVTs) on XHTML elements. In Orbeon Forms, AVTs were already supported on some XForms elements, but you can now use AVTs on XHTML elements as well. This allows you to use XForms to influence aspects of XHTML such as class, style or colspan attributes, and all this without script.

• Action context enhancements. We added the exforms:iterate attribute, XForms 1.1 context() function, and a generalized context attribute on XForms elements (which may appear in XForms 1.2).

• Events enhancements. We now have full support for ev:observer and ev:target attributes on action handlers. We also added support for lists of event names, observers, and targets (an extension which we hope will be available in XML Events 2, currently in Working Draft at W3C). Finally, you can now pass context information to custom events with the extension xxforms:contex child element of actions such as xforms:dispatch or xforms:event.

• Image handling. xforms:output supports displaying images when used with xs:anyURI or xs:base64Binary datatypes and an image mediatype, as per XForms 1.1. It has never been easier to upload and display images from XForms!

• XPath extension functions. We added XPath functions to access headers and request parameters directly from XForms (xxforms:get-request-parameter() and xxforms:get-request-header()), as well as extension functions to create elements and attributes by name (xxforms:element() and xxforms:attribute()).

• Submissions. Submissions allow PUT and POST of binary content directly from XForms. This makes it very easy to upload a binary file using xforms:upload and then send it out to a service through xforms:submission.

For more information, check the temporary list of changes for the upcoming Orbeon Forms 3.7! We already have over one hundred enhancements implemented since version 3.6.

XForms 1.1’s small number of actions are surprisingly powerful. By combining them, you can do almost any operation you can think of within an XForms page. Yeah, XForms is Turing complete.

There are however a few use cases that remain difficult to achieve. For example, you can easily repeat an action while a condition is satisfied with the while attribute. However, you cannot yet directly iterate over a sequence of node and perform a group of actions for each iteration. As the XForms Working Group is working on fixing this, we have implemented in Orbeon Forms the exforms:iterate attribute (also available as xxforms:iterate attribute) on XForms action elements. This attribute was first proposed as part of the eXforms extensions.

So how does it work? Consider the following three XForms instances:

    <xforms:instance id="main-instance">
        <instance/>
    </xforms:instance>
	
    <xforms:instance id="source-instance">
        <instance>
            <title>Don Quixote de la Mancha</title>
            <author>Miguel de Cervantes Saavedra</author>
            <title>Jacques le fataliste et son maître</title>
            <author>Denis Diderot</author>
            <title>Childhood's End</title>
            <author>Arthur C. Clarke</author>
        </instance>
    </xforms:instance>
	
    <xforms:instance id="template-instance">
        <book>
            <title />
            <author />
        </book>
    </xforms:instance>

The goal is to take data from source-instance, and copy it over to main-instance but formatted differently. Here is a way to do it:

    <xforms:action xxforms:iterate="instance('source-instance')/title">
        <xforms:insert context="instance('main-instance')" nodeset="book" origin="instance('template-instance')"/>
        <xforms:setvalue ref="instance('main-instance')/book[last()]/title" value="context()"/>
        <xforms:setvalue ref="instance('main-instance')/book[last()]/author" value="context()/following-sibling::author"/>
    </xforms:action>

The outermost action iterates over the <title> elements of source-instance. For each of those, a new <book> element, copied from the template stored in template-instance, is inserted into main-instance. Then values from the <title> and <author> elements are copied over to the new structure. The XForms 1.1 context() function, also recently implemented in Orbeon Forms, provides access to each of the iterated nodes. The resulting main-instance is as follows:

    <instance>
        <book>
            <title>Don Quixote de la Mancha</title>
            <author>Miguel de Cervantes Saavedra</author>
        </book>
        <book>
            <title>Jacques le fataliste et son maître</title>
            <author>Denis Diderot</author>
        </book>
        <book>
            <title>Childhood's End</title>
            <author>Arthur C. Clarke</author>
        </book>
    </instance>

It looks easy this way, doesn’t it?

A couple of weeks ago I attempted to order prints from Flickr. I added a few photos to my cart and was redirected to a service called QOOP. This has a neat interface that allows you organize photos for printing in a book format. I played a bit with the interface and then was caught by other tasks. When I came back to my browser window and attempted to preview the photo album, I got a wonderful error message about a session having expired. The result: my album was lost. I gave up.

My adventure with expired sessions is quite common and I can bet you have had this kind of issues in the past. The least that can be said about this is that it is not user-friendly!

To address this kind of issues, we recently introduced a new feature in Orbeon Forms called the “session heartbeat”. The idea is that if you happen to leave a browser window open on your computer, chances are that you will get back to that window and keep using the application. The last thing you want to happen when you come back is lose your session and therefore your data, as happened to me.

This is not always a correct guess of course: you may just happen to leave a window or tab open without planning to use it again. Conversely you may have a page which is not actually visible, for example in your browser history, yet you will come back to it. This approach wouldn’t be good for banking applications either. Still, in many situations, such as filling-out large forms, it sounds like a good idea to keep your session alive for open pages.

To achieve this goal you could make all server sessions longer. However this is harder to configure for the application developer, and this won’t discriminate between pages that are actually open on a client and the ones that are not. And while it may be ideal to have infinitely long sessions, unfortunately many applications are not ready for this kind of approach.

So we turned to the idea of implementing a “session heartbeat”. Here is how this works:

• When this feature is enabled, an open XForms page regularly pings the server through Ajax to keep the current session alive.

• The ping delay is automatically computed based on the server’s session timeout. The client pings the server at 80% of the session expiration time after the last interaction with the server.

• We are careful not to hit the XForms engine too much, in fact we do a very limited amount of work on the server for each ping, so they should run fast.

• XForms state information for pages hit with the heartbeat just migrates to the disk store over time if RAM is used by other pages, so keeping even large numbers of pages open should not have any negative impact on server RAM.

• When a user gets back to using the page, state information migrates back from disk to RAM, and the page will be live again.

• Sessions do eventually expire as nobody keeps a browser open forever.

Note that whenever an application keeps sessions alive for a long time, it is a good idea to keep as little data as possible in the session. The Orbeon Forms XForms engine itself uses a global state store and does not use session objects for storage, but do remember to keep your sessions small!

Hopefully, this will help prevent many occurrences of the infamous “session expired” error message. The beauty of it is that as an Orbeon Forms application developer you don’t have to worry about anything: the session heartbeat is enabled by default (but it can be turned off globally in properties.xml or individually for each page).

I recently booked a flight on the Continental web site. I knew approximately the flight I wanted to book, so dug right away into the search to select it. I encountered the following issues:

• When the results came, the prices were in Swiss francs. Since I was in Switzerland but using a US credit card, I decided to switch my location using the quite nicely done language and location menu available on the result page.

  I first noticed a funny glitch: for Switzerland, the language selection was English and German. English was selected by default, which suited me, but English was the second radio button in the list. When I switched to the United States location, the second radio button remained selected. However, since for that location the language selection was English first, then Spanish, the result was that the Spanish language was selected. Oops.

• This was just a little bug though. The bigger problem was that as the language was switched, somehow the search page kept my flight dates, but lost the selected flights. So I had to go through them again.

• When doing the final checkout, I omitted a digit in the credit card number. It took the web site almost one minute, after I pressed the submit button, to tell me that there was a problem with that number.

• When that was fixed, I also happened to have forgotten to enter the billing address. The same thing happened: I had to wait for quite a long time after submitting the form to get a message telling me that that was wrong.

Now let’s say the developers of the Continental web site decide to switch to Orbeon Forms and XForms for the next version of the web site:

• With XForms, they easily set constraints, such as saying that the value of the radio button should be the currently selected value (say, ‘en’) if available, or then the default value if unavailable.

• Localized user interface resources are stored in a separate XForms instance (which is an XML document). When switching the resources document, the entire user interface updates without losing the other data captured by the user.

• XForms has several mechanisms to check input data: you can use XML Schema types or XPath constraints (XForms 1.1 even supports a credit card number type) to make sure that a Visa or Mastercard number is at least approximately correct in length and that the redundancy digit applies correctly.

• XForms also support ways to mark fields as required so the user doesn’t forget to enter some information.

And this is all interactive too, meaning that the user doesn’t need to wait one minute to know that some fields are missing or incorrect.

Yahoo! today announced a new mobile platform called “Mobile Widgets”. The amazing part is that the language used to describe widgets, called Blueprint, is based on XForms:

Much of Blueprint’s philosophy and syntax comes from XForms. We opted for a full declarative language because it was the only way we could effectively run on the wide range of devices out there, some of which have no scripting at all. By using declarative syntax, we can encapsulate and hide the scripting specifics. In some cases, the code could run on the phone, in other case, such as XHTML, we can put the logic on our servers. It’s the perfect way to deal with the various environments and their capabilities.

The Blueprint Philosophy and roadmap highlights the benefits of declarative vs. imperative, “because it was the only way we could effectively run on the wide range of devices out there, some of which have no scripting at all”. For now, only very basic XForms constructs are supported, but “In our second release of the Blueprint language you’ll be able to take full advantage of the power of XForms-style MVC programming”.

Although Orbeon does not particularly focuses on mobile devices, this philosophy is entirely in line with our transformations- and Ajax-based approach to deliver XForms everywhere (in our case to mainstream browsers). It is excellent news that a company like Yahoo! not only recognizes the benefits of the XForms approach, but also decides to leverage the standard instead of reinventing the wheel (like Google did with their Google Mashup Editor, which looks very much like an incompatible clone of XForms).

In short this is extremely positive for the XForms community. We can only hope that Blueprint lives up to its promises.

It is with great pleasure that we are announcing the final release of Orbeon Forms 3.6!

Orbeon Forms is an open source forms solution that handles the complexity of forms typical of the enterprise or government. It is delivered to standard web browsers thanks to XForms and Ajax technology, with no need for client-side software or plugins.

Orbeon Forms allows you to build fully interactive forms with features that include as-you-type validation, optional and repeated sections, always up-to-date error summaries, PDF output, full internationalization, and controls like auto-completion, tabs, dialogs, trees and menus.

Orbeon Forms 3.6 features over 170 improvements since Orbeon Forms 3.5.1, including major improvements in the areas of state handling, XML Schema validation, error handling, deployment within Java applications, and performance. The complete list of changes is available at:

http://www.orbeon.com/ops/doc/home-changes-36

The latest example applications are online at:

http://www.orbeon.com/ops/

You can get Orbeon Forms 3.6 from the download page:

http://www.orbeon.com/forms/download

The Orbeon team hopes you will enjoy this release!

We had great success with the XForms Evening a the XML 2007 conference on Monday. We were wondering if people would really want to attend sessions that late in the evening (from 7:30 to 9:00 PM), but the attendance beat our expectations, with easily over 60 people cramped in the allocated room and some actually standing.

The initial feedback we are getting is excellent. I believe that the format of the sessions was one of the reasons for this: it consisted in having a one and a half hour session on a specific topic (XForms) but split into a series of very short 15-minute presentations by different speakers. This made it almost impossible to bore the audience, and also constrained the speakers to be concise and to the point. So kudos to the organizers, in particular to our working group chair, John Boyer, for making this happen.

The main point of my presentation was that with XForms and eXist, you don’t need a middle tier for a wide category of applications: XForms can directly talk to the persistence layer, in this case eXist, with xforms:submission and REST, while providing all the CRUD and search operations (through XQuery). This point resonated with other speakers’ point of view as well.

Elliotte Rusty Harold ended the evening with an excellently delivered but provocative keynote about what XForms proponents need to do to ensure the sucess of XForms. At Orbeon, we disagree with Elliotte’s contention that it is necessary to bring XForms natively to the web browser:

• After all, Java, PHP and Ruby are a success today on the web without being delivered to the browser.

• XForms implemented as a combination of server-side and client-side software in fact has considerable benefits, such as ensuring the security of the data handled by your application, ease of upgrade, and especially the fact that this allows to deliver XForms today to mainstream web browsers.

• Finally, XForms as a forms technology (as opposed to a general-purpose UI technology) fills a huge gap in the industry. This, along with quality implementations (among which we hope Orbeon Forms stands) able to deliver XForms to the web, will secure its future.

You can read the Forms Working Group’s short debriefing in our meeting notes for today, and the slides of my talk are available on slideshare.