XForms Everywhere

If you are using XInclude in IntelliJ, you might have seen errors like this:

An XInclude errors reported by IntelliJ 8.1

As of IntelliJ 8.1, you can’t disable this behavior so rather than become numb to errors reported by IntelliJ, you can setup it up to look for those files in your resources folder by adding resources a Source Folder or Test Source Folder.

Adding resource paths as source folders in Project Structure (screenshot in IntelliJ 8.1)

A negative side-effect of this configuration is that when compiling your source with IntelliJ as described earlier, it will copy all the files that it considers to be resources in the output path. Mostly, this means that compiling will start taking more time than it should. You can disable this behavior by editing the pattern that describes what IntelliJ considers to be a resource. You can change this pattern under Settings, Compiler, Resource patterns. Entirely removing the pattern will do.

Removing the resource pattern under Settings, Compiler (screenshot in IntelliJ 8.1)

Here is a summary of the Orbeon Forms developments for March 2009:

1. Form Builder and Form Runner

• Form Runner L&F improvements. The Form Runner look and feel is improved! We got rid of the overwhelming orange color in favor of a more appealing blue theme, put nicer-looking buttons, and reworked the positioning of the logo, title and description areas. Let us know how you like it! Form Builder is next in line for UI improvements.

• Form Runner and Form Builder packaging. In order to facilitate Orbeon Forms updates as well as modularity, Form Runner and Form Builder are now packaged in separate JAR files in the distribution. For development, they are still under RESOURCES/apps/fr and RESOURCES/forms/orbeon/builder.

• More flexibility with sections. Form Runner now has a bit more flexibility with sections: you can disable open/close in Ajax mode, or just disable animations.

2. XForms engine improvements

• XBL support is maturing. We put in lots of XBL bug-fixes last month. We have also setup a consistent directory hierarchy for XBL components.

• Map component. Nightly builds now feature a new map component written in XBL.

• More AVTs on controls. xforms:textarea supports xxforms:cols and xxforms:rows attributes as AVTs. All controls support the style attribute as AVT.

3. XForms tips from our scratchpad

• Up/down buttons. Here is a simple XForms 1.1 howto: Up/Down buttons in a table with xforms:repeat, xforms:insert, and xforms:delete.

• Optional empty item. XForms 1.1 tip to optionally add a first empty item to a select1 based on some condition.

4. Testing

• JavaScript unit test driver. Our JavaScript unit testing framework is now able to run all the tests at once.

• Running server-side tests from IntelliJ. We added some developer documentation about running Orbeon Unit Tests from IntelliJ IDEA.

That’s it for today. Enjoy!

Here is a summary of the recent Orbeon Forms developments:

1. Form Builder and Form Runner

We keep polishing Form Runner and Form Builder. In addition to the usual bug-fixes:

• Form attachments are attached to emails, and Form Runner is able to include values from form controls in email subjects.

• The Form Runner summary page is more configurable: there are new properties to hide the Created and Last Modified columns, values of type xs:dayTimeDuration have been added (eg “10 min 25 sec”), and documents that use namespaces are better supported.

• In noscript (accessible) mode, Form Runner properly shows all existing errors when the user presses the Save or Review buttons.

• Form Builder’s dialogs are all internationalized and localized in English and French.

• Form Builder has a clearer mechanism to enter an “application name” and “form name”.

• Form Builder supports a global components library as well as a library per application.

• Our Bookcast example form based on Form Runner is now called Bookshelf. It supports image uploads too!

2. Performance

• Local, optimized submissions, i.e. submissions to local pages/services without actually using HTTP, are improved and work better in separate deployment. A new property also allows optimized local submission using the servlet include mechanism. This is useful for submissions with replace="instance|text|none". See the new Documentation on local submissions.

• Orbeon Forms supports two different client-side event modes, including a new “deferred” mode which reduces the number of Ajax requests.

• In some situations where multiple Ajax requests were sent, we now just send one and avoid duplication of events in the request.

2. Notable features and bug-fixes

• Orbeon Forms deploys on Liferay 5 in addition to Liferay 4. We fixed lots of Liferay-related issues, and Orbeon Forms even supports running Form Runner and Form Builder within a portlet. Support has been added for Portlet 2 (JSR-268) resources (use OrbeonPortlet2Delegate portlet to enable JSR-268 mode).

• XBL components with local models and instances work within xforms:repeat, better handle HTML directly within XBL templates, and support xbl:script and xbl:style.

• xforms:input bound to a node of type xs:boolean works within xforms:repeat.

• xforms:submission supports submitting form values and uploaded files as multipart/form-data.

• File uploads, as well as xs:date, xs:time and xs:dateTime input controls work properly in noscript mode.

• xforms:output supports a new appearance, xxforms:download, which displays a link for, well, downloading the data pointed to by the control.

• xforms:input’s extension attributes (xxforms:maxlength, xxforms:size, xxforms:autocomplete) can be dynamic using AVTs.

• xforms:input supports changes to the type of the bound node (xs:string, xs:date, xs:time, xs:dateTime, xs:boolean).

• xxforms:modal="true" on triggers is improved to prevent double submissions using the keyboard.

• xforms:message actions are executed during page initialization.

• A new configuration property allows you to enable the XForms Inspector without having to change your page.

2. Other

• Orbeon participated in the XForms face-to-face meeting at Google. See this Open Source at Google blog entry for more details.

• We have started running a series of XForms 1.1 test suite tests on Orbeon Forms. Our wiki shows the work in progress (your help on this is welcome!).

• We use the Orbeon Forms wiki more and more and we have also reorganized the XForms documentation a little.

That’s it for this time. Enjoy, and don’t forget to follow our updates on Twitter.

It is common to see, in web applications, Gmail-like UIs, where you show a list of items (emails in the case of Gmail), and where you’d like users to be able to perform actions on multiple items at a time. For this, users need to be able to select items. This can be done by adding a checkbox at the beginning of each line. It is easy enough to list your items in XForms with an <xforms:repeat>, but developers often wonder how to add that checkbox at the beginning of every line.

Let’s consider a practical case:

• You have an instance with a list of countries. Each country has a code, which can be used as a key to identify the country.
• You’d like the selected countries to be stored in a separate instance, as a space separated list of country codes.
• You don’t want to change the instance that contains the list of countries in any way (no adding of an attribute or element “selected” for each country).
• You want to generate a UI like the one shown along this text.

The trick here is to have:

• One <xforms:select appearance="full"> control per line.
• All the <xforms:select appearance="full"> have exactly one item, each with a different value (the country code) and they are all bound to the same node.

Show me the code! The space separated list of country codes will go in the root element of the selected instance:

<xforms:instance id="selected">
    <selected />
</xforms>

And finally the <xforms:repeat> which iterates over the countries and generates the <xforms:select appearance="full"> looks like:

<xforms:repeat nodeset="instance('countries')/country">
    <xxforms:variable name="country" select="."/>
    <xhtml:tr>
        <xhtml:td>
            <xforms:select appearance="full" ref="instance('selected')">
                <xforms:item>
                    <xforms:label/>
                    <xforms:value value="$country/us-code"/>
                </xforms:item>
            </xforms:select>
        </xhtml:td>
        <xhtml:td>
            <xforms:output value="name"/>
        </xhtml:td>
        <xhtml:td>
            <xforms:output value="us-code"/>
        </xhtml:td>
    </xhtml:tr>
</xforms:repeat>

This works because the XForms select is non-destructive: when you uncheck a a checkbox from a specific XForms select, the engine will only remove the value corresponding to the checkbox you unchecked, and will leave other values in the space separated list unchanged. Job done!

For those of you who don’t yet follow our updates on Twitter, here is a summary of some of the new developments since our last post:

1. Form Builder and Form Runner

Work on Form Builder and Form Runner is ongoing. We have added important features:

• Support for custom components in the toolbox, which allows for supporting small components like phone numbers or reusing entire form sections.

• Options to control form submission in Form Runner (directly saving into a database, taking you to a confirmation page, showing a confirmation dialog, etc.).

• Support for sending emails with XML and/or PDF attachments.

At this point we don’t plan to add any new major feature for Form Builder version 1.0. We are working on a couple of deployments, which means that we are concentrating on bug-fixes and performance.

We have lots of ideas for Form Builder past version 1.0. Feel free to send your suggestions to the ops-users mailing-list, and keep in mind the various ways features get into Orbeon Forms.

2. XForms engine features and bug-fixes

Our focus has been on Form Builder and Form Runner, but we have also worked hard on the XForms engine, including:

• Improvements to the XBL-based component system, which is getting more solid.

• New extension XPath functions, including xxforms:is-user-in-role(), xxforms:get-remote-user(), xxforms:invalid-binds(), xxforms:encode-iso9075-14(), xxforms:decode-iso9075-14(), xxforms:doc-base64().

• Localized date picker (English, French, Spanish).

• Split the good old properties.xml file into several files, including properties-local.xml which facilitates upgrades.

• Label, help, hint, alert for a control can be located anywhere on a page with the @for attribute.

• Upgraded YUI to version 2.6.0 and integrated the YUI Rich Text Editor (RTE).

• Added support for the xforms:email and other XForms 1.1 datatypes.

Check out the latest change list to get an idea of how much has happened in the XForms engine since version 3.6.

3. When is the next release of Orbeon Forms coming out?

As always the answer is “when it is ready”, but we understand this is not enough for our users! The immediate plan is to release a second beta of 3.7 if not in December, then in January. Hopefully that can be the last beta before a 3.7 release candidate.

Nightly builds are quite stable now as we use them on various deployments. We don’t know yet if the next release of Orbeon Forms will coincide with Form Builder version 1.0. That certainly would be nice, provided it happens soon!

That’s it for this time. Enjoy!

Now that Orbeon Forms 3.7 beta 1 is out, we have committed a series of changes that intend to improve the performance of the XForms engine. These particular changes initially focused on <xforms:insert> and <xforms:delete>, but they ended up requesting a fairly major refactoring of the XForms controls code on the server. Some (probably obscure) developer information is available on our wiki.

In short, we used to duplicate the tree of controls on the server at every possible turn. Now we:

• never do it during page initialization

• do it at most once per Ajax request

Instead we update the tree of controls incrementally upon insert/delete and upon refresh.

The result is that the performance has in fact improved quite a bit for large forms, especially those that use <xforms:insert> and <xforms:delete> heavily. In particular, one of the main goals of this work was to make Form Builder faster. There is still work to do there, but performance has already improved quite a bit!

This is a good step forward: both the code and the processing model for the controls are now much clearer. We are also more XForms 1.1 compliant (in particular regarding repeat index updates). This also fixed along the way a few bugs related to insert/delete/repeat and UI events dispatching.

What’s even better, this lays down a healthy foundation for further improvements, including at some point implementing a dependency mechanism for the controls which should yield even better performance, especially for large forms.

In the last few months, we have been regularly posting updates to Twitter so we have had less of a need to write full blog entries. But now we feel that owe the followers of this blog a higher-level update:

• Form Builder and Form Runner. A big priority is our continued work on Form Builder and Form Runner. We are getting closer from a 1.0 version, and we have a great incentive as we have customers sponsoring features and planning to deploy very soon.

• Oracle XML support in Form Runner. Initially, Form Runner only supported the built-in eXist database. However, we designed from the get-go a REST-based API for handling persistence. We have now implemented a connector for storing form types and form data into Oracle.

• Noscript mode. The “noscript mode” in Orbeon Forms allows you to use a single XHTML+XForms description of a page and to target browser that have JavaScript disabled. This is now integrated into Form Runner, which means you don’t need JavaScript on the client to fill-out forms created by Form Builder. This is also the basis for the accessibility features we are working on (including WCAG AA compliance).

• Offline mode. We have worked quite a bit on an offline mode for Orbeon Forms, based on Google Gears. This is very exciting, but it is still a bit too early for us to integrate that into Form Runner.

• Components. An important aspect of any UI platform is the ability to reuse code. XForms should not be different. So we have started implementing a component system in the XForms engine, based on XBL 2. Simple components already work, and we are adding locally-scoped XForms models.

• Date and time controls. We have completely overhauled the date picker. You can now type in values and the date picker is by default the YUI date picker. We have also added controls for xs:time and xs:dateTime types, which was long overdue.

• Access request and session directly from XForms. In the past, accessing request and session information from XForms required using the page flow controller or possibly a page model written in XPL. Now, you can use XPath functions to do so.

• New YUI. We have upgraded to YUI 2.5.2. It’s good to be up to date and to benefit from the latest features and bug-fixes.

• Control the HTML title from XForms. Orbeon Forms now supports xforms:output within xhtml:title. This is very convenient to change the HTML page’s title directly from XForms.

• More flexible placement of AVTs and XForms models. AVTs are supported outside the HTML body. This allows for example controlling the lang attribute on the root HTML element. XForms models can alse now be placed outside the HTML head (e.g. within dialogs).

• Refactoring. A lot of refactoring of the Java code (and some of the JavaScript code) has taken place.

• Loads of bug-fixes. See the list of changes since Orbeon Forms 3.6. We hope that version 3.7 is not too far.

• New wiki. We now use Google Sites to host our development documentation.

This does sound like a lot, doesn’t it? And while all this is a lot of fun to work on, we need to mention that these new features and developments are driven by projects for our great customers.

Now, on to polishing Form Builder and getting 3.7 out!

With XForms 1.0, xforms:output was limited to displaying plain text. With the introduction of XForms 1.1, xforms:output adds powerful capabilities which include handling images, HTML content and more, depending on your XForms implementation.

To achieve this, XForms 1.1 introduces a new mediatype attribute. You use it to specify the type of the content to display, using the familiar Internet media type (or “MIME type”) syntax. This includes values such as image/* (any image content) or text/html (HTML content).

Orbeon Forms already supported the text/html mediatype, so we had to add support for image mediatypes. To achieve this, we made Orbeon Forms a little smarter.

• Consider that XForms allows you to refer to images not only as URLs (e.g. http://example.org/my/image.png), but as binary data embedded in an XForms instance. With Orbeon Forms, this data is handled on the server. So somehow, this information has to be brought to the client. In addition, browsers typically do not support providing image content as binary (although with some clients this is possible with the data: URI scheme)

• Even when using URLs, consider the following scenario: an image resides on your client computer, you upload it to the server through xforms:upload, at which point the image is available, in your XForms instance, as a temporary server-side URL. Now bind an xforms:output to that image URL: clearly, your browser cannot access that URL directly since it is server-side and therefore private.

Orbeon Forms solves these problems by creating proxy URLs for dynamically produced image content. Concretely, you see in the resulting HTML page paths of the form /xforms-server/dynamic/7FF0AD061DD9BAD27BCA1FF4674C75D6. Orbeon Forms takes care of handling these URLs as needed: you don’t need to worry a bit about this.

That was for the “smart” bit, so how does it look in the end to you, the form author? It’s as simple as this:

<xforms:upload ref="my-image">
  <xforms:label>Logo</xforms:label>
  <xforms:hint>Please upload an image</xforms:hint>
</xforms:upload>
<xforms:output mediatype="image/*" ref="my-image"/>

Yes that’s right: the few lines above handle uploading an image, attaching it to your data, and displaying the uploaded image to the user. Beat that.

We are glad to announce that we will be present at JavaOne 2008 on the OW2 Consortium stand. This year, JavaOne will take place May 6-9.

We have some great Orbeon Forms demos in the works so be sure to stop by!

Wow, it’s been a month since our last post in this blog, and one of the reasons for this is that we have been very busy enhancing Orbeon Forms! Here is a quick list of the most notable additions:

• XPath variables. XPath variables can be used within controls, models and actions, with the xxforms:variable (or exforms:variable) element. Variables are extremely useful, for example to avoid repeating long XPath expressions, or to give an XPath expression unambiguous access to data computed in enclosing xforms:group or xforms:repeat elements. If you are familiar with XSLT variables, you will feel right at home. And once you start using them, you won’t be able to go back!

• XML Schema validation improvements. Inline XML Schema support is now implemented. This allows you to embed XML Schema directly within the xforms:model element. This is particularly useful when you create a new schema which is just used within an XForms document, or to define a series of simple types used for validation. We have also added support for the xsi:type attribute in XForms instances. This is a convenient shortcut which allows you to avoid using binds just to specify types.

• Attribute Value Templates (AVTs) on XHTML elements. In Orbeon Forms, AVTs were already supported on some XForms elements, but you can now use AVTs on XHTML elements as well. This allows you to use XForms to influence aspects of XHTML such as class, style or colspan attributes, and all this without script.

• Action context enhancements. We added the exforms:iterate attribute, XForms 1.1 context() function, and a generalized context attribute on XForms elements (which may appear in XForms 1.2).

• Events enhancements. We now have full support for ev:observer and ev:target attributes on action handlers. We also added support for lists of event names, observers, and targets (an extension which we hope will be available in XML Events 2, currently in Working Draft at W3C). Finally, you can now pass context information to custom events with the extension xxforms:contex child element of actions such as xforms:dispatch or xforms:event.

• Image handling. xforms:output supports displaying images when used with xs:anyURI or xs:base64Binary datatypes and an image mediatype, as per XForms 1.1. It has never been easier to upload and display images from XForms!

• XPath extension functions. We added XPath functions to access headers and request parameters directly from XForms (xxforms:get-request-parameter() and xxforms:get-request-header()), as well as extension functions to create elements and attributes by name (xxforms:element() and xxforms:attribute()).

• Submissions. Submissions allow PUT and POST of binary content directly from XForms. This makes it very easy to upload a binary file using xforms:upload and then send it out to a service through xforms:submission.

For more information, check the temporary list of changes for the upcoming Orbeon Forms 3.7! We already have over one hundred enhancements implemented since version 3.6.