All in the <head>

– Ponderings & code by Drew McLellan –

– Live from The Internets since 2003 –

About

The Best Forms Implementation I've Ever Built

19 August 2011

The one thing that will really kick your developers’ butts when building an interaction-heavy web app or site is the forms. Forms can be a lot of work to implement. Get the technical design of your form generation/handling/validation system right and your project can fly along. Get it wrong, and you’re sunk in work that becomes tedious and demotivates everyone involved.

Now, before you tell me that this is a non-issue because no one builds individual forms anymore and that they’re all auto-generated by frameworks, I’m talking about the bit that goes on inside the framework. My consideration is how you design a system that outputs forms without resulting in a sucky literal representation of a database row in HTML.

Anyone who cares about interaction design and gathering accurate data carefully designs and tunes forms individually to suit the task in hand. Auto-generated data entry forms are fine for routine back-office jam-this-data-in-a-database-table tasks, but they suck for anything that matters.

So the problem becomes one of how to feed the form engine with the data it needs whilst still giving designers control of the markup and user experience.

Attributes of a form handling system

There are a few core things that a form handling system needs to do.

Firstly, it needs to generate the HTML for the form itself, including all input fields, labels, default values, surrounding help and tips and error messages.

Secondly, it needs to be able to detect that the form has been submitted, and validate the data for required fields, check the format of any data, and verify any co-dependancies between fields (such as two password fields matching). If the validation fails, errors need to be set and the fields all need to be repopulated with the data that was just submitted.

Lastly, when the form passes validation checks, it needs to collect all the data up and pass it along the line for the next step of the application to deal with.

The processing of this generally falls into two places – MVC-types would say in the Controller and the View – but can simply be thought of as before any browser output and down in the page.

What happens down in the page could be dismissed as basic template conditionals, but there’s such a level of complexity with repopulated vs. default values, messages, errors and such that I really consider it to be worthy of more detailed consideration than just basic templating. If the goal is to reduce work, then it needs more thought.

Bad designs are easy

Due to this split nature of forms, the usual design for a system of this nature is declare all the fields in code, and then have some templating system handle the output down in the page.

A system I used for a while was PHP’s HTML_QuickForm (now QuickForm2) which defines fields like this (from their hello-world tutorial):

$fieldset = $form->addElement('fieldset')->setLabel('QuickForm2 tutorial example');
$name = $fieldset->addElement('text', 'name', array('size' => 50, 'maxlength' => 255))
                 ->setLabel('Enter your name:');
$fieldset->addElement('submit', null, array('value' => 'Send!'));

The shape of the form is defined in code and then piped into a standard template for output. If you want to customise the HTML output, you can code up your own custom renderer. I’m an experienced PHP developer, and that makes my toes curl.

Designers need control

In my ideal world, a designer should be able to put a form together on the fly, working in HTML as much as is possible. When every form could have a small area of uniqueness, custom renderers or overriding templates isn’t the way to go – let the app deal with the input tags, but the designer needs direct control over the HTML.

That’s the principal I’ve stuck with for the last five or six years (since deciding QuickForm wasn’t the way to go) and have designed my form systems around it. I say systems, but really it’s just one which has evolved and finds use in both the edgeofmyseat.com web app and CMS frameworks, and ultimately in the control panel of Perch.

That system took me part way there, with the layout of a form being generated directly in the HTML, but with validation and processing rules being declared up in the PHP code before output. So it was a good step forward, but still required form declaration be split across two places.

Adding forms to Perch

When it came to designing a way for Perch users to add forms to their websites, I knew I’d need to do better still. One of our design principals is that we try not to abstract the designer away from the page. If you want to add something to a page during site build, you go into the page and add a region.

We also wanted designers to be able to throw in a form into pretty much any situation without needing to think too much about the technical implications. If you’re listing out products, you should be able to throw in an “add to cart” form, or a booking form for an event. That sort of thing. Wherever you’re outputting content, you should also be able to output a form.

I quickly came to the conclusion that our forms would have to be completely declarative. Rather than specifying a form in code and have a template turn that into HTML, we’d let designers create the form in as-close-to-HTML as we could and let the code figure the rest out.

Enter HTML5

One of the best things about HTML5 for me is the improvements that have been made to forms. You can now specify a field as, e.g.

<input id="email" name="email" type="email" required="true" />

and a supporting browser will prevent the form being posted until the field is completed and contains a valid email address. No fuss, no tangle of ugly server-side code, just simple, easy to use declarations. I thought this was the perfect model for making forms simple, so I copied it. I wrote a complete server-side implementation of HTML5 forms.

Inevitably, we had to add a bit of magic around any forms, and it was never going to be a case of just using straight HTML, but I tried to make things as natural-feeling as possible.

<perch:label for="email">Email</perch:label>
<perch:input id="email" type="email" required="true" />

Outputs:

<label for="form1-email">Email</label>
<input id="form1-email" name="email" type="email" required="true" />

Enhanced with the magic needed to pre-fill and re-fill field values, automatically ensure that IDs are unique in the page and so on. If you want to generate an error message:

<perch:error for="email" type="required">
	Please enter your email address.
</perch:error>

Change type="required" to type="format" and specify an error if the format isn’t correct. These can include any markup you need, and go anywhere in the form – near the field or at the top, whichever is your preference. And when the form is successfully submitted,

<perch:success>Thank you for filling out the form!</perch:success>

specifies the response. (Of course, as this is all in content templates, any of the text or even form attributes can be content managed.)

Making it work

It’s one thing to have forms, but you need to be able to process them with something. Perch has a system of ‘apps’ (add-on functionality, e.g. Blog, Events, Gallery etc) which now all have the opportunity to make use of forms.

When specifying a form, the designer adds an app attribute:

<perch:form id="add_to_cart" app="perch_products">

When a form is submitted for a named app, the app is notified and handed a prepared, validated set of fields and files that have been uploaded, along with the ID and a copy of the form template all loaded up and ready for further inspection if required. Creating a app using forms is about as trivial as it gets, which is great.

So what’s your point?

My point is this. I’ve built a lot of different form handling systems over the years, in a few different languages, and they’re hard to get right. If experience has taught me anything, it’s that a design that doesn’t put the web designer in control of the output is going to end up being a burden to your project.

This design shifts form configuration into the template and I think it really works well. Writing forms is fast and simple because by the time you’ve built your template you’ve defined the form. I really do think this is the best form implementation I’ve ever built, and so I thought it would be useful to share.

- Drew McLellan

Comments

  1. § David Carrington:

    I really like the idea of using the existing HTML5 elements and using those as the basis of your form markup design, although I think it would need to be adapted slightly away from the View in a stricter MVC framework. Thanks for taking the time not only to post this, but the video too is a great demo.

  2. § Tom Wright:

    Having also gone through myriad ways of achieving this myself over the last few years, I have to give a double thumbs up to most of this. Having a declarative way of throwing forms around is pretty much essential if you want to keep non-hardcore-back-end-developer types happy.

    Do you also have a programmatic interface to these widgets or is it all black box magic in the template renderer?
    The reason I ask is I’ve spent quite a lot of time playing with the QuickForm-like paradigm; there are quite a lot of really neat combinative things you can build with that system that are as difficult and messy to code in declarative tag representation as basic html forms are normally.

    There’s also the issue of trying to spit out certain portions of your page / form as responses to ajax actions. Have you toyed with this much? I could imagine it could start to get a bit fiddly!

    The sweet spot seems to be a very flexible QuickForm-like library that can be used as the underlying engine for the meta tags in the template. Then it’s down to context and personal choice as to which approach you take.

    Also, one little thing, I’d be careful with changing element IDs (in the example above it is translated from “email” to “form1-email”) – it’s fine when you’re prepared for it, but boy, that could get confusing when you’re trying to target form inputs in your javascript / css.

  3. § Robert:

    Drew, do you intend to add pattern or regexp validation (e.g. <perch:input id=“number” valid=”[0-9]{4,5}” />) ?

  4. § Drew McLellan:

    Yes, it’s already there – HTML5 uses the pattern attribute for that.

  5. § Chris Garrett:

    The trick we’re finding with anything auto-generated is to always allow hooks for easy override. If you adopt an aspect oriented approach then you can rapidly scaffold class properties (or however you describe your form) and add a filter when you want to override something specific. Much like filters in WordPress. This works incredibly well for a language that supports closures too…

Photographs

Work With Me

edgeofmyseat.com logo

At edgeofmyseat.com we build custom content management systems, ecommerce solutions and develop web apps.

Follow me

Affiliation

  • Web Standards Project
  • Britpack
  • 24 ways

Perch - a really little cms

About Drew McLellan

Photo of Drew McLellan

Drew McLellan (@drewm) has been hacking on the web since around 1996 following an unfortunate incident with a margarine tub. Since then he’s spread himself between both front- and back-end development projects, and now is Director and Senior Web Developer at edgeofmyseat.com in Maidenhead, UK (GEO: 51.5217, -0.7177). Prior to this, Drew was a Web Developer for Yahoo!, and before that primarily worked as a technical lead within design and branding agencies for clients such as Nissan, Goodyear Dunlop, Siemens/Bosch, Cadburys, ICI Dulux and Virgin.net. Somewhere along the way, Drew managed to get himself embroiled with Dreamweaver and was made an early Macromedia Evangelist for that product. This lead to book deals, public appearances, fame, glory, and his eventual downfall.

Picking himself up again, Drew is now a strong advocate for best practises, and stood as Group Lead for The Web Standards Project 2006-08. He has had articles published by A List Apart, Adobe, and O’Reilly Media’s XML.com, mostly due to mistaken identity. Drew is a proponent of the lower-case semantic web, and is currently expending energies in the direction of the microformats movement, with particular interests in making parsers an off-the-shelf commodity and developing simple UI conventions. He writes here at all in the head and, with a little help from his friends, at 24 ways.