All in the <head>

– Ponderings & code by Drew McLellan –

– Live from The Internets since 2003 –

About

The Best Forms Implementation I've Ever Built

162 days ago

The one thing that will really kick your developers’ butts when building an interaction-heavy web app or site is the forms. Forms can be a lot of work to implement. Get the technical design of your form generation/handling/validation system right and your project can fly along. Get it wrong, and you’re sunk in work that becomes tedious and demotivates everyone involved.

Now, before you tell me that this is a non-issue because no one builds individual forms anymore and that they’re all auto-generated by frameworks, I’m talking about the bit that goes on inside the framework. My consideration is how you design a system that outputs forms without resulting in a sucky literal representation of a database row in HTML.

Anyone who cares about interaction design and gathering accurate data carefully designs and tunes forms individually to suit the task in hand. Auto-generated data entry forms are fine for routine back-office jam-this-data-in-a-database-table tasks, but they suck for anything that matters.

So the problem becomes one of how to feed the form engine with the data it needs whilst still giving designers control of the markup and user experience.

Attributes of a form handling system

There are a few core things that a form handling system needs to do.

Firstly, it needs to generate the HTML for the form itself, including all input fields, labels, default values, surrounding help and tips and error messages.

Secondly, it needs to be able to detect that the form has been submitted, and validate the data for required fields, check the format of any data, and verify any co-dependancies between fields (such as two password fields matching). If the validation fails, errors need to be set and the fields all need to be repopulated with the data that was just submitted.

Lastly, when the form passes validation checks, it needs to collect all the data up and pass it along the line for the next step of the application to deal with.

The processing of this generally falls into two places – MVC-types would say in the Controller and the View – but can simply be thought of as before any browser output and down in the page.

What happens down in the page could be dismissed as basic template conditionals, but there’s such a level of complexity with repopulated vs. default values, messages, errors and such that I really consider it to be worthy of more detailed consideration than just basic templating. If the goal is to reduce work, then it needs more thought.

Bad designs are easy

Due to this split nature of forms, the usual design for a system of this nature is declare all the fields in code, and then have some templating system handle the output down in the page.

A system I used for a while was PHP’s HTML_QuickForm (now QuickForm2) which defines fields like this (from their hello-world tutorial):

$fieldset = $form->addElement('fieldset')->setLabel('QuickForm2 tutorial example');
$name = $fieldset->addElement('text', 'name', array('size' => 50, 'maxlength' => 255))
                 ->setLabel('Enter your name:');
$fieldset->addElement('submit', null, array('value' => 'Send!'));

The shape of the form is defined in code and then piped into a standard template for output. If you want to customise the HTML output, you can code up your own custom renderer. I’m an experienced PHP developer, and that makes my toes curl.

Designers need control

In my ideal world, a designer should be able to put a form together on the fly, working in HTML as much as is possible. When every form could have a small area of uniqueness, custom renderers or overriding templates isn’t the way to go – let the app deal with the input tags, but the designer needs direct control over the HTML.

That’s the principal I’ve stuck with for the last five or six years (since deciding QuickForm wasn’t the way to go) and have designed my form systems around it. I say systems, but really it’s just one which has evolved and finds use in both the edgeofmyseat.com web app and CMS frameworks, and ultimately in the control panel of Perch.

That system took me part way there, with the layout of a form being generated directly in the HTML, but with validation and processing rules being declared up in the PHP code before output. So it was a good step forward, but still required form declaration be split across two places.

Adding forms to Perch

When it came to designing a way for Perch users to add forms to their websites, I knew I’d need to do better still. One of our design principals is that we try not to abstract the designer away from the page. If you want to add something to a page during site build, you go into the page and add a region.

We also wanted designers to be able to throw in a form into pretty much any situation without needing to think too much about the technical implications. If you’re listing out products, you should be able to throw in an “add to cart” form, or a booking form for an event. That sort of thing. Wherever you’re outputting content, you should also be able to output a form.

I quickly came to the conclusion that our forms would have to be completely declarative. Rather than specifying a form in code and have a template turn that into HTML, we’d let designers create the form in as-close-to-HTML as we could and let the code figure the rest out.

Enter HTML5

One of the best things about HTML5 for me is the improvements that have been made to forms. You can now specify a field as, e.g.

<input id="email" name="email" type="email" required="true" />

and a supporting browser will prevent the form being posted until the field is completed and contains a valid email address. No fuss, no tangle of ugly server-side code, just simple, easy to use declarations. I thought this was the perfect model for making forms simple, so I copied it. I wrote a complete server-side implementation of HTML5 forms.

Inevitably, we had to add a bit of magic around any forms, and it was never going to be a case of just using straight HTML, but I tried to make things as natural-feeling as possible.

<perch:label for="email">Email</perch:label>
<perch:input id="email" type="email" required="true" />

Outputs:

<label for="form1-email">Email</label>
<input id="form1-email" name="email" type="email" required="true" />

Enhanced with the magic needed to pre-fill and re-fill field values, automatically ensure that IDs are unique in the page and so on. If you want to generate an error message:

<perch:error for="email" type="required">
	Please enter your email address.
</perch:error>

Change type="required" to type="format" and specify an error if the format isn’t correct. These can include any markup you need, and go anywhere in the form – near the field or at the top, whichever is your preference. And when the form is successfully submitted,

<perch:success>Thank you for filling out the form!</perch:success>

specifies the response. (Of course, as this is all in content templates, any of the text or even form attributes can be content managed.)

Making it work

It’s one thing to have forms, but you need to be able to process them with something. Perch has a system of ‘apps’ (add-on functionality, e.g. Blog, Events, Gallery etc) which now all have the opportunity to make use of forms.

When specifying a form, the designer adds an app attribute:

<perch:form id="add_to_cart" app="perch_products">

When a form is submitted for a named app, the app is notified and handed a prepared, validated set of fields and files that have been uploaded, along with the ID and a copy of the form template all loaded up and ready for further inspection if required. Creating a app using forms is about as trivial as it gets, which is great.

So what’s your point?

My point is this. I’ve built a lot of different form handling systems over the years, in a few different languages, and they’re hard to get right. If experience has taught me anything, it’s that a design that doesn’t put the web designer in control of the output is going to end up being a burden to your project.

This design shifts form configuration into the template and I think it really works well. Writing forms is fast and simple because by the time you’ve built your template you’ve defined the form. I really do think this is the best form implementation I’ve ever built, and so I thought it would be useful to share.

- Drew McLellan

The Lure of On-page Editing

305 days ago

I get asked quite often why Perch doesn’t offer any sort of on-page editing. It’s something we’ve given a lot of consideration to, and as it’s an interesting software design issue I thought I would document some of my reasoning here.

So what do I mean by on-page editing? In the context of a content management system, we’re talking about the ability to make edits to a page directly by clicking on the content (or a nearby Edit link) and manipulating it in place on the front end of a website. The alternative approach in contrast to this — and the approach we currently take with Perch — is for the user to go to a dedicated control panel to edit the content away from the page.

There are a couple of issues with on-page editing in my mind.

The technical

The big technical hurdle with on-page editing is the necessity for the CMS to inject its UI into the front-end page. At first that doesn’t sound too bad, but the more you think about it, the bigger a challenge that is. The pages own CSS and JavaScript are all going to influence the editing UI in unpredictable ways. The CSS you might think can just be dealt with by using very specific selectors and some sort of local CSS-reset, and you might be right, but what about z-index, overlapping absolutely positioned elements, dealing with Flash? JavaScript is another issue – if the editing UI uses JavaScript, will it conflict with anything on the page? Will the page’s own JavaScript conflict with it? What if the page attaches event listeners to the editing UI by mistake? What happens if two versions of the same library get loaded? What about sIFR?

These are all thorny technical issues that require effort to work around. Are they insurmountable? No – they can all be addressed. However, it becomes a Red Queen’s race to address issues as they arise, requiring a considerable amount of development time (not to mention the associated support) in order to maintain the status quo. If you have limited development resources, taking on that sort of burden could really inhibit your ability to move the product forward. As a developer I want to spend my time making a product better, not keeping it the same.

The touchy-feely stuff

More important than the technical considerations (which can all be dealt with if you choose to take them on) are the usability considerations. Will this be good for my clients editing their site? On-page editing certainly gives good demo, but does it go further than that?

Obviously, the immediate usability issue that has to be overcome is how clients edit content that’s not always visible on the page. Things like modal overlays and JavaScript carousels all requires a lot of thought, especially when you have no idea of the sort of sites where the CMS will be implemented.

That’s not my main concern with on-page editing, however. My main concern is this. One of the primary benefits of a CMS is being able to reuse bits of content around your site whilst maintaining it in one central place. A client can create, for example, a listing of news items on a News page, and also have the latest item or headline display on the home page. This is where on-page editing falls down.

On-page editing creates a model whereby the client is told they are directly manipulating the content on the page. It tells them they are taking a ‘physical’ page and remoulding it to their requirements before setting it down. That’s a really powerful metaphor, and I can clearly see why clients would like it.

However, as soon as you attempt any content reuse across the site, the metaphor is broken. The physical model becomes metaphysical, as a change to content in one place and affect content in another. This leads to unpredictability in the interface, which is the very worst thing for less technically-confident clients. Unpredictability leads to a distrust of the software, which leads to it just not getting used.

The control panel editing experience creates an abstraction from the page quite deliberately. That abstraction is needed in order to be able to make use of the technology in the way that best serves the client. I don’t believe the direct-manipulation metaphor (i.e. on-page editing) can be maintained whilst still offering the labour-saving benefits of managed content.

It’s about giving users confidence

Much of what we do with Perch is designed to give those less technically-confident clients confidence in editing their sites. We don’t want to see designers handing off a CMS to their clients only to be getting a phone call every time some content needs changing. The abstraction of using a control panel for editing enables content reuse around the site – that means the client doesn’t need to remember a long task checklist each time they make a change.

Features like preview, drafts and undo are all designed to give clients the confidence that they can’t make a mess of the site, or if they do, they can back out of it.

On-page editing is a technical and a design challenge, but we’re not afraid of those. There are plenty of instances where we’ve taken on difficult problems because it will make the user experience much better. It’s more that I believe that direct-manipulation is a flawed approach because it is in direct conflict with the goal of using software to reduce tedious, repetitive tasks.

If we can find a way in the future of implementing on-page editing without those downsides, we’ll certainly give it some attention. I want the editing experience with Perch to be the best it can be for clients. At the moment, although on-page editing looks great in demos, I don’t think that implementing it is at all in the client’s interests. In fact, I believe it’s against their interests.

- Drew McLellan

Ideas of March

319 days ago

When I started this site in 2003 — as best as I can tell it was 2003 — a individual’s personal site or blog was pretty much their primary method for joining the online conversation. If you had something you wanted to share, you wrote a blog post. Others would read it, and if they found it to be of value they might link to it in a post of their own. Or leave a comment. That’s how we all communicated.

At that time there was an amazing wealth of content being posted to blogs, and truthfully, it was the primary way I picked up information around the subject of web design, and spurred me to form my own opinions on subjects I wouldn’t have otherwise thought about. Because you can’t blog without an opinion.

One thing I’ve found over the last four-and-a-half years of using Twitter (where did that time go?) is that the opinions and ideas that I used to consider for a while and then focus into a blog post now get fired off into a 140 character tweet and forgotten. Twitter has in many ways replaced the role of blogs as the simplest outlet for the shared thought. Rather than going through the process of refining thoughts and reasoning into something (hopefully) coherent, we condense those thoughts into a single terse headline and move on.

Whilst I really love Twitter, I do think it’s a shame that we now mainly get to hear people’s opinions, but without hearing the reasoning behind those opinions that you would normally find in a longer blog post. Not to mention that it’s far easier to tweet without an opinion, and I think the conversation is fundamentally weakened that way.

We need a blog revival

This isn’t a backlash against Twitter, however. There’s room for both — for quick headline thoughts and for more reasoned posts. I think it would be a shame to have only the former and none of the latter. As such, I’ve been making a bit more of an effort to dust off my own blog and to post some of the things I would normally just tweet. Prompted by Chris, I’m making the pledge to post more for the rest of March, as I have already begun to this last week.

Here’s how you can join in the blog revival:

  • Write a post called Ideas of March.
  • List some of the reasons you like blogs.
  • Pledge to blog more the rest of the month.
  • Share your thoughts on Twitter with the #ideasofmarch hashtag.

Let’s see if we can tip the scales back a little and find a better balance between tweets and posts. Will you join us?

- Drew McLellan

Tags

Photographs

Work With Me

edgeofmyseat.com logo

At edgeofmyseat.com we build custom content management systems, ecommerce solutions and develop web apps.

Follow me

Recent Links

Affiliation

  • Web Standards Project
  • Britpack
  • 24 ways

I made

Perch - a really little cms

About Drew McLellan

Photo of Drew McLellan

Drew McLellan (@drewm) has been hacking on the web since around 1996 following an unfortunate incident with a margarine tub. Since then he’s spread himself between both front- and back-end development projects, and now is Director and Senior Web Developer at edgeofmyseat.com in Maidenhead, UK (GEO: 51.5217, -0.7177). Prior to this, Drew was a Web Developer for Yahoo!, and before that primarily worked as a technical lead within design and branding agencies for clients such as Nissan, Goodyear Dunlop, Siemens/Bosch, Cadburys, ICI Dulux and Virgin.net. Somewhere along the way, Drew managed to get himself embroiled with Dreamweaver and was made an early Macromedia Evangelist for that product. This lead to book deals, public appearances, fame, glory, and his eventual downfall.

Picking himself up again, Drew is now a strong advocate for best practises, and stood as Group Lead for The Web Standards Project 2006-08. He has had articles published by A List Apart, Adobe, and O’Reilly Media’s XML.com, mostly due to mistaken identity. Drew is a proponent of the lower-case semantic web, and is currently expending energies in the direction of the microformats movement, with particular interests in making parsers an off-the-shelf commodity and developing simple UI conventions. He writes here at all in the head and, with a little help from his friends, at 24 ways.