With the advent of Google Suggest it seems that the industry has deemed that client-side XML HTTP is ready for the prime time. The technology is nothing new, of course, and has been part of every server-side developer’s standard toolkit for years, but whilst some browsers have maintained support for XML HTTP for a few years, it’s only recently that support has been widespread enough to utilise.
Interestingly enough, the XMLHttpRequest is not part of any public standard. The W3C DOM Level 3 ‘Load and Save’ spec covers similar ground, but you know how long these things take to get implemented. At the time of writing, if you need to use XML HTTP from a user agent, then the XMLHttpRequest object is the only way you can do it.
So what is XML HTTP?
The idea itself is very simple. By using JavaScript, a web page can make requests to a web server and get responses in the background. The user stays on the same page, and generally has no idea that script running on the page might be requesting pages (using GET) or sending data (using POST) off to a server behind the scenes.
This is useful because it enables a web developer to change a page with data from the server after the page has already been sent to the browser. In a nutshell, this means that the page can change based on user input, without having to have pre-empted and pre-loaded that data when the page was generated.
Example: Google Suggest. There’s no way Google can have any idea what you might be about to search for, so when you start typing, JavaScript in the page sends the letters off to the server and gets back a list of suggestions. If the JavaScript wasn’t able to talk to the server, the page would have to have been created by the server initially to hold every single search term you might type – which would obviously be impractical!
So what can we use this for?
Obviously, the technology has a place in creating a better use experience in terms of user interface. That is not its primary use, however. Its primary use is reducing load on the server.
That may sound mad considering that, in the example of Google Suggest, an additional request is being made to the server with every letter you type. Surely that increases the load by a magnitude, no? Well, no. Each ‘suggestion’ list served by Google is a very inexpensive list to produce. It’s not particularly time sensitive, and it’s not computing anything to get you there – it’s simply giving you a snapshot of a search phrase list sorted alphabetically and then by PageRank (or something similar, but it’s just a list).
Consider what happens if you don’t make a selection from the list and you keep typing. You perform your search and get back 6 million results – no problems there. But if you made a typo, Google’s just had to retrieve 6 million results that you have no need for. You retype and get the 6 million results you’d originally hoped for.
Now consider what happens when you do pick a selection from the list. Well, first thing is that the list doesn’t contain typos, so that problem has been eliminated straight off. Presuming that you clicked on the item you intended to select, you get a perfect set of results straight off. More importantly, however, is that you get a set of results that Google’s already got. Because you pick from the list, there’s no need to reperform the search.
An example
Take the example of searching for information on Britney Spears’ undergarments. I might think of searching on “Britney Spears knickers”, but if I see “Britney Spears panties” in the list, then I’m going to go ahead and select that instead. If Google already have a search for “Britney Spears panties” cached (and believe me, they do) then the reduction on load on the server for retrieving that search vs performing a new search on the uncached “Britney Spears knickers” is significant.
Side note: I checked both of the above search terms, and yes, they both appear in the list. I’d like to say I’m shocked, but amused will have to suffice. Interestingly, this proves the suggestions aren’t more than a simple list lookup (i.e. they’re not intelligent), as a search for “Margaret Thatcher panties” yields no such suggestions.
So, another example on how this technique reduces load. Say that you have a couple of select lists on your page, and in a drill-down style the user’s selection in the first list determines the options available in the second. There are two traditional ways to do this. The simple method, if the number of permutations for the second list isn’t too great, is to pre-load all the options as arrays when the page is built. Of course, for many applications this simply isn’t practical, as either the permutations are too great or are too lengthy to preload into the page at build time. In this case the second option is to have JavaScript post the entire page back to the server after the first selection is made, so that on the reload the server can build the second list with the appropriate options.
Now, this practise, especially the second option, doesn’t sound too heavy on the server. Consider the case, however, where the page containing the two select lists holds a much larger form with all sorts of data that has to be retrieved from a database and processed for display. Some of the other data might contain complex calculations or enormous queries that are very expensive to run. Consider also the amount of work involved in a simple post-back of a large form. All the data has to be re-read from the post and written back into the form in case the user is part-way through completing.
By utilising XML HTTP to fetch the options for the second list behind the scenes, you not only make the experience a little more slick for the user (no page reloads), but you also reduce the load on the server as it doesn’t have to rebuild that page.
Unifying front- and back-end processing
Another neat trick you can perform using this technology is using the server to perform any tricky processes that have until now been left to the client.
Take the example of input validation. In modern web apps, all validation of user input is typically performed twice – once on the client with JavaScript, and once at the server in case the client-side process failed. What this means is that the validation routines have to be written twice and maintained in two places. That’s ok if it’s just a case of checking the user has entered their surname, but if the process is any more complex than that (consider evil numbers with checksums etc), you’re into writing two bits of code to do the same thing.
If you use XML HTTP to tackle this problem, any complex validations with their checksums and wotnot can be posted off to the server during the validation routine, and the server can check it using its own process (the same processes it will re-check it with in a fraction of a second’s time) and spit back a result. And that, as they say, is magic.
So what does it all mean?
Well, now that Google are at it, every bugger’s gunna want it. We should prepare ourselves for an onslaught of badly implemented assisted searches with filthy client-side code the likes of which have not been seen since some utter twit figured it’d be a nice idea to create drop-down navigation using JavaScript. If it’s not on DynamicDrive already, give it a few days.
For us, it means that we should be reading up, trying it out and adding the technique to our ever-expanding toolbox o’tricks. XML HTTP is a useful device and certainly one that it pays to be aware of, especially when it comes to reducing the amount of work your servers have to do. What it’s not is some big revolution that’s going to change the way we build web apps. It will help us build better ones, but you may never notice.
Comments
Anyway, good overview. I’m also enjoying the rise in popularity of the word “twit”. Excellent.
I’ve been using this technique for quite a while to populate drop down lists based on other selections on the page (as you discussed), so I would say this technology primarily facilitates dynamically loading portions of a page in the background.
Have to disagree about the performance aspect though. The performance difference between getting a list or computing a result on the back end for someone like Google would be relatively trivial, whereas the overall load (including network infrastructure) will increase due to the extra volume of requests and network traffic you are generating.
I’m not saying this is a bad technology (it clearly has a lot of great potential), I’m just wary about how it will be used.
I think that the benefits will far outweight the pitfalls.
Of course the possibilities seem very interesting, but I’m paranoid. If anyone needs me I’ll be searching the web for more info for the next few hours.
Actually, since I have seen Google Suggest, I have been thinking of ways I could use this at work in an accessible manner. Meaning, I don’t want to give the users with Javascript enabled so much more than those without it that it would diminish the usefulness/purpose of the application.
I see great potential for making more rich web applications using this.
I’d like to see developers taking all the advantages of Macromedia’s RIA concept and actually making it work with standard, usable technologies.
:)
Hear, hear!
>> of Macromedia’s RIA concept and actually making
>> it work with standard, usable technologies.”
>
> Hear, hear!
Well, you’re welcome to do so… the more good sites with varied technologies, the better. Go for it! What project do you have in mind…?
Regards,
John Dowdell
Macromedia Support
I have a dozen local web pages set up to pull content (HTML scraping?) from other sites so I can read everything in one place. Basically, I have made my own huge comics pages (about 120 comics from different sources). It’s cheating, I know, but it started out as a real project and just kind of degenerated into selfish gratification.
I know that XMLHttpRequest has practical uses, but it’s also tons of fun for hacks…
I also would like to see more about this and accessibility implications. Just when it was starting to look like accessibility was gaining popularity and JavaScript menus were losing it. Doh!
Most designers will ignore intended audience and use it for the wow factor, I’m assuming.
The SVG group seems to be the only one trying to improve things, so they are the ones on top of things enough to put it in at least some spec.
That being said, I’m sad that google has stolen my thunder. No more wowing customers (“Hey, how’d you do that without a page refresh, that’s slick, I love it, here’s a big fat cheque”)
Well, at backbase we are doing just that. e.g check for example sourcecode of:
http://www.heinekenmusic.com/
-m
Still 99.999% of web applications reload entire datasets into html tables if only one row or cell has changed, xmlHttp has given me the ability to do only that row, instead of reloading the entire page again.
Hmm… the front page says, “This broadband site is optimised for IE 6 for Windows and Mozilla 1.4 for Mac,” yet the inside seems to use four or five SWFs. Is there special JavaScript which limits the audience, or…?
(btw, anyone have a matrix of the current levels of XMLHttp support across various browser brand/version/platform mixes?)
jd/mm
Like the first comment, we already use this feature in our Enterprise Web Applications to provide a wonder ful user experience, where users get a VB (or a client server app) kind of experience without the normal page refreshes.
Previously it used to work only on IE, but it is good to note that FireFox has implemented capability for XMLHttp Requests.
This is incorrect. The latest Firefox supports DOM 3 Load and Save (not sure how long that’s been there), and so do the latest Opera betas. Konqueror also seems to have an (incomplete) implementation.
It’s very similar to the hidden frame method, but much more flexible. You have access to status codes, header information, and the response as either XML or text. I may be wrong, but I don’t think you can get all that with the hidden frame method.
it’s a magic :-)
the speed is incredible, compared with page refreshes. ALso I can get back only the “value” like 0 or 1 wich in it’s self makes everyhitng much much faster.
great, http://www.google.com/webhp?complete=1
Thanks for an excellent articles.
XMLHttpRequest is great and all, but I challenge somebody to this problem I have been experiencing:
Create an XMLHttpRequest that populates it’s data into a html list in javascript then say, change the selectedIndex to “2” underneath it, it won’t let you. It will change, but then default back to -1 by itself, I believe when it goes to make the open call, and then select the entry in the drop down list, it doesn’t realise the select list has been populated yet therefore selecting null. If i add a simple alert(‘test’); above this line, allowing everything to populate and waste a few seconds, it would then populate when i hit ok. If anybody DOES figure out a solution to this, could you please e-mail me on graham at resonline dot com dot au
Anyone who has worked with Microsoft’s special style filters might have come across this: If a user disables Active X components, Microsoft’s style filters don’t work.
It looks like the XMLHttpRequest object is part of the Active X objects too. I’d strongly recommend testing web applications that are going to use the XMLHttpRequest object, with Active X disabled. I believe anything higher than (and possibly including) Medium security in MSIE will block Active X components.
It’s been a while since I used MSIE so I can’t recall the specific settings.
(sorry, thought you had trackback)
I’ve actually been using this method since about April ‘04. In fact – as Sr. Developer at my previous co. I mandated it’s usage.
I think that intensive web applications not using the XML Request are intrisically less capable of massive adoption without seeing massive overhead. I’ve done some benchmarking, and concluded that in one of our applications (roughly 28,000 unique users per hour) we were able to decrease server load, and bandwidth by over 87%. That is STAGGERING! We figured that had we used traditional methods, we would have needed a least one more server to handle the load, and our client would have been paying roughly 8k a month in bandwidth costs…
Since that time I’ve used it in thin client’s for network hardware devices, hundreds of sites, dozens of applications… I thank God every day for it…
Sometime back I had trouble getting a keystroke on a page. In IE its done with event.KeyCode and for mozilla class of browsers, it is Keystroke.which.
Would I have similar problems like these with XMLhttpRequest??
Regards
Arun Kumar
http://www.backbase.com/
Nice components.
It’s an implemention of the object in javascript & java.
http://www.scss.com.au/family/andrew/webdesign/xmlhttprequest/
regards,
Rodrigo
Cheers,
Although i fear it may be too late :(
hehe, thanks for that :)
With XMLHTTPRequest on the client side and XML web services on the server side developers will finally have a straightforward method for consuming web services that doesn’t require a any additional controls on the client side. I think this is going to be a big deal. Well, if I have a say it will be anyway.
Great article.
M@
I really enjoyed reading this; I hadn’t really thought about XMLHttpRequest from that perspective before. Now I’m codind an auction script using this technology.