Skip navigation

Tag Archives: Django

Brett and I recently began refactoring a significant amount of the JavaScript that is currently in MyFrontSteps and Homebook. One of the areas we identified as needing improvement was controlling when scripts get loaded in the page; it’s a challenging subject especially when utilizing Django templates which can extend and include bits of HTML that are both static and dynamic. We’re not finished the refactoring quite yet but I thought it would be valuable to blog about the lessons we’ve learned early on about how to manage JavaScript loading without having script tags all over the place.

Manual Dependency Management is Hard

Not too long ago, Yahoo put out a list of guidelines that web developers can use to enhance the performance of their websites. I won’t get into it in this post but there is a Firefox plugin called YSlow available that can automate some of the performance checking. The recommendation I’m going to focus on here is the one regarding moving scripts as close to the bottom of the page as possible. I’ll just quote the Yahoo doc as they explain it very clearly:

The problem caused by scripts is that they block parallel downloads. The HTTP/1.1 specification suggests that browsers download no more than two components in parallel per hostname. If you serve your images from multiple hostnames, you can get more than two downloads to occur in parallel. While a script is downloading, however, the browser won’t start any other downloads, even on different hostnames.

In some situations it’s not easy to move scripts to the bottom. If, for example, the script uses document.writeto insert part of the page’s content, it can’t be moved lower in the page. There might also be scoping issues. In many cases, there are ways to workaround these situations.

In the case of the code we’re using on MyFrontSteps we had all kinds of Django templates with includes and templatetags that include other little bits of dynamic JavaScript that was adding markup to the DOM and manipulating DOM content. Some of this code would appear prior to certain dependent scripts being loaded which created a real nightmare for us as we had to manually track down the position of the dependent scripts in the page that was fed to the browser after a variable number of layers of templates and includes. We decided to research a better way to manage all these scripts and since we had been using the YUI library for many other parts of the website we settled on the YUI loader.

YUI Loader Makes Dependency Management Easy

The YUI Loader Utility is a client-side JavaScript component that allows you to load specific YUI components and their dependencies into your page via script. YUI Loader can operate as a holistic solution by loading all of your necessary YUI components, or it can be used to add one or more components to a page on which some YUI content already exists.

The great thing about the YUI Loader is that you can use it to manage dependency sorting for all of the YUI components you use on a page but the killer feature imho is the fact that it allows you to create custom modules to load your own JS and CSS libraries. The basic pattern we’re using in our root level Django template is as follows:

  1. Load all CSS Files at the top of the template
  2. Place all YUI Loader and other statically included scripts at the bottom of the page prior to the closing BODY tag
  3. Define the YUI Loader instance
  4. Define custom modules for the YUI loader
  5. Provide a Django template block to allow manipulation of the onSuccess callback
  6. Call the YUI Loaders .insert() method to trigger insertion of all dependency sorted scripts into the DOM

The advantage to using the loader in conjunction with our root Django template is clear; all of our scripts are loaded at the bottom of the page and CSS is loaded at the top. When a user requests the pages they will start receiving the content from the server immediately and not have to wait for scripts to process as the loader dynamically inserts them into the HEAD element after the page content has been rendered. Another trick we’re using to control when scripts get executed is by manipulating how the onSuccess callback of the loader works. Here’s the code:

// Instantiate and configure YUI Loader:
MFSLoader = new YAHOO.util.YUILoader({
    base: "",
    require: ["base","reset-fonts-grids","MFS"],
    loadOptional: false,
    combine: true,
    filter: "MIN",
    allowRollup: true,
    onSuccess: function() {
        while( MFSLoader.onReady.length )  {
            MFSLoader.onReady.shift()();
        }
    }
});

// Define a list of executables that we
// can add to prior to page load completion
MFSLoader.onReady = [];

// Custom Modules for Loader
MFSLoader.addModule({
    name: 'MFS', type: 'js', varName: 'MFS',
    path: '{% vurl "/static/script/MFS.js" %}',
    requires: ['jQuery', 'jQueryUI']
});

{# Override this block if you need script at the global level. #}
{% block global.script %}{% endblock %}
// Trigger insertion of all dependency sorted scripts into the DOM
MFSLoader.insert();

As you can see we have a few of the custom modules defined here, which include a ‘requires’ attribute in the configuration object. This lets the YUI Loader know that these files require the other modules to be loaded. The Loader then determines sort order for inclusion based on all required modules and goes to work inserting the scripts for you.

Tying it all Together

The key to making this work in the varying levels of Django templates is to manipulate the onSuccess callback. When the YUI Loader finishes loading all the dependency sorted scripts it will execute this callback and allow you to then execute any additional code that depends on the dynamically inserted scripts. We have a number of namespaced JS objects for MyFrontSteps (MFS.js, MFS.DataGrid.js etc..) and when we need to call functions defined in these objects we do so by extending the global.script block in our child template and pushing a new anonymous function onto the MFSLoader.onReady list we defined in the root template. Here’s a simple example:

{# A CHILD TEMPLATE #}
{% block global.script %}
    #include the parent templates global.script contents
    {{ block.super }} 

    // Load our required features
    MFSLoader.require( 'MFS.DataGrid', 'uploader', 'MFS.Uploader' );
    // Push an anonymous function onto the list
    // to get executed when all required scripts have been loaded
    MFSLoader.onReady.push( function()  {
        MFS.DataGrid.initPhotosGrid();
    });
{% endblock %}

Once the page is rendered and the YUI Loader has finished parsing the loaded scripts the code we have in the onSuccess callback pops all the anonymous functions off the list which causes them to be executed.

{# THE ONSUCCESS CALLBACK #}
onSuccess: function() {
    while( MFSLoader.onReady.length )  {
        MFSLoader.onReady.shift()();
    }
}

We still have a number of files to refactor, but the performance benefits we’ve seen so far using this approach have really made us confident that this is the way to go. Using a dependency manager that works for custom JS / CSS libraries is just so much more liberating than having to manually keep track of where scripts are getting executed. Because we’re also migrating all of our code to external library files it makes things that much easier to debug in Firebug as well. 🙂

On Work

I’ve been at VendAsta for a week now and I’m happy to say that my initial impressions haven’t changed all that much. It’s really fulfilling to go in every day and feel totally engaged and encouraged by both the work you’re doing and the people you’re working with. I’ve heard some people say negative things about VendAsta; things like “they won’t be around longer than 8 months” or “they’re just a start up, how can they hope to accomplish anything”. I find most of the people saying these things are either uninformed about what it is we’re working on or just spiteful for some reason. Well, to those of you in either camp let me enlighten you.

VendAsta is about building quality software. VendAsta is about empowering people by allowing them to work on the things they are passionate about. Right now the team I’m working on is building a social marketing tool that will enable homeowners and home industry service providers to share their experiences on whatever social network they happen to be using. There is a lot of room for movement in this area of the social networking sphere, particularly because nobody else has really taken advantage of the online experience as it relates to our homes and home experiences. Rennovations, improvements, parties, appliances you purchased, decorating tips… the list of things people can share about their homes goes on and on. So when I hear people predicting the downfall of VendAsta I really think they have no clue about how much potential there is in the market segment we’re working in. Oh and did I mention, that’s only 1/2 of what we do? 🙂

On Macs

I made the switch to working on a Mac at home about 8 months ago. It was more of a novelty at the time, but I needed a change and was tired of “tweaking” my PC when I got home from work. I did manage to learn some important tricks about OS X and get a glimpse of what actually “working” on one would be like. Flash forward some 9 months and I work full time on a MacBook Pro developing with some of the most intriguing technologies out there. The only thing holding me back to Windows for the longest time was the games, and I find now that most of my gaming plays better on my iMac anyways. The keyboard shortcuts are kind of a pain to get used to, but once you realize you actually have an additional modifier key to work with things just kind of click 🙂

On Python

Python is neat. It’s kind of like JavaScript with less of the syntax cruft (hooray for no more semicolons). It’s also kind of like C (I think some of the base libraries are actually written in C) and while I don’t have a lot of C experience, the one class I did take was very enjoyable. It’s like you get all the power in the lower level like C but without all the syntax nightmares; you also have access to a lot of libraries. Writing python is kind of like writing pseudo code, which is a good fit for me because I always thought I was better at coming up with pseudo code algorithms than actually programming.

If you’re interested in learning I can’t recommend this book enough: Dive Into Python. The best part is, you can download a full PDF of the book entirely free! I’m only about half way through so far but it’s written in a very easy to understand style and comes with many code examples in the zip file. If you’re following along using the examples on a Mac it’s even better because Leopard comes already installed with Python 2.5. Yes, working with Python is good stuff 🙂

On Google App Engine (GAE)

Google App Engine is an interesting beast. I’ve read some articles already about how it’s changing development paradigms and forcing developers to think about scalability and data access in a completely different way. I don’t purport to be a database expert but from what I’ve read GAE uses a storage system called BigTable that differs radically from typical RDBS (Relational Database Systems). While this doesn’t affect me directly (yet) I find it interesting to read about problems like scalability. The GAE DataStore API is really simple yet effective, and so far the documentation has been really easy to read and the code examples really easy to follow. A few posts in the Google Groups section for GAE have some further detail on performance testing and the results are intriguing.

My only beef so far with GAE is with running Django on it and the fact that there seems to be a few established ways of running it but no “standard” way:

  1. Google Code – Running Django on Google App Engine
  2. GC Project – Google-AppEngine-Django
  3. Guido van Rossum – Rietveld (sample django on GAE project)

I’ve been working hard to try and figure out some of the best practices for running Django on GAE but I suppose a lot of this is subject to potential change considering GAE is still in PREVIEW RELEASE mode and the Django 1.0 release is still a few weeks away. I don’t anticipate too much change though as there is quite a bit of the API already established. I guess it’s a good thing we’re using the latest SVN releases of Django instead of the default 0.96, 0.97 releases GAE comes with by default.

On Django

I tried doing some tutorials on the Django website a few months back and while it was really cool seeing all the “magic” stuff that you got for free with things like the admin interface, ORM layer etc… I don’t think I really appreciated the power of what it can do until I started reading Dive Into Python. Since Django is written in Python it really helps to have an understanding of the language at a more rudimentary level to see exactly how Django works and to avoid the “wtf?! how did it do that?” questions I was having previously. It seems to me that the guys who built Django really understand what’s tedious about building web applications and they’ve created this set of tools to make web application development much more enjoyable.

I personally enjoy working on backend stuff like configuration, deployment, ORM setup and working with Django + GAE is really nice for all of that, but at the same time part of me really enjoys constructing the CSS, XHTML templates, implementing jQuery and YUI for effects and interface widgets, and actually designing the graphics… all of the more frontend type work. I consider myself a Sweeper (read the first paragraph), although more of a frontend leaning Sweeper, but working with the tools and technologies I am able to work with here at VendAsta has really empowered me to “sweep” from frontend to backend and picking up all kinds of knowledge everywhere in between. Just one more thing that makes me really feel like I’ve finally found a job that fits with my never ending thirst to work in both worlds. Did I mention, we’re hiring? 🙂