Async Ads with HTML Imports

November 16, 2013 5:05 am | 15 Comments

Scripts loaded in the typical way (<script src="a.js"></script>) block rendering which is bad for performance. The solution to this problem is to load scripts asynchronously. This is the technique used by the best 3rd party snippets, for example, Google Analytics, Tweet button, Facebook SDK, and Google+ Follow button.

Ads are probably the most prevalent 3rd party content on the Web. Unfortunately, most ads can’t be loaded asynchronously because they use document.write. (Doing document.write in an async script causes the entire page to be erased. We’ll see this in the examples below.) Some teams (GhostWriter, PageSpeed) have tried to solve the async document.write problem but it requires a lot of code and edge cases exist.

In my recent investigations into the performance of HTML Imports I discovered a way to make ads load asynchronously – even ads that use document.write. Let’s take a look at how HTML imports block, how to make HTML imports asynchronous, the dangers of doing this with document.write, and the workaround to make this all work.

HTML imports block rendering

HTML imports block rendering if used in the default way. This is unfortunate, especially given that this is a recent addition to HTML when the importance of not blocking rendering is well known. The HTML Imports spec is still a working draft, so it’s possible this could be switched so that they load asynchronously by default.

Nevertheless, HTML imports are typically created like this:

<link rel="import" href="import.php">

Content from the imported HTML document is inserted like this:

<div id=import-container></div>
<script>
var link = document.querySelector('link[rel=import]');
var content = link.import.querySelector('#imported-content');
document.getElementById('import-container').appendChild(content.cloneNode(true));
</script>

The LINK tag itself doesn’t block rendering – the browser knows the import can be downloaded asynchronously. But rendering is blocked when the browser encounters the first SCRIPT tag following the LINK. This behavior is demonstrated in the sync.php test page. To make the blocking observable, the import takes five seconds to download and then the pink “IMPORTED CONTENT” is displayed. The SCRIPT block is in the middle of the page so the first paragraph IS rendered, but the last paragraph IS NOT rendered until after five seconds. This demonstrates that HTML imports block rendering.

Running the examples: Currently HTML imports only work in Chrome Canary and you have to turn on the following flags in chrome://flags/: Experimental Web Platform features, Experimental JavaScript, and HTML Imports.

Making HTML imports async

It’s not too hard to make HTML imports asynchronous thus avoiding the default blocking behavior. Instead of using the LINK tag in markup, we create it using JavaScript:

var link = document.createElement('link');
link.rel = 'import';
link.onload = function() {
    var link = document.querySelector('link[rel=import]');
    var content = link.import.querySelector('#imported-content');
    document.getElementById('import-container').appendChild(content.cloneNode(true));
};
link.href = url;
document.getElementsByTagName('head')[0].appendChild(link);

The async.php test page shows how using this asynchronous pattern doesn’t block rendering – the last paragraph is rendered immediately, then after five seconds we see the pink “IMPORTED CONTENT” from the HTML import. This shows that HTML imports can be used without blocking the page from rendering.

HTML imports with document.write – watch out!

This is kind of weird and might be hard to grok: HTML imports have their own HTML document. BUT (here’s the complex part) any JavaScript within the HTML import is executed in the context of the main page. At least that’s the way it works now in Chrome. The spec doesn’t address this issue.

This is important because some 3rd party content (especially ads) use document.write. Some people might think that a document.write inside an HTML import would write to the HTML import’s document. But that’s not the case. Instead, document refers to the main page’s document. This can produce surprising (as in “bad”) results.

As shown in the sync docwrite.php and async docwrite.php test pages, when the HTML import contains a script that uses document.write it erases the content of the main page. If you’re uncertain whether the imported content uses document.write then it’s risky to use HTML imports. Or is it?

Safely using HTML imports with document.write

Since much 3rd party content (especially ads) use document.write, there’s a motivation to make them work with HTML imports. However, as shown by the previous examples, this can have disastrous results because when the HTML import does document.write it’s actually referencing the main page’s document and erases the main page.

There’s a simple “hack” to get around this. We can’t redefine document, but we CAN redefine document.write within the HTML import:

// inside the HTML import
document.write = function(msg) {
    document.currentScript.ownerDocument.write(msg);
};

With this change, all document.write output from scripts inside the HTML import goes to the HTML import’s document. This eliminates the problem of the HTML import clearing the main page. This fix is shown by the sync docwrite-override.php and async docwrite-override.php test pages.

Async (document.write) ads with HTML imports

Let’s see this technique in action. The async-ads.php test page includes Google’s show_ads.js to load real ads. The overridden version of document.write also echoes the output to the page so you can verify what’s happening. The document.write works and the ad is shown even though it’s loaded asynchronously.

This is a major accomplishment but there are some big caveats:

  • Although we’ve overridden document.write, there might be other JavaScript in the HTML import that assumes it’s running in the main page’s context (e.g., location.href, document.title).
  • Some people believe it would be good to disable document.write inside HTML imports, in which case ads wouldn’t work.
  • We need a fallback as support for HTML imports grows. This is possible by detecting support for HTML imports and reverting to the current (blocking) technique for ads.

Perhaps the biggest caveat is whether it’s realistic to expect website owners to do this. I don’t think a majority of websites would adopt this technique, but I like having an option to make ads async for websites that are willing to do the work. Right now, motivated website owners don’t have good alternatives for loading ads without blocking their own content on their page. I know some sites that have loaded ads at the bottom of the page in a hidden div and then clone them to the top when done, but this usually results in a drop in ad revenue because the ads load later. Using HTML imports allows the ad to be loaded at the top so we can have asynchronous behavior without a loss in ad revenue.

The goal of this post is to suggest that we find a way to solve one of today’s biggest obstacles to fast web pages: ads. The spec for HTML imports is a working draft and there’s only one implementation, so both are likely to change. My hope is we can make HTML imports asynchronous by default so they don’t block rendering, and use them as technique for achieving asynchronous ads.

15 Responses to Async Ads with HTML Imports

  1. I’m not sure if I want this technique, but I want a technique that works like this. Since I’m working on the frontend of a website, I stumble upon Ads that do document.write. It’s annoying and bad for performance. I see the point about ad revenues when loading them later, but a technique loading them asynchronously would be magnificent.

  2. Great post Steve!
    Maybe one day we’ll fix the ads thing on the web, without workarounds.
    This technique seem to be promising, maybe more than Friendly Iframes.

    I have a question:
    Polymer and Xtags are using some kind of HTML Import polyfill to simulate full Web Components feature, right?
    What you think, we can use this polyfill to apply the HTML import ads technique?

    Thanks!

  3. Florian: I agree this technique is hacky (overriding document.write for example). As you suggest it’s more of a suggestion that we keep the ads problem in mind. For example, perhaps JavaScript in HTML imports should be executed in the context of the HTML import’s document, eliminating the need for the override.

    Jaydson: The issue with ads is the website owner often has little knowledge about what the ad is going to do (for example, expando ads). Polymer and X-Tag are written from the perspective of providing a *new* widget that presumably has well known behavior (and hopefully doesn’t use document.write). Also, I believe Polymer and X-Tag are loaded synchronously and thus block rendering themselves. For example, the second step in Polymer’s Quick start is to add a blocking script to the page, so this clearly is not providing an asynchronous widget.

  4. Can’t ad networks deliver constructed ads themselves via html-import fetches? I presume that request themselves have sufficient information to construct the ads. Pre-constructed ads would also enable them to be cached, etc.

  5. Leonard: This is indeed possible. However, advertisers still use document.write. We shouldn’t assume they’ll adopt better implementations unless there’s a strong motivation.

  6. Great post! What is the advantage of this technique over HTML5 iframe srcdoc & seamless attributes? Obviously rendering iframe is expensive but I’m not clear if there are other reasons?

    Regarding pre-constructed ads: the reason this usually fails in practice is because of the “daisy chaining” of different ad networks in play, the eventual ad is not one that is known ahead of time.

  7. Matt: Great question!! I love the HTML5 upgrades to iframe. The ones relevant to this discussion IMO are:

    • srcdoc – Better for performance because we can embed the content of the (small) HTML document that contains the ad. However, we can also do this with data: URIs.
    • sandbox – Yay! Restrict 3rd party access!
    • allowfullscreen – For expando ads (I don’t love them, but we have to deal with them).
    • seamless – In case we want the main document’s styles to extend to the iframe.

    The main advantage of HTML imports over HTML5 iframes is getting the content to start loading sooner. LINK (by definition) is in the HEAD so the browser can start parsing it right away. The iframe, on the other hand, has to be placed where you want the content (ad) to go – so it’ll be later than HEAD. In some case it might be better to load 3rd party content later, but for ads it’s been shown to improve revenue if the ad is loaded sooner.

    Other small tradeoffs include browser support and DOM creation time. Looking beyond the world of ads, HTML imports are much more flexible than iframes for sharing web components.

  8. Hi, small remark regarding async ads. GPT which is Google Publisher Tag is based on async loading: https://support.google.com/dfp_sb/answer/1649768?hl=en. Kind of a better way then rendering hidden then clone. This may lead to revenue drop but as far as it’s recommended by major ad market player I suspect it will be all over the place sooner or later.

  9. Eduard: GPT and one other advertiser (I forget who) are the only ones I know that support async ads. I agree it’s better – but if you think it’ll be all over the place sooner or later, the answer is later.

    I spoke to the IAB in 2007 about the importance of ad performance. They established the Ad Load Performance Working Group whose main accomplishments has been a document of Ad Load Performance Best Practices. That document came out in 2008 and doesn’t even suggest avoiding the use of document.write. That was five years ago.

    GPT went async in October 2011 – over two years ago.

    The incentives necessary to get advertisers to move to async do not exist. Until that time, I believe we’ll continue to see synchronous, blocking ads that use document.write from most advertisers.

  10. What you think about this? Implementation is:

    <div class="banneritem">
    <object width="260" height="177" data="/media/public/....swf" type="application/x-shockwave-flash">
    <param name="movie" value="/media/public/....swf" />
    <param name="quality" value="high" />
    <param name="wmode" value="transparent" />
    <embed
    src="/media/public/....swf"
    type="application/x-shockwave-flash"
    width="260"
    height="177"
    quality="high"
    wmode="transparent"
    />
    </object> <div class="clr"></div>
    </div>

    And on resize only for desktop banners are decoded to html and placed to ‘html’ div. This way there is one request less (async load of banners).

  11. I would think that compatibility with `history.pushState`-enabled-sites is an incentive for everything javascript to be decoupled from `document.write` and `window.onload`.

    On these sites, ads which are delivered using `document.write` can’t be updated when the user navigates to another page.

  12. It’s interesting to get knowledge about HTML imports. What about another technique which is very well supported by all major browsers ? What about injecting main external ads javascript with defer attribute or in asynchronous way like Google Analytics at first step ? At second step when main code will be loaded perform “asynchronous” document.write into iframe without src ?
    That will solve cross-domain security policy and iFrame is “asynchronous” relative to main doc.
    I used this technique when I was working as principal front-end engineer of Yandex Ads Network.

  13. @Imad: that’s a great idea and thanks for your write-up about the IE problems and work-arounds you found.

    The direct link for Imad’s write-up:

    http://kidjs.com/rendering-html-in-iframe/

  14. Imad: Writing to an iframe’s document is a great way to load content asynchronously – even content that does document.write. The two main drawbacks are expando ads (until HTML5 iframes are fully supported) and ads that require access to the global JavaScript namespace. For example, some major ads services need this to coordinate unique ads across multiple ad locations on the same page.

  15. You are welcome, Sean.

    Steve: What is expando ad ? An ad that will stretch to fill up all available space of the container ? In this case yes, an iframe will not provide stretching. But those ads are very rare, as far as I know only Yandex has such benefit. Other major ads were always fixed size.

    Usually, such problems might happen when ads networks use third party ad providers. For example Real Time Bidding Networks. But RTB networks are “selling” only fixed size ads, so an iframe will work perfectly and will protect the main document from any malformed third party ad content.
    On other hand, because an iframe is in the same domain, javascript will have full access to the main document and will be able to provide fully support for ads filtering. Of course it will require additional work-around in advertising showing code.