Another package install performance boost

I came across an interesting approach for improving the performance of automated package installs the other day. In the space of one day of internet reading I saw the Sitecore.Data.BulkUpdateContext class mentioned in a blog post and it came up in MVP forum discussions. In both cases, it was mentioned in the context of making item operations faster. Hence it seemed like an interesting thing to investigate, to see what sort of difference it would make to my previous work.

How do you use it?

The class is simple to use – it’s an IDisposable which can be placed into a using block wrapped around the rest of the logic for installing a package. The Page_Load() code from my previous blog post‘s package install endpoint can be updated simply by wrapping this class around the existing set of using statements:

public void Page_Load(object sender, EventArgs e)
{
    var files = WebUtil.GetQueryString("modules").Split('|');
    if (files.Length == 0)
    {
        Response.Write("No Modules specified");
        return;
    }
    Sitecore.Context.SetActiveSite("shell");

    using(new BulkUpdateContext())
    {
        using (new SecurityDisabler())
        {
            using (new ProxyDisabler())
            {
                using (new SyncOperationContext())
                {
                    foreach (var file in files)
                    {
                        Install(Path.Combine(Sitecore.Shell.Applications.Install.ApplicationContext.PackagePath, file));
                        Response.Write("Installed Package: " + file + "<br>");
                    }
                }
            }
        }
    }
}

What difference does it make?

The simplest way to work out what difference this code change makes is to measure the time taken to install some packages with each approach.

In PowerShell we can time code with the .Net Stopwatch class:

$sw = [Diagnostics.Stopwatch]::StartNew()

# Put the code to time here...

$sw.Stop()
Write-Host "Elapsed time: $($sw.Elapsed)"

If we wrap that around the PowerShell code which is calling the ASP.Net code above, we can measure the elapsed time for installing a set of packages:

function Add-Packages() {
    $siteName = Get-ConfigParam "InstanceName"
    $sitecoreFolder = "C:\Inetpub\wwwroot\$($siteName)\Data\packages"   
    
    $sw = [Diagnostics.Stopwatch]::StartNew()

    foreach($module in Get-ConfigModules) {
        # make querystring
        $query = (Split-Path -Path $module -Leaf)
        
        # Copy the package
        Write-Host "Copying package $query to Sitecore package folder..."
        Copy-Item $module $sitecoreFolder -force
        
        Write-Host "Calling package upload tool for $query"
    
        # call tool
        $url = "http://$siteName/PackageDeploy.aspx?modules=$query"
        $result = Invoke-WebRequest -Uri $url -TimeoutSec 600 -OutFile ".\$siteName-PackageResponse-$query.log" -PassThru
        
        if($result.StatusCode -ne 200) {
            Write-Host "StatusCode: $($result.StatusCode)"
            throw "Package install failed for $($query)"
        }
        
        # Removing installed package file
        $file = Join-Path $sitecoreFolder $query
        Remove-Item $file -Force
    }

    $sw.Stop()
    Write-Host "Package install time: $($sw.Elapsed)"

    write-host "Package tool done..."
}

If I run this as part of a scripted install of a new instance of Sitecore 6.6 and adding SPEAK and Email Campaign Manager packages then, an average of 10 runs gives me:

  • Without BulkUpdateContext: 1 minute 25 seconds
  • With BulkUpdateContext: 52 seconds

Repeating just the package installs a further 10 times on top of the already-installed instance gives an average elapsed time of:

  • Without BulkUpdateContext: 48 seconds
  • With BulkUpdateContext: 29 seconds

So while that’s not a statistically rigorous analysis, it does suggest that it’s well worth adding that extra line of code to improve the performance when installing big packages, or multiple packages…


Edited to add: As pointed out by @kamsar on twitter, the underlying code for BulkUpdateContext disables the publishing queue while the install proceeds – so you need to remember to publish things yourself after you’ve finished with your package installs.

Another point I failed to add in my original write-up was that you’ll want to update your search indexes after using this approach. (Thanks to Pavel for his comment below for reminding me) I’m looking at the code above in the context of my recent series of posts on development install automation via PowerShell – so I’d make use of the post I wrote on index builds to address this.

Advertisements

2 thoughts on “Another package install performance boost

  1. Great read, Jeremy, as always! One note about using BulkUpdateContext. Many event handlers respect it and silence themselves . No indexes are updated either. It may be a good idea to trigger index rebuilds if you expect the contents of the installed package to be immediately searchable.

    • A very good point – I was thinking about that in the context of the scripts I’m experimenting with, but I’ve not said so explicitly here. I’ll add a note.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s