An Async Html cache – part II – Testing the cache

Other posts:

• Part I – Writing the cache

Let’s try out our lit­tle cache. First I want to write a syn­chro­nous ver­sion of it as a base­line.

    Private Shared Sub TestSync(ByVal sites() As String, ByVal sitesToDownload As Integer, ByVal howLong As Integer)
        Dim syncCache As New Dictionary(Of String, String)
        Dim count = sites.Count()
        Dim url1 = "http://moneycentral.msn.com/investor/invsub/results/statemnt.aspx?Symbol="
        For i = 0 To sitesToDownload - 1
            Dim html As String = ""
            Dim url = url1 & sites(i Mod count)
            If Not syncCache.TryGetValue(url, html) Then
                html = LoadWebPage(url)
                syncCache(url) = html
            End If
            DoWork(html, howLong)
        Next
    End Sub

This is a loop that loads web­pages in the cache if they are not al­ready there. sites is a list of tick­ers used to com­pose the urls; sitesToDown­load is the to­tal num­ber of sites to down­load, so that a sin­gle url can be loaded mul­ti­ple times; how­Long rep­re­sents the work to be done on each loaded page.

In this ver­sion the cache is sim­ply a Dictionary and there is no par­al­lelism. The two bold lines is where the cache is man­aged.

DoWork is this.

    Public Shared Sub DoWork(ByVal html As String, ByVal howLong As Integer)
        Thread.Sleep(howLong)
    End Sub

Let’s take a look at the asyn­chro­nous ver­sion.

    Private Shared Sub TestAsync(ByVal sites() As String, ByVal sitesToDownload As Integer, ByVal howLong As Integer)
        Dim htmlCache As New HtmlCache
        Dim count = sites.Count()
        Dim url = "http://moneycentral.msn.com/investor/invsub/results/statemnt.aspx?Symbol="
        Using ce = New CountdownEvent(sitesToDownload)
            For i = 1 To sitesToDownload
                htmlCache.GetHtmlAsync(
                    url & sites(i Mod count),
                    Sub(s)
                        DoWork(s, howLong)
                        ce.Signal()
                    End Sub)
            Next
            ce.Wait()
        End Using

There are sev­eral points worth mak­ing on this:

• The lambda used as second parameter for GetHtmlAsync is invoked on a different thread whenever the html has been retrieved (which could be immediately if the cache has downloaded the url before)
• CountDownEvent allows a thread to wait for a certain number of signals to be sent. The waiting happens on the main thread in the ce.Wait() instruction. The triggering of the event happens in the lambda described in the point above (the ce.Signal() instruction)

This is the dri­ver for the over­all test­ing.

    Private Shared Sub TestPerf(ByVal s As String, ByVal a As Action, ByVal iterations As Integer)
        Dim clock As New Stopwatch
        clock.Start()
        For i = 1 To iterations
            a()
        Next
        clock.Stop()
        Dim ts = clock.Elapsed
        Dim elapsedTime = String.Format(s & ": {0:00}:{1:00}:{2:00}.{3:00}", ts.Hours, ts.Minutes, ts.Seconds, ts.Milliseconds / 10)
        Console.WriteLine(elapsedTime, "RunTime")
    End Sub

There is not much to say about it. Start the clock, per­form a bunch of it­er­a­tions of the passed lambda, stop the clock, print out per­for­mance.

And fi­nally the main method. Note that all the ad­justable pa­ra­me­ters are fac­tored out be­fore the calls to TestPerf.

    Public Shared Sub Main()
        Dim tickers = New String() {"mmm", "aos", "shlm", "cas", "abt", "anf", "abm", "akr", "acet", "afl", "agl", "adc", "apd",
                                           "ayr", "alsk", "ain", "axb", "are", "ale", "ab", "all"}
        Dim sitesToDownload = 50
        Dim workToDoOnEachUrlInMilliSec = 20
        Dim perfIterations = 5
        TestPerf("Async", Sub() TestAsync(tickers, sitesToDownload, workToDoOnEachUrlInMilliSec), perfIterations)
        TestPerf("Sync", Sub() TestSync(tickers, sitesToDownload, workToDoOnEachUrlInMilliSec), perfIterations)
    End Sub

Feel free to change (tick­ers, sitesToDown­load, work­ToDoOnEachUrlIn­Mil­liSec, per­fIt­er­a­tions). Depending on the ra­tios be­tween these pa­ra­me­ters and the num­ber of cores on your ma­chine, you’re go­ing to see dif­fer­ent re­sults. Which high­lights the fact that par­al­leliz­ing your al­go­rithms can yield per­for­mance gains or not de­pend­ing on both soft­ware and hard­ware con­sid­er­a­tions. I get ~3X im­prove­ment on my box. I at­tached the full source file for your amuse­ment.

AsyncCache.vb

Tags

• VB

2 Comments

Comments

An Async Html cache ??? part I

2009-05-08T11:52:28Z

PingBack from http://​mi­crosoft-share­point.sim­plynet­dev.com/​an-async-html-cache-%e2%80%93-part-ii-%e2%80%93-test­ing-the-cache/

DotNetShoutout

2009-05-08T14:41:50Z

Thank you for sub­mit­ting this cool story - Trackback from DotNetShoutout