Ben Forta

AdobeStock_455007340

ColdFusion 8 Performance Numbers

ColdFusion 8 is fast, really fast, and in the CFUnited keynote this morning I presented some actual numbers (all reported as requests per second, the higher the number the more load the server can handle):

  • Structure manipulation: CF6.1=99, CF7=101, CF8=200. In other words, CF8 manipulated structures twice as fast as CF6 and 7.
  • List manipulation: CF6.1=33.6, CF7=34.4, CF8=103.4. CF8 list manipulation is over three times faster than that of CF6 and 7.
  • CFSWITCH/CFCASE: CF6.1=66.2, CF7=88.9, CF8=244.4. CF8 is three times faster than CF7.
  • CFPARAM: CF6.1=8.9, CF7=9.6, CF8=341.4. Yep, thirty five times as fast!
  • Date functions: CF6.1=230, CF7=251, CF8=1423. About six times faster in CF8.
  • RegEx functions: CF6.1=390, CF7=392, CF8=987. Over double the speed in CF8.
  • IsDefined(): CF6.1=136, CF7=611, CF8=1243. Double as fast as CF7, and nearly ten times the speed of CF6.
  • CFC creation: CF6.1=9.7, CF7=3, CF8=69.4. Yep, performance here took a hit in CF7, but it’s more than made up for in CF8.

Of course, in the real world you need to test whole applications, and not just specific language elements. So:

  • BlogCFC: CF6.1=271, CF7=255, CF8=355. About a 30% performance gain.
  • Adobe.com store: CF6.1=275, CF7=290, CF8=435. 30% performance gain there too.

In general we are expecting an average of 30%-40% performance gain across all application. And we have seen some apps where the gain is double and even triple performance.
Oh, and all of these tests were performed using CF6 and CF7 CFML, no new CF8 stuff. When those are added to the mix CF8 apps will run even quicker.

34 responses to “ColdFusion 8 Performance Numbers”

  1. Samuel Smith Avatar
    Samuel Smith

    Ben,
    That is awesome! Thanks for sharing. I can’t wait to see the speed improvements myself.

    Reply
  2. arthur Avatar
    arthur

    i just tried coldfusion wizard on eclipse. it just made nice ajax stuff. But I have to tell you when I checked the file, there was 900kb code. yes 900kb. it is like people needs to download some small application just for running basic coldfusion apps with ajax specs. i think this is very important on the performance. Am I right?

    Reply
  3. Hemant Avatar
    Hemant

    We have worked on reducing the size considerably in the final release.
    Thanks,
    Hemant

    Reply
  4. Luke Avatar
    Luke

    will createuuid() receive a speed boost in the spirit of the one Joe mentions here http://www.firemoss.com/blog/index.cfm?mode=entry&entry=BC614E9B-3048-55C9-432594FC1E443986 ? Thanks for these stats Ben you guys are doing an absolutely outstanding job!

    Reply
  5. Dave Shuck Avatar
    Dave Shuck

    Ben, thanks for posting these. Aaron Lynch called me right after the keynote and was telling me in general terms. Now I get to see specifics!

    Reply
  6. Patrick Whittingham Avatar
    Patrick Whittingham

    Ben –
    Did these number use JVM 1.4,1.5 or 1.6? Also, do you have numbers for cfdocument (pdf / flash) for various amount of memory (512,1g,2g,3g,4g). I think all of these numbers should be in a white paper

    Reply
  7. Dan Wilson Avatar
    Dan Wilson

    These are excellent numbers. I thank the CF Team for their hard work optimizing our language.
    Any word on a release date, it is almost *MID* year 😉

    Reply
  8. hibiscusroto Avatar
    hibiscusroto

    quit your braggin! in all seriousness, CF8 is really solid. long live cold fusion!

    Reply
  9. tony petruzzi Avatar
    tony petruzzi

    Is there any word if the implicit structure and array bug was fixed as noted by Ben Nadel:
    http://www.bennadel.com/blog/740-Learning-ColdFusion-8-Implicit-Struct-And-Array-Creation.htm
    Also is the team going to change the syntax from using "=" signs to ":" as noted in the comments from Ben’s posts? I would welcome this change since I do a fair amount of jQuery Javascript programming and my biggest gripe against CF is the fact that Adobe doesn’t follow standards when it comes to the language. If Javascript and all other languages use a ":", then why would Adobe take it apon itself to use a "="?

    Reply
  10. Ben Forta Avatar
    Ben Forta

    Patrick, I assume they used the default JVM for each, for CF8 that’s 1.6. We actually plan on publishing a white paper with exact test results, including reproducable steps. Stay tuned.
    Dan, very soon. 😉
    Tony, bug fixes are still being applied. As for changes, I highly doubt it at this point, but you never know …
    More to follow.
    — Ben

    Reply
  11. Fred B Avatar
    Fred B

    I’d like to know which JVM was used in each test. Is CF8 on Java 1.6 faster than on 1.4? Also, can you provide examples of the CFML used in each test?

    Reply
  12. Fred B Avatar
    Fred B

    Ben – just saw your comment about the white paper, I look forward to reading it.

    Reply
  13. Justin Avatar
    Justin

    Hopefully the test were run under the same JVM, so that its more of comparing apples to apples, than apples to oranges. Is MX7 foward compatible?

    Reply
  14. Ben Forta Avatar
    Ben Forta

    Justin, no, CFMX6 and CFMX7 are not supported on 1.5 and 1.6.
    — Ben

    Reply
  15. Ben Forta Avatar
    Ben Forta

    Neil, nice to see that most of your tests do indeed show improved performance in CF8. But we’re not quite done yet. The team is still testing and optimizing, and they are testing your specific examples as well. Stay tuned.
    — Ben

    Reply
  16. Neil Middleton Avatar
    Neil Middleton

    I was playing with testing performance of various bits of the code a while ago and came up with the following (all times are averages over a number of requests):
    Very simplistic testing, but results nonetheless:
    <cfloop from="1" to="10000" index="i">
    <cfset foo = createObject("component","foo")>
    </cfloop>
    CF7 14 secs
    CF8 2.1 secs
    —————————————————————
    <cfloop from="1" to="10000" index="i">
    <cfparam name="myName" default="dave">
    </cfloop>
    CF7 31ms
    CF8 16ms
    —————————————————————
    <cfloop from="1" to="1000000" index="i">
    <cfset foo = "poo">
    </cfloop>
    CF7 47ms
    CF8 31ms
    —————————————————————
    <cfloop from="1" to="100000" index="i">
    <cfset foo = arrayNew(1)>
    <cfset arrayAppend(foo, "This is an item")>
    </cfloop>
    CF7 350ms
    CF8 1500ms
    —————————————————————
    <cfloop from="1" to="100000" index="i">
    <cfset foo = structNew()>
    <cfset foo.poo = "1">
    </cfloop>
    CF7 500ms
    CF8 1266ms
    —————————————————————
    <cfscript>
    for (i=1;i lte 100000;i=i+1)
    {
    foo = structNew();
    foo.poo = "1";
    }
    </cfscript>
    CF7 157ms
    CF8 78ms
    In some places, I found CF8 to be a lot slower. Any ideas?

    Reply
  17. MIke Rankin Avatar
    MIke Rankin

    Does this mean we can go back to using IsDefined instead of StructKeyExists as a best practice? I find it’s easier to quickly grasp what is going on with the IsDefined method.

    Reply
  18. Jason McMinn Avatar
    Jason McMinn

    Are there any online webcasts of his presentation??

    Reply
  19. Aegis Avatar
    Aegis

    So does this mean CF8 will be released 30-40% sooner? =)

    Reply
  20. Ben Forta Avatar
    Ben Forta

    Neil, QA tested your examples, and sent this to me:
    I am *not* able to reproduce the results neil talks about. I am seeing better performance with both CF8+JDK1.4.2_09 (and CF8+1.6_01) as compared to CF702/CF7 in all the areas he is talking about.
    I am not sure what might have differed in his testing and mine. Please ask him to check:
    – If he has both CF8 and CF7 (or CF702) on the same machine.
    – Settings like Whitespace management, trusted cache, Number of JVM threads are the same.
    – Debugging is turned off on both
    — Ben

    Reply
  21. Tim McAuliffe Avatar
    Tim McAuliffe

    Very interesting Ben – thanks for the numbers.
    I’ve found that both CF6 & 7 have very poor performance on large lists (thousands of items). Performance seems to degrade exponentially with list size. My wild guess is it’s related to the "Shlemiel the painter’s algorithm" issues that Joel Spolsky describes so elequently here:
    http://www.joelonsoftware.com/articles/fog0000000319.html
    Is that close to the mark at all? It’d be interesting to see a performance comparison with CF8 on large lists.

    Reply
  22. Ben Forta Avatar
    Ben Forta

    Tim, well …
    Lists are slow because there is no list data type. A list is actually a string, any string. The list functions parse the string looking for delimiters, so as to allow you to extract values. If you want element 100 from the string it needs to be parsed (using , by default, but you could use any delimiter actually) and that member needs to be found. Every access is a string parsing exercise. And this has to be performed on every access, because the delimiter for one list operation may not be the same as the delimiter on the next. And so on. As a rule, lists are great because A) SQL uses them, B) HTML form fields come back as them. But for any longer list, or for any data accessed and manipulated frequently, use arrays or structures.
    — Ben

    Reply
  23. Stephen Avatar
    Stephen

    I just took my app that I have running on CF7 with a shared host and uploaded it to the CF8 beta server at Hostmysite.com. Wow, the performance is incredible. It "feels" like it’s running 10X faster.
    I’m using the query2csv script from CFLIB.org, and it literally takes about 2-5 minutes to download data from my current production environment on 7. Seems to take just seconds on 8!! Awesome!

    Reply
  24. Vishnuprasad Avatar
    Vishnuprasad

    I found an issue with Coldfusion 6.1 as well as Coldfusion 8 beta need to test the same in the Coldfusion 8 Final version
    Following are the issue description
    Have an Oracle Database with Character set as AL32UTF8 and NLS_LENGTH_SEMANTICS= Char
    I have created the table Table1 with the Columns as col1 char (1 char)
    Inserted the values as "A"
    When I use the query Select Col1 from table1 in Toad and SQL Plus it just returning data without the spaces
    Returned
    "A"
    When I put the same query from CFM template it adding trailing space
    Eg: ‘A ‘

    Reply
  25. Vishnuprasad Avatar
    Vishnuprasad

    in the earlier post forget to mention the enviornment and Driver
    I found an issue with Coldfusion 6.1 as well as Coldfusion 8 beta need to test the same in the Coldfusion 8 Final version
    Enviornment : Coldfusion Mx 6.1 , 8 Beta
    OS : windows 2000
    Driver :
    Coldfusion Oracle Native option
    Coldfusion Oracle Thin driver option
    ODBC Driver
    Following are the issue description
    Have an Oracle Database with Character set as AL32UTF8 and NLS_LENGTH_SEMANTICS= Char
    I have created the table Table1 with the Columns as col1 char (1 char)
    Inserted the values as "A"
    When I use the query Select Col1 from table1 in Toad and SQL Plus it just returning data without the spaces
    Returned
    "A"
    When I put the same query from CFM template it adding trailing space
    Eg: ‘A ‘

    Reply
  26. Andrew Avatar
    Andrew

    I see CF 8 shiped with JRE 1.6.0_01 and I had quite a lot of problems with JRE error "EXCEPTION_FLT_STACK_CHECK, Problematic frame" and this seems to be common with this JRE version. To save others the headache when I downloaded and installed JRE 1.6.0_02 (JRE 6 update 2) it by default installs to C:/Program Files/Java/jre1.6.0_02.
    Under this it creates a folder "bin/client/jvm.dll" file. I edited the C:ColdFusion8runtimebinjvm.config for java home to:
    java.home=C:/Program Files/Java/jre1.6.0_02
    And then CF 8 refused to start. After a bit of head scratching I realised CF 8 looks for a folder called "server/jvm.dll" so I simply copied the "client" folder to a new folder called "server" and hey presto – it worked.
    I hope this helps others not have the same problem I did trying to work out why it wouldn’t work and why they may be having problems with CF randomly crashing and re-starting under CF 8.

    Reply
  27. Mike Causer Avatar
    Mike Causer

    Speaking of CF8 performance, for creating structures, check out the difference between:
    {}
    structNew()
    deserializeJSON(‘{}’)
    You might be pleasantly surprised!

    Reply
  28. Still life Artist Avatar
    Still life Artist

    These are excellent numbers. I thank the CF Team for their hard work optimizing our language.
    Andrzej Filipowicz
    Still life Artist, still life art
    http://www.andrzejfilipowicz.com

    Reply
  29. David Avatar
    David

    Just a heads up – I have been evaluating CF8 against the Bluedragon.NET port of CF and am finding that CF8 outperforms Bluedragon by 10-20% on the sample code I am using – which includes a number of cfc calls.
    I contacted Bluedragon directly for some advice on how to tune it. They replied (said to remove the JRun Connector Filter from the ISAPI Filters in IIS) and I did see some improvement, but I was still unable to get .NET to run faster than the JVM in CF8.
    Our company is getting a lot of input from outside consultants that CF does not scale – so I am trying to fight back. Was very encouraged by my personal result – curious if there are any benchmarks that I could reference that are recent and industry accepted.

    Reply
  30. peakhk Avatar
    peakhk

    I just took my app that I have running on CF7 with a shared host and uploaded it to the CF8 beta server at Hostmysite.com. Wow, the performance is incredible. It "feels" like it’s running 10X faster.
    <a href=http://www.recyclebag.net/eng/html/pp_bag.htm >Promotion bag</a>,
    <a href=http://www.recyclebag.net/eng/html/tote_bag.htm >Tote Bag</a>,
    <a href=http://www.recyclebag.net/eng/html/tote_bag.htm >Cotton Bag</A>,
    <a href=http://www.recyclebag.net/eng/html/paper_bag.htm >Paper Bag</a>,
    <a href=http://www.recyclebag.net/ >Non-woven Bag</a>,
    <a href=http://www.recyclebag.net/ >Shopping Bag</a>,
    <a href=http://www.recyclebag.net/ >Bag Manufacturer</a>,
    <a href=http://www.recyclebag.net/ >Recycle Bag Manufacturer</a>,
    <a href=http://www.cuhk.edu.hk/clc/summer.htm >Summer Study Abroad Program</a>,
    <a href=http://www.cuhk.edu.hk/clc/e_putonghua.htm >study in china</a>,
    <a href=http://www.cuhk.edu.hk/clc/ >Chinese Language</a>,
    <a href=http://www.huangjinjiage.org/ >http://www.huangjinjiage.org/</a&gt;,
    <a href=http://www.cuhk.edu.hk/clc/e_ssp.htm >learn mandarin</a>,
    <a href=http://www.cuhk.edu.hk/clc/e_cantonese.htm >cantonese</a>,
    <a href=http://www.9lolo.com/ >http://www.huangjinjiage.com/</a&gt;,

    Reply
  31. 369 Avatar
    369

    So does this mean CF8 will be released 30-40% sooner? =)
    http://www.recyclebag.net/eng/html/pp_bag.htm
    recyclebag
    http://www.recyclebag.net/eng/html/tote_bag.htm
    recyclebag
    http://www.recyclebag.net/eng/html/paper_bag.htm
    recyclebag
    http://www.recyclebag.net
    recyclebag
    http://www.cuhk.edu.hk/clc/summer.htm
    summer
    http://www.cuhk.edu.hk/clc/e_putonghua.htm
    putonghua
    http://www.cuhk.edu.hk/clc/
    cuhk
    http://www.huangjinjiage.org/
    huangjinjiage
    http://www.cuhk.edu.hk/clc/e_ssp.htm
    ssp
    http://www.cuhk.edu.hk/clc/e_cantonese.htm
    cantonese
    http://www.9lolo.com/
    9lolo
    http://huangjinjiage.blogspot.com/
    blogspot
    http://huangjinjiage.spaces.live.com/

    Reply
  32. mwinnie Avatar
    mwinnie

    <a href=http://www.recycle-bag.com/en>recycle-bag</a>
    <a href=http://www.hkpointer.com/>hkpointer</a>
    <a href=http://www.easyflowers.com.hk>easyflowers</a>
    <a href=http://www.everydayflowers.com.hk>everydayflowers</a>
    <a href=http://www.print101.net>print101</a>
    <a href=http://www.flag100.com/>flag100</a>
    <a href=http://www.uniwebdesign.com.hk>uniwebdesign</a>
    <a href=http://www.belisport.com>belisport</a>
    <a href=http://www.carton100.com>carton100</a>
    <a href=http://www.tableware100.com>tableware100</a>
    <a href=http://www.brothersglobal.com>brothersgloba</a>

    Reply
  33. glex Avatar
    glex

    http://www.96169.org/
    http://www.seo-labs.cn/
    http://www.yahoo-seo.cn/
    http://blog.dslrbiz.com/
    http://blog.go123go.com/
    http://netvouz.adv88.com/
    http://alexking.iblog.com
    http://bokardo.uwants.com
    http://blog.qooza.hk/murich
    http://chronicu.blogspot.com/
    http://friendfinder.96169.org/
    http://chainki.mysinablog.com/
    http://lullabot.wordpress.com/
    http://dslrsreview.blogspot.com/
    http://findmatch.mysinablog.com/
    http://lullabot.spaces.live.com/
    http://hk.myblog.yahoo.com/plsyoga
    http://hk.myblog.yahoo.com/spudart
    http://www.diigo.com/user/lullabot
    http://profile.typekey.com/nesthone/

    Reply
  34. cf coder Avatar
    cf coder

    hi all,
    Is it possible to compare performance of cf8 server, .net server and a java server

    Reply

Leave a Reply