Blog

27Jun
2007
ColdFusion 8 Performance Numbers

ColdFusion 8 is fast, really fast, and in the CFUnited keynote this morning I presented some actual numbers (all reported as requests per second, the higher the number the more load the server can handle):

  • Structure manipulation: CF6.1=99, CF7=101, CF8=200. In other words, CF8 manipulated structures twice as fast as CF6 and 7.
  • List manipulation: CF6.1=33.6, CF7=34.4, CF8=103.4. CF8 list manipulation is over three times faster than that of CF6 and 7.
  • CFSWITCH/CFCASE: CF6.1=66.2, CF7=88.9, CF8=244.4. CF8 is three times faster than CF7.
  • CFPARAM: CF6.1=8.9, CF7=9.6, CF8=341.4. Yep, thirty five times as fast!
  • Date functions: CF6.1=230, CF7=251, CF8=1423. About six times faster in CF8.
  • RegEx functions: CF6.1=390, CF7=392, CF8=987. Over double the speed in CF8.
  • IsDefined(): CF6.1=136, CF7=611, CF8=1243. Double as fast as CF7, and nearly ten times the speed of CF6.
  • CFC creation: CF6.1=9.7, CF7=3, CF8=69.4. Yep, performance here took a hit in CF7, but it's more than made up for in CF8.
Of course, in the real world you need to test whole applications, and not just specific language elements. So:
  • BlogCFC: CF6.1=271, CF7=255, CF8=355. About a 30% performance gain.
  • Adobe.com store: CF6.1=275, CF7=290, CF8=435. 30% performance gain there too.

In general we are expecting an average of 30%-40% performance gain across all application. And we have seen some apps where the gain is double and even triple performance.

Oh, and all of these tests were performed using CF6 and CF7 CFML, no new CF8 stuff. When those are added to the mix CF8 apps will run even quicker.

Related Blog Entries

Comments (34)



  • Dave Shuck

    Ben, thanks for posting these. Aaron Lynch called me right after the keynote and was telling me in general terms. Now I get to see specifics!

  • Samuel Smith

    Ben,

    That is awesome! Thanks for sharing. I can't wait to see the speed improvements myself.

  • arthur

    i just tried coldfusion wizard on eclipse. it just made nice ajax stuff. But I have to tell you when I checked the file, there was 900kb code. yes 900kb. it is like people needs to download some small application just for running basic coldfusion apps with ajax specs. i think this is very important on the performance. Am I right?

    #3Posted by arthur | Jun 27, 2007, 02:01 PM
  • Hemant

    We have worked on reducing the size considerably in the final release.

    Thanks,
    Hemant

    #4Posted by Hemant | Jun 27, 2007, 02:25 PM
  • Luke

    will createuuid() receive a speed boost in the spirit of the one Joe mentions here http://www.firemoss.com/blog/index.cfm?mode=entry&... ? Thanks for these stats Ben you guys are doing an absolutely outstanding job!

    #5Posted by Luke | Jun 27, 2007, 03:31 PM
  • Patrick Whittingham

    Ben -

    Did these number use JVM 1.4,1.5 or 1.6? Also, do you have numbers for cfdocument (pdf / flash) for various amount of memory (512,1g,2g,3g,4g). I think all of these numbers should be in a white paper

    #6Posted by Patrick Whittingham | Jun 27, 2007, 04:33 PM
  • Dan Wilson

    These are excellent numbers. I thank the CF Team for their hard work optimizing our language.

    Any word on a release date, it is almost *MID* year ;)

  • hibiscusroto

    quit your braggin! in all seriousness, CF8 is really solid. long live cold fusion!

  • tony petruzzi

    Is there any word if the implicit structure and array bug was fixed as noted by Ben Nadel:

    http://www.bennadel.com/blog/740-Learning-ColdFusi...

    Also is the team going to change the syntax from using "=" signs to ":" as noted in the comments from Ben's posts? I would welcome this change since I do a fair amount of jQuery Javascript programming and my biggest gripe against CF is the fact that Adobe doesn't follow standards when it comes to the language. If Javascript and all other languages use a ":", then why would Adobe take it apon itself to use a "="?

  • Ben Forta

    Patrick, I assume they used the default JVM for each, for CF8 that's 1.6. We actually plan on publishing a white paper with exact test results, including reproducable steps. Stay tuned.

    Dan, very soon. ;-)

    Tony, bug fixes are still being applied. As for changes, I highly doubt it at this point, but you never know ...

    More to follow.

    --- Ben

    #10Posted by Ben Forta | Jun 28, 2007, 09:36 AM
  • Fred B

    I'd like to know which JVM was used in each test. Is CF8 on Java 1.6 faster than on 1.4? Also, can you provide examples of the CFML used in each test?

    #11Posted by Fred B | Jun 28, 2007, 09:45 AM
  • Fred B

    Ben - just saw your comment about the white paper, I look forward to reading it.

    #12Posted by Fred B | Jun 28, 2007, 09:47 AM
  • Justin

    Hopefully the test were run under the same JVM, so that its more of comparing apples to apples, than apples to oranges. Is MX7 foward compatible?

    #13Posted by Justin | Jun 28, 2007, 11:01 AM
  • Ben Forta

    Justin, no, CFMX6 and CFMX7 are not supported on 1.5 and 1.6.

    --- Ben

    #14Posted by Ben Forta | Jun 28, 2007, 02:19 PM
  • Neil Middleton

    I was playing with testing performance of various bits of the code a while ago and came up with the following (all times are averages over a number of requests):

    Very simplistic testing, but results nonetheless:

    <cfloop from="1" to="10000" index="i">
    <cfset foo = createObject("component","foo")>
    </cfloop>

    CF7 14 secs
    CF8 2.1 secs
    ---------------------------------------------------------------

    <cfloop from="1" to="10000" index="i">
    <cfparam name="myName" default="dave">
    </cfloop>

    CF7 31ms
    CF8 16ms
    ---------------------------------------------------------------

    <cfloop from="1" to="1000000" index="i">
    <cfset foo = "poo">
    </cfloop>

    CF7 47ms
    CF8 31ms
    ---------------------------------------------------------------

    <cfloop from="1" to="100000" index="i">
    <cfset foo = arrayNew(1)>
    <cfset arrayAppend(foo, "This is an item")>
    </cfloop>

    CF7 350ms
    CF8 1500ms
    ---------------------------------------------------------------

    <cfloop from="1" to="100000" index="i">
    <cfset foo = structNew()>
    <cfset foo.poo = "1">
    </cfloop>

    CF7 500ms
    CF8 1266ms
    ---------------------------------------------------------------
    <cfscript>
    for (i=1;i lte 100000;i=i+1)
    {
    foo = structNew();
    foo.poo = "1";
    }
    </cfscript>

    CF7 157ms
    CF8 78ms


    In some places, I found CF8 to be a lot slower. Any ideas?

  • Ben Forta

    Neil, nice to see that most of your tests do indeed show improved performance in CF8. But we're not quite done yet. The team is still testing and optimizing, and they are testing your specific examples as well. Stay tuned.

    --- Ben

    #16Posted by Ben Forta | Jun 29, 2007, 10:03 AM
  • MIke Rankin

    Does this mean we can go back to using IsDefined instead of StructKeyExists as a best practice? I find it's easier to quickly grasp what is going on with the IsDefined method.

  • Jason McMinn

    Are there any online webcasts of his presentation??

    #18Posted by Jason McMinn | Jun 29, 2007, 06:04 PM
  • Aegis

    So does this mean CF8 will be released 30-40% sooner? =)

    #19Posted by Aegis | Jul 1, 2007, 08:27 PM
  • Ben Forta

    Neil, QA tested your examples, and sent this to me:

    I am *not* able to reproduce the results neil talks about. I am seeing better performance with both CF8+JDK1.4.2_09 (and CF8+1.6_01) as compared to CF702/CF7 in all the areas he is talking about.

    I am not sure what might have differed in his testing and mine. Please ask him to check:
    - If he has both CF8 and CF7 (or CF702) on the same machine.
    - Settings like Whitespace management, trusted cache, Number of JVM threads are the same.
    - Debugging is turned off on both

    --- Ben

    #20Posted by Ben Forta | Jul 1, 2007, 10:28 PM
  • Tim McAuliffe

    Very interesting Ben - thanks for the numbers.

    I've found that both CF6 & 7 have very poor performance on large lists (thousands of items). Performance seems to degrade exponentially with list size. My wild guess is it's related to the "Shlemiel the painter's algorithm" issues that Joel Spolsky describes so elequently here:
    http://www.joelonsoftware.com/articles/fog00000003...

    Is that close to the mark at all? It'd be interesting to see a performance comparison with CF8 on large lists.

    #21Posted by Tim McAuliffe | Jul 2, 2007, 10:47 PM
  • Ben Forta

    Tim, well ...

    Lists are slow because there is no list data type. A list is actually a string, any string. The list functions parse the string looking for delimiters, so as to allow you to extract values. If you want element 100 from the string it needs to be parsed (using , by default, but you could use any delimiter actually) and that member needs to be found. Every access is a string parsing exercise. And this has to be performed on every access, because the delimiter for one list operation may not be the same as the delimiter on the next. And so on. As a rule, lists are great because A) SQL uses them, B) HTML form fields come back as them. But for any longer list, or for any data accessed and manipulated frequently, use arrays or structures.

    --- Ben

    #22Posted by Ben Forta | Jul 2, 2007, 10:58 PM
  • Stephen

    I just took my app that I have running on CF7 with a shared host and uploaded it to the CF8 beta server at Hostmysite.com. Wow, the performance is incredible. It "feels" like it's running 10X faster.

    I'm using the query2csv script from CFLIB.org, and it literally takes about 2-5 minutes to download data from my current production environment on 7. Seems to take just seconds on 8!! Awesome!

    #23Posted by Stephen | Jul 13, 2007, 04:45 PM
  • Vishnuprasad

    I found an issue with Coldfusion 6.1 as well as Coldfusion 8 beta need to test the same in the Coldfusion 8 Final version

    Following are the issue description

    Have an Oracle Database with Character set as AL32UTF8 and NLS_LENGTH_SEMANTICS= Char

    I have created the table Table1 with the Columns as col1 char (1 char)

    Inserted the values as "A"

    When I use the query Select Col1 from table1 in Toad and SQL Plus it just returning data without the spaces
    Returned
    "A"

    When I put the same query from CFM template it adding trailing space
    Eg: ‘A ‘

    #24Posted by Vishnuprasad | Jul 31, 2007, 07:51 AM
  • Vishnuprasad

    in the earlier post forget to mention the enviornment and Driver

    I found an issue with Coldfusion 6.1 as well as Coldfusion 8 beta need to test the same in the Coldfusion 8 Final version

    Enviornment : Coldfusion Mx 6.1 , 8 Beta
    OS : windows 2000

    Driver :
    Coldfusion Oracle Native option
    Coldfusion Oracle Thin driver option
    ODBC Driver

    Following are the issue description

    Have an Oracle Database with Character set as AL32UTF8 and NLS_LENGTH_SEMANTICS= Char

    I have created the table Table1 with the Columns as col1 char (1 char)

    Inserted the values as "A"

    When I use the query Select Col1 from table1 in Toad and SQL Plus it just returning data without the spaces
    Returned
    "A"

    When I put the same query from CFM template it adding trailing space
    Eg: ‘A ‘

    #25Posted by Vishnuprasad | Jul 31, 2007, 08:34 AM
  • Andrew

    I see CF 8 shiped with JRE 1.6.0_01 and I had quite a lot of problems with JRE error "EXCEPTION_FLT_STACK_CHECK, Problematic frame" and this seems to be common with this JRE version. To save others the headache when I downloaded and installed JRE 1.6.0_02 (JRE 6 update 2) it by default installs to C:/Program Files/Java/jre1.6.0_02.

    Under this it creates a folder "bin/client/jvm.dll" file. I edited the C:\ColdFusion8\runtime\bin\jvm.config for java home to:
    java.home=C:/Program Files/Java/jre1.6.0_02

    And then CF 8 refused to start. After a bit of head scratching I realised CF 8 looks for a folder called "server/jvm.dll" so I simply copied the "client" folder to a new folder called "server" and hey presto - it worked.

    I hope this helps others not have the same problem I did trying to work out why it wouldn't work and why they may be having problems with CF randomly crashing and re-starting under CF 8.

    #26Posted by Andrew | Aug 8, 2007, 09:47 PM
  • Mike Causer

    Speaking of CF8 performance, for creating structures, check out the difference between:
    {}
    structNew()
    deserializeJSON('{}')

    You might be pleasantly surprised!

    #27Posted by Mike Causer | Aug 23, 2007, 09:32 PM
  • Still life Artist

    These are excellent numbers. I thank the CF Team for their hard work optimizing our language.

    Andrzej Filipowicz
    Still life Artist, still life art
    http://www.andrzejfilipowicz.com

  • David

    Just a heads up - I have been evaluating CF8 against the Bluedragon.NET port of CF and am finding that CF8 outperforms Bluedragon by 10-20% on the sample code I am using - which includes a number of cfc calls.

    I contacted Bluedragon directly for some advice on how to tune it. They replied (said to remove the JRun Connector Filter from the ISAPI Filters in IIS) and I did see some improvement, but I was still unable to get .NET to run faster than the JVM in CF8.

    Our company is getting a lot of input from outside consultants that CF does not scale - so I am trying to fight back. Was very encouraged by my personal result - curious if there are any benchmarks that I could reference that are recent and industry accepted.

    #29Posted by David | May 16, 2008, 05:21 PM
  • peakhk

    I just took my app that I have running on CF7 with a shared host and uploaded it to the CF8 beta server at Hostmysite.com. Wow, the performance is incredible. It "feels" like it's running 10X faster.

    <a href=http://www.recyclebag.net/eng/html/pp_bag.htm >Promotion bag</a>,
    <a href=http://www.recyclebag.net/eng/html/tote_bag.htm >Tote Bag</a>,
    <a href=http://www.recyclebag.net/eng/html/tote_bag.htm >Cotton Bag</A>,
    <a href=http://www.recyclebag.net/eng/html/paper_bag.htm" target="_blank">http://www.recyclebag.net/eng/html/paper_bag.htm >Paper Bag</a>,
    <a href=http://www.recyclebag.net/ >Non-woven Bag</a>,
    <a href=http://www.recyclebag.net/ >Shopping Bag</a>,
    <a href=http://www.recyclebag.net/ >Bag Manufacturer</a>,
    <a href=http://www.recyclebag.net/ >Recycle Bag Manufacturer</a>,
    <a href=http://www.cuhk.edu.hk/clc/summer.htm >Summer Study Abroad Program</a>,
    <a href=http://www.cuhk.edu.hk/clc/e_putonghua.htm >study in china</a>,
    <a href=http://www.cuhk.edu.hk/clc/ >Chinese Language</a>,
    <a href=http://www.huangjinjiage.org/ >http://www.huangjinjiage.org/</a>,
    <a href=http://www.cuhk.edu.hk/clc/e_ssp.htm >learn mandarin</a>,
    <a href=http://www.cuhk.edu.hk/clc/e_cantonese.htm" target="_blank">http://www.cuhk.edu.hk/clc/e_cantonese.htm >cantonese</a>,
    <a href=http://www.9lolo.com/ >http://www.huangjinjiage.com/</a>;,

    #30Posted by peakhk | Jun 3, 2008, 02:35 AM
  • 369

    So does this mean CF8 will be released 30-40% sooner? =)

    http://www.recyclebag.net/eng/html/pp_bag.htm
    recyclebag   
    http://www.recyclebag.net/eng/html/tote_bag.htm
    recyclebag
    http://www.recyclebag.net/eng/html/paper_bag.htm
    recyclebag
    http://www.recyclebag.net
    recyclebag
    http://www.cuhk.edu.hk/clc/summer.htm
    summer
    http://www.cuhk.edu.hk/clc/e_putonghua.htm
    putonghua
    http://www.cuhk.edu.hk/clc/
    cuhk
    http://www.huangjinjiage.org/
    huangjinjiage
    http://www.cuhk.edu.hk/clc/e_ssp.htm
    ssp
    http://www.cuhk.edu.hk/clc/e_cantonese.htm" target="_blank">http://www.cuhk.edu.hk/clc/e_cantonese.htm
    cantonese
    http://www.9lolo.com/
    9lolo
    http://huangjinjiage.blogspot.com/
    blogspot
    http://huangjinjiage.spaces.live.com/

    #31Posted by 369 | Jun 3, 2008, 03:12 AM
  • mwinnie

    <a href=http://www.recycle-bag.com/en>recycle-bag</a...;
    <a href=http://www.hkpointer.com/>hkpointer</a>;
    <a href=http://www.easyflowers.com.hk>easyflowers</a...;
    <a href=http://www.everydayflowers.com.hk>everydayflowe...;
    <a href=http://www.print101.net>print101</a>;
    <a href=http://www.flag100.com/>flag100</a>;
    <a href=http://www.uniwebdesign.com.hk>uniwebdesign<...;
    <a href=http://www.belisport.com>belisport</a>;
    <a href=http://www.carton100.com>carton100</a>;
    <a href=http://www.tableware100.com>tableware100</a&...;
    <a href=http://www.brothersglobal.com>brothersgloba<...;

    #32Posted by mwinnie | Jul 9, 2008, 04:28 AM
  • glex

    http://www.96169.org/
    http://www.seo-labs.cn/
    http://www.yahoo-seo.cn/
    http://blog.dslrbiz.com/
    http://blog.go123go.com/
    http://netvouz.adv88.com/
    http://alexking.iblog.com
    http://bokardo.uwants.com
    http://blog.qooza.hk/murich
    http://chronicu.blogspot.com/
    http://friendfinder.96169.org/
    http://chainki.mysinablog.com/
    http://lullabot.wordpress.com/
    http://dslrsreview.blogspot.com/
    http://findmatch.mysinablog.com/
    http://lullabot.spaces.live.com/
    http://hk.myblog.yahoo.com/plsyoga
    http://hk.myblog.yahoo.com/spudart
    http://www.diigo.com/user/lullabot
    http://profile.typekey.com/nesthone/

    #33Posted by glex | Aug 14, 2008, 12:32 AM
  • cf coder

    hi all,
    Is it possible to compare performance of cf8 server, .net server and a java server

    #34Posted by cf coder | Dec 19, 2008, 12:56 AM