Ben Forta

AdobeStock_455007340

More ColdFusion 8 Performance Data

Last week at CFUnited we publicly discussed the performance enhancements in ColdFusion 8, and I also posted some specifics.
I also said that we had seen some apps where the gain is double and even triple performance. Well, sorry, I lied. I just got stats comparing Ray Camden’s Canvas Wiki running on ColdFusion MX 7.0.2 and ColdFusion 8. Same code, same hardware, some CF settings, same everything … CFMX7.0.2 processed 42.7 requests per second, and CF8 processed 181.57 requests per second. That’s more than 4 times faster!
Also, several of you have asked if the performance gains are merely the result of ColdFusion 8 running on JDK1.6. Good question. Take a look at the following numbers:

  • CreatObject in CFLOOP (10,000 iterations): CFMX7.0.2(JDK1.4.2_09)=5344ms, CF8(JDK1.4.2_09)=172ms, CF8(JDK1.6)=110ms.
  • CFPARAM in CFLOOP (10,000 iterations): CFMX7.0.2(JDK1.4.2_09)=828ms, CF8(JDK1.4.2_09)=250ms, CF8(JDK1.6)=171ms.
  • CFSET in CFLOOP (10,000 iterations): CFMX7.0.2(JDK1.4.2_09)=843ms, CF8(JDK1.4.2_09)=260ms, CF8(JDK1.6)=180ms.
  • Array creation in CFLOOP (10,000 iterations): CFMX7.0.2(JDK1.4.2_09)=130ms, CF8(JDK1.4.2_09)=60ms, CF8(JDK1.6)=33ms.
  • Struct creation in CFLOOP (10,000 iterations): CFMX7.0.2(JDK1.4.2_09)=156ms, CF8(JDK1.4.2_09)=94ms, CF8(JDK1.6)=51ms.

As you can see, JDK1.6 does indeed improve performance (comparing CF8 and CF7.0.2 both running on the same JDK 1.4.2_09), but that improvement is far less significant that the core CF processing improvements.
We plan to publish all of these numbers, and much more, complete with details on how to recreate these tests. More details to follow.

6 responses to “More ColdFusion 8 Performance Data”

  1. Sam Farmer Avatar
    Sam Farmer

    Awesome. Great work by the CF engineering team.
    (And, Ben, have you ever heard of the cfchart tag? It can take boring numbers and make them look good! 😉 )

    Reply
  2. Sam Curren Avatar
    Sam Curren

    Thanks for the numbers! I’m very excited to see CF8 released so I can use this performance in my production applications.
    Nobody has said much about any speed improvements of cfdocument. I have a few scripts that run painfully slow, and I’d love to see an increase there. Any hints?

    Reply
  3. Ben Forta Avatar
    Ben Forta

    Sam F, true, but these numbers are anything but boring. 😉
    Sam C, not yet, but will post when I do.
    — Ben

    Reply
  4. Alexandre Madurell Avatar
    Alexandre Madurell

    Wow!
    Those numbers are great indeed!
    I’m definitely installing CF8 right nnnn… hmmm… right after some sleep, might be better (3:30AM already!!!)
    Best regards from Barcelona (we’ll see you around here next October, right?) 😉

    Reply
  5. robi sen Avatar
    robi sen

    One of the greatest sins in performance testing is micro benchmarks. As you know what your actually testing is how well JVM performs a task that it can cache and optimize to no acutely how the application performance will truly behave in a normal application under load.
    To that effect though CF 8 does improve performance and I have run several performance tests on applications and seen some good performance increase but nothing like the orders of magnitude increase one might infer from these numbers. Best way to test is to take a app that you have good benchmarks for and test it following testing best practices (which would include testing it modeling realistic usage and behavior for at least a few hours and then repeating the test at least 5 times to get decent data) then compare the same code running CF 8.

    Reply
  6. Kate M Avatar
    Kate M

    I’m moving an app from 4.5 to 8 and have found no performance increase using the same code, same everything. My CFUG organizer said that was probably because CF8 increased speed from CFCs and since 4.5 didn’t have CFCs I wasn’t going to see a significant increase.
    I hope my managers will think the pretty graphs make up for this . . .

    Reply

Leave a Reply