Ben Forta

AdobeStock_455007340

Improving <CFDIRECTORY> Performance

Mark Kruger posted comments about the performance of (see post 1 and post 2). He noted that returning full directory information with all file details (especially over network connections) can be painfully slow. As per his post, using Java APIs to return directory lists is far quicker, which is odd as actually uses the same Java APIs internally.
In a comment on Mark’s blog, Tom Jordahl pointed out that the reason for this is that for to return all file details it needs to first retrieve the file list and then perform a Java stat() for each file, and that stat() call is very time consuming (especially over network connections). The solution, if you just need file names and not other details, is to use the undocumented LISTINFO attribute. LISTINFO defaults to “all”, but specify LISTINFO=”name” and will only return file names and will run just as quickly as the Java APIs it relies on.

13 responses to “Improving <CFDIRECTORY> Performance”

  1. Andrea Veggiani Avatar
    Andrea Veggiani

    This can be done on all versions of CFMX?

    Reply
  2. Anthony Batchelor Avatar
    Anthony Batchelor

    Hi,
    Any chance of this being added to the documentation?
    Cheers,
    Tony

    Reply
  3. Ben Forta Avatar
    Ben Forta

    Andrea, I don’t know about CFMX, it is definitley in CFMX7.
    Anthony, yes, it’ll be added, and is already in a note at the bottom of that page in LiveDocs.
    — Ben

    Reply
  4. Dan G. Switzer, II Avatar
    Dan G. Switzer, II

    Does the listinfo attribute take a list of columns to return? If so, what columns can be returned w/out the stat() call being invoked?

    Reply
  5. Ben Forta Avatar
    Ben Forta

    Dan,
    Right now it is NAME or ALL, as far as I know.
    — Ben

    Reply
  6. Nathan Mische Avatar
    Nathan Mische

    I tested and this does not work on CFMX 6.1, only 7. Also, it looks like the only valid values are NAME and ALL. (Anything other than name appears to give you all the info.) It would be nice if you could include a list of columns as Dan suggests.

    Reply
  7. Michael Avatar
    Michael

    Well hopefully the cf team will add the other parameters to scorpio. It would be nice to give multiple parameters like name and size or something like that. Also a expanded filter would nice, eg <cfdirectory name="getImages" folder="C:images" filter=" *.jpg | *.gif | *.png " />
    I’m not thinking any of this will be released in any hotfix… Or maybe they will… Ben??
    btw, you got some comment spam up here…

    Reply
  8. Javier Julio Avatar
    Javier Julio

    Yes I have great interest in knowing if the filter attribute in cfdirectory will be expanded as Michael has suggested. For example, I currently use cfdirectory to automate some things in my application for me such as initializing my controllers (use cfdirectory to read and list my controllers directory) and initializing my models (simply reads the models directory). With that approach I’d like to do a little bit more with the filter other than *.cfc so I can filter out specific files, for example those that start with an underscore "_" or some familiar set of letters. Thanks though for this great post! All I’ve needed is to the name of the file and no other data so its nice to know I can speed it up now using the listinfo attribute. Thanks for sharing with us Ben!

    Reply
  9. CJ Avatar
    CJ

    Is there anyway that I can get just the NAME and DATELASTMODIFIED from cfdirectory?
    Thanks
    CJ

    Reply
  10. Bruce L. Jacobs Avatar
    Bruce L. Jacobs

    I have discovered that the use of the sort attribute of CFDirectory is a significant contributor to it’s poor response with large file lists (CFMX 6.1).
    If you need to sort a large CFDirectory listing, you may find a very significant improvement if you…
    – CFDirectory but do NOT use the sort attribute.
    – Do a Query of Query on the CFDirectory results and sort within the Q of Q.
    – Do not use SELECT * in the Q of Q. Only select the fields you need such as the file Name and Date.
    – If you need to filter the results, it can be done in the same Q of Q.
    Be sure to test both ways and see which is better. (We had a 10 times faster response in one case.)
    (Reminder: CFDirectory results are stored as a Query object in memory.)
    The above is a "CF only" workaround using documented calls (as compared to some of the known JAVA workarounds).
    Hope This helps some folks!
    Bruce L. Jacobs
    “The comments and opinions expressed are my own
    and do not represent the views of United Space Alliance.”

    Reply
  11. Alex Avatar
    Alex

    Thanks though for this great post! All I’ve needed is to the name of the file and no other data so its nice to know I can speed it up now using the listinfo attribute.Anything other than name appears to give you all the info

    Reply
  12. Abhijeet Avatar
    Abhijeet

    Was just curious what all parameter LISTINFO could have that are as useful as this "name". By now, I am sure this is in documented info
    —
    Abhijeet
    http://www.lifeiscolourful.com

    Reply
  13. Techie Traveler Avatar
    Techie Traveler

    Wow, that’s a good number of spam comments from various people. I am not sure if Ben is busy or something else, but I never saw this happening few months back

    Reply

Leave a Reply