CF2016 timeout on cfdirectory and directoryList to s3 AWS but works on CommandBox/Lucee

Question

This command

<cfset thedirlist = directoryList(theimgthumbpath, false, "name", "", "asc", "dir")>

and this command

<cfdirectory directory="#theimgthumbpath#" action="list" recurse="false" name="thedirlist">

run into a timeout on our CF2016 server. When I run the command from my local CommandBox/Lucee to the exact same bucket with the exact same key and secret it works ok.

theimgthumbpath :

s3://#s3AccessKey#:#s3SecretKey#@#companybucket#/#projectname#/files/thumbnails/

The same on both environments.

theimgthumbpath contains 14 folders.

directoryExists give a true , also on CF2016

If I dive deeper, for example theimgthumbpath/1200 (which contains 1800 files) again on CommandBox/Lucee it works fine and on CF2016 it runs into timeout.

I'd like to know what could be the cause of the timeout on CF2016 unlike on CommandBox/Lucee. Since I have no idea where to start looking, I don't know what other details to provide.

Edit: On CF2016 I can perform actions on specific files in theimgthumbpath subfolders like copy up and down, setStoreACL() etc. But I cannot fe create a directory. I can on CommandBox/Lucee.

Answer 1

We had similar slowness issues with S# access in Coldfusion 2016. So I created the following function to check for files with AWS SDK(You need to download SDK jar file and place it in CF_HOME\cfusion\lib for this to work). Using SDK for file operations are much more faster than Coldfusion 2016-18 implementations.

//Create SDK Client object
public any function getS3SDKClient(){
  local.objBasicAWSCredentials = createObject('java', 'com.amazonaws.auth.BasicAWSCredentials').init(application.s3key, application.s3secret);
  local.objAwsCreds = createObject('java', 'com.amazonaws.auth.AWSStaticCredentialsProvider').init(local.objBasicAWSCredentials);
  local.objRegions = createObject('java', 'com.amazonaws.regions.Regions');
  local.s3Client = createObject('java', 'com.amazonaws.services.s3.AmazonS3ClientBuilder').standard()
                    .withRegion(local.objRegions.US_WEST_2)
                    .withCredentials(local.objAwsCreds)
                    .build();
  return local.s3Client;
}

//Check for file
public boolean function s3CheckFileExists(
  required string bucket,
  required string sourcePath
){
  local.s3Client = getS3SDKClient();
  return local.s3Client.doesObjectExist(arguments.bucket, arguments.sourcePath);
}

//Get list of files in path as an Aray
public array function s3GetPathFileList(
  required string bucket,
  required string sourcePath,
  string extensions = '' // list of extensions seperated by | pipe
){
  local.s3Client = getS3SDKClient();
  local.result = local.s3Client.listObjectsV2(arguments.bucket, arguments.sourcePath);
  local.fileList = [];
  for(local.item in local.result.getObjectSummaries()){
    if(len(trim(arguments.extensions)) == 0 || reFindNoCase("^.*\.(" & arguments.extensions & ")$", local.item.getKey())){
      local.fileList.append(local.item.getKey());
    }
  }
  return local.fileList;
}

I have heard S3 access in Coldfusion 2020 is way better than 2018. But if you cannot upgrade, then its easier to use this approach.