RevoScaleR can not find file, directory that exists

Question

I am using RevoscaleR and I have successfully converted csv files to xdf files which I have saved to my local disk.

However, when I try to run functions that call these xdf files I get an error message that there is no such file or directory:

The file or directory 'P:/PROPENSITY/CL_Generic_Retail_201506' cannot be found.

Let me expose the whole process:

My working directory:

> getwd()
[1] "P:/PROPENSITY"

I used this code to convert csv file to xdf:

rx_CL_Generic_Retail_201506 <- rxImport(

  inData = "CL_Generic_Retail_201506_23-05-2017.csv",

  outFile = "CL_Generic_Retail_201506.xdf",

  overwrite = TRUE

)

Then I used this code to check that the conversion was successful:

rxSummary(formula = ~ Avg_Deposits + Total_Num_ + Sumof_CC_AVGBAL_,  

          data = "CL_Generic_Retail_201506.xdf"

)


Summary Statistics Results for: ~Avg_Deposits + Total_Num_ + Sumof_CC_AVGBAL_
Data: "CL_Generic_Retail_201506.xdf" (RxXdfData Data Source)
File name: CL_Generic_Retail_201506.xdf
Number of valid observations: 7155413 

Name             Mean        StdDev      Min        Max        ValidObs MissingObs
Avg_Deposits     4562.914627 128614.5683 -325684032 69317080.0 7155413        0   
Total_Num_          7.062068    247.1506          1   224579.0  831567  6323846   
Sumof_CC_AVGBAL_  951.484138   2249.3149          0   164746.6  601304  6554109  

Up to that point everything was fine.

I continued to convert files to xdf files.

Then I returned to that same file and tried to run the same function (summary) and I got the following error message:

> rxSummary(formula = ~ Avg_Deposits + Total_Num_ + Sumof_CC_AVGBAL_,  
+           
+           data = "CL_Generic_Retail_201506.xdf"
+           
+ )

The file or directory 'CL_Generic_Retail_201506.xdf' cannot be found.

In case I repeat the process and run again rxImport the rxSummary function runs again. But then after a while, the same error repeats.

Could this have to do with back slashes?

Ie: The message is:

The file or directory 'P:\PROPENSITY\CL_Generic_Retail_201506.xdf' cannot be found.

But when I ask R to print the working directory it returns:

> getwd()
[1] "P:/PROPENSITY"

Observe that in the RevoScaleR error message the slashes are \\ while R's output of getwd() has /.

If this is the problem what I could do about it?

By the way this problem occurs in a workstation where Windows and RevoScaleR are installed. In a notebook running also RevoScaleR the problem does not appear.

I would appreciate any suggestion.

---------------------------------------------------------------------------

Here is an image of the directory where it is apparent that the files exist. Image of the PROPENSITY folder with the xdf files

Answer 1

Try using append = "rows". The last csv is probably empty, resulting in overwritting a xdf with an empty xdf which is no file.

rx_CL_Generic_Retail_201506 <- rxImport(inData = "CL_Generic_Retail_201506_23-05-2017.csv", outFile = "CL_Generic_Retail_201506.xdf",  overwrite = TRUE,
append = "rows"
)