Copying Data from SAP BW to Azure Data Lake Store using Azure Data Factory

Question

I've been trying to copy some data from an on-premises SAP BW to a cloud Azure Data Lake Store. I've already configured the sink as the Data Lake Store but i'm having trouble to configure the source. Already downloaded the netweaver library and put the dlls in my system 32 folder and created the integration runtime which is running on my local machine. Has anyone tried this before?

Thanks

Answer 1

I recommend using SAP Open Hub function SAP BW to generate flat file if you do not have any other SAP Data Services or HANA tools in place. Then the files can be loaded into HDFS, which is Azure data lake storage.

The reason for this recommendation: 1. SAP BW Open hub is easy for developer and even for non-SAP-BW person. 2. I do not recommend use netweaver RFC (dll library) approach to integration with SAP BW as this mostly uses MDX to read BW data with significant coding and understanding of BW metadata. 3. This can ensure no violation of SAP data distribution licensing.

Hope this can help.

Lei

Answer 2

@lei the nw library Fabricio was mentioning has to be the nwlibrfc32.dll which has to be manually injected by customers such that ADF could work with BW data source. This is the shared framework that PowerBI service accesses BW data source. Technically speaking, it's doable, and Fabricio has to ensure the right DLL is injected (32bit vs. 64bit). Without error log we have to speculate.

However, from solution standpoint, please understand that this is not going to be a performing way due to the bottleneck at the underlying MDX engine and result processor of the BW connector. If the volume is small, then there is nos issue. Otherwise, we need to review other options. Open Hub could be an option, if the user is ok to deal with a set of batch job management in BW and separate set of ADF job management in Azure. From IT agility standpoint, coupling two sets of operational processes is not a best approach.

Another option to consider is to hold off on ADF but opt to SSIS instead. Use SSIS to load SAP data with Azure Feature pack, like we do. But this may not be the best approach either, as there is no ADF any more if Fabricio's team already invested in ADF. Or maybe they favor SSIS. All in all, there has to be some level of tradeoff towards sustainable solution.

Back to the original question, please post the error detail and we can help investigate.