将数据从 Excel 导出到 SQL Server 并生成报告

Question

I am working on a project where we would like to export data from a set of complex multi sheet Excel files in Sharepoint. There are multiple tables as well as single kpi values to be extracted from each other Excel file.

The data will fit into the data model (of another database) we already have in SQL Server using an ID value present in the Excel file and SQL Server tables. All data is then consumed in a PowerBI report.

I have considered using powerapps, power automate or Azure Functions to achieve this. I have experience with Azure Functions but not the other two. I am proficient in Python and SQL so I would prefer using those languages if possible.

Other database types than SQL Server are also fine so long as the ID value is present it will be easy to join the data to the tables existing in the SQL Server database inside PowerBI.

My question is what would be the easiest, most user friendly way of achieving this? As much should be automated as possible but the issue with using Excel as an initial data source is that one small change in the Excel file, or data type may cause the automated process to fail. The inherent, breakable nature of excel files means that all it would take would be for one person to change something and the whole system is down, requiring additional work and money to fix it.

Does anyone have experience if a similar project?

I thought a power app would be easy to make and include in PowerBI. The main KPIs could be manually entered but this would be an annoying and boring task for the user.

Using functions could pick up aa change or addition of a file in Sharepoint, extract the data using python and dump the data into SQL or any other database. But this is open to being broken by changes in the Excel file.

Cheers

Answer 1

One trick I've done in such situations is to build INSERT SQL statements per row as a formula in the excel file itself using Excel string functions. You can hide the column from users. If the user adds columns, the formula will still work as Excel will handle rewriting it. It's easier to extract a single column from each tab you care about, as you can find it by the header name you choose.

This also works for quick-and-dirty one time manual data loads.