有没有办法在 Databricks SQL (Spark SQL) 中按表、模式和目录计算行数？

Question

I need to create a dashboard inside Databricks that summarizes the number of rows in the current workspace right now.

Is there a way to create a SQL query to calculate the number of rows by table, schema, and catalog? The expected result would be:

Catalog	Schema	Table	Rows
example_catalog_1	Finance	table_example_1	1567000
example_catalog_1	Finance	table_example_2	67000
example_catalog_2	Procurement	table_example_1	45324888
example_catalog_2	Procurement	table_example_2	89765987
example_catalog_2	Procurement	table_example_3	145000

Currently, I am working on a pure SQL workflow. So I would like to understand if it's possible to execute such an action using SQL, because as much as I know, the dashboards in Databricks do not accept PySpark Codes.

I was looking for a way to do that. I know that it's possible to access the tables in the workspace by using system.information_schema.tables but how to use it to count to total rows for each table presented there?

I was checking that via SQL Server it's possible via sys schema , dynamic query , or BEGIN...END clause. I couldn't find a way in Databricks to do that.

Answer 1

I strongly doubt if you can run that kind of query in the databricks dashboard. The link shared by @Sharma is more as to how to get the record count using dataframe and not how to link that with the databricks dashboard.