Why did my SQL Server database fill up so fast, and how can I get back my space?

Question

Have a SQL Server environment in which we are using Microsoft SQL Server Management Studio to perform data analysis. The server capacity is 50GB and we loaded in about 8GB worth of '.csv' datasets using the Import Wizard. The 8GB worth of '.csv' files are no longer stored on the server environment.

However, after a few days of querying and creating three new tables, the full 50GB is taken up.

I tried to delete all interim tables, restart the server multiple times, turn off backup, and limit log autogrowth--all to no avail.

How can I get back my storage space? Or at least figure out what is using it all up?

Answer 1

The 8GB worth of '.csv' files...

However, after a few days of querying and creating three new tables, the full 50GB is taken up.

If you're using nvarchar instead of varchar then you're using UCS-2 to store textual data, so your 8GB of data becomes 16GB right there (I wish SQL Server natively supported UTF-8...), then with 3 tables that's 48GB right there - not to mention space used by transaction logs.

When you create a table, then fill it, then delete it, then create a new table, the previously-occupied space is not necessarily reclaimed or overwritten by the new table - this is what probably what's happening. "Normal" tables are expensive in relational database servers - that's why in-memory tables and temporary-tables should be used for ephemeral data (as I believe, though I don't have a reference, that temporary-tables have different storage behavior, and their disk space is more aggressively reclaimed compared to normal tables).

Note that you should use BULK INSERT or at least INSERT INTO ... SELECT FROM instead of repeating a single INSERT INTO ... VALUES... statement because that will hit your transaction log pretty hard (though I'll concedede that BULK INSERT in SQL Server does not offer much flexibility when processing CSV files)

Answer 2

In Simple mode , the log files should cycle on themselves and cause few issues unless you have really big transactions.

In Full mode , the log will grow until you backup the database. Indeed, SQL Server wants to keep all the logs until the datas are safely stored somewhere else. So, rule of the thumb, in Full mode , you MUST backup your database.

Now, when you delete a record or a table, it is not physically deleted from the storage until you clean the database. Use the command DBCC SHRINKDATABASE to do that. You can also shrink a specific file, just lookup the various commands offered.

For more informations:
https://docs.microsoft.com/en-us/sql/relational-databases/databases/shrink-a-database
https://docs.microsoft.com/en-us/sql/t-sql/database-console-commands/dbcc-shrinkdatabase-transact-sql
https://docs.microsoft.com/en-us/sql/t-sql/database-console-commands/dbcc-shrinkfile-transact-sql