The best way to manage records in a table

Question

I am sorry that they ask this question has been asked many times but I still have not yet found the best answer.

I am worried applications take a long time to download the record or filter the records . Assuming I have a table called tbl_customer . And records in tbl_customer more than 10,000 rows .

The first question , I am using Data Grid View to display the records . Would be ideal if I download all the records up to 10,000 rows into the Data Grid View? Or perhaps I had better put the record row limit?

Second question , what is the best way to filter records in tbl_customer . Do we just need to query using SQL? or using LINQ? or maybe there is a better way?

For now, I only use this way:

DataTable dtCustomer = new DataTable();

 using (SqlConnection conn = new SqlConnection(cs.connString))
        {
            string query = "SELECT customerName,customerAddress FROM tbl_customer WHERE customerAddress = '"+addressValue+"' ORDER BY customerName ASC;";

            using (SqlDataAdapter adap = new SqlDataAdapter(query, conn))
            {
                adap.Fill(dtCustomer);

            }
        }

dgvListCustomer.DataSource = dtCustomer

Then I learn about LINQ so i do like this

    DataTable dtCustomer = new DataTable();

         using (SqlConnection conn = new SqlConnection(cs.connString))
                {
                    string query = "SELECT * FROM tbl_customer ORDER BY customerName ASC;";

                    using (SqlDataAdapter adap = new SqlDataAdapter(query, conn))
                    {
                        adap.Fill(dtCustomer);

                    }
                }

var resultCustomer = from row in dtCustomer.AsEnumerable()
                                         where row.Field<string>("customerAddress") == addressValue
                                         select new
                                         {
                                             customerName = row["customerName"].ToString(),
                                             customerAddress = row2["customerAddress"].ToString(),

                                         };

 dgvListCustomer.DataSource = resultCustomer;

Workflow SQL> DATATABLE> LINQ > DataGridView is suitable to filter records? Or if there are better suggestions are most welcome.

Thanks you..:)

Answer 1

I am worried applications take a long time to download the record or filter the records.

Welcome - you seem to live in a world like me where performance ms measured in milliseconds, and yes, on a low power server it will take likely more than a millisecond (0.001 seconds) to hot load and filter 10.000 rows.

As such, my advice is not to put that database on a tablet or mobile phone but to use at least a decent desktop level compute r or VM for the database server.

As a hint: I am regularly making queries on a billion row table and it is fast. Anything below a million rows is a joke these days - in fact it was nothing worth mentioning when I started with databases more than 15 years ago. You are the guy asking whether it is better to have a ferrari or a porsche becauese you are concerned whether any of those case goes more than 20km/h.

Would be ideal if I download all the records up to 10,000 rows into the Data Grid View?

In order to get fired? Yes. Old rule with databases: never load more data than you have to, especially when you have no clue. Forget the SQL side - you will get UI problems with 10.000 rows and more, especially usability issues.

Do we just need to query using SQL? or using LINQ?

Hint: Linq is also using SQL under the hood. The question is more - how much time do you want to spend writing boring repetitive code for handwritten SQL like in your examples? Espeically given that you also do "smart" things like referencing fields by name, not ordinal, and asking for "select *" instead of a field list, bot obvious beginner mistakes.

What you should definitely not do - but you do - is using a DataTable. Get a decent book about programming databases. RTFSM may help - both LINQ (which I am not sure what you mean - LINQ is a language for the compiler, you need an implementor, so that could be NHibernate, Entity Framework, Linq2Sql, BlToolkit, to name just a FEW tha t go from a LINQ query to a sql statement).

Workflow SQL> DATATABLE> LINQ > DataGridView is suitable to filter records?

A Ferrari is also suitable to transport 20 tons of coal from A to B - just the worst possible car for it. GSour stack is likely the worst I have seen, but it is suuitable in that you CAN do it - slow, lots f mmemoory use, but you will get a result and hopefully fired. You pull the data from a high performance database into a data table, then use a non integrating technology (LINQ) to filter (not using the indices in the data table) to go into yet another layer.

Just to give you an idea - this would get you removed from quite some "beginning programming" courses.

What about:

LINQ

Point.

Pulls a collection of business objects that go to the UI. Period.

Read at least some of the sample code for the technologies you use.