Handle >10 million rows for one table (postgresql)

Question

I imported 11 Million location names from geonames.org into my postgresql. However when I try to just view the data for instance in TablePlus it is extremely slow. Executing a simple select for one row, takes like 2 minutes. What can I do with large data, so that it won't be too slow and I can select it very fast?

I think I don't have any indexes, would that make a difference?

This is my table:

create table geoname (
    geonameid   int,
    name varchar(200),
    asciiname varchar(200),
    alternatenames text,
    latitude float,
    longitude float,
    fclass char(1),
    fcode varchar(10),
    country varchar(2),
    cc2 varchar(120),
    admin1 varchar(20),
    admin2 varchar(80),
    admin3 varchar(20),
    admin4 varchar(20),
    population bigint,
    elevation int,
    gtopo30 int,
    timezone varchar(40),
    moddate date
 );

Answer 1

You need to specify what the query looks like.

Indexes would definitely make a difference. But the type of index depends on the query you are using and the columns used for selecting one or more rows.

The place to start is by defining a primary key on the table. Presumably, geonameid is the primary key. You can do this:

alter table geonames add constraint pk_geonames_geonameid primary key (geonameid);

You should really do this when you create the table, but better late than never.

If you are searching by geonameid , then you will notice a significant speed-up.

If you want to search by other columns, such as name or asciiname , then add indexes for those:

create index idx_geonames_name on geonames(name);
create index idx_geonames_asciiname on geonames(aciiname);

This doesn't work for all searches. If your criteria is like with wildcards, you may need a different indexing strategy. Similarly, if it is by latitude and longitude , you'll want a GIS index.