按属性搜索的最佳数据库类型和架构

Question

I know that this question may not have an easy answer or at least many possible correct ones.

I am developing a weather web app to search cities by summary, temperature, humidity, precipitation, wind speed, visibility, pressure and some other weather indicators. I will also include the weather station set up that, for making things easier, let's consider it is unique in every city. I would also like to include some city data such as: population, afforestation index as well as latitude, longitude.

Continent, Country and Region will also be needed.

Weather station will include the model number of every sensor installed in it.

There will be around 5.000 cities.

Most used query will be to search the cities by a temperature, humidity, precipitation, wind speed, visibility and pressure range as well as filtering by population, etc. and weather station sensor model name.

A query would look like:

• summary = “Clear”

• and temperature > 6 and temperature < 10

• and pressure > 900 and pressure <1000

• and visibility > 5 and visibility < 7

• and humidity > 0.60 and humidity < 0.90

• and population is > 20.000

• and afforestation index is > 3

• and country = France

• and “sensor1” = “string”

The question is: What database type and schema fit the best my search needs regarding to performance? As you can see I need to search by attributes and not by the city name itself. I am completely free to use Relational or NoSQL database rather that I would like to use an asynchronous system.

I don't know if a NoSQL db like MongoDB is intended to be used like this, if this is the case, would this schema be fast enough? I am worried as everything is nested and indexes can be huge.

"continents": 
[
    {
        "name": "Europe",
        "countries": 
        [
            {
                "name": "France",
                "regions": 
                [
                    {
                        "name": "Île-de-France"
                        "cities": 
                        [
                            {
                                "name": "Paris",
                                "coordinates": {"lat": 48.856614, "lon": 2.352222},
                                "summary":"Clear",
                                "temperature": 9.4,
                                "pressure": 976,
                                "visibility" : 6.8,
                                "humidity" : 0.84,
                                "afforestation": 6,
                                "population": 2249975,
                                ...
                                "weather_station": {
                                    "name": "name",
                                    "sensor 1": "string",
                                    "sensor 2": "string",
                                    "sensor 3": "string",
                                    "sensor 4": "string",
                                }
                            },
                            ...
                        ]
                    },
                    ...
                ]                   
            },
            ...
        ]
    },
    ...
]

I guess this use case has been developed in many other apps that require a search by element attributes.

Oh! I forgot to say that I am using Python and Tornado web framework.

Many thanks for your help!

Answer 1

The Following Schema May be what you are Looking for.

Note that in document DB's you will need to denormalize your data slightly to match the way its accessed the most

this would be 1 row in a City Collection

{
    "City": "Paris",
    "coordinates": {"lat": 48.856614, "lon": 2.352222},
    "summary":"Clear",
    "temperature": 9.4,
    "pressure": 976,
    "visibility" : 6.8,
    "humidity" : 0.84,
    "afforestation": 6,
    "population": 2249975,
    ...
    "weather_station": {
        "name": "name",
        "sensor 1": "string",
        "sensor 2": "string",
        "sensor 3": "string",
        "sensor 4": "string",
    }
    "region": "Île-de-France",
    "country":"France",
    "continent":"Europe"
}

Answer 2

5000 rows in one table? About 20 metrics? No "history"?

Make a single table with 5000 rows and 20 columns. No INDEXes other than the minimal PRIMARY KEY for UPDATEing a row when a weather station reports in. Build a SELECT from the desired conditions, then let to optimizer do a full table scan.

Everything will stay in RAM, and the SELECTs will be "brute force". It should take only a few milliseconds. (I ran a similar SELECT on a 2.7M-row table; it took 1.3 seconds.)

If you are keeping history, then we need to talk further.