Hive JSON Serde-未填充空格的键

Question

I have an external table that is built off of a json file. All of the json keys are columns and are populated as expected except for one key that has a space.

Here is the DDL:

CREATE EXTERNAL TABLE foo.bar ( event ARRAY <STRUCT value:STRING ,info:STRUCT <id:STRING ,event_source:STRING>> ROW FORMAT SERDE 'org.openx.data.jsonserde.JsonSerDe' WITH SERDEPROPERTIES("mapping.event_source"="event source") STORED AS INPUTFORMAT 'org.apache.hadoop.mapred.TextInputFormat' OUTPUTFORMAT 'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat' LOCATION 'foo/bar'

All of the values show up as expected except for event_source, which shows up as NULL. The original form of event_source in the json file is 'event source' without the single quotes. Is there something I need to do different with the WITH SERDEPROPERTIES setting in order to get the key to work properly?

Thanks

Answer 1

you mean that the json has data like

{ id: "myid", event source: "eventsource" } 

If so, there's not much that can be done since it's simply broken JSON. If not, can you post a sample of the JSON you're trying to read ?

Answer 2

I have encountered a similar problem as above but with a slight variation that the input data is correct json.

I have an external table that is built off of a json file. All of the json keys are populated except one msrp_currency Here is the DDL:

CREATE EXTERNAL TABLE foo.bar
( id string,
variants array<struct<pid:string, msrp_currency:string>>
)
ROW FORMAT SERDE 'org.openx.data.jsonserde.JsonSerDe'
WITH SERDEPROPERTIES ( "ignore.malformed.json" = "true" ,
'mapping.variants.msrp_currency' = 'variants.msrpcurrency')
LOCATION 'foo/bar'

All of the values show up as expected except for msrp_currency, which shows up as NULL. The reason I need to introduce underscore is because later I need to extract the same field value as msrpCurrecny using brickhouse to_json UDF.

sample values:

{ "pid": "mypid", "msrpCurrency": "USD" }