删除Hive中的前导特殊字符

Question

I am trying to remove leading special characters (could be -"$&^@_) from "Persi és Levon Cnatówóeez using Hive.

select REGEXP_REPLACE('“Persi és Levon Cnatówóeez', '[^a-zA-Z0-9]+', '') but this removes all special characters.

I am expecting an output similar to

Persi és Levon Cnatówóeez

Answer 1

Try this:

select REGEXP_REPLACE('"Persi és Levon Cnatówóeez', '[^a-zA-Z0-9\u00E0-\u00FC ]+', '');

I tried it on Hive and it replaces any character that is not a letter (a-zA-Z) a number (0-9) or an accented character (\à-\ü).

0: jdbc:hive2://localhost:10000> select REGEXP_REPLACE('"Persi és Levon Cnatówóeez', '[^a-zA-Z0-9\u00E0-\u00FC ]+', '');
+----------------------------+--+
|            _c0             |
+----------------------------+--+
| Persi és Levon Cnatówóeez  |
+----------------------------+--+
1 row selected (0.104 seconds)
0: jdbc:hive2://localhost:10000>

Answer 2

From the Hive documentation:

regexp_replace(string INITIAL_STRING, string PATTERN, string REPLACEMENT)

Returns the string resulting from replacing all substrings in INITIAL_STRING that match the java regular expression syntax defined in PATTERN with instances of REPLACEMENT. For example, regexp_replace("foobar", "oo|ar", "") returns 'fb.' Note that some care is necessary in using predefined character classes: using '\\s' as the second argument will match the letter s; '\\s' is necessary to match whitespace, etc.

Reference: https://cwiki.apache.org/confluence/display/Hive/LanguageManual+UDF

You should do something like this:

select REGEXP_REPLACE('“Persi és Levon Cnatówóeez', '^[\!-\/\[-\`]+', '')

I haven't Hive right know to try this code, but the idea should be correct. In the second field you must put what you want to substitute, not what you want to keep in your string. In this specific case, this should remove (substitute with empty string '') every consequent character in the beginning of the line, that is in the range from ! to /, or in the range [ to ` referring to the ASCII table.