复杂的字符串解析

Question

I am working with a Chinese database in text that saves entries in this format:

Traditional Simplified [pin1 yin1] /English equivalent 1/equivalent 2/

I've tried parsing it using delimiters (in Java).

This is what I have so far:

                    String delims = "[\\[\\]/]+";
                    String tokens[] = str.split(delims);

The problem is that the English equivalent also contains delimiter tokens.

For instance:

⿔ ⿔ [gui1] /variant of 龜|龟[gui1]/

How would someone parse this String?

I'm trying to get the following information from the String:

Simplified : ⿔

Traditional : ⿔

Pinyin : gui1

English Equivalent : variant of 龜|龟[gui1]

Answer 1

Try using regex to cleanup the whole string.

String text = "⿔ ⿔ [gui1] /variant of 龜|龟[gui1]/";

String pattern =    "(\\S+)\\s*(\\S+)\\s*\\[(.+?)\\]\\s*/(.+?)/";

text = text.replaceAll(pattern, "$1;$2;$3;$4"));

(\\\\S+) ---> ⿔
find continuous non-white space group

\\\\s* --->
find continuous white space

\\\\[(.+?)\\\\] ---> gui1
find everything inside [ bla bla bla ] .
'?' will match shortest possible answer.
eg [ bla bla ] rather than [ bla bla] [ble ble ]

/(.+?)/ ---> variant of 龜|龟[gui1]
same as above, but find everything inside / bla bla /
'?' will match shortest

You can test the regex here

---

Now text becomes:
⿔;⿔;gui1;variant of 龜|龟[gui1]

Next you can continue to use ; as delims to split them

String tokens[] = text.split(";");

Answer 2

The regex pattern is just a tad more complex, as there are often several definitions in the CEDICT :

矮小 矮小 [ai3 xiao3] /short and small/low and small/undersized/

So the regex would be:

^(\S+)\s+(\S+)\s+\[[^]]+\]\s+(/[^/\r]*){1,19}/$