spark regexp_extract function 的 groupIdx 参数

Question

I don't understand how last parameter groupIdx works in below function, I can't find any details in documentation. I am using this function with groupIdx = 0, when I changed this value to > 0, I've received an error java.lang.IndexOutOfBoundsException: No group 1 . Can someone explain how it works and when groupIdx > 0 could be applied?

regexp_extract(e: Column, exp: String, groupIdx: Int): Column

Answer 1

The argument extracts the part of a match that was captured with the specified capturing group .

See the docs :

regexp_extract(str, regexp[, idx]) - Extracts a group that matches regexp.
Examples:
> SELECT regexp_extract('100-200', '(\d+)-(\d+)', 1);
100

The 100 substring is captured with the first (\d+) in the regex pattern, and the 1 argument makes the function return just this part of the whole match (which is 100-200 ).