I am trying to convert informatica transformation to pyspark transformation, but I am stuck in replacing char in the code shown below:
"DECODE(TRUE,
ISNULL(v_check_neg_**) OR v_check_neg_** = '',
i_default,
NOT IS_NUMBER(v_check_neg_** ,
i_default,
REPLACECHR(0,v_check_neg_**, '+-0123456789.' ,'')<>'',
i_default,
TO_DECIMAL(v_check_neg_**,5))
v_check_neg_** = IIF(INSTR(i_string_**,'-')!=0,'-'||SUBSTR(i_string_**,1,INSTR(i_string_**,'-')-1),i_string_**)"
This is what I tried:
def is_digit(value):
if value:
return value.isdigit()
else:
return False
is_digit_udf = udf(is_digit, BooleanType())
df_informatica=df_informatica.withColumn(column_name,when((isnull(col(column_name)) |(col(column_name==' ')),i_default).when(is_digit_udf(col(column_name)),i_default)
df_informatica=df_informatica.withColumn
Please help me convert informatica to pyspark transformation.
i cant see whole statement but your decode logic is - if (v_check_neg_** is null or v_check_neg_ ='' or v_check_neg_ is not number or v_check_neg_** has anything other than numbers) then i_default else TO_DECIMAL(v_check_neg_**,5)
Use python to check above cases and you should be good to go. Like you can use string.isnumeric() to check positive number. And use try except to check -ve, decimal etc. Example to check negative number-
def check_negative(s):
try:
f = float(s)
if (f < 0):
return True
# Otherwise return false
return False
except ValueError:
return False
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.