How to configure Hive Cli to automatically get the kerberos ticket and renew/request new if expires by it own

Question

Hi I am new to Hive and kerberos. I have some hive jobs which run more then life time of ticket. how can I configure hive so that when I start hive shell if ticket is not cached it automatically request for ticket. After acquiring ticket lets suppose if ticket expire is the middle then automatically acquire new one and also I may have simultaneous job running by same user so may be one cached ticket can be used by many jobs.

Any Solutions or direction to look upon will be highly appreciated. Thanks in Advance.

I am looking for a solution in which hive cli or shell can automatically acquire or renew Kerberos credentials.

Answer 1

What you need to look into is Java Authentication and Authorization Service ( JAAS ) It's how to enable java to use kerberos without adding anything to your code. Specifically here you might want to look at how beeline uses kerberos config as an example .

Create setEnv.sh file and save it inside "bin" folder. Paste below content inside it:

 export HADOOP_HOME=/home/user/beeline/hadoop-2.5.1 export HIVE_HOME=/home/user/beeline/apache-hive-1.2.1-bin export JAVA_HOME=/home/user/beeline/jre PATH=$PATH:$HIVE_HOME/bin:$JAVA_HOME/bin export HADOOP_OPTS="$HADOOP_OPTS -Dsun.security.krb5.debug=true -Djava.security.krb5.conf=/home/user/beeline/conf/krb5.conf -Djavax.security.auth.useSubjectCredsOnly=false -Djava.security.auth.login.config=/home/user/beeline/conf/jaas.conf"

jaas.conf File:

Create and save jaas.conf file under conf folder

Client { com.sun.security.auth.module.Krb5LoginModule required useKeyTab=false useTicketCache=true; };

krb5.conf File:

Create and save krb5.conf File under conf folder. Modify this file as per your environment.

 [logging] default = FILE:~/krb5libs.log kdc = FILE:~/krb5kdc.log admin_server = FILE:~/kadmind.log kdc_rotate = {"period"=>"1d", "versions"=>200} admin_server_rotate = {"period"=>"1d", "versions"=>201} [libdefaults] default_realm = DOMAIN.COM dns_lookup_realm = false dns_lookup_kdc = false forwardable = true renew_lifetime = 30d ticket_lifetime = 30d renewable = yes service = yes kdc_timeout = 5000 default_tgs_enctypes = aes256-cts-hmac-sha1-96 aes128-cts arcfour-hmac-md5 des-cbc-crc des-cbc-md5 des-hmac-sha1 default_tkt_enctypes = aes256-cts-hmac-sha1-96 aes128-cts arcfour-hmac-md5 des-cbc-crc des-cbc-md5 des-hmac-sha1 allow_weak_crypto = yes udp_preference_limit = 1 [realms] DOMAIN.COM = { kdc = kdcserver.domain.com:88 default_domain = domain.com } [domain_realm].domain.com = DOMAIN.COM domain.com = DOMAIN.COM [appdefaults] pam = { debug = false forwardable = true renew_lifetime = 36000 ticket_lifetime = 36000 krb4_convert = false }

It should be noted that the above config doesn't use a renewable kerberos ticket but that's just and example and you can make it renewable.