堆棧內存溢出
  簡體   English   中英

蜂巢：SHOW CONF不遵守hive-site.xml

[英]Hive: SHOW CONF does not obey hive-site.xml

 原文  2016-04-27 21:05:03  7   1    hadoop/ hive

問題描述

我有以下配置單元站點配置： 

[hadoop@ip-10-102-201-205 ~]$ cat /etc/hive/conf.dist/hive-site.xml
<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>

<!-- Licensed to the Apache Software Foundation (ASF) under one or more       -->
<!-- contributor license agreements.  See the NOTICE file distributed with    -->
<!-- this work for additional information regarding copyright ownership.      -->
<!-- The ASF licenses this file to You under the Apache License, Version 2.0  -->
<!-- (the "License"); you may not use this file except in compliance with     -->
<!-- the License.  You may obtain a copy of the License at                    -->
<!--                                                                          -->
<!--     http://www.apache.org/licenses/LICENSE-2.0                           -->
<!--                                                                          -->
<!-- Unless required by applicable law or agreed to in writing, software      -->
<!-- distributed under the License is distributed on an "AS IS" BASIS,        -->
<!-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -->
<!-- See the License for the specific language governing permissions and      -->
<!-- limitations under the License.                                           -->

<configuration>

<!-- Hive Configuration can either be stored in this file or in the hadoop configuration files  -->
<!-- that are implied by Hadoop setup variables.                                                -->
<!-- Aside from Hadoop setup variables - this file is provided as a convenience so that Hive    -->
<!-- users do not have to edit hadoop configuration files (that may be managed as a centralized -->
<!-- resource).                                                                                 -->

<!-- Hive Execution Parameters -->


<property>
  <name>hbase.zookeeper.quorum</name>
  <value>ip-10-102-201-205.ec2.internal</value>
  <description>http://wiki.apache.org/hadoop/Hive/HBaseIntegration</description>
</property>

<property>
  <name>hive.execution.engine</name>
  <value>mr</value>
</property>

  <property>
    <name>fs.defaultFS</name>
    <value>hdfs://ip-10-102-201-205.ec2.internal:8020</value>
  </property>

<property>
  <name>hive.metastore.uris</name>
  <value>thrift://ip-10-102-201-205.ec2.internal:9083</value>
  <description>JDBC connect string for a JDBC metastore</description>
</property>

<property>
    <name>javax.jdo.option.ConnectionURL</name>
    <value>jdbc:mysql://ip-10-102-201-205.ec2.internal:3306/hive?createDatabaseIfNotExist=true</value>
    <description>username to use against metastore database</description>
</property>

<property>
    <name>javax.jdo.option.ConnectionDriverName</name>
    <value>org.mariadb.jdbc.Driver</value>
    <description>username to use against metastore database</description>
</property>

<property>
  <name>javax.jdo.option.ConnectionUserName</name>
  <value>hive</value>
  <description>username to use against metastore database</description>
</property>

<property>
  <name>javax.jdo.option.ConnectionPassword</name>
  <value>nsqiBHEs6Wn8mz9b</value>
  <description>password to use against metastore database</description>
</property>

  <property>
    <name>datanucleus.fixedDatastore</name>
    <value>true</value>
  </property>

  <property>
    <name>mapred.reduce.tasks</name>
    <value>-1</value>
  </property>

  <property>
    <name>mapred.max.split.size</name>
    <value>256000000</value>
  </property>

  <property>
    <name>hive.metastore.connect.retries</name>
    <value>5</value>
  </property>

  <property>
    <name>hive.optimize.sort.dynamic.partition</name>
    <value>true</value>
  </property>

  <property>
    <name>hive.aux.jars.path</name>
    <value>file:///etc/hive/conf/jar/json-serde-1.3.8-SNAPSHOT-jar-with-dependencies.jar</value>
  </property>

</configuration>

這只是AWS EMR的默認hive-site.xml，但在底部設置了“ hive.aux.jars.path”。

當我開始一個會話時，Hive聲稱已經加載了正確的配置文件，但是未設置或設置不正確： 

[hadoop@ip-10-102-201-205 ~]$ echo 'SHOW CONF "hive.aux.jars.path"; EXIT;' | hive --hiveconf hive.root.logger=DEBUG,console
...
16/04/27 20:49:43 [main()]: DEBUG common.LogUtils: Using hive-site.xml found on CLASSPATH at /etc/hive/conf.dist/hive-site.xml
...

hive> SHOW CONF "hive.aux.jars.path"; EXIT;
    STRING  The location of the plugin jars that contain implementations of user defined functions and serdes.
Time taken: 0.741 seconds, Fetched: 1 row(s)
...

第一列（空）是該值應在的位置。 這與“ hive.optimize.sort.dynamic.partition”的故事相同：在配置中將其設置為“ true”，但在我實際運行控制台時顯示為“ false”。 DEBUG和TRACE日志不顯示任何配置錯誤。 為什么未設置這些值？

完整的調試輸出，以防萬一有人好奇： 

[hadoop@ip-10-102-201-205 ~]$ echo 'SHOW CONF "hive.aux.jars.path"; EXIT;' | hive --hiveconf hive.root.logger=DEBUG,console
OpenJDK 64-Bit Server VM warning: ignoring option MaxPermSize=128m; support was removed in 8.0
OpenJDK 64-Bit Server VM warning: ignoring option MaxPermSize=128m; support was removed in 8.0
16/04/27 20:49:43 [main()]: DEBUG common.LogUtils: Using hive-site.xml found on CLASSPATH at /etc/hive/conf.dist/hive-site.xml

Logging initialized using configuration in file:/etc/hive/conf.dist/hive-log4j.properties
16/04/27 20:49:43 [main()]: INFO SessionState: 
Logging initialized using configuration in file:/etc/hive/conf.dist/hive-log4j.properties
16/04/27 20:49:43 [main()]: DEBUG parse.VariableSubstitution: Substitution is on: hive
16/04/27 20:49:43 [main()]: DEBUG lib.MutableMetricsFactory: field org.apache.hadoop.metrics2.lib.MutableRate org.apache.hadoop.security.UserGroupInformation$UgiMetrics.loginSuccess with annotation @org.apache.hadoop.metrics2.annotation.Metric(about=, always=false, sampleName=Ops, type=DEFAULT, valueName=Time, value=[Rate of successful kerberos logins and latency (milliseconds)])
16/04/27 20:49:43 [main()]: DEBUG lib.MutableMetricsFactory: field org.apache.hadoop.metrics2.lib.MutableRate org.apache.hadoop.security.UserGroupInformation$UgiMetrics.loginFailure with annotation @org.apache.hadoop.metrics2.annotation.Metric(about=, always=false, sampleName=Ops, type=DEFAULT, valueName=Time, value=[Rate of failed kerberos logins and latency (milliseconds)])
16/04/27 20:49:43 [main()]: DEBUG lib.MutableMetricsFactory: field org.apache.hadoop.metrics2.lib.MutableRate org.apache.hadoop.security.UserGroupInformation$UgiMetrics.getGroups with annotation @org.apache.hadoop.metrics2.annotation.Metric(about=, always=false, sampleName=Ops, type=DEFAULT, valueName=Time, value=[GetGroups])
16/04/27 20:49:43 [main()]: DEBUG impl.MetricsSystemImpl: UgiMetrics, User and group related metrics
16/04/27 20:49:43 [main()]: DEBUG util.KerberosName: Kerberos krb5 configuration not found, setting default realm to empty
16/04/27 20:49:43 [main()]: DEBUG security.Groups:  Creating new Groups object
16/04/27 20:49:43 [main()]: DEBUG util.NativeCodeLoader: Trying to load the custom-built native-hadoop library...
16/04/27 20:49:43 [main()]: DEBUG util.NativeCodeLoader: Loaded the native-hadoop library
16/04/27 20:49:43 [main()]: DEBUG security.JniBasedUnixGroupsMapping: Using JniBasedUnixGroupsMapping for Group resolution
16/04/27 20:49:43 [main()]: DEBUG security.JniBasedUnixGroupsMappingWithFallback: Group mapping impl=org.apache.hadoop.security.JniBasedUnixGroupsMapping
16/04/27 20:49:43 [main()]: DEBUG security.Groups: Group mapping impl=org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback; cacheTimeout=300000; warningDeltaMs=5000
16/04/27 20:49:43 [main()]: DEBUG security.UserGroupInformation: hadoop login
16/04/27 20:49:43 [main()]: DEBUG security.UserGroupInformation: hadoop login commit
16/04/27 20:49:43 [main()]: DEBUG security.UserGroupInformation: using local user:UnixPrincipal: hadoop
16/04/27 20:49:43 [main()]: DEBUG security.UserGroupInformation: Using user: "UnixPrincipal: hadoop" with name hadoop
16/04/27 20:49:43 [main()]: DEBUG security.UserGroupInformation: User entry: "hadoop"
16/04/27 20:49:43 [main()]: DEBUG security.UserGroupInformation: UGI loginUser:hadoop (auth:SIMPLE)
16/04/27 20:49:43 [main()]: INFO hive.metastore: Trying to connect to metastore with URI thrift://ip-10-102-201-205.ec2.internal:9083
16/04/27 20:49:43 [main()]: INFO hive.metastore: Connected to metastore.
16/04/27 20:49:44 [main()]: DEBUG hdfs.BlockReaderLocal: dfs.client.use.legacy.blockreader.local = false
16/04/27 20:49:44 [main()]: DEBUG hdfs.BlockReaderLocal: dfs.client.read.shortcircuit = false
16/04/27 20:49:44 [main()]: DEBUG hdfs.BlockReaderLocal: dfs.client.domain.socket.data.traffic = false
16/04/27 20:49:44 [main()]: DEBUG hdfs.BlockReaderLocal: dfs.domain.socket.path = 
16/04/27 20:49:44 [main()]: DEBUG retry.RetryUtils: multipleLinearRandomRetry = null
16/04/27 20:49:44 [main()]: DEBUG ipc.Server: rpcKind=RPC_PROTOCOL_BUFFER, rpcRequestWrapperClass=class org.apache.hadoop.ipc.ProtobufRpcEngine$RpcRequestWrapper, rpcInvoker=org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker@258d79be
16/04/27 20:49:44 [main()]: DEBUG ipc.Client: getting client out of cache: org.apache.hadoop.ipc.Client@7fcf2fc1
16/04/27 20:49:44 [Finalizer()]: DEBUG azure.NativeAzureFileSystem: finalize() called.
16/04/27 20:49:44 [Finalizer()]: DEBUG azure.NativeAzureFileSystem: finalize() called.
16/04/27 20:49:44 [client DomainSocketWatcher()]: DEBUG unix.DomainSocketWatcher: org.apache.hadoop.net.unix.DomainSocketWatcher$2@4ce9d42d: starting with interruptCheckPeriodMs = 60000
16/04/27 20:49:44 [main()]: DEBUG util.PerformanceAdvisory: Both short-circuit local reads and UNIX domain socket are disabled.
16/04/27 20:49:44 [main()]: DEBUG sasl.DataTransferSaslUtil: DataTransferProtocol not using SaslPropertiesResolver, no QOP found in configuration for dfs.data.transfer.protection
16/04/27 20:49:44 [main()]: DEBUG ipc.Client: The ping interval is 60000 ms.
16/04/27 20:49:44 [main()]: DEBUG ipc.Client: Connecting to ip-10-102-201-205.ec2.internal/10.102.201.205:8020
16/04/27 20:49:44 [IPC Client (1309129055) connection to ip-10-102-201-205.ec2.internal/10.102.201.205:8020 from hadoop()]: DEBUG ipc.Client: IPC Client (1309129055) connection to ip-10-102-201-205.ec2.internal/10.102.201.205:8020 from hadoop: starting, having connections 1
16/04/27 20:49:44 [IPC Parameter Sending Thread #0()]: DEBUG ipc.Client: IPC Client (1309129055) connection to ip-10-102-201-205.ec2.internal/10.102.201.205:8020 from hadoop sending #0
16/04/27 20:49:44 [IPC Client (1309129055) connection to ip-10-102-201-205.ec2.internal/10.102.201.205:8020 from hadoop()]: DEBUG ipc.Client: IPC Client (1309129055) connection to ip-10-102-201-205.ec2.internal/10.102.201.205:8020 from hadoop got value #0
16/04/27 20:49:44 [main()]: DEBUG ipc.ProtobufRpcEngine: Call: getFileInfo took 45ms
16/04/27 20:49:44 [IPC Parameter Sending Thread #0()]: DEBUG ipc.Client: IPC Client (1309129055) connection to ip-10-102-201-205.ec2.internal/10.102.201.205:8020 from hadoop sending #1
16/04/27 20:49:44 [IPC Client (1309129055) connection to ip-10-102-201-205.ec2.internal/10.102.201.205:8020 from hadoop()]: DEBUG ipc.Client: IPC Client (1309129055) connection to ip-10-102-201-205.ec2.internal/10.102.201.205:8020 from hadoop got value #1
16/04/27 20:49:44 [main()]: DEBUG ipc.ProtobufRpcEngine: Call: getFileInfo took 1ms
16/04/27 20:49:44 [main()]: DEBUG session.SessionState: HDFS root scratch dir: /tmp/hive, permission: rwx-wx-wx
16/04/27 20:49:44 [IPC Parameter Sending Thread #0()]: DEBUG ipc.Client: IPC Client (1309129055) connection to ip-10-102-201-205.ec2.internal/10.102.201.205:8020 from hadoop sending #2
16/04/27 20:49:44 [IPC Client (1309129055) connection to ip-10-102-201-205.ec2.internal/10.102.201.205:8020 from hadoop()]: DEBUG ipc.Client: IPC Client (1309129055) connection to ip-10-102-201-205.ec2.internal/10.102.201.205:8020 from hadoop got value #2
16/04/27 20:49:44 [main()]: DEBUG ipc.ProtobufRpcEngine: Call: getFileInfo took 1ms
16/04/27 20:49:44 [main()]: DEBUG nativeio.NativeIO: Initialized cache for IDs to User/Group mapping with a  cache timeout of 14400 seconds.
16/04/27 20:49:44 [main()]: INFO session.SessionState: Created local directory: /mnt/tmp/f113c3f9-3316-4998-8e80-9bd4c3fc079c_resources
16/04/27 20:49:44 [IPC Parameter Sending Thread #0()]: DEBUG ipc.Client: IPC Client (1309129055) connection to ip-10-102-201-205.ec2.internal/10.102.201.205:8020 from hadoop sending #3
16/04/27 20:49:44 [IPC Client (1309129055) connection to ip-10-102-201-205.ec2.internal/10.102.201.205:8020 from hadoop()]: DEBUG ipc.Client: IPC Client (1309129055) connection to ip-10-102-201-205.ec2.internal/10.102.201.205:8020 from hadoop got value #3
16/04/27 20:49:44 [main()]: DEBUG ipc.ProtobufRpcEngine: Call: getFileInfo took 2ms
16/04/27 20:49:44 [main()]: DEBUG hdfs.DFSClient: /tmp/hive/hadoop/f113c3f9-3316-4998-8e80-9bd4c3fc079c: masked=rwx------
16/04/27 20:49:44 [IPC Parameter Sending Thread #0()]: DEBUG ipc.Client: IPC Client (1309129055) connection to ip-10-102-201-205.ec2.internal/10.102.201.205:8020 from hadoop sending #4
16/04/27 20:49:44 [IPC Client (1309129055) connection to ip-10-102-201-205.ec2.internal/10.102.201.205:8020 from hadoop()]: DEBUG ipc.Client: IPC Client (1309129055) connection to ip-10-102-201-205.ec2.internal/10.102.201.205:8020 from hadoop got value #4
16/04/27 20:49:44 [main()]: DEBUG ipc.ProtobufRpcEngine: Call: mkdirs took 3ms
16/04/27 20:49:44 [main()]: INFO session.SessionState: Created HDFS directory: /tmp/hive/hadoop/f113c3f9-3316-4998-8e80-9bd4c3fc079c
16/04/27 20:49:44 [IPC Parameter Sending Thread #0()]: DEBUG ipc.Client: IPC Client (1309129055) connection to ip-10-102-201-205.ec2.internal/10.102.201.205:8020 from hadoop sending #5
16/04/27 20:49:44 [IPC Client (1309129055) connection to ip-10-102-201-205.ec2.internal/10.102.201.205:8020 from hadoop()]: DEBUG ipc.Client: IPC Client (1309129055) connection to ip-10-102-201-205.ec2.internal/10.102.201.205:8020 from hadoop got value #5
16/04/27 20:49:44 [main()]: DEBUG ipc.ProtobufRpcEngine: Call: getFileInfo took 2ms
16/04/27 20:49:44 [main()]: INFO session.SessionState: Created local directory: /mnt/tmp/hadoop/f113c3f9-3316-4998-8e80-9bd4c3fc079c
16/04/27 20:49:44 [IPC Parameter Sending Thread #0()]: DEBUG ipc.Client: IPC Client (1309129055) connection to ip-10-102-201-205.ec2.internal/10.102.201.205:8020 from hadoop sending #6
16/04/27 20:49:44 [IPC Client (1309129055) connection to ip-10-102-201-205.ec2.internal/10.102.201.205:8020 from hadoop()]: DEBUG ipc.Client: IPC Client (1309129055) connection to ip-10-102-201-205.ec2.internal/10.102.201.205:8020 from hadoop got value #6
16/04/27 20:49:44 [main()]: DEBUG ipc.ProtobufRpcEngine: Call: getFileInfo took 1ms
16/04/27 20:49:44 [main()]: DEBUG hdfs.DFSClient: /tmp/hive/hadoop/f113c3f9-3316-4998-8e80-9bd4c3fc079c/_tmp_space.db: masked=rwx------
16/04/27 20:49:44 [IPC Parameter Sending Thread #0()]: DEBUG ipc.Client: IPC Client (1309129055) connection to ip-10-102-201-205.ec2.internal/10.102.201.205:8020 from hadoop sending #7
16/04/27 20:49:44 [IPC Client (1309129055) connection to ip-10-102-201-205.ec2.internal/10.102.201.205:8020 from hadoop()]: DEBUG ipc.Client: IPC Client (1309129055) connection to ip-10-102-201-205.ec2.internal/10.102.201.205:8020 from hadoop got value #7
16/04/27 20:49:44 [main()]: DEBUG ipc.ProtobufRpcEngine: Call: mkdirs took 2ms
16/04/27 20:49:44 [main()]: INFO session.SessionState: Created HDFS directory: /tmp/hive/hadoop/f113c3f9-3316-4998-8e80-9bd4c3fc079c/_tmp_space.db
16/04/27 20:49:44 [IPC Parameter Sending Thread #0()]: DEBUG ipc.Client: IPC Client (1309129055) connection to ip-10-102-201-205.ec2.internal/10.102.201.205:8020 from hadoop sending #8
16/04/27 20:49:44 [IPC Client (1309129055) connection to ip-10-102-201-205.ec2.internal/10.102.201.205:8020 from hadoop()]: DEBUG ipc.Client: IPC Client (1309129055) connection to ip-10-102-201-205.ec2.internal/10.102.201.205:8020 from hadoop got value #8
16/04/27 20:49:44 [main()]: DEBUG ipc.ProtobufRpcEngine: Call: getFileInfo took 1ms
16/04/27 20:49:44 [main()]: INFO session.SessionState: No Tez session required at this point. hive.execution.engine=mr.
16/04/27 20:49:44 [main()]: DEBUG exec.Utilities: Use session specified class loader
16/04/27 20:49:44 [main()]: DEBUG exec.Utilities: Use session specified class loader
16/04/27 20:49:44 [main()]: DEBUG exec.Utilities: Use session specified class loader
<this line repeats many times>
hive> SHOW CONF "hive.aux.jars.path"; EXIT;
16/04/27 20:49:45 [main()]: INFO log.PerfLogger: <PERFLOG method=Driver.run from=org.apache.hadoop.hive.ql.Driver>
16/04/27 20:49:45 [main()]: INFO log.PerfLogger: <PERFLOG method=TimeToSubmit from=org.apache.hadoop.hive.ql.Driver>
16/04/27 20:49:45 [main()]: INFO ql.Driver: Concurrency mode is disabled, not creating a lock manager
16/04/27 20:49:45 [main()]: INFO log.PerfLogger: <PERFLOG method=compile from=org.apache.hadoop.hive.ql.Driver>
16/04/27 20:49:45 [main()]: DEBUG parse.VariableSubstitution: Substitution is on: SHOW CONF "hive.aux.jars.path"
16/04/27 20:49:45 [main()]: INFO log.PerfLogger: <PERFLOG method=parse from=org.apache.hadoop.hive.ql.Driver>
16/04/27 20:49:45 [main()]: INFO parse.ParseDriver: Parsing command: SHOW CONF "hive.aux.jars.path"
16/04/27 20:49:45 [main()]: INFO parse.ParseDriver: Parse Completed
16/04/27 20:49:45 [main()]: INFO log.PerfLogger: </PERFLOG method=parse start=1461790185397 end=1461790185846 duration=449 from=org.apache.hadoop.hive.ql.Driver>
16/04/27 20:49:45 [main()]: DEBUG ql.Driver: Encoding valid txns info 9223372036854775807:
16/04/27 20:49:45 [main()]: INFO log.PerfLogger: <PERFLOG method=semanticAnalyze from=org.apache.hadoop.hive.ql.Driver>
16/04/27 20:49:45 [main()]: INFO ql.Driver: Semantic Analysis Completed
16/04/27 20:49:45 [main()]: INFO log.PerfLogger: </PERFLOG method=semanticAnalyze start=1461790185852 end=1461790185931 duration=79 from=org.apache.hadoop.hive.ql.Driver>
16/04/27 20:49:45 [main()]: DEBUG exec.Utilities: Use session specified class loader
16/04/27 20:49:46 [main()]: DEBUG lazy.LazySimpleSerDe: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe initialized with: columnNames=[default, type, desc] columnTypes=[string, string, string] separator=[[B@1e63d216] nullstring=  lastColumnTakesRest=false
16/04/27 20:49:46 [main()]: INFO exec.ListSinkOperator: Initializing Self OP[0]
16/04/27 20:49:46 [main()]: DEBUG exec.Utilities: Use session specified class loader
16/04/27 20:49:46 [main()]: DEBUG lazy.LazySimpleSerDe: org.apache.hadoop.hive.serde2.DelimitedJSONSerDe initialized with: columnNames=[] columnTypes=[] separator=[[B@1280851e] nullstring=  lastColumnTakesRest=false
16/04/27 20:49:46 [main()]: INFO exec.ListSinkOperator: Operator 0 OP initialized
16/04/27 20:49:46 [main()]: INFO exec.ListSinkOperator: Initialization Done 0 OP
16/04/27 20:49:46 [main()]: DEBUG exec.Utilities: Use session specified class loader
16/04/27 20:49:46 [main()]: DEBUG lazy.LazySimpleSerDe: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe initialized with: columnNames=[default, type, desc] columnTypes=[string, string, string] separator=[[B@5e840abf] nullstring=  lastColumnTakesRest=false
16/04/27 20:49:46 [main()]: INFO ql.Driver: Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:default, type:string, comment:from deserializer), FieldSchema(name:type, type:string, comment:from deserializer), FieldSchema(name:desc, type:string, comment:from deserializer)], properties:null)
16/04/27 20:49:46 [main()]: INFO log.PerfLogger: </PERFLOG method=compile start=1461790185344 end=1461790186055 duration=711 from=org.apache.hadoop.hive.ql.Driver>
16/04/27 20:49:46 [main()]: INFO log.PerfLogger: <PERFLOG method=Driver.execute from=org.apache.hadoop.hive.ql.Driver>
16/04/27 20:49:46 [main()]: INFO ql.Driver: Starting command: SHOW CONF "hive.aux.jars.path"
16/04/27 20:49:46 [main()]: INFO log.PerfLogger: </PERFLOG method=TimeToSubmit start=1461790185344 end=1461790186062 duration=718 from=org.apache.hadoop.hive.ql.Driver>
16/04/27 20:49:46 [main()]: INFO log.PerfLogger: <PERFLOG method=runTasks from=org.apache.hadoop.hive.ql.Driver>
16/04/27 20:49:46 [main()]: INFO log.PerfLogger: <PERFLOG method=task.DDL.Stage-0 from=org.apache.hadoop.hive.ql.Driver>
16/04/27 20:49:46 [main()]: INFO ql.Driver: Starting task [Stage-0:DDL] in serial mode
16/04/27 20:49:46 [main()]: INFO log.PerfLogger: </PERFLOG method=runTasks start=1461790186062 end=1461790186078 duration=16 from=org.apache.hadoop.hive.ql.Driver>
16/04/27 20:49:46 [main()]: INFO log.PerfLogger: </PERFLOG method=Driver.execute start=1461790186055 end=1461790186078 duration=23 from=org.apache.hadoop.hive.ql.Driver>
OK
16/04/27 20:49:46 [main()]: INFO ql.Driver: OK
16/04/27 20:49:46 [main()]: INFO log.PerfLogger: <PERFLOG method=releaseLocks from=org.apache.hadoop.hive.ql.Driver>
16/04/27 20:49:46 [main()]: INFO log.PerfLogger: </PERFLOG method=releaseLocks start=1461790186079 end=1461790186079 duration=0 from=org.apache.hadoop.hive.ql.Driver>
16/04/27 20:49:46 [main()]: INFO log.PerfLogger: </PERFLOG method=Driver.run start=1461790185344 end=1461790186079 duration=735 from=org.apache.hadoop.hive.ql.Driver>
16/04/27 20:49:46 [main()]: INFO Configuration.deprecation: mapred.input.dir is deprecated. Instead, use mapreduce.input.fileinputformat.inputdir
16/04/27 20:49:46 [main()]: INFO lzo.GPLNativeCodeLoader: Loaded native gpl library
16/04/27 20:49:46 [main()]: INFO lzo.LzoCodec: Successfully loaded & initialized native-lzo library [hadoop-lzo rev 72c57a1c06c471da40827b432ecff0de6a5c6dcc]
16/04/27 20:49:46 [main()]: DEBUG mapred.FileInputFormat: Time taken to get FileStatuses: 3
16/04/27 20:49:46 [main()]: INFO mapred.FileInputFormat: Total input paths to process : 1
16/04/27 20:49:46 [main()]: DEBUG mapred.FileInputFormat: Total # of splits generated by getSplits: 1, TimeTaken: 12
16/04/27 20:49:46 [main()]: DEBUG lazy.LazySimpleSerDe: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe initialized with: columnNames=[default, type, desc] columnTypes=[string, string, string] separator=[[B@2dbd803f] nullstring=  lastColumnTakesRest=false
16/04/27 20:49:46 [main()]: DEBUG lazy.LazySimpleSerDe: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe initialized with: columnNames=[default, type, desc] columnTypes=[string, string, string] separator=[[B@3e48e859] nullstring=  lastColumnTakesRest=false
16/04/27 20:49:46 [main()]: DEBUG exec.FetchOperator: Creating fetchTask with deserializer typeinfo: struct<default:string,type:string,desc:string>
16/04/27 20:49:46 [main()]: DEBUG exec.FetchOperator: deserializer properties:
table properties: {columns=default,type,desc, serialization.null.format= , serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, serialization.format=9, columns.types=string,string,string}
partition properties: {columns=default,type,desc, serialization.null.format= , serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, serialization.format=9, columns.types=string,string,string}
    STRING  The location of the plugin jars that contain implementations of user defined functions and serdes.
16/04/27 20:49:46 [main()]: INFO exec.ListSinkOperator: 0 finished. closing... 
16/04/27 20:49:46 [main()]: INFO exec.ListSinkOperator: 0 Close done
16/04/27 20:49:46 [main()]: DEBUG ql.Driver: Shutting down query SHOW CONF "hive.aux.jars.path"
Time taken: 0.741 seconds, Fetched: 1 row(s)
16/04/27 20:49:46 [main()]: INFO CliDriver: Time taken: 0.741 seconds, Fetched: 1 row(s)
16/04/27 20:49:46 [main()]: DEBUG session.SessionState: Removing resource dir /mnt/tmp/f113c3f9-3316-4998-8e80-9bd4c3fc079c_resources
16/04/27 20:49:46 [IPC Parameter Sending Thread #0()]: DEBUG ipc.Client: IPC Client (1309129055) connection to ip-10-102-201-205.ec2.internal/10.102.201.205:8020 from hadoop sending #9
16/04/27 20:49:46 [IPC Client (1309129055) connection to ip-10-102-201-205.ec2.internal/10.102.201.205:8020 from hadoop()]: DEBUG ipc.Client: IPC Client (1309129055) connection to ip-10-102-201-205.ec2.internal/10.102.201.205:8020 from hadoop got value #9
16/04/27 20:49:46 [main()]: DEBUG ipc.ProtobufRpcEngine: Call: delete took 3ms
16/04/27 20:49:46 [IPC Parameter Sending Thread #0()]: DEBUG ipc.Client: IPC Client (1309129055) connection to ip-10-102-201-205.ec2.internal/10.102.201.205:8020 from hadoop sending #10
16/04/27 20:49:46 [IPC Client (1309129055) connection to ip-10-102-201-205.ec2.internal/10.102.201.205:8020 from hadoop()]: DEBUG ipc.Client: IPC Client (1309129055) connection to ip-10-102-201-205.ec2.internal/10.102.201.205:8020 from hadoop got value #10
16/04/27 20:49:46 [Thread-0()]: DEBUG ipc.ProtobufRpcEngine: Call: getFileInfo took 2ms
16/04/27 20:49:46 [IPC Parameter Sending Thread #0()]: DEBUG ipc.Client: IPC Client (1309129055) connection to ip-10-102-201-205.ec2.internal/10.102.201.205:8020 from hadoop sending #11
16/04/27 20:49:46 [IPC Client (1309129055) connection to ip-10-102-201-205.ec2.internal/10.102.201.205:8020 from hadoop()]: DEBUG ipc.Client: IPC Client (1309129055) connection to ip-10-102-201-205.ec2.internal/10.102.201.205:8020 from hadoop got value #11
16/04/27 20:49:46 [Thread-0()]: DEBUG ipc.ProtobufRpcEngine: Call: getFileInfo took 1ms
16/04/27 20:49:46 [Thread-0()]: DEBUG ipc.Client: stopping client from cache: org.apache.hadoop.ipc.Client@7fcf2fc1
16/04/27 20:49:46 [Thread-0()]: DEBUG ipc.Client: removing client from cache: org.apache.hadoop.ipc.Client@7fcf2fc1
16/04/27 20:49:46 [Thread-0()]: DEBUG ipc.Client: stopping actual client because no more references remain: org.apache.hadoop.ipc.Client@7fcf2fc1
16/04/27 20:49:46 [Thread-0()]: DEBUG ipc.Client: Stopping client
16/04/27 20:49:46 [IPC Client (1309129055) connection to ip-10-102-201-205.ec2.internal/10.102.201.205:8020 from hadoop()]: DEBUG ipc.Client: IPC Client (1309129055) connection to ip-10-102-201-205.ec2.internal/10.102.201.205:8020 from hadoop: closed
16/04/27 20:49:46 [IPC Client (1309129055) connection to ip-10-102-201-205.ec2.internal/10.102.201.205:8020 from hadoop()]: DEBUG ipc.Client: IPC Client (1309129055) connection to ip-10-102-201-205.ec2.internal/10.102.201.205:8020 from hadoop: stopped, remaining connections 0

1 個解決方案

解決方案1
 1 已采納  2016-04-28 21:08:04

從Hive 管理員手冊中 ：

您可以使用SHOW CONF命令顯示有關配置變量的信息 。

從Hive 語言手冊中 ：

set <key>=<value>設置特定配置變量（密鑰）的值 。

手冊中沒有明確指出的是，您還可以使用set <key> 顯示 當前值 。

底線：運行set hive.aux.jars.path ; 在Hive CLI中顯示當前值，該值是（a）在先前的set命令中set ，或者（b）作為命令行參數，或者（c）在hive-site.xml ，或者（d）失敗所有這些，以硬編碼默認值表示。

僅當您需要一些有關硬編碼默認值的技術文檔並且無法訪問互聯網時，SHOW CONF才有用。

暫無
暫無

聲明:本站的技術帖子網頁，遵循CC BY-SA 4.0協議，如果您需要轉載，請注明本站網址或者原文地址。任何問題請咨詢:yoyou2525@163.com.

相關問題 hive-site.xml被hive忽略 在hive-site.xml中查找_HOST的值 在類路徑上找不到hive-site.xml 火花不讀hive-site.xml嗎？ WARN conf.HiveConf：在CLASSPATH-TOS上找不到大數據的hive-site.xml 從Spark連接到Hive，無需使用“ hive-site.xml” 如何在hive-site.xml中映射Hive倉庫路徑？ 沒有hive-site.xml文件的蜂巢如何運行？ Hive-site.xml 中缺少 Hive 元存儲數據庫詳細信息 當我添加hive-site.xml時，Hive Shell無法打開
相關標簽
 
粵ICP備18138465號  © 2020-2024 STACKOOM.COM