flume configuration java exception

Question

I'm having trouble running a flume agent (basic example from the apache flume site). The command I'm running is

---

./bin/flume-ng agent --conf conf --conf-file test.conf --name a1 -Dflume.root.logger=INFO,console

---

my test.conf file looks like

# example.conf: A single-node Flume configuration

# Name the components on this agent
a1.sources = r1
a1.sinks = k1
a1.channels = c1

# Describe/configure the source
a1.sources.r1.type = netcat
a1.sources.r1.bind = localhost
a1.sources.r1.port = 44444

# Describe the sink
a1.sinks.k1.type = logger

# Use a channel which buffers events in memory
a1.channels.c1.type = memory
a1.channels.c1.capacity = 1000
a1.channels.c1.transactionCapacity = 100

# Bind the source and sink to the channel
a1.sources.r1.channels = c1
a1.sinks.k1.channel = c1

---

my env vars are here:

TERM_PROGRAM=Apple_Terminal
SHELL=/bin/bash
TERM=xterm-256color
TMPDIR=/var/folders/qf/lnn77ppn1mv51vg67hlljpqw0000gn/T/
Apple_PubSub_Socket_Render=/tmp/launch-h7JcOn/Render
TERM_PROGRAM_VERSION=309
OLDPWD=/Users/wolf/Downloads/apache-flume-1.5.0-src/conf
TERM_SESSION_ID=58D255EF-6DF2-4887-8598-844F2BBF0370
USER=wolf
COMMAND_MODE=unix2003
SSH_AUTH_SOCK=/tmp/launch-OdJEdz/Listeners
Apple_Ubiquity_Message=/tmp/launch-x6Ws50/Apple_Ubiquity_Message
__CF_USER_TEXT_ENCODING=0x1F5:0:0
MAVEN_OPTS=-Xms512m -Xmx1024m -XX:PermSize=256m -XX:MaxPermSize=512m
PATH=/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/usr/local/git/bin
PWD=/Users/wolf/Downloads/apache-flume-1.5.0-src
JAVA_HOME=/System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Home
LANG=en_US.UTF-8

---

The errors I'm getting are

Info: Sourcing environment configuration script /Users/wolf/Downloads/apache-flume-1.5.0-   src/conf/flume-env.sh
Info: Including Hadoop libraries found via (/usr/local/bin/hadoop) for HDFS access
Exception in thread "main" java.lang.NoClassDefFoundError:  org/apache/flume/tools/GetJavaProperty
Caused by: java.lang.ClassNotFoundException: org.apache.flume.tools.GetJavaProperty
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:247)

---

AND

-Djava.library.path= org.apache.flume.node.Application --conf-file test.conf --name a1
Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/flume/node/Application
 Caused by: java.lang.ClassNotFoundException: org.apache.flume.node.Application
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:247)

---

Anybody know what I may be missing?

Answer 1

Well that's spectacular.

I'll state the obvious: Java isn't finding the Flume libraries.

I'd be hunting into the flume-ng script to see why it's not getting the classpath right.

Answer 2

Got this patch/info from a Cloudera Engineer. Going to see if it resolves the issue. https://issues.apache.org/jira/secure/attachment/12628911/HBASE-10518.00.patch