How can I get Jetty to scan the Maven dependency jar files in the class path?

Question

Eclipse automatically adds all Maven dependencies to the class path.

How can I get Jetty to scan the jar files on the class path for web fragments, when started from Eclipse?

When packaged and placed in webapp/WEB-INF/lib/*.jar, the scanning works.

Server server = new Server(8080);

WebAppContext webapp = new WebAppContext();
webapp.setContextPath("/myapp");
webapp.setBaseResource(Resource.newResource(new File("webapp")));

// https://stackoverflow.com/questions/11768113/cant-get-jetty-to-scan-for-annotated-classes
webapp.setConfigurations(new Configuration[] { //
    new AnnotationConfiguration(), // @WebServlet, @WebListener...
    new WebXmlConfiguration(), // webapp/WEB-INF/web.xml
    new WebInfConfiguration(), // ?
    new MetaInfConfiguration(), // ?
    new FragmentConfiguration(), // e.g. zkwebfragment-9.6.0.1.jar!/META-INF/web-fragment.xml
});

if(RUNNING_FROM_IDE)
{
  // add project classes to classpath (e.g. target/classes)
  webapp.getMetaData().setWebInfClassesDirs(Arrays.asList(Resource.newResource(MyAppMainClass.class.getProtectionDomain().getCodeSource().getLocation())));

  // how to add maven dependencies?
  // webapp.getMetaData().addWebInfJar(???);
}

server.setHandler(webapp);
server.start();

Answer 1

Don't manually set the configurations, that's the first mistake.

Your effort basically broke the configurations, as you didn't respect the original default configurations, or the delicate order requirements.

This is wrong way, by setting the webapp configurations directly.

webapp.setConfigurations(new Configuration[] { //
    new AnnotationConfiguration(), // @WebServlet, @WebListener...
    new WebXmlConfiguration(), // webapp/WEB-INF/web.xml
    new WebInfConfiguration(), // ?
    new MetaInfConfiguration(), // ?
    new FragmentConfiguration(), // e.g. zkwebfragment-9.6.0.1.jar!/META-INF/web-fragment.xml
});

This is the correct way, by adjusting the default Configuration list at the Server level.

Configuration.ClassList classlist = Configuration.ClassList
  .setServerDefault(server);
classlist.addBefore(
  "org.eclipse.jetty.webapp.JettyWebXmlConfiguration",
  "org.eclipse.jetty.annotations.AnnotationConfiguration");

This should be enough.

You can see the default configuration order on the server dump.

server.setDumpAfterStart(true);
server.start();

// and then on the output (see your logger) ...
|  |  |  +> Configurations Servlet Dump WebApp@4c163e3 size=5
|  |  |  |  +> org.eclipse.jetty.webapp.WebInfConfiguration@5e403b4a
|  |  |  |  +> org.eclipse.jetty.webapp.WebXmlConfiguration@5117dd67
|  |  |  |  +> org.eclipse.jetty.webapp.MetaInfConfiguration@5be49b60
|  |  |  |  +> org.eclipse.jetty.webapp.FragmentConfiguration@2931522b
|  |  |  |  +> org.eclipse.jetty.webapp.JettyWebXmlConfiguration@7674b62c
|  |  |  +> Handler attributes Servlet Dump WebApp@4c163e3 size=3

This dump feature is useful, as it will likely tell you that the the $HOME/.m2/repository jar files are not present in the webapp's own classpath.

|  |  |  +> WebAppClassLoader{Servlet Dump WebApp}@5fb759d6
|  |  |  |  +> URLs size=1
|  |  |  |  |  +> file:/tmp/jetty-0_0_0_0-8080-ROOT_war-_-any-15598896298108484560/webapp/WEB-INF/classes/

If you don't see your maven repository files here, then it's time for you to configure your WebAppContext.setExtraClasspath(String) to include them before you start your webapp.

Another feature to be aware of are the 2 scan regexes used to identify which content (class files) are scanned. One for the webapp classes, and one for the server classes. Both of these are relevant for the discovery of anything in the servlet spec, be it annotated listeners, annotated servlets, web fragments, servlet container initializers, jsp level details, websocket level details, etc)

Both are set as attributes on the WebAppContext.setAttribute(String key, String value) if you want to configure them.

By default, these values are empty, meaning all jars are scanned.

// This one controls server/container level discovery.

// An example of limited scanning would be
webappContext.setAttribute(
  "org.eclipse.jetty.server.webapp.ContainerIncludeJarPattern"
  ".*/[^/]*servlet-api-[^/]*\\.jar$|.*[^/]*jstl-.*\\.jar$|.*/[^/]*taglibs.*\\.jar$");


// This one controls webapp specific discovery.
webappContext.setAttribute(
  "org.eclipse.jetty.server.webapp.WebInfIncludeJarPattern"
  ".*/[^/]*foo-[^/]*\\.jar$|.*/[^/]*bar.*\\.jar$");

An example of this would be...

You have a webapp with...

WEB-INF/lib/
           foo-api-1.2.3.jar
           foo-1.2.3.jar
           bar-0.9.jar
           boo-1.0.jar

And defined the webapp scanning like...

webappContext.setAttribute(
    "org.eclipse.jetty.server.webapp.WebInfIncludeJarPattern",
    ".*/.*foo-api-[^/]\.jar$|./.*bar-[^/]\.jar$|./.*wibble[^/]*\.jar$");

Then the following files would match and be scanned:

WEB-INF/lib/
           foo-api-1.2.3.jar
           bar-0.9.jar

Answer 2

I got it to work somehow. Still wondering if there is a more standard way to achieve this:

// launching from IDE: manually add WEB-INF/classes and WEB-INF/lib/*.jar entries
webapp.getMetaData().setWebInfClassesDirs(Arrays.asList(Resource.newResource(MyAppMainClass.class.getProtectionDomain().getCodeSource().getLocation())));
for(URL url : ((URLClassLoader) MyAppMainClass.class.getClassLoader()).getURLs()) {
  if(url.getPath().endsWith(".jar")) {
    webapp.getMetaData().addWebInfJar(Resource.newResource(url));
  }
}

I also found MavenWebInfConfiguration in the jetty-maven-plugin . I uses:

((WebAppClassLoader)  webapp.getClassLoader()).addClassPath(...)

Answer 3

Based on Joakim's answer, I now have:

Server server = new Server(port);
Configuration.ClassList classlist = Configuration.ClassList.setServerDefault(server);
classlist.addAfter(JettyWebXmlConfiguration.class.getName(), AnnotationConfiguration.class.getName());

WebAppContext webapp = new WebAppContext();
webapp.setContextPath("/myapp");
webapp.setBaseResource(Resource.newResource(new File(isRunningFromIde ? "./webapp" : "/path/to/webapp").getCanonicalFile()));    
webapp.setWelcomeFiles(new String[] { "index.html" });
//    webapp.setInitParameter(..., ...));
//    webapp.setAttribute("org.eclipse.jetty.server.webapp.ContainerIncludeJarPattern", "");
//    webapp.setAttribute("org.eclipse.jetty.server.webapp.WebInfIncludeJarPattern", "");

if(isRunningFromIde)
{
  List<String> extraClassPath = new ArrayList<>();
  // target/classes
  extraClassPath.add(Paths.get(MyAppMain.class.getProtectionDomain().getCodeSource().getLocation().toURI()).toString());
  // maven dependencies
  for(URL url : ((URLClassLoader) MyAppMain.class.getClassLoader()).getURLs()) {
    if(url.getPath().endsWith(".jar")) {
      extraClassPath.add(Paths.get(url.toURI()).toString());
    }
  }
  webapp.setExtraClasspath(extraClassPath.stream().collect(Collectors.joining(";")));
}

server.setHandler(webapp);
server.setDumpAfterStart(true);
server.start();

Thanks to the server dump tip, I think I now also understand why I am getting replicate resource warnings from ZK . The WebAppClassLoader and the AppClassLoader both contain the same entries. The IDE launch because of the above extra classpath. The docker container probably because I start it with java -cp <webapp>/WEB-INF/lib/* MyAppMain.class .

|  +> WebAppClassLoader{403716510}@1810399e
|  |  +> URLs size=41
|  |  |  +> file:<maven-repo-or-webapp-dir>/WEB-INF/lib/zk-9.6.0.1.jar
|  |  |  ...
|  |  +> sun.misc.Launcher$AppClassLoader@4e0e2f2a
|  |  |  +> URLs size=41
|  |  |  |  +> file:<maven-repo-or-webapp-dir>/WEB-INF/lib/zk-9.6.0.1.jar
|  |  |  |  ...

Answer 4

Attempt 3 (also based on Joakim's answer): Setting the ContainerIncludeJarPattern to scan everything that is already on the classpath. No need for any additional class path modifications.

    Server server = new Server(8080);
    WebAppContext webapp = new WebAppContext();
    webapp.setContextPath("/test");
    webapp.setBaseResource(Resource.newResource(new File("webapp").getCanonicalFile()));
    // https://www.eclipse.org/jetty/documentation/jetty-9/index.html#configuring-webapps
    // order apparently important
    webapp.setConfigurations(new Configuration[] { //
        new WebInfConfiguration(), //
        new WebXmlConfiguration(), //
        new MetaInfConfiguration(), //
        new FragmentConfiguration(), //
        new AnnotationConfiguration(), //
    });
    webapp.setAttribute("org.eclipse.jetty.server.webapp.ContainerIncludeJarPattern", ".*");
    server.setHandler(webapp);
    server.setDumpAfterStart(true);
    server.start();

My other problems seem to be ZK-related, so I started a new question for that.