Linking to javadoc.io using Javadoc -link option

Question

I am trying to link into some Javadocs hosted at javadoc.io (specifically, PowerMock's Javadocs) using the @link option. I have tried to add the URL to PowerMock's Javadocs to my -link flag, but can't get Javadoc to recognize it. I am using external links to other Javadocs just fine (eg Guava, Java SE 7) with Gradle as my build system. I have tried the following options:

-link http://static.javadoc.io/org.powermock/powermock-core/1.6.3/

^ I have confirmed that there is a package-list file in this directory

-link http://static.javadoc.io/org.powermock/powermock-core/

-link http://javadoc.io/doc/org.powermock/powermock-core/1.6.3/

-link http://javadoc.io/doc/org.powermock/powermock-core/

All of these result in the following error (URL changed accordingly):

javadoc: warning - Error fetching URL: http://static.javadoc.io/org.powermock/powermock-core/1.6.3/

Does anyone have advice on how to make this work?

As far as I can tell this is some sort of javadoc.io specific problem, though likely a usage issue on my end - for example I am currently using -link http://junit.org/javadoc/latest/ without issue, but -link http://static.javadoc.io/junit/junit/4.12/ doesn't work.

Answer 1

From the command line, use an argument like -J-Dhttp.agent=javadoc .

In Maven, use something like:

<additionalJOption>-J-Dhttp.agent=maven-javadoc-plugin-${pom‌​.name}</additionalJO‌​ption>

The background: As Danilo Pianini suggests in another answer , the problem is the User-Agent header. However, the problem isn't an empty User-Agent ; it's the default Java User-Agent , which looks something like " Java/1.8.0_112 ":

$ URL=https://static.javadoc.io/org.checkerframework/checker-qual/2.2.2/package-list

# default Java User-Agent:
$ wget -U Java/1.8.0_112 "$URL" 2>&1 | grep response
HTTP request sent, awaiting response... 403 Forbidden

# no User-Agent:
$ wget -U '' "$URL" 2>&1 | grep response
HTTP request sent, awaiting response... 200 OK

# custom User-Agent:
$ wget -U javadoc "$URL" 2>&1 | grep response
HTTP request sent, awaiting response... 200 OK

So the fix is to tell Javadoc to use a different User-Agent . Java won't let you omit the User-Agent , so you'll have to provide a value, which Java will prepend to its default agent.

As best I can tell, the blocking of Javadoc isn't intentional: Javadoc just (probably unwisely) uses the default Java User-Agent , and the content delivery network that javadoc.io uses blocks that by default.

(One more note about Maven: Everything works fine with -link . It also works fine with -linkoffline if you download the package-list file and tell Javadoc to read it from disk. However, if you use -linkoffline but tell Javadoc to fetch package-list from the javadoc.io URL (this is an unusual thing to do), it may fail . The problem: Maven tries to pre-validate the package-list file but, under some versions of Java, fails because it rejects the SSL certificate of javadoc.io , a certificate that Javadoc itself accepts.)

(Oh, and it appears to be important to use a URL specifically from static.javadoc.io , not javadoc.io . Also, I would recommend https , not http , in case http://static.javadoc.io someday starts issuing redirects to https://static.javadoc.io , as Javadoc currently doesn't handle such redirects . Also, https is a good thing :))

Answer 2

It's strange: I could see in the browser eg http://static.javadoc.io/org.pegdown/pegdown/1.6.0/package-list but when I add http://static.javadoc.io/org.pegdown/pegdown/1.6.0 as javadoc's link option it says

Error fetching URL: http://static.javadoc.io/org.pegdown/pegdown/1.6.0/package-list

I use next workaround:

1. With maven-dependency-plugin unapack the javadoc of desired dependency.

2. Link it with linkoffline option.

    <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-dependency-plugin</artifactId> <version>2.8</version> <executions> <execution> <id>unpack-javadoc</id> <phase>package</phase> <goals> <goal>unpack</goal> </goals> <configuration> <artifactItems> <artifactItem> <groupId>org.pegdown</groupId> <artifactId>pegdown</artifactId> <classifier>javadoc</classifier> <version>${pegdownVersion}</version> <overWrite>false</overWrite> <outputDirectory>${project.build.directory}/pegdown-javadoc</outputDirectory> </artifactItem> </artifactItems> </configuration> </execution> </executions> </plugin> <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-javadoc-plugin</artifactId> <configuration> <links> <link>http://www.slf4j.org/apidocs/</link> </links> <offlineLinks> <offlineLink> <url>http://static.javadoc.io/org.pegdown/pegdown/${pegdownVersion}</url> <location>${project.build.directory}/pegdown-javadoc</location> </offlineLink> </offlineLinks> </configuration> </plugin> 

Answer 3

I have investigated the problem, the issue here is that a user agent must be set (an empty string is ok) in order for the connection to javadoc.io to complete successfully.

I worked the problem around and wrote a Gradle plugin that may be of help for those who rely on that build system.

Unfortunately, the work around can not get ported to the regular javadoc -link command invocation.

Answer 4

我最终只是使用-linkoffline来解决这个问题，我认为它具有在构建时不需要互联网连接的良好属性，但是如果有人进一步想到如何使用-link进行这项工作，我会全力以赴。

Answer 5

I am running javadoc.io.

This is reported as this github issue and it's solved just now. There is no need to override user agent string any more.

Feel free to re-open github issue if things are still not working. This thread is not actively monitored.

curl -I -A "Java/1.6.0_14" https://static.javadoc.io/org.checkerframework/checker-qual/2.2.2/package-list
HTTP/1.1 200 OK
Date: Mon, 08 Apr 2019 13:06:04 GMT
Content-Type: text/plain