Spring Cloud Eureka server restarts as soon as eureka client service is started. (After upgrading JDK and Spring Boot version.)

Question

I am developing a set of microservices that discover each other using a eureka server.

Yesterday I upgraded our projects from JDK 1.8 to JDK 14 and adapted all version numbers of the dependencies to the most recent ones. I am now using spring-boot-starter-parent as parent POM in version 2.2.6.RELEASE.

All projects are building fine, also all unit tests are running through. However today I discovered that the service discovery using eureka doesn't work anymore.

For testing, I usually start my eureka server at first and wait for it to come up completely. Afterwards I start the clients so they can register to it. The problem is: As soon as I start the first client service, the eureka server shuts down and attempts to restart, throwing lots of exceptions that weren't there the first it started.

In the end, when it seems to have settled, I can't call my services, probably because they are not discoverable.

---

Here some details about my setup:

I use a custom parent POM, derived from spring-boot-starter-parent.

Parent POM:

<project xmlns="http://maven.apache.org/POM/4.0.0"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <parent>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-parent</artifactId>
        <version>2.2.6.RELEASE</version>
        <relativePath />
    </parent>

    <groupId>com.whatever</groupId>
    <artifactId>parent</artifactId>
    <version>0.0.1-SNAPSHOT</version>
    <packaging>pom</packaging>

    <properties>
        <java.version>14</java.version>
        <spring-cloud.version>Hoxton.RELEASE</spring-cloud.version>
<!-- Also tried these...
        <spring-cloud.version>Hoxton.SR1</spring-cloud.version>
        <spring-cloud.version>Hoxton.SR2</spring-cloud.version>
        <spring-cloud.version>Hoxton.SR3</spring-cloud.version>
-->
    </properties>

    <dependencyManagement>
        <dependencies>
            <dependency>
                <groupId>org.springframework.cloud</groupId>
                <artifactId>spring-cloud-starter-parent</artifactId>
                <version>${spring-cloud.version}</version>
                <type>pom</type>
                <scope>import</scope>
            </dependency>
        </dependencies>
    </dependencyManagement>

 <!-- ... more stuff ... -->

</project>

The POM of my eureka server service is as follows:

Eureka-Server POM:

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <parent>
        <groupId>com.whatever</groupId>
        <artifactId>parent</artifactId>
        <version>0.0.1-SNAPSHOT</version>
        <relativePath/>
    </parent>

    <groupId>com.whatever</groupId>
    <artifactId>service-directory</artifactId>
    <version>0.0.1-SNAPSHOT</version>

    <properties>
        <java.version>14</java.version>
    </properties>

    <dependencies>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-netflix-eureka-server</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-configuration-processor</artifactId>
            <optional>true</optional>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-devtools</artifactId>
            <scope>runtime</scope>
        </dependency>
        <dependency>
            <groupId>org.projectlombok</groupId>
            <artifactId>lombok</artifactId>
            <optional>true</optional>
        </dependency>
        <dependency>
            <groupId>io.springfox</groupId>
            <artifactId>springfox-swagger2</artifactId>
        </dependency>
        <dependency>
            <groupId>io.springfox</groupId>
            <artifactId>springfox-swagger-ui</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <scope>test</scope>
        </dependency>
    </dependencies>

    <build>
        <plugins>
            <plugin>
                <groupId>org.jacoco</groupId>
                <artifactId>jacoco-maven-plugin</artifactId>
                <!-- version managed in parent POM -->
            </plugin>
        </plugins>
    </build>
</project>

One thing worthy to note: My eureka server service is a eureka client itself.

This is my application.yml from the eureka server service:

application.yml of Eureka Server:

spring:
  profiles:
    active: test
  application:
    name: ServiceDirectory

server:
  port: 8099

logging.level:
    org.springframework:
      web: DEBUG
      web.servlet.view.freemarker.FreeMarkerConfigurer: INFO

eureka:
  client:
    service-url:
      defaultZone: http://localhost:${server.port}/eureka/

management:
  endpoint:
    health:
      show-details: always
  endpoints:
    web:
      exposure:
        include:
        - '*'

---

spring:
  profiles: test

spring.output.ansi.enabled: ALWAYS

As an example for a client application.yml, this is the application.yml of my API-Gateway service, that registers itself at the eureka server:

application.yml of Eureka Client (API-Gateway):

spring:
  profiles:
    active: test
  application:
    name: gateway

server:
  port: 8088

eureka:
  client:
    service-url:
      defaultZone: http://localhost:8099/eureka/

---

spring:
  profiles: test

zuul:
  routes:
    service-a:
      path: /service-a/**
      url: http://localhost:8083
      sensitiveHeaders: 
      strip-prefix: true
    service-b:
      path: /service-b/**
      url: http://localhost:9920
      sensitiveHeaders: 
      strip-prefix: true
    service-directory:
      path: /service-directory/**
      url: http://localhost:8099
      sensitiveHeaders: 
      strip-prefix: true
    eureka-home:
      path: /eureka/**
      url: http://localhost:8099
      sensitiveHeaders: 
      strip-prefix: false

spring.output.ansi.enabled: ALWAYS      

It doesn't seem to matter, which client service I start.

Let me know if you need more details about the setup.

Here is the output of my eureka server:

Eureka server output 1:

 . ____ _ __ _ _
 /\\ / ___'_ __ _ _(_)_ __ __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
 \\/ ___)| |_)| | | | | || (_| | ) ) ) )
 ' |____| .__|_| |_|_| |_\__, | / / / /
 =========|_|==============|___/=/_/_/_/
 :: Spring Boot :: (v2.2.6.RELEASE)

13:22:18.244 c.w.s.ServiceDirectoryApplication : The following profiles are active: test
13:22:19.625 o.s.boot.actuate.endpoint.EndpointId : Endpoint ID 'service-registry' contains invalid characters, please migrate to a valid format.
...
13:22:20.769 c.netflix.config.DynamicPropertyFactory : DynamicPropertyFactory is initialized with configuration sources: com.netflix.config.ConcurrentCompositeConfiguration@235cb2ea
 WARNING: An illegal reflective access operation has occurred
 WARNING: Illegal reflective access by org.springframework.cglib.core.ReflectUtils (file:/home/xxx/.m2/repository/org/springframework/spring-core/5.2.1.RELEASE/spring-core-5.2.1.RELEASE.jar) to method java.lang.ClassLoader.defineClass(java.lang.String,byte[],int,int,java.security.ProtectionDomain)
 WARNING: Please consider reporting this to the maintainers of org.springframework.cglib.core.ReflectUtils
 WARNING: Use --illegal-access= to enable ings of further illegal reflective access operations
 WARNING: All illegal access operations will be denied in a future release
13:22:21.412 c.s.j.s.i.a.WebApplicationImpl : Initiating Jersey application, version 'Jersey: 1.19.1 03/11/2016 02:08 PM'
...
13:22:23.207 ockingLoadBalancerClientRibbon Logger : You already have RibbonLoadBalancerClient on your classpath. It will be used by default. As Spring Cloud Ribbon is in maintenance mode. We recommend switching to BlockingLoadBalancerClient instead. In order to use it, set the value of `spring.cloud.loadbalancer.ribbon.enabled` to `false` or remove spring-cloud-starter-netflix-ribbon from your project.
13:22:23.237 iguration$LoadBalancerCaffeine Logger : Spring Cloud LoadBalancer is currently working with default default cache. You can switch to using Caffeine cache, by adding it to the classpath.
13:22:23.254 o.s.c.n.eureka.Instance Factory : Setting initial instance status as: STARTING
13:22:23.321 com.netflix.discovery.DiscoveryClient : Initializing Eureka in region us-east-1
...
13:22:23.601 com.netflix.discovery.DiscoveryClient : Getting all instance registry from the eureka server
13:22:23.655 ERROR c.n.d.s.t.d.RedirectingEurekaHttpClient : Request execution error. endpoint=DefaultEndpoint{ serviceUrl='http://localhost:8099/eureka/}

com.sun.jersey.api.client.ClientHandlerException: java.net.ConnectException: Connection refused
    at com.sun.jersey.client.apache4.ApacheHttpClient4Handler.handle(ApacheHttpClient4Handler.java:187) ~[jersey-apache-client4-1.19.1.jar:1.19.1]
    ...
    at com.netflix.discovery.shared.transport.decorator.EurekaHttpClientDecorator$6.execute(EurekaHttpClientDecorator.java:137) ~[eureka-client-1.9.13.jar:1.9.13]
    ...
Caused by: java.net.ConnectException: Connection refused
    at java.base/sun.nio.ch.Net.pollConnect(Native Method) ~[na:na]
    ...
    at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180) ~[httpclient-4.5.12.jar:4.5.12]
    ...

13:22:23.656 c.n.d.s.t.d.RetryableEurekaHttpClient : Request execution failed with message: java.net.ConnectException: Connection refused
13:22:23.658 ERROR com.netflix.discovery.DiscoveryClient : DiscoveryClient_SERVICEDIRECTORY/xxxComputer.fritz.box:ServiceDirectory:8099 - was unable to refresh its cache! status = Cannot execute request on any known server

com.netflix.discovery.shared.transport.TransportException: Cannot execute request on any known server
    at com.netflix.discovery.shared.transport.decorator.RetryableEurekaHttpClient.execute(RetryableEurekaHttpClient.java:112) ~[eureka-client-1.9.13.jar:1.9.13]
    ...
    at org.springframework.cloud.netflix.eureka.EurekaClientAutoConfiguration$RefreshableEurekaClientConfiguration.eurekaClient(EurekaClientAutoConfiguration.java:324) ~[spring-cloud-netflix-eureka-client-2.2.0.RELEASE.jar:2.2.0.RELEASE]
    ...
    at org.springframework.boot.devtools.restart.RestartLauncher.run(RestartLauncher.java:49) ~[spring-boot-devtools-2.2.6.RELEASE.jar:2.2.6.RELEASE]

13:22:23.659 com.netflix.discovery.DiscoveryClient : Using default backup registry implementation which does not do anything.
...
13:22:23.691 c.n.eureka.cluster.PeerEurekaNodes : Adding new peer nodes [http://localhost:8099/eureka/]
...
13:22:23.805 c.n.eureka.cluster.PeerEurekaNodes : Replica node URL: http://localhost:8099/eureka/
13:22:23.810 c.n.e.registry.AbstractInstanceRegistry : Finished initializing remote region registries. All known remote regions: []
13:22:23.811 c.n.eureka.DefaultEurekaServerContext : Initialized
13:22:23.854 o.s.c.n.e.s.EurekaServiceRegistry : Registering application SERVICEDIRECTORY with eureka with status UP
13:22:23.855 com.netflix.discovery.DiscoveryClient : Saw local status change event StatusChangeEvent [timestamp=1586344943855, current=UP, previous=STARTING]
13:22:23.857 com.netflix.discovery.DiscoveryClient : DiscoveryClient_SERVICEDIRECTORY/xxxComputer.fritz.box:ServiceDirectory:8099: registering service...
13:22:23.860 o.s.c.n.e.server.EurekaServerBootstrap : Setting the eureka configuration..
13:22:23.860 d.s.w.p.DocumentationPluginsBootstrapper : Context refreshed
13:22:23.860 o.s.c.n.e.server.EurekaServerBootstrap : Eureka data center value eureka.datacenter is not set, defaulting to default
13:22:23.861 o.s.c.n.e.server.EurekaServerBootstrap : Eureka environment value eureka.environment is not set, defaulting to test
13:22:23.863 c.n.d.s.t.d.RedirectingEurekaHttpClient : Request execution error. endpoint=DefaultEndpoint{ serviceUrl='http://localhost:8099/eureka/}

com.sun.jersey.api.client.ClientHandlerException: java.net.ConnectException: Connection refused
    ...
    at com.netflix.discovery.EurekaIdentityHeaderFilter.handle(EurekaIdentityHeaderFilter.java:27) ~[eureka-client-1.9.13.jar:1.9.13]
    at com.sun.jersey.api.client.Client.handle(Client.java:652) ~[jersey-client-1.19.1.jar:1.19.1]
    ...
    at com.netflix.discovery.shared.transport.decorator.EurekaHttpClientDecorator.register(EurekaHttpClientDecorator.java:56) ~[eureka-client-1.9.13.jar:1.9.13]
    ...
Caused by: java.net.ConnectException: Connection refused
    ...
 ...

13:22:23.864 c.n.d.s.t.d.RetryableEurekaHttpClient : Request execution failed with message: java.net.ConnectException: Connection refused
13:22:23.869 com.netflix.discovery.DiscoveryClient : DiscoveryClient_SERVICEDIRECTORY/xxxComputer.fritz.box:ServiceDirectory:8099 - registration failed Cannot execute request on any known server

com.netflix.discovery.shared.transport.TransportException: Cannot execute request on any known server
    at com.netflix.discovery.shared.transport.decorator.EurekaHttpClientDecorator$1.execute(EurekaHttpClientDecorator.java:59) ~[eureka-client-1.9.13.jar:1.9.13]
    ...

13:22:23.870 c.n.discovery.Instance Replicator : There was a problem with the instance replicator

com.netflix.discovery.shared.transport.TransportException: Cannot execute request on any known server
    at com.netflix.discovery.shared.transport.decorator.RetryableEurekaHttpClient.execute(RetryableEurekaHttpClient.java:112) ~[eureka-client-1.9.13.jar:1.9.13]
    ...

...
13:22:24.193 o.s.b.w.embedded.tomcat.TomcatWebServer : Tomcat started on port(s): 8099 (http) with context path ''
13:22:24.194 .s.c.n.e.s.EurekaAutoServiceRegistration : Updating port to 8099
13:22:24.197 c.w.s.ServiceDirectoryApplication : Started ServiceDirectoryApplication in 6.846 seconds (JVM running for 7.369)

This is the output it produces by itself, without any other service started.

After letting it run for a while, it adds more output like this:

Eureka server output 2:

13:22:53.662 com.netflix.discovery.DiscoveryClient : Disable delta property : false
...
13:22:53.783 c.n.e.registry.AbstractInstanceRegistry : DS: Registry: lease doesn't exist, registering resource: SERVICEDIRECTORY - xxxComputer.fritz.box:ServiceDirectory:8099
13:22:53.783 c.n.eureka.resources.InstanceResource : Not Found (Renew): SERVICEDIRECTORY - xxxComputer.fritz.box:ServiceDirectory:8099
13:22:53.803 com.netflix.discovery.DiscoveryClient : DiscoveryClient_SERVICEDIRECTORY/xxxComputer.fritz.box:ServiceDirectory:8099 - Re-registering apps/SERVICEDIRECTORY
13:22:53.804 com.netflix.discovery.DiscoveryClient : DiscoveryClient_SERVICEDIRECTORY/xxxComputer.fritz.box:ServiceDirectory:8099: registering service...
13:22:53.815 com.netflix.discovery.DiscoveryClient : The response status is 200
...
13:22:53.856 c.n.e.registry.AbstractInstanceRegistry : Registered instance SERVICEDIRECTORY/xxxComputer.fritz.box:ServiceDirectory:8099 with status UP (replication=false)
13:22:53.859 com.netflix.discovery.DiscoveryClient : DiscoveryClient_SERVICEDIRECTORY/xxxComputer.fritz.box:ServiceDirectory:8099 - registration status: 204
...
13:22:54.398 c.n.e.registry.AbstractInstanceRegistry : Registered instance SERVICEDIRECTORY/xxxComputer.fritz.box:ServiceDirectory:8099 with status UP (replication=true)
13:23:23.817 com.netflix.discovery.DiscoveryClient : Disable delta property : false
...
13:23:23.824 o.s.w.s.handler.SimpleUrlHandlerMapping : Mapped to ResourceHttpRequestHandler ["classpath:/META-INF/resources/", "classpath:/resources/", "classpath:/static/", "classpath:/public/", "/"]
13:23:23.829 com.netflix.discovery.DiscoveryClient : The response status is 200
13:23:23.867 o.s.w.s.handler.SimpleUrlHandlerMapping : Mapped to ResourceHttpRequestHandler ["classpath:/META-INF/resources/", "classpath:/resources/", "classpath:/static/", "classpath:/public/", "/"]
13:23:23.881 c.n.e.registry.AbstractInstanceRegistry : Registered instance SERVICEDIRECTORY/xxxComputer.fritz.box:ServiceDirectory:8099 with status UP (replication=true)
13:23:23.881 c.n.e.r.PeerAwareInstanceRegistryImpl : Got 1 instances from neighboring DS node
13:23:23.881 c.n.e.r.PeerAwareInstanceRegistryImpl : Renew threshold is: 1
13:23:23.881 c.n.e.r.PeerAwareInstanceRegistryImpl : Changing status to UP
13:23:23.893 e.s.EurekaServerInitializerConfiguration : Started Eureka Server
13:23:24.385 o.s.w.s.handler.SimpleUrlHandlerMapping : Mapped to ResourceHttpRequestHandler ["classpath:/META-INF/resources/", "classpath:/resources/", "classpath:/static/", "classpath:/public/", "/"]

I can let it run for as long as I want. As soon as I start another service, the following is printed in the eureka server output:

Eureka server error output:

13:24:25.899 o.s.c.n.e.s.EurekaServiceRegistry : Unregistering application SERVICEDIRECTORY with eureka with status DOWN
13:24:25.899 com.netflix.discovery.DiscoveryClient : Saw local status change event StatusChangeEvent [timestamp=1586345065899, current=DOWN, previous=UP]
13:24:25.900 com.netflix.discovery.DiscoveryClient : DiscoveryClient_SERVICEDIRECTORY/xxxComputer.fritz.box:ServiceDirectory:8099: registering service...
13:24:25.903 c.n.eureka.DefaultEurekaServerContext : Shutting down ...
13:24:25.907 o.s.w.s.handler.SimpleUrlHandlerMapping : Mapped to ResourceHttpRequestHandler ["classpath:/META-INF/resources/", "classpath:/resources/", "classpath:/static/", "classpath:/public/", "/"]
13:24:25.908 c.n.eureka.DefaultEurekaServerContext : Shut down
13:24:25.910 c.n.e.registry.AbstractInstanceRegistry : Registered instance SERVICEDIRECTORY/xxxComputer.fritz.box:ServiceDirectory:8099 with status DOWN (replication=false)
13:24:25.911 o.s.b.f.support.DisposableBeanAdapter : Invocation of destroy method failed on bean with name 'tomcatMetricsBinder': java.lang.NoSuchMethodError: 'void io.micrometer.core.instrument.binder.tomcat.TomcatMetrics.close()'
13:24:25.912 com.netflix.discovery.DiscoveryClient : DiscoveryClient_SERVICEDIRECTORY/xxxComputer.fritz.box:ServiceDirectory:8099 - registration status: 204
13:24:25.913 o.s.s.concurrent.ThreadPoolTaskExecutor : Shutting down ExecutorService 'applicationTaskExecutor'
13:24:25.923 com.netflix.discovery.DiscoveryClient : Shutting down DiscoveryClient ...
13:24:28.924 com.netflix.discovery.DiscoveryClient : Unregistering ...
13:24:28.930 o.s.w.s.handler.SimpleUrlHandlerMapping : Mapped to ResourceHttpRequestHandler ["classpath:/META-INF/resources/", "classpath:/resources/", "classpath:/static/", "classpath:/public/", "/"]
13:24:28.933 c.n.e.registry.AbstractInstanceRegistry : Cancelled instance SERVICEDIRECTORY/xxxComputer.fritz.box:ServiceDirectory:8099 (replication=false)
13:24:28.935 com.netflix.discovery.DiscoveryClient : DiscoveryClient_SERVICEDIRECTORY/xxxComputer.fritz.box:ServiceDirectory:8099 - deregister status: 200
13:24:28.953 com.netflix.discovery.DiscoveryClient : Completed shut down of DiscoveryClient
13:24:29.111 trationDelegate$BeanPostProcessorChecker : Bean 'org.springframework.cloud.autoconfigure.ConfigurationPropertiesRebinderAutoConfiguration' of type [org.springframework.cloud.autoconfigure.ConfigurationPropertiesRebinderAutoConfiguration] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)

 . ____ _ __ _ _
 /\\ / ___'_ __ _ _(_)_ __ __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
 \\/ ___)| |_)| | | | | || (_| | ) ) ) )
 ' |____| .__|_| |_|_| |_\__, | / / / /
 =========|_|==============|___/=/_/_/_/
 :: Spring Boot :: (v2.2.6.RELEASE)

13:24:29.170 c.w.s.ServiceDirectoryApplication : The following profiles are active: test
...
13:24:31.186 com.netflix.discovery.DiscoveryClient : Getting all instance registry from the eureka server
13:24:31.190 ERROR c.n.d.s.t.d.RedirectingEurekaHttpClient : Request execution error. endpoint=DefaultEndpoint{ serviceUrl='http://localhost:8099/eureka/}

com.sun.jersey.api.client.ClientHandlerException: java.net.ConnectException: Connection refused
    at com.sun.jersey.client.apache4.ApacheHttpClient4Handler.handle(ApacheHttpClient4Handler.java:187) ~[jersey-apache-client4-1.19.1.jar:1.19.1]
    ...
    at com.netflix.discovery.EurekaIdentityHeaderFilter.handle(EurekaIdentityHeaderFilter.java:27) ~[eureka-client-1.9.13.jar:1.9.13]
    ...
    at com.netflix.discovery.shared.transport.jersey.AbstractJerseyEurekaHttpClient.getApplications(AbstractJerseyEurekaHttpClient.java:165) ~[eureka-client-1.9.13.jar:1.9.13]
    ...
Caused by: java.net.ConnectException: Connection refused
    at java.base/sun.nio.ch.Net.pollConnect(Native Method) ~[na:na]
    ..
    at java.base/java.net.SocksSocketImpl.connect(SocksSocketImpl.java:333) ~[na:na]
    at java.base/java.net.Socket.connect(Socket.java:648) ~[na:na]
    ...

13:24:31.191 c.n.d.s.t.d.RetryableEurekaHttpClient : Request execution failed with message: java.net.ConnectException: Connection refused
13:24:31.193 ERROR com.netflix.discovery.DiscoveryClient : DiscoveryClient_SERVICEDIRECTORY/xxxComputer.fritz.box:ServiceDirectory:8099 - was unable to refresh its cache! status = Cannot execute request on any known server

com.netflix.discovery.shared.transport.TransportException: Cannot execute request on any known server
    at com.netflix.discovery.shared.transport.decorator.SessionedEurekaHttpClient.execute(SessionedEurekaHttpClient.java:77) ~[eureka-client-1.9.13.jar:1.9.13]
    ...

13:24:31.193 com.netflix.discovery.DiscoveryClient : Using default backup registry implementation which does not do anything.
13:24:31.194 com.netflix.discovery.DiscoveryClient : Starting heartbeat executor: renew interval is: 30
13:24:31.195 c.n.discovery.Instance Replicator : Instance Replicator onDemand update allowed rate per min is 4
13:24:31.197 com.netflix.discovery.DiscoveryClient : Discovery Client initialized at timestamp 1586345071197 with initial instances count: 0
13:24:31.210 c.n.eureka.DefaultEurekaServerContext : Initializing ...
13:24:31.210 c.n.eureka.cluster.PeerEurekaNodes : Adding new peer nodes [http://localhost:8099/eureka/]
...
13:24:31.304 c.n.eureka.cluster.PeerEurekaNodes : Replica node URL: http://localhost:8099/eureka/
13:24:31.308 c.n.e.registry.AbstractInstanceRegistry : Finished initializing remote region registries. All known remote regions: []
13:24:31.310 c.n.eureka.DefaultEurekaServerContext : Initialized
13:24:31.346 o.s.c.n.e.s.EurekaServiceRegistry : Registering application SERVICEDIRECTORY with eureka with status UP
13:24:31.347 com.netflix.discovery.DiscoveryClient : Saw local status change event StatusChangeEvent [timestamp=1586345071347, current=UP, previous=STARTING]
13:24:31.347 com.netflix.discovery.DiscoveryClient : DiscoveryClient_SERVICEDIRECTORY/xxxComputer.fritz.box:ServiceDirectory:8099: registering service...
13:24:31.349 o.s.c.n.e.server.EurekaServerBootstrap : Setting the eureka configuration..
13:24:31.350 d.s.w.p.DocumentationPluginsBootstrapper : Context refreshed
13:24:31.350 o.s.c.n.e.server.EurekaServerBootstrap : Eureka data center value eureka.datacenter is not set, defaulting to default
13:24:31.350 o.s.c.n.e.server.EurekaServerBootstrap : Eureka environment value eureka.environment is not set, defaulting to test
...
13:24:31.351 c.n.d.s.t.d.RedirectingEurekaHttpClient : Request execution error. endpoint=DefaultEndpoint{ serviceUrl='http://localhost:8099/eureka/}

com.sun.jersey.api.client.ClientHandlerException: java.net.ConnectException: Connection refused
    at com.sun.jersey.client.apache4.ApacheHttpClient4Handler.handle(ApacheHttpClient4Handler.java:187) ~[jersey-apache-client4-1.19.1.jar:1.19.1]
    ...
    at com.netflix.discovery.EurekaIdentityHeaderFilter.handle(EurekaIdentityHeaderFilter.java:27) ~[eureka-client-1.9.13.jar:1.9.13]
    at com.sun.jersey.api.client.Client.handle(Client.java:652) ~[jersey-client-1.19.1.jar:1.19.1]
    ...
    at com.netflix.discovery.shared.transport.jersey.AbstractJerseyEurekaHttpClient.register(AbstractJerseyEurekaHttpClient.java:56) ~[eureka-client-1.9.13.jar:1.9.13]
    ...
Caused by: java.net.ConnectException: Connection refused
    at java.base/sun.nio.ch.Net.pollConnect(Native Method) ~[na:na]
    ...
    at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:121) ~[httpclient-4.5.12.jar:4.5.12]
    ...

13:24:31.352 c.n.d.s.t.d.RetryableEurekaHttpClient : Request execution failed with message: java.net.ConnectException: Connection refused
13:24:31.353 com.netflix.discovery.DiscoveryClient : DiscoveryClient_SERVICEDIRECTORY/xxxComputer.fritz.box:ServiceDirectory:8099 - registration failed Cannot execute request on any known server

com.netflix.discovery.shared.transport.TransportException: Cannot execute request on any known server
    at com.netflix.discovery.shared.transport.decorator.RetryableEurekaHttpClient.execute(RetryableEurekaHttpClient.java:112) ~[eureka-client-1.9.13.jar:1.9.13]
    ...

13:24:31.354 c.n.discovery.Instance Replicator : There was a problem with the instance replicator

com.netflix.discovery.shared.transport.TransportException: Cannot execute request on any known server
    at com.netflix.discovery.shared.transport.decorator.RetryableEurekaHttpClient.execute(RetryableEurekaHttpClient.java:112) ~[eureka-client-1.9.13.jar:1.9.13]
    ...

13:24:31.371 s.d.s.w.s.ApiListingReferenceScanner : Scanning for api listing references
13:24:31.457 o.s.b.w.embedded.tomcat.TomcatWebServer : Tomcat started on port(s): 8099 (http) with context path ''
13:24:31.458 .s.c.n.e.s.EurekaAutoServiceRegistration : Updating port to 8099
...
13:24:34.203 c.n.e.registry.AbstractInstanceRegistry : Registered instance GATEWAY/xxxComputer.fritz.box:gateway:8088 with status UP (replication=false)
...
13:24:34.722 c.n.e.registry.AbstractInstanceRegistry : Registered instance GATEWAY/xxxComputer.fritz.box:gateway:8088 with status UP (replication=true)

Then it keeps running.

On part that occurs multiple times in this output is the followWARNING:

13:22:23.658 ERROR com.netflix.discovery.DiscoveryClient : DiscoveryClient_SERVICEDIRECTORY/xxxComputer.fritz.box:ServiceDirectory:8099 - ...(more stuff)

As you can see from the fritz.box (my router) in the hostname, I work from home. I left it inside the logs because I wonder why it is trying to register itself within the whole LAN using my router. Shouldn't it be simply using "localhost"?

On the client side, the following is printed:

Eureka client output:

 . ____ _ __ _ _
 /\\ / ___'_ __ _ _(_)_ __ __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
 \\/ ___)| |_)| | | | | || (_| | ) ) ) )
 ' |____| .__|_| |_|_| |_\__, | / / / /
 =========|_|==============|___/=/_/_/_/
 :: Spring Boot :: (v2.2.6.RELEASE)

13:24:29.584 com.whatever.ApiGatewayApplication : The following profiles are active: test
...
13:24:32.004 o.s.b.w.embedded.tomcat.TomcatWebServer : Tomcat initialized with port(s): 8088 (http)
...
13:24:34.122 com.netflix.discovery.DiscoveryClient : Starting heartbeat executor: renew interval is: 30
13:24:34.126 c.n.discovery.Instance Replicator : Instance Replicator onDemand update allowed rate per min is 4
13:24:34.132 com.netflix.discovery.DiscoveryClient : Discovery Client initialized at timestamp 1586345074130 with initial instances count: 0
13:24:34.137 o.s.c.n.e.s.EurekaServiceRegistry : Registering application GATEWAY with eureka with status UP
13:24:34.137 com.netflix.discovery.DiscoveryClient : Saw local status change event StatusChangeEvent [timestamp=1586345074137, current=UP, previous=STARTING]
13:24:34.139 com.netflix.discovery.DiscoveryClient : DiscoveryClient_GATEWAY/xxxComputer.fritz.box:gateway:8088: registering service...
13:24:34.181 o.s.b.w.embedded.tomcat.TomcatWebServer : Tomcat started on port(s): 8088 (http) with context path ''
13:24:34.182 .s.c.n.e.s.EurekaAutoServiceRegistration : Updating port to 8088
13:24:34.186 com.whatever.ApiGatewayApplication : Started ApiGatewayApplication in 5.556 seconds (JVM running for 6.113)
13:24:34.206 com.netflix.discovery.DiscoveryClient : DiscoveryClient_GATEWAY/xxxComputer.fritz.box:gateway:8088 - registration status: 204
...
13:25:04.125 com.netflix.discovery.DiscoveryClient : Getting all instance registry from the eureka server
13:25:04.168 com.netflix.discovery.DiscoveryClient : The response status is 200

Which seems to me ok on the first glance.

---

What I tried so far is to test if the problem occurs with different versions of spring cloud. As I'm using the 2.2.x release of spring boot, only the Hoxton releases of spring cloud come into consideration. But maybe I should broaden that assumption.

I restarted my computer.

I tested if the problem persists with different clients being started.

One funny thing is: The eureka server restarts as soon as I start a client. It doesn't even seem wait for a request to be received.

Can anyone help me to find the solution to this problem?

Answer 1

I found a workaround for me. It seems like this issue is connected to Netbeans, which I use in version 11.3 (currently the newest).

When I use mvn from the command line and start the services with java in seperate terminals, the problem is gone.

I don't know why Netbeans produces this issue.

For now this workaround is ok for me, so I will close this issue.