Upstart script for Tomcat respawn on Ubuntu

Question

I'm using foreman export upstart /etc/init to setup a tomcat app server, such that availability can be monitored and maintained in Ubuntu.

But there are a few challenges that I would like some advice on:

1) After starting my instance with sudo service tomcat start ... if i try to kill off the process using kill <pid> ... I can see that its instantly respawned by the Upstart utility ... BUT its too fast! The ports required have not been released yet. What would be a workaround to overcome this?

SEVERE: Failed to initialize end point associated with ProtocolHandler ["http-nio-8443"]
8 java.net.BindException: Address already in use
9       at sun.nio.ch.Net.bind0(Native Method)
43 Caused by: java.net.BindException: Address already in use
44      at sun.nio.ch.Net.bind0(Native Method)

2) I use initctl list | grep tomcat initctl list | grep tomcat to figure out the pid (3518 in this case) and kill the process:

$ initctl list | grep tomcat
  tomcat start/running
  tomcat-web-1 start/running, process 3518
  tomcat-web start/running

Is it possible that killing tomcat-web-1 is actually an incorrect way to test Upstart's respawn capabilities?

3) Another challenge is that if I do this just once or twice, I quickly run into memory problems:

376 Error occurred during initialization of VM
377 Could not reserve enough space for object heap

4) Lastly the default script seems to handle respawn only for exit conditions that are erroneous. But a machine reboot does not count as one of those so how should an export from foreman export upstart /etc/init be enhanced to take care of this scenario?

I wonder if anyone else has run into similar challenges before and/or has workarounds for them?

---

UPDATE # 1 (on Mar 4th 2013):

To accomplish (4), I just had to edit the appname.conf file's start on statement to include start on (... OR runlevel [2345])

Answer 1

In Step2, all three results listed are actually unix processes. None of them is the tomcat java webserver. So you are killing one of the processes, which helps you police the actual tomcat/java process and when it comes back up .. it tries to start tomcat again. The ports on which tomcat is running have never been released and the memory is still in use because you never shutdown the original tomcat! Therefore you experience (1) and (3).

A better/easier way to identify the java process is to look at it a utility such as System Monitor. You'll see a process with name 'java' and the process id will be listed there. Now issue a kill for that pid, which will demo to you that foreman+upstart respawn works just fine.