How can we debug our production Struts/Hibernate apps?

Question

So, we use Netbeans 6.9.1 , push our code to a SVN server (dev) and run local webservers (Tomcat) that connect to MySQL on a DEV machine. We can run in debug mode and step through our code.

However, we then take our WAR file, push to a production machine (on a production database). This machine is completely separate from dev.

But we run into constant problems. Things like null id in Hibernate when we have triple-checked all columns, etc.

What we need is a way to attach a debugger to a PRODUCTION user and step through the code.

What are our options?

Thanks for any advice.

EDIT

I wanted to post a little more information. Thanks for those that have offered suggestions.

First, our production server hosts 3 apps (along with the prod db). The app we are having the most problems with just has one user and the data is almost identical to what we have in dev. We are having transaction problems with JDBC and can't find out where they are coming from.

Answer 1

If you really want to attach debugger to a running production server, Java is fully capable of doing this. First, run you server/JVM with the following commands:

-Xrunjdwp:transport=dt_socket,address=8778,server=y,suspend=n

Now you can attach your IDE using prd_server:8788 address in Eclipse or IntelliJ IDEA .

However there are few gotchas:

• You need to have TCP/IP access to production server on a specified port. Probably some firewall/SSH tunnelling will be involved.

• By default your debugger might suspend all threads when it hits the breakpoint. Make sure you only suspend the current one, otherwise you will freeze the whole server.

• ...yes, remote debugging is dangerous , especially on production.

• JVM in debug mode/listening for debug connections might be slightly slower (haven't seen it though).

Answer 2

If you are willing to debug the production environment remotely than beware it will slow down your applicaton. But in case if you are not able to replicate the issue on development server than please follow the steps to remotely debug the applicaton : -

1. Make sure your production server is having debug port open to connect debug remotely.
2. Debug setting differ for different-different application server
3. If you are using Eclispe than right click on the project and go to debug configuration enter the production server name/IP and the port number and than hit the debug button. Mkae sure your server is having debug port open

Let us know which production server you are using.

Answer 3

You can configure Tomcat to allow remote debugger connections , but I would not advise doing so as you will significantly slow down execution of the JVM while a debugger is attached, and I believe whenever a breakpoint is hit the execution of all threads will be paused.

Much better to recreate the problem using the production data on a dev server, add more logging, etc etc.

Answer 4

Before you attempt to debug Prod, migrate your Prod data down to a Dev env and debug there. If you have a security policy against Prod data in a non-Prod env, sanitize the data as needed.

If the issue is data related, you should hit it fairly quickly.