Could this be a Java threading or Hibernate Issue?

Question

I am facing below issue - unable to reproduce it in Performance or lower environment but only in Prod has made debugging it bit difficult and hence posting here if I could get some clue(s). Tried to keep it as short as possible, still quite long..

---

Issue : In a 2 steps order creation process viz Create Customer + Submit order - we are getting Unique Constrain Violation where a single Customer Id is getting assigned to Orders created in Multiple Locations.

Not sure where it is going wrong. Few things that I could find

1. Oracle is unique in generating sequences
2. Hibernate has default allocation size of 50 and is pretty much optimized in terms of handing out those 50 sequence
3. Which makes me thing perhaps there is some thread that hijacking this sequence while the entity copy is happening somewhere , around last 2 methods below.

Pom Entry is as below for versions of Spring/Hibernate stack, Database is Oracle

<properties>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
        <spring.version>3.2.1.RELEASE</spring.version>
        <hibernate.version>4.1.10.Final</hibernate.version>
        <thymeleaf.version>2.1.0.RELEASE</thymeleaf.version>
        <tiles.version>2.2.2</tiles.version>
    </properties>

The Customer Object is as below

@Entity
@Table(name = "CUSTOMER")
public class Customer implements Serializable {
    private static final long serialVersionUID = 1L;

    @SequenceGenerator(name="CUSTOMER_SEQUENCE_GENERATOR", 
        sequenceName="CUSTOMER_SEQ", allocationSize = 1)

    @Id
    @GeneratedValue(strategy=GenerationType.SEQUENCE, 
        generator="CUSTOMER_SEQUENCE_GENERATOR")
    @Column(name = "CUSTOMER_ID", unique = true, nullable = false)
    private Integer id;

---

This Code snippet is called from various places like 3 Controllers and 2 Service methods - could say all over the code.

 Customer customer = createCustomerEntity(newcustomer);
 customer = customerManagementDAO.addUpdateCustomer(customer);

The Details of 2 methods are following

I am not sure if something in below code appears wrong from a threading stand point

    private Customer createCustomerEntity(CustomerBean newcustomer) 
                 throws InternalBusinessException {
          Customer customer = null;
            try{
            // If Loop1
            if(newcustomer.getInternalcustomerid() == null){
                 customer = new Customer();
              customer.setCustomerNumber(newcustomer.getExternalSystemCustomerid());
            }
            // If Loop2
            if(newcustomer.getInternalcustomerid() != null){
                //if a new customer going for new  enrollment , 
                // we lookup up the customer in DB based on 
                // InternalCustomerId and merge the customer.
                Integer customerIdInternal = null;
                // Below Call goes to Internal Database to find existing Customer
                Customer customerEntity  = orderManagementDAO.findExistingCustomerByCustomerId(
                                newcustomer.getEoecustomerid().toString());
                if(customerEntity != null){
                customer = new Customer();  
                customer.setCustomerNumber(
                                newcustomer.getExternalSystemCustomerid()); 
                customerIdInternal = customerEntity.getId();
                if (customerIdInternal != null) {
                     customer.setId(customerIdInternal);
                }
            }
            //In case of customer look up if customer with same 
            // External System's id exists we merge the same customer.
            // If Loop3
if(newcustomer.getExternalSystemCustomerid() != null) {   
                // Below call goes to external System to find existing Customer Number
                Customer customerEntity = orderManagementDAO.findExistingCustomerByCustomerNumber(
                                newcustomer.getExternalSystemCustomerid().trim());
                Integer customerIdInternal = null;
                if(customerEntity != null) {
                    customer = new Customer();
                    customer.setCustomerNumber(
                                newcustomer.getExternalSystemCustomerid());
                    customerIdInternal = customerEntity.getId();
                 }
                if (customerIdInternal != null) {
                    customer.setId(customerIdInternal);
                }            
            }
            // End of If loop3
            customer.setEmail(newcustomer.getEmail()); //<== We have null pointers at this point in logs
            customer.setFirstName(newcustomer.getFirstname());
            customer.setLastName(newcustomer.getLastname());

            } catch (RuntimeException e) {
                throw new InternalBusinessException("createCustomerEntityForEOE failed", e);
            }
            return customer;
        }

• Logs show Printing of Same Customer Ids in Loop 2 & Loop3 in serveral instances with different threads

• Null Pointers at at End of Loop3 at customer.setEmail

Dao Code is as below

@SuppressWarnings("unchecked")
    @Override
    public Customer addUpdateCustomer(Customer customer) 
            throws InternalDataAccessException {
        if (customer.getId() != null && customer.getId() != 0)
            saveMerge(customer);
        else
            save(customer);
        return customer;
    }

Please let me know if you feel some other part of code could add clarity to this question.

Answer 1

This issue ended up being more of a Business case scenario issue and was resolved when the whole create Customer process was moved to a Stored Prod in Oracle DB

We had a Table Cust#Relinquished to pickup Active Customer No from a Range, do a few steps to set new range if they are consumed or Increment the no and do SaveMerge()

This 3 Hibernate query step were changed to One Stored Proc and it worked fine.