JPA batch inserts (aka bulk inserts) may seem trivial at first. However, if you’re not careful you won’t see the performance gains you expect even though your application works “just fine”. If you follow the below guidelines your JPA batch inserts should be blazingly fast, though.

JPA batch inserts with Hibernate

All the material presented in this chapter is a summary of the official Hibernate documentation on batch processing. It starts with an example how not to do JPA batch inserts:

Session session = sessionFactory.openSession();
Transaction tx = session.beginTransaction();
for (int i=0; i<100000; i++) {
  Customer customer = new Customer(.....);
  session.save(customer);
}
tx.commit();
session.close();

This will most likely throw an OutOfMemoryException before the loop is done. That is because Hibernate caches all the newly inserted Customer instances in the session-level cache.

The first step to fix this is to enable JDBC batching with Hibernate. Set the hibernate.jdbc.batch_size property to a “sensible” value, commonly between 10 and 50.

Then you need to update your code to update and clear the session in regular intervals. ‘regular’ in this respect means at the same intervals as the underlying JDBC implementation batches your insert statements:

Session session = sessionFactory.openSession();
Transaction tx = session.beginTransaction();
for ( int i=0; i<100000; i++ ) {
  Customer customer = new Customer(.....);
  session.save(customer);
  if ( i % 20 == 0 ) { // 20, same as the JDBC batch size
    // flush a batch of inserts and release memory
    session.flush();
    session.clear();
  }
}
// Flush one last time to catch those beyond that last full batch.
session.flush();
session.clear();
tx.commit();
session.close();

Verify that your JPA batch inserts work fine

Hibernate may fool you if you look at the SQL statements it dumps, assuming you enabled those (nice StackOverflow answer related to this). I learned to rather trust the trace messages of org.hibernate.engine.jdbc.batch.internal.AbstractBatchImpl (or org.hibernate.jdbc.AbstractBatcher for Hibernate <v4.0) than the SQL log statements. So, make sure your logging framework is configured to log messages of that class on TRACE level.

Furthermore, if your data set is just slightly more complicated than in the above example you may not see any JDBC batching at all.

Customer customer = new Customer(.....);
X x = new X(...);
customer.setX(x)
// note we are adding an X to the customer, so that object 
// needs to be persisted as well
session.save(customer);

This results in SQL statements like

insert into Customer values (...)
insert into X values(...)

The problem is Hibernate looks at each SQL statement and checks to see if it is the same statement as the previously executed statement. If they are and if it hasn’t reached batch_size it will batch those two statements together using JDBC batch. However, if your statements look like the example above, Hibernate will see alternating insert statements and will flush an individual insert statement for each record processed.

To fix this you need to set hibernate.order_inserts=true and hibernate.order_updates=true.

JPA batch inserts with Spring Data JPA

A casual observer could be fooled into thinking that Spring Data JPA offers JPA batch inserts out of the box transparently behind the scenes. It’s true that CrudRepository does have a save(Iterable) method that calls save(Entity) in a loop. However, since it does not flush and clear the session it suffers from the problems explained above. I use the following code to work around that:

@PersistenceContext
private EntityManager entityManager;

@Value("${hibernate.jdbc.batch_size}")
// @Value("${spring.jpa.properties.hibernate.jdbc.batch_size}") for Spring Boot
private int batchSize;

public <T extends MyClass> Collection<T> bulkSave(Collection<T> entities) {
  final List<T> savedEntities = new ArrayList<T>(entities.size());
  int i = 0;
  for (T t : entities) {
    savedEntities.add(persistOrMerge(t));
    i++;
    if (i % batchSize == 0) {
      // Flush a batch of inserts and release memory.
      entityManager.flush();
      entityManager.clear();
    }
  }
  // Flush one last time to catch those beyond that last full batch.
  entityManager.flush();
  entityManager.clear();
  return savedEntities;
}

private <T extends MyClass> T persistOrMerge(T t) {
  if (t.getId() == null) {
    entityManager.persist(t);
    return t;
  } else {
    return entityManager.merge(t);
  }
}