Java heap space when trying to display 1M rows in a table

General
1

Hi,
We are currently checking out vaadin for our next generation project.

I tried to create a very quick and dirty app that uses the skeleton of the phonebook example.
but the datasource im using is from our staging db (via mysql jdbc connector)

When i’m inserting into the IndexedContainer 1,000,000 entries (with around 8 fields each) i’m getting the :

javax.servlet.ServletException: com.vaadin.server.ServiceException: java.lang.OutOfMemoryError: Java heap space
com.vaadin.server.VaadinServlet.service(VaadinServlet.java:240)
javax.servlet.http.HttpServlet.service(HttpServlet.java:723)

error.

I’m new to vaadin and my code probably not optimal:

here is my major change:

private static IndexedContainer createDummyDatasource() {
IndexedContainer ic = new IndexedContainer();
try {
Class.forName(“com.mysql.jdbc.Driver”);
// Setup the connection with the DB

    Connection  connect =   DriverManager.getConnection("jdbc:mysql://localhost:33306/OurDB", "root", "password");
      // Statements allow to issue SQL queries to the database
      Statement statement = connect.createStatement();
      // Result set get the result of the SQL query
     ResultSet  resultSet = statement
          .executeQuery("select p.firstName as firstName, p.lastName as lastName, e.* from Event e join Player p on p.id = e.player_id where e.id < 1000000");
     	    
     for (String p : fieldNames) {
		ic.addContainerProperty(p, String.class, "");
     }
     
     while( resultSet.next() ) {	
	      // It is possible to get the columns via name
	      // also possible to get the columns via the column number
	      // which starts at 1
	      // e.g. resultSet.getSTring(2);		 		
	      
	      Object oid = ic.addItem();					
		
	      
	      ic.getContainerProperty(oid, FNAME).setValue(resultSet.getString("firstName") == null? "No name" : resultSet.getString("firstName"));
	      ic.getContainerProperty(oid, LNAME).setValue(resultSet.getString("lastName")== null? "No Last name" : resultSet.getString("lastName"));
	      ic.getContainerProperty(oid, "id").setValue(resultSet.getString("id"));
	      ic.getContainerProperty(oid, "sum").setValue(resultSet.getString("sum"));
	      ic.getContainerProperty(oid, "totalNum").setValue(resultSet.getString("totalNum"));
	      ic.getContainerProperty(oid, "brandId").setValue(resultSet.getString("brandId"));
	      ic.getContainerProperty(oid, "currency").setValue(resultSet.getString("currency"));
	      ic.getContainerProperty(oid, "type").setValue(resultSet.getString("type"));
	      ic.getContainerProperty(oid, "total").setValue(resultSet.getString("total"));
	      ic.getContainerProperty(oid, "player_id").setValue(resultSet.getString("player_id"));
	      		      
	    }
	
    }
    catch(Exception e) {
	System.out.println("sdf");
    }
    return ic;
}

any idea why that happens?

Thanks

2

This is common issue. Please run your application (server) with param -Xmx1024m. You can use greater values of memory i.e. -Xmx6g, but remember that you need to have enough RAM.

3

Yes - you’ve loaded 1,000,000 rows into memory! This is a very bad idea.

You really don’t want to use the IndexedContainer for a table of this size : you’ll want to use a Container with paging (i.e. it only loads the items from the database as it needs it).

IndexedContainer retains every Item/Property in memory. That’s alot of data.

As an example, the SQLContainer only loads items those that it needs.
Another container with lazy loading is
LazyQueryContainer

FWIW, for our project, we’ve written our own lazy-loading containers - and it’s not as difficult as you might think. (The above referenced were not available when we started - and anyway, this project is “n”-tier with no direct access to the database from the webapp).

Cheers,

Charles.

4

And one more note: the client side Table implementation doesn’t really handle 1M rows due to how scrolling is implemented and some browser limitations. The practical limit (depending on the browser, row heights etc.) might be around 400000.

Then again, navigating in a single table with 1M rows by scrolling is probably not going to be a pleasant experience for the user, either, so some filtering or such will probably also improve the user experience considerably.

5

Thanks for your replies. i will see into it.