Any chance of bug fix for TableQuery+SQLContainer and Connection pools?

I note this existing bug

http://dev.vaadin.com/ticket/12370

I see the same thing when trying to use HikariCP https://github.com/brettwooldridge/HikariCP

Barfs with a TableQuery, works fine with FreeFormQuery, but I lose the sorting/filtering etc.

i.e. I just want it to work

I need to use a proper connection pool as the Maria DB is setup with wait_timeout of 10 seconds

So if I use SimpleJDBCConnectionPool, my connections die after 10 seconds and don’t repair.

If there anything else I can do to get Vaadin to retry/repair the connection if the server has closed it?

Thanks.

My workaround to this was to replace all my TableQuery’s. Instead of using FreeFormQuery I defined FreeformStatementDelegate’s, in order to get filtering and sorting, as described here: https://vaadin.com/book/-/page/sqlcontainer.freeform.html

Granted it looks like a lot of work to write a FreeformQueryDelegate or FreeformStatementDelegate for every grid query. But it can be simplified as follows.

  1. The deprecated methods can simply throw UnsupportedOperationException.
  2. The same exception can be thrown for inserts and updates, if the list is going to be read-only (as mine were.)
  3. Here is a sample implementation you can crib off of. http://dev.vaadin.com/svn/addons/SQLContainer/trunk/demo/src/com/vaadin/addon/sqlcontainer/demo/DemoFreeformQueryDelegate.java

Still seems like a lot of work, right? But note that most of the filtering and sorting code is generic, using a QueryBuilder to turn column names into ORDER BY and WHERE clauses. The only bits that aren’t generic are the lines containing “SELECT * FROM PEOPLE” or “SELECT COUNT(*) FROM PEOPLE”. So what I did was make my own abstract class called AbstractStatementDelegate. It’s basically this demo code, but with two new methods:

abstract StringBuffer getSelectQueryClause(); abstract StringBuffer getCountQueryClause(); Anywhere the demo code has a SELECT string, I replace the string with a call to one of the methods above.

Then for each of my table queries, I simply declare a new subclass. Here’s a simplified example:

public class CustomersStatementDelegate extends AbstractStatementDelegate implements FreeformStatementDelegate {
    @Override
    StringBuffer getSelectQueryClause() {
        return new StringBuffer("SELECT customer_id, name, is_active FROM customer");
    }
    @Override
    StringBuffer getCountQueryClause() {
        return new StringBuffer("SELECT COUNT(customer_id) FROM customer");
    }
}

As you can see, once it’s set up adding new queries is painless. (I was also able to add security to the queries in a pretty nice way: I automatically add new WHERE clauses to the query based on the user’s privileges. That code is not shown.)

Hope this helps. Like you, I think Vaadin should prioritize that pool problem higher. There were several StackOverflow questions about it, plus these forum posts. I guess no one on a support contract has complained about it yet.

I also find it odd that FreeformQueryDelegate and FreeformStatementDelegate are so ungainly to use. They lack the simplicity of FreeformQuery and TableQuery. Fortunately in my case I was able to make it simpler using the trick I described above. It feels like something that could have been built-in though. If I hadn’t found the Demo example on the internet (I think it was added to the manual later on) then I would have been completely hosed… But maybe there’s an easier way I am overlooking.

Good luck,
Matt

Thanks for that. Any chance of your code in a gist or attachment? i.e. the less I have to write the less mistakes I get to make.

I honestly think you are better off taking the Demo code and making the modifications I described. It’s possible this Vaadin Demo code is more recent than what I used, which I found by searching the internet.

Having said that, I will paste the code here.

Here again is the delegate CustomersStatementDelegate. This time I include the role-based filter method. If the user is not an admin they only get back one row-- the company they work for. Those lines will not compile for you but should give you an idea how to use the filter. If you don’t need row-level security then just return null from getRoleBasedFilter (or remove the code that calls this method from AbstractStatementDelegate.)

public class CustomersStatementDelegate extends AbstractStatementDelegate implements FreeformStatementDelegate {
    @Override
    StringBuffer getSelectQueryClause() {
        return new StringBuffer("SELECT customer_id, name, is_active FROM customers");
    }

    @Override
    StringBuffer getCountQueryClause() {
        return new StringBuffer("SELECT COUNT(customer_id) FROM customers");
    }

    @Override
    Filter getRoleBasedFilter() {
        Filter filter = null;
        User me = User.getCurrentUser();
        if (!me.isSystemAdmin()) {
            filter = new Compare.Equal("customer_id", me.getCompanyId());
        }
        return filter;
    }
}

Here is the main thing you wanted: the class AbstractStatementDelegate.

[code]
public abstract class AbstractStatementDelegate implements FreeformStatementDelegate {

protected List<Filter> filters;
private List<OrderBy> orderBys;

abstract StringBuffer getSelectQueryClause();

abstract StringBuffer getCountQueryClause();

abstract Filter getRoleBasedFilter();

public AbstractStatementDelegate() {
    super();
}

private List<Filter> combineFilters(List<Filter> first, Filter second) {
    if (second == null) {
        return first;
    } else if (first == null) {
        return Arrays.asList(second);
    }
    ArrayList<Filter> union = new ArrayList<Filter>(first.size() + 1);
    union.addAll(first);
    union.add(second);
    return union;
}

public StatementHelper getQueryStatement(int offset, int limit) throws UnsupportedOperationException {
    StatementHelper sh = new StatementHelper();
    StringBuffer query = getSelectQueryClause();
    List<Filter> allFilters = combineFilters(filters, getRoleBasedFilter());
    if (allFilters != null) {
        query.append(QueryBuilder.getWhereStringForFilters(allFilters, sh));
    }
    query.append(getOrderByString());
    if (offset != 0 || limit != 0) {
        query.append(" LIMIT ").append(limit);
        query.append(" OFFSET ").append(offset);
    }
    sh.setQueryString(query.toString());
    return sh;
}

private String getOrderByString() {
    StringBuffer orderBuffer = new StringBuffer("");
    if (orderBys != null && !orderBys.isEmpty()) {
        orderBuffer.append(" ORDER BY ");
        OrderBy lastOrderBy = orderBys.get(orderBys.size() - 1);
        for (OrderBy orderBy : orderBys) {
            orderBuffer.append(SQLUtil.escapeSQL(orderBy.getColumn()));
            if (orderBy.isAscending()) {
                orderBuffer.append(" ASC");
            } else {
                orderBuffer.append(" DESC");
            }
            if (orderBy != lastOrderBy) {
                orderBuffer.append(", ");
            }
        }
    }
    return orderBuffer.toString();
}

public StatementHelper getCountStatement() throws UnsupportedOperationException {
    StatementHelper sh = new StatementHelper();
    StringBuffer query = getCountQueryClause();
    List<Filter> allFilters = combineFilters(filters, getRoleBasedFilter());
    if (allFilters != null) {
        query.append(QueryBuilder.getWhereStringForFilters(allFilters, sh));
    }
    sh.setQueryString(query.toString());
    return sh;
}

public void setFilters(List<Filter> filters) throws UnsupportedOperationException {
    this.filters = filters;
}

public void setFilter(Filter filter) throws UnsupportedOperationException {
    this.filters = Arrays.asList(filter);
}

public void setOrderBy(List<OrderBy> orderBys) throws UnsupportedOperationException {
    this.orderBys = orderBys;
}

@Override
public StatementHelper getContainsRowQueryStatement(Object... keys) throws UnsupportedOperationException {
    throw new UnsupportedOperationException(""); // default WHERE clause is good enough for our simple queries
}

public int storeRow(Connection conn, RowItem row) throws SQLException {
    throw new UnsupportedOperationException("List is read-only.");
}

public boolean removeRow(Connection conn, RowItem row) throws UnsupportedOperationException, SQLException {
    throw new UnsupportedOperationException("List is read-only.");
}

@Deprecated
public String getQueryString(int offset, int limit) throws UnsupportedOperationException {
    throw new UnsupportedOperationException("Use getQueryStatement method.");
}

@Deprecated
public String getCountQuery() throws UnsupportedOperationException {
    throw new UnsupportedOperationException("Use getCountStatement method.");
}

@Deprecated
public String getContainsRowQueryString(Object... keys) throws UnsupportedOperationException {
    throw new UnsupportedOperationException("Please use getContainsRowQueryStatement method.");
}

}
[/code]Lastly, here is sample code that creates a SQLContainer, given the query and connection pool. You’ll need to substitute your own connection pool obviously.

        AbstractStatementDelegate stmtDelegate = new CustomersStatementDelegate();
        FreeformQuery query = stmtDelegate.getSelectQueryClause(), JPAUtilities.getConnectionPool(), "customer_id");
        query.setDelegate(stmtDelegate);

        container = new SQLContainer(query);


OPTIONAL IMPROVEMENT:
I found it annoying to have to supply both the query string and the statement delegate when creating the FreeformQuery. They are redundant. So I hid that in yet another class:

[code]
public class DelegateQuery extends FreeformQuery {
/**
* Creates a new freeform query delegate to be used with the
* {@link SQLContainer}. Constructed with a prepared statement instead of a query
* string. Gets around the validation requiring the query string to be
* non-null.
*/
public DelegateQuery(AbstractStatementDelegate freeformQueryDelegate, JDBCConnectionPool connectionPool,
String… primaryKeyColumns) {
super(freeformQueryDelegate.getSelectQueryClause().toString(), connectionPool, primaryKeyColumns);

    this.setDelegate(freeformQueryDelegate);
}

}
[/code]The sample code for creating the SQLContainer then becomes:

[code]
AbstractStatementDelegate stmtDelegate = new CustomersStatementDelegate();
FreeformQuery query = new DelegateQuery(stmtDelegate, JPAUtilities.getConnectionPool(), “customer_id”);

    container = new SQLContainer(query)

[/code]It only saves one line of code, but it is arguably cleaner. (The fact that FreeformQuery requires a query string even if you want to construct it from a delegate is one more way that statement delegates feel less finished/polished than other parts of the Vaadin framework.)

Hope this helps,
Matt

Thanks ever so much. Hopefully this will also help others who have found the same problem. Just amazed this hasn’t been fixed. Previously I was using IndexedContainer and Beans so never saw this.

You might find my StackOverflow.com Question interesting:
How to use Vaadin SQLContainer when I already have a JDBC connection pool
. Links back to this thread.

I had used SimpleJDBCConnectionPool and SQLContainers for all database access due to its simplicity. During a round of optimization, I found HikariCP and implemented some PreparedStatement-based access methods to speed ad-hoc database interactions. Big improvement. Having done that, I wondered if I could use the same fast connection pool as a basis for TableQuery/SQLContainer/Grid.

// JDBCConnectionPool connectionPool = SimpleJDBCConnectionPool ( ... );  // by replacing thi
hikariDataSource = new HikariDataSource(hikariConfiguration);
JDBCConnectionPool connectionPool = new J2EEConnectionPool(hikariDataSource);  // with this

The first resulting SQLContainer gets created but fails at the a container.size() statement. So let me add my voice to the choir that would like to see #12370 worked, if that’s what it would take.

SimpleJDBCConnectionPool works fine for me, but I’d like to consolidate on one connection pool and HikariCP looks like the one.

Had to put

orderBuffer.append(“"); orderBuffer.append(SQLUtil.escapeSQL(orderBy.getColumn())); orderBuffer.append("”);

as my column names have spaces in them.

But thanks for the code, its a life saver.

Vaadin 7.4.7 with fix for above

https://dev.vaadin.com/ticket/12370

Haven’t tried it yet.

Just tried out 7.4.7, works fine for me. So no need for above fix now.

Dino have you tried mySQL → HikariCP → TableQuery → SQLContainer? I’ve tried both HikariCP and Tomcat connection pool with Vaadin 7.4.7. Both form connections, and get data ( I think with the FreeformQueries I use) , but neither populate an SQLContainer with items. SimpleJDBCConnectionPool works perfectly with all the same code. I was hoping to experiment with different connection pools to determine if the performance difference matters.

I’m doing just that. Works fine.

Mariadb 10.x, mysql driver 5.1.35, Hikari 2.3.7

e.g.

val pool = new J2EEConnectionPool(ds)
val tq = new com.vaadin.data.util.sqlcontainer.query.TableQuery(“users”,pool)
val usersContainer = new SQLContainer(tq)

val usersTable=new com.vaadin.ui.Table{

  setContainerDataSource(usersContainer)

Maybe an issue with at the driver level?

https://vaadin.com/forum#!/thread/4573208

When I get my data source

val cpConfig = new com.zaxxer.hikari.HikariConfig();
cpConfig.setJdbcUrl(jdbcUrl)
cpConfig.setUsername(name)
cpConfig.setPassword(PASSWORD)
cpConfig.addDataSourceProperty(“databaseName”,dbname)
cpConfig.setAutoCommit(false)
cpConfig.setConnectionInitSql(“SET SESSION sql_mode=‘ANSI’”)
new com.zaxxer.hikari.HikariDataSource(cpConfig)

Thanks very much Dino. The configuration method

setConnectionInitSql("SET SESSION sql_mode='ANSI'")

was previously absent, leading to some queries working but those involving TableQuery to fail.