Help! Memory leak problem.

My current setup consists of JRockit JVM + Tomcat 7. There is a possible memory leak problem. Here are the profiling results, please help me to solve it.

Pic.1 Tomcat started and before login.

Pic.2 1st time login and initialized some tables. (IndexedContainer) Then, logout.

Pic.3 2nd time login and initialized same tables.(IndexedContainer) Then, logout.

As you can see, com.vaadin.data.util.MethodProperty started to build up and it never went down even I’ve logged out or the session has time out. On my production environment, this class could occupied hundreds of MB in memory. Here is my code that close the application:


        logoutBtn.addListener(new ClickListener(){
			private static final long serialVersionUID = 3441527571087478137L;

			@Override
			public void buttonClick(ClickEvent event) 
			{
            	               ContextApplication.currentApplication().setLogoutURL(ContextApplication.currentApplication().getURL().toExternalForm());
            	               ContextApplication.currentRequest().getSession().invalidate();
            	               ContextApplication.currentApplication().close();
			}});

What might be cause of this memory leak? Is it the correct way to close the the application?
I am assuming that by closing the application, every window or table within application will be GCed.

Anyone? This possible memory leak problem is killing me!
BTW, I use Hibernate and BeanContainer.

As it is not a known problem, without source-code I (and everybody else I presume) can only guess where the problem might be. As MethodProperty is closely related to the BeanContainers my guess is that you keep the container’s instance (for instance in a static-field or in a Spring Application-Context) and just refill it with new Beans. I didn’t see any instances of BeanItem which might stand against that szenario, but it might worth a shot.

Thanks for the feedback. I paste the source code of my custom component here. (being used as a tab)
LazyBeanItem or LazyBeanContainer is no different than BeanItem or BeanContainer except they can take argument of exclusion list on bean properties. They are not the cause of this problem, because I’ve tried stock BeanItem or BeanContainer with same result.
The business logic is simple. There is a table which will be loaded with search result by pressing search button. Each time the items in the container will be removed and refilled with newer result. I’ve noticed that if I do many search during the same session, the memory consumption (MethodProperty) will not increase. With different session, the memory will start to build up and never released even the sessions were released!


package com.kymco.resume.composite;

import java.util.List;
import java.util.Set;

import javax.annotation.PostConstruct;
import javax.annotation.PreDestroy;
import javax.annotation.Resource;

import org.hibernate.criterion.CriteriaSpecification;
import org.hibernate.criterion.DetachedCriteria;
import org.hibernate.criterion.MatchMode;
import org.hibernate.criterion.Order;
import org.hibernate.criterion.Restrictions;
import org.hibernate.sql.JoinFragment;
import org.springframework.beans.factory.annotation.Configurable;
import org.springframework.util.StringUtils;
import org.vaadin.dialogs.ConfirmDialog;

import com.kymco.model.resume.Application;
import com.kymco.model.resume.Resume;
import com.kymco.resume.common.AppCountColumnGenerator;
import com.kymco.resume.common.GenderColumnGenerator;
import com.kymco.resume.common.ProcessColumnGenerator;
import com.kymco.resume.common.UserCodeContainerFactory;
import com.kymco.resume.dao.ResumeDAO;
import com.kymco.resume.window.ProcessChangeWindow;
import com.kymco.resume.window.ResumeEditWindow;
import com.kymco.vaadin.data.util.LazyBeanContainer;
import com.kymco.vaadin.data.util.LazyBeanItem;
import com.kymco.vaadin.table.DateColumnGenerator;
import com.vaadin.annotations.AutoGenerated;
import com.vaadin.data.Property;
import com.vaadin.data.Property.ValueChangeEvent;
import com.vaadin.ui.AbstractSelect;
import com.vaadin.ui.Alignment;
import com.vaadin.ui.Button;
import com.vaadin.ui.ComboBox;
import com.vaadin.ui.CustomComponent;
import com.vaadin.ui.HorizontalLayout;
import com.vaadin.ui.PopupDateField;
import com.vaadin.ui.Table;
import com.vaadin.ui.TextField;
import com.vaadin.ui.VerticalLayout;
import com.vaadin.ui.Window;
import com.vaadin.ui.AbstractSelect.Filtering;
import com.vaadin.ui.Button.ClickEvent;
import com.vaadin.ui.Button.ClickListener;
import com.vaadin.ui.Window.Notification;

@Configurable
public class MemberComp extends CustomComponent 
{
	@AutoGenerated
	private VerticalLayout mainLayout;

	@AutoGenerated
	private Table table;

	@AutoGenerated
	private HorizontalLayout hl2;

	@AutoGenerated
	private Button removeBtn;

	@AutoGenerated
	private Button resumeBtn;

	@AutoGenerated
	private HorizontalLayout hl1;

	@AutoGenerated
	private Button searchBtn;

	@AutoGenerated
	private TextField tf1;

	@AutoGenerated
	private ComboBox combo2;

	@AutoGenerated
	private PopupDateField toDate;

	@AutoGenerated
	private PopupDateField fromDate;

	@AutoGenerated
	private TextField tf2;

	@AutoGenerated
	private ComboBox combo1;


	private static int MAX_COUNT=2000;
	@Resource private ResumeDAO dao;
	private LazyBeanContainer<String,Resume> ctr;
	@Resource private UserCodeContainerFactory ucFactory;

	private static final long serialVersionUID = 178821687368574870L;
	
	/**
	 * The constructor should first build the main layout, set the
	 * composition root and then do any custom initialization.
	 *
	 * The constructor will not be automatically regenerated by the
	 * visual editor.
	 */
	public MemberComp() {
		buildMainLayout();
		setCompositionRoot(mainLayout);
		// TODO add user code here
	}
	
	public DetachedCriteria buildCriteria()
	{
		DetachedCriteria dc = DetachedCriteria.forClass(Resume.class,"r")
			.createCriteria("r.applications", "app", JoinFragment.LEFT_OUTER_JOIN)
			.createCriteria("app.job", "j", JoinFragment.LEFT_OUTER_JOIN)
			.createCriteria("app.process", "p", JoinFragment.LEFT_OUTER_JOIN);
		if(combo1.getValue()!=null)
			dc.add(Restrictions.eq("j.jobUuid", combo1.getValue().toString()));
		if(StringUtils.hasLength(tf2.getValue().toString()))
			dc.add(Restrictions.like("j.jobTitle", tf2.getValue().toString(), MatchMode.ANYWHERE));
		if(fromDate.getValue()!=null)
			dc.add(Restrictions.ge("app.applyDate", fromDate.getValue()));
		if(toDate.getValue()!=null)
			dc.add(Restrictions.le("app.applyDate", toDate.getValue()));
		if(combo2.getValue()!=null)
			dc.add(Restrictions.eq("p.processUuid", combo2.getValue().toString()));
		if(StringUtils.hasLength(tf1.getValue().toString()))
			dc.add(Restrictions.like("r.name", tf1.getValue().toString(), MatchMode.ANYWHERE));
		dc.addOrder(Order.desc("app.applyDate"));
		dc.setResultTransformer(CriteriaSpecification.DISTINCT_ROOT_ENTITY);
		return dc;
	}
		
	@PostConstruct
	public void init()
	{
		//initialize search ui
		combo1.setContainerDataSource(dao.getJobCtr(40));
		combo1.setItemCaptionPropertyId("jobTitle");
		combo1.setItemCaptionMode(AbstractSelect.ITEM_CAPTION_MODE_PROPERTY);
		combo1.setFilteringMode(Filtering.FILTERINGMODE_CONTAINS);
		
		combo2.setContainerDataSource(dao.getProcessCtr(20));
		combo2.setItemCaptionPropertyId("processName");
		combo2.setItemCaptionMode(AbstractSelect.ITEM_CAPTION_MODE_PROPERTY);
		combo2.setFilteringMode(Filtering.FILTERINGMODE_CONTAINS);
		
		DetachedCriteria dc = buildCriteria();
		List<Resume> result = dao.getRsumeByDC(dc, MAX_COUNT);
		ctr = new LazyBeanContainer<String,Resume>(Resume.class,null,null);
		ctr.setBeanIdProperty("resumeUuid");
//		ctr.addAll(result);
		table.setSelectable(true);
		table.setMultiSelect(false);
		table.setImmediate(true);
		table.setColumnHeader("name", "應徵人員");
		table.setColumnWidth("name", 50);
		table.setColumnHeader("gender", "性別");
		table.setColumnWidth("gender", 25);
		table.setColumnHeader("nid", "身分證號");
		table.setColumnWidth("nid", 80);
		table.setColumnHeader("birthday", "生日");
		table.setColumnWidth("birthday", 70);
		table.setColumnHeader("createDate", "建立日期");
		table.setColumnWidth("createDate", 70);
		table.setColumnHeader("applications", "應徵次數");
		table.setColumnWidth("applications", 60);
		table.setColumnHeader("isVerify", "是否認證");
		table.setColumnWidth("isVerify", 50);
		table.addGeneratedColumn("createDate", new DateColumnGenerator("yyyy/MM/dd"));
		table.addGeneratedColumn("birthday", new DateColumnGenerator("yyyy/MM/dd"));
		table.addGeneratedColumn("gender", new GenderColumnGenerator());
		table.addGeneratedColumn("applications", new AppCountColumnGenerator());
		table.setPageLength(30);
		table.setContainerDataSource(ctr);
		table.setVisibleColumns(new String[]{"name","applications","gender","nid","birthday","createDate","isVerify"});
		table.setRowHeaderMode(Table.ROW_HEADER_MODE_INDEX);
		table.setColumnExpandRatio("isVerify", 1.0f);
		table.addListener(new Property.ValueChangeListener()
		{
			private static final long serialVersionUID = 4523110264052715214L;
			public void valueChange(ValueChangeEvent event) 
			{
				String selected = (String) table.getValue();
				
				if(selected==null)
				{
					resumeBtn.setEnabled(false);
					removeBtn.setEnabled(false);
				}
				else
				{
					resumeBtn.setEnabled(true);
					removeBtn.setEnabled(true);
				}
			}
		});
		
		searchBtn.addListener(new ClickListener()
		{
			private static final long serialVersionUID = 5384262902530395296L;
			@Override
			public void buttonClick(ClickEvent event) 
			{
				DetachedCriteria dc = buildCriteria();
				List<Resume> result = dao.getResumeByDC(dc, MAX_COUNT);
				ctr.removeAllItems();
				ctr.addAll(result);
			}
		});
		
		resumeBtn.setEnabled(false);
		resumeBtn.addListener(new ClickListener()
		{
			private static final long serialVersionUID = 7436262902530396271L;
			@SuppressWarnings("unchecked")
			@Override
			public void buttonClick(ClickEvent event) 
			{
				Window main = event.getButton().getApplication().getMainWindow();
				LazyBeanItem<Resume> i = (LazyBeanItem<Resume>) table.getItem(table.getValue());
				ResumeEditWindow rewin = new ResumeEditWindow(i.getBean());
				rewin.setModal(true);
				main.addWindow(rewin);
			}
		});
		
		removeBtn.setEnabled(false);
		removeBtn.addListener(new ClickListener()
		{
			private static final long serialVersionUID = -5639801170771445147L;
			@Override
			public void buttonClick(ClickEvent event) 
			{
				final Window main = event.getButton().getApplication().getMainWindow();
				
				
				ConfirmDialog.show(main, "注意", "確定要刪除本應徵人員與所有紀錄?", "確定", "取消",
			        new ConfirmDialog.Listener() {
			            @SuppressWarnings("unchecked")
						public void onClose(ConfirmDialog dialog) {
			                if (dialog.isConfirmed()) 
			                {
			                	try 
			                	{
			                		LazyBeanItem<Resume> i = (LazyBeanItem<Resume>) table.getItem(table.getValue());
									dao.removeResumeAssociates(i.getBean());
									table.removeItem(table.getValue());
									main.showNotification("刪除應徵人員成功!", Notification.TYPE_HUMANIZED_MESSAGE);
								} 
			                	catch(Exception e) 
								{
			                		e.printStackTrace();
									main.showNotification("刪除失敗!請洽系統管理員", Notification.TYPE_ERROR_MESSAGE);
								}
			                } 
			            }
			        });
			}
			
		});
	}

	@AutoGenerated
	private VerticalLayout buildMainLayout() {
		// common part: create layout
		mainLayout = new VerticalLayout();
		mainLayout.setImmediate(false);
		mainLayout.setWidth("100%");
		mainLayout.setHeight("100%");
		mainLayout.setMargin(false);
		
		// top-level component properties
		setWidth("100.0%");
		setHeight("100.0%");
		
		// hl1
		hl1 = buildHl1();
		mainLayout.addComponent(hl1);
		
		// hl2
		hl2 = buildHl2();
		mainLayout.addComponent(hl2);
		
		// table
		table = new Table();
		table.setImmediate(false);
		table.setWidth("100.0%");
		table.setHeight("100.0%");
		mainLayout.addComponent(table);
		mainLayout.setExpandRatio(table, 1.0f);
		
		return mainLayout;
	}

	@AutoGenerated
	private HorizontalLayout buildHl1() {
		// common part: create layout
		hl1 = new HorizontalLayout();
		hl1.setImmediate(false);
		hl1.setWidth("-1px");
		hl1.setHeight("-1px");
		hl1.setMargin(false);
		hl1.setSpacing(true);
		
		// combo1
		combo1 = new ComboBox();
		combo1.setCaption("近期職缺選擇");
		combo1.setImmediate(false);
		combo1.setWidth("250px");
		combo1.setHeight("-1px");
		hl1.addComponent(combo1);
		
		// tf2
		tf2 = new TextField();
		tf2.setCaption("職缺名稱");
		tf2.setImmediate(false);
		tf2.setWidth("100px");
		tf2.setHeight("-1px");
		hl1.addComponent(tf2);
		
		// fromDate
		fromDate = new PopupDateField();
		fromDate.setCaption("送出日期(起)");
		fromDate.setImmediate(false);
		fromDate.setWidth("-1px");
		fromDate.setHeight("-1px");
		fromDate.setInvalidAllowed(false);
		fromDate.setResolution(4);
		hl1.addComponent(fromDate);
		
		// toDate
		toDate = new PopupDateField();
		toDate.setCaption("送出日期(迄)");
		toDate.setImmediate(false);
		toDate.setWidth("-1px");
		toDate.setHeight("-1px");
		toDate.setInvalidAllowed(false);
		toDate.setResolution(4);
		hl1.addComponent(toDate);
		
		// combo2
		combo2 = new ComboBox();
		combo2.setCaption("處理狀態");
		combo2.setImmediate(false);
		combo2.setWidth("100px");
		combo2.setHeight("-1px");
		hl1.addComponent(combo2);
		
		// tf1
		tf1 = new TextField();
		tf1.setCaption("姓名");
		tf1.setImmediate(false);
		tf1.setWidth("80px");
		tf1.setHeight("-1px");
		hl1.addComponent(tf1);
		
		// searchBtn
		searchBtn = new Button();
		searchBtn.setCaption("搜尋");
		searchBtn.setImmediate(true);
		searchBtn.setWidth("-1px");
		searchBtn.setHeight("-1px");
		hl1.addComponent(searchBtn);
		hl1.setComponentAlignment(searchBtn, new Alignment(9));
		
		return hl1;
	}

	@AutoGenerated
	private HorizontalLayout buildHl2() {
		// common part: create layout
		hl2 = new HorizontalLayout();
		hl2.setImmediate(false);
		hl2.setWidth("-1px");
		hl2.setHeight("-1px");
		hl2.setMargin(false);
		
		// resumeBtn
		resumeBtn = new Button();
		resumeBtn.setCaption("履歷內容");
		resumeBtn.setImmediate(true);
		resumeBtn.setWidth("-1px");
		resumeBtn.setHeight("-1px");
		hl2.addComponent(resumeBtn);

		
		// removeBtn
		removeBtn = new Button();
		removeBtn.setCaption("刪除應徵紀錄");
		removeBtn.setImmediate(true);
		removeBtn.setWidth("-1px");
		removeBtn.setHeight("-1px");
		hl2.addComponent(removeBtn);
		
		return hl2;
	}
}

Finally, I think I found the problem. It’s the Tabsheet which will not release its sub components (mostly tables).
The following is the logic of my logout button. I added “tabsheet.removeAllComponents()” just right before closing the application.
Voila! All MethodProperty instances are gone! No more memory leaks. Is it a bug?


logoutBtn.addListener(new ClickListener(){
			private static final long serialVersionUID = 3441527571087478137L;

			@Override
			public void buttonClick(ClickEvent event) 
			{
				tabsheet.removeAllComponents();  //IMPORTANT!
				ContextApplication.currentApplication().setLogoutURL(ContextApplication.currentApplication().getURL().toExternalForm());
				ContextApplication.currentApplication().close();
				ContextApplication.currentRequest().getSession().invalidate(); 					
			}});

Then the question is: what is keeping the TabSheet in memory? Otherwise, its subcomponents etc. should get garbage collected automatically.

That might be good to check with a memory profiler if possible even if you already managed to substantially limit the leak.

Henri,

My application is simple. A Window with a Tabsheet which contains 4 CustomComponents.
Each CustomComponent has a table with some textfields and buttons. That’s it!
It looks like the Table on a CustomComponent will not get GCed from its parent TabSheet.

You can use this guy to find the root of you problem: https://www.eclipse.org/mat/

I came here after days struggling with a similar problem to Po-ting’s one.

I’m experiencing a huge memory leak on my Linux production server which ends up consuming all the memory (4 GB) very quickly and usually I have to restart Tomcat after a day or so. Trying to track down the problem I’ve done some memory profiling, apart from noticing that on my development Windows machine all the memory consumption is located after entering some specifing screen from my application, and never being freed up after abandoning it.

My app is Tab based, so when I enter the screen I’m talking about, all components get (re)created and put inside a new tab.
This tab is displaying a Grid with elements, and, guess what… the Grid is using a BeanItemContainer. I implemented some nullifying of the list of objects which feeds the container, and I currently also nullify the full component that is put inside that Tab sheet which is holding all the components (apart from the Grid there are some controls) when the Tab detach event is detected. Nothing has helped releasing any memory after the tab is closed or even I log off from my app and close the browser tab that is holding it. I’ve even put some System.gc() calls after nullifying in order to try to “force” some garbage collection. Nothing.

The problem is greatly exacerbated due to the fact that my Grid is being filled up with tens of hundreds of elements. Currently between 50,000 and 80,000 or so.

As with Po-ting’s report, my memory profiler suddenly puts com.vaading.data.util.MethodProperty on top of the memory profiler “Live Bytes” and “Live Objects” list. I have plenty other screens following the same schema, the only difference being that I use other kind of containers (SQLContainer and or in-memory), and I don’t see this memory-related behaviour with them, so after finding this topic directly after googling with the proper word list combination, I’m almost sure it’s Grid/com.vaadin.data.util.MethodProperty+TabSheet related.

I haven’t still experimented with Po-ting’s solution, as I don’t need to free memory just after loging out from my app but after closing the offending tab, so I cannot use TabSheet.removeAllComponents() or I’d close all app’s tabs. I’m going to try with just .removeComponent() (I’m writing this as I’m doing my findings).

Anyway, I’m aware that this probably won’t get much attention from Vaadin staff due to containers having been deprecated, but again it’s such a horrible vehaviour to take notice of…

Thanks a lot for helping, Denis.

I just tried your UI.close() suggestion after realizing that any TabSheet component removement is unsuccessfull. Previously I even attempted to nullify the UI on the detach event for the main UI (without any success).

But no, doing an UI.close(); doesn’t release any memory, even though I see the code is executed when I do a trace.

Oh, by the way, I see that you ¿edited? your reply while trying to post my own reply, as I though I had posted it 30 minutes ago, but I didn’t due to these forums ·)$%()·$"%( bugs.

I know of Mat, but instead I’m using Netbeans’ built-in memory profiler for the moment. But maybe I need to look into it. Thanks a lot.

yes forum a bit buggy. Please post the problem cause. Mat is quit handy, After ui is closed and session is invalidated all must be gone, its some strange issue :slight_smile:

main.addWindow(rewin);

this can cause your problem

Sorry Denis, I don’t understand what you mean with that.

Anyway, after messing up with Netbeans’ built-in memory profiler, which in my opinion works very well, I was able to trace some memory leak sources and greatly diminish any voracious depletion. At least that seems so while profiling locally. Heap size is less prone to lower once it has grown, but I see the “used heap” memory keeps very low in my local tests after finishing operations with the screen that makes use of my 80,000 row Grid or after closing it (I can see how memory is freed).

I cannot spot the difference between heap size and used heap with a simply “top” command on my production server, but once it grows until 50% memory or so, it seems to keep stable, so I’m confident that I’ve hit the spot.

UPDATE: ---------

After a quick googling, I found a couple command line tools that worked very well to deeply look into java memory usage, being able to discern between HeapSize and HeapUsed, thanks to this StackOverflow thread:

https://stackoverflow.com/questions/12797560/command-line-tool-to-find-java-heap-size-and-memory-used-linux

I recommend jvmtop and, specially, jvm-mon.

It seems that garbage collection is efectively working on my production server :slight_smile:


Many thanks for helping!

you can use for example https://prometheus.io/ with java agent in order to monitor your server in production. with https://grafana.com/ you can easly visualize your metrics, and predict problems, and check the difference with passed days.

it depends on which GarbageCollector you use heap will behave differently.

PS i pointed to window because memory leaks is made like this you register somewhere reference to you object via anonymous class and its not GC because some objects may be like this Window = global and exist until user closed browser tab etc. (I am not quit sure how vaadin Window is bounded to UI thats why i said it could be a problem.)

Hi Denis and everyone,

Well, my memory leak problems still exist. They are well located; apart from being suspicious from the beginning, been playing with Eclipse Mat during last days and it goes down to the Grid with 50,000+ elements with a BeanItemContainer. That’s consistent with what I was already noticing by simply lookint at top command (Linux) or task admin (Windows) by simply entering and exiting the screen in my app that creates and displays that grid.

The think is that the manual nullification of the object list I use to populate the grid, and the manual System.gc() garbage collector calls I’m now doing manually (for instance, when I close the tab which holds the grid) seemed to work to an extent to mitigate the problem - but eventually the app will run out of memory because they don’t seem to work always.

It’s very strange because memory won’t get freed even after logging out from the app and even closing the implied browser tab.

I see lots of references ot objects, methods and all while playing with Mat which show a clear idea of the problem, and it’s probably due because there may be some cycling references which I cannot identify that don’t get clean.

I’m attaching one of the thousands of screens I’ve been browsing through. Maybe someone can give me some hint on how to use this tool to get closer to the code that is causing the leak…
42107.png

Manually nulling objects is neat but not neccessary. Same is for manual call GC. Its
always a ROOT object
which holds reference to those BeanContainers.

How big is your dumb? I can take a look.

Thanks for your offer, Denis, but I cannot share the dump. Appart from being hugely sized (about ~2 GB), it’s filled with sensible data which belongs to my company.

Anyway, although my app keeps growing and growing its memory consumption, I even have my doubts that this is really a memory leak. I’m attaching another capture of Mat after analizing a memory dump I took an hour ago or so. Memory heap size is 2 GB, and was about 50% used at that moment. The memory consumption of the suspicious leak is 270 MB - which is consistent with the size of the suspicious screen I told you before. But it could be also that one user is on that screen so that could be perfectly normal. In fact I would expect that if there were really a problem this memory compsution would by 4x or 6x at least.

Could this in fact be due to normal memory consumption? Another thing that I noticed is that when memory consumption reaches its critical point (which belongs to about 70% memory use by the process when issuing a top command) and the app hangs, it’s CPU usage jumps from very low (between 1% and less than 10%) to nearly 200% and becomes totally unresponsive, maybe displaying the “connection lost” message (but sometimes failing silently, simply not responding neither updating displayed info on screen), but I’m not getting any “OutOfMemory” exceptions…
42302.png

Hi,

you can make dump from local pc with some generated data. but never mind. you are describing normal behaviour when memory is full. 70% is a threshold which id configurated for you GC. After this threshold is reached GC is constantly trying to free memory. Freeing memory is CPU cost operation. After it cant free anything it retries, so you got CPU getting very busy.

2 things

  1. What is on MAT is you DTOs which you are loading from DB. I dono what happens but seems only one user can burn your memory just sliding enough pages. You need is to make sure those DTOs which are not used anymore are exposed, so when user switch to next page of DTOs previous are gone, and you data provider has only current page of dtos.

  2. you can do is to install software local and run with one user, close session and check if things are disposed. Manually run GC and check if memory drops. If not make a dump and find root object where all this things are hanging. Your DataContainer hangs on something you need to understand why its happening.

PS 2gigs of memory dump is not rly so big belive me :))))