2009-03 - Memory not released in SoftRefFilesCache...

Info iconThis preview shows pages 1–3. Sign up to view the full content.

View Full Document Right Arrow Icon

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: Memory not released in SoftRefFilesCache Implementation of method putFile(final FileObject file) does not remove references from refReverseMap when adding a new file. #current implementation {noformat} synchronized (files) { files.put(file.getName(), ref); synchronized(refReverseMap) { refReverseMap.put(ref, key); } } {noformat} #should become: {noformat} synchronized (files) { Reference old = files.put(file.getName(), ref); synchronized(refReverseMap) { refReverseMap.remove(old); refReverseMap.put(ref, key); } } {noformat} HTTP only allows reading from one file at a time VFS 164 modified HttpClientFactory to use a single connection per thread. The consequence of this is that only a single file can be accessed at a time. Several applications, such as Commons Configuration and XML includes will read a second file while processing the first. In the case of Commons Configuration an IOException is being thrown when the first file is closed because it was already closed by ThreadLocalHttpConnectionManager. Memory leaks in DIH If delta-import is executed many times, the heap utilization grows up and finally OutOfMemoryError occurs. When delta-import is executed with SqlEntityProcessor, the instances of TemplateString cached in VariableResolverImpl#TEMPLATE_STRING#cache. If the deltaQuery contains variable like `last_index_time', the cached values never used increases. Similarly, I guess that the cache increases when fetching each modified row with primary key. I think these queries should not be cached. I came up with two solutions: 1) Not to cache queries to get modified rows. 2) Make VariableResolverImpl#TEMPLATE_STRING non-static. Or clear cache on finishing delta-import. I think that #1 is better for performance than #2, but #2 is easier to solve the problem. I made a patch in #2 way, and then tested two solr applications with `-XX: +PrintClassHistgram' option. The result after importing several million rows from a MySQL database is as follows: * original solr-1.3: num #instances #bytes class name----------------------------------------------... 6: 2983024 119320960 org.apache.solr.handler.dataimport.TemplateString ... * patched solr-1.3: num #instances #bytes class name----------------------------------------------... 748: 3 120 org.apache.solr.handler.dataimport.TemplateString ... Though it is version 1.3 that I tested, perhaps current nightly version has same problem. Farm deployment of configurations using JNDI resource references does not work Due to the transformation of the name of a configuration when it is distributed to a server farm (i.e. the _G_SLAVE suffix is appended) JNDI resource references can not be resolved. Here is an user provided stack trace which illustrates this problem: {code} java.lang.IllegalStateException: No configuration found for id: clustering/clustering/2.1/war at org.apache.geronimo.naming.reference.AbstractEntryFactory.getConfiguration(Abstr actEntryFactory.java:110) at org.apache.geronimo.naming.reference.AbstractEntryFactory.resolveTargetName(Abst org....
View Full Document

This document was uploaded on 10/12/2012.

Page1 / 285

2009-03 - Memory not released in SoftRefFilesCache...

This preview shows document pages 1 - 3. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online