Too many open file Error - Alfresco community 4.0

cancel
Showing results for 
Search instead for 
Did you mean: 
sanketmohite
Active Member

Too many open file Error - Alfresco community 4.0

Hi All,

We are running Alfresco community version 4.0 on Linux environment.

Since last few days we are regularly getting too many open files error in alfresco.log file.(snapshot of log file is pasted in the message below)

We tried tuning tomcat "nofile" parameter and maximum possible 65536

We suspect that it might be able to some customization but not able to pin point to any so far

Any pointers/suggestions to analyze issue will be highly appreciated as we our alfresco is getting filled due to this error

Also if you can suggest on following

1. How can we identify which files are being open?

2. Can we prevent the reindexing which might be causing this error?

12:10:55,019 ERROR [org.quartz.core.JobRunShell] Job DEFAULT.org.springframework.scheduling.quartz.JobDetailBean#5570984c threw an unhandled Exception:
org.alfresco.error.AlfrescoRuntimeException: 0017151442 Reindex failure for org.alfresco.repo.node.index.AVMRemoteSnapshotTracker
 at org.alfresco.repo.node.index.AbstractReindexComponent.reindex(AbstractReindexComponent.java:315)
 at org.alfresco.repo.node.index.IndexRecoveryJob.execute(IndexRecoveryJob.java:59)
 at org.quartz.core.JobRunShell.run(JobRunShell.java:216)
 at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:549)
Caused by: org.alfresco.repo.search.impl.lucene.LuceneIndexException: 0017151441 Failed to flush index
 at org.alfresco.repo.search.impl.lucene.AbstractLuceneIndexerImpl.flushPending(AbstractLuceneIndexerImpl.java:934)
 at org.alfresco.repo.search.impl.lucene.AVMLuceneIndexerImpl.getLastAsynchronousSnapshot(AVMLuceneIndexerImpl.java:2013)
 at org.alfresco.repo.search.impl.lucene.AVMLuceneIndexerImpl.getLastAsynchronousSnapshot(AVMLuceneIndexerImpl.java:1991)
 at org.alfresco.repo.search.impl.lucene.AVMLuceneIndexerImpl.getLastIndexedSnapshot(AVMLuceneIndexerImpl.java:1867)
 at org.alfresco.repo.search.AVMSnapShotTriggeredIndexingMethodInterceptor.getLastIndexedSnapshot(AVMSnapShotTriggeredIndexingMethodInterceptor.java:316)
 at org.alfresco.repo.search.AVMSnapShotTriggeredIndexingMethodInterceptor.getLastIndexedSnapshot(AVMSnapShotTriggeredIndexingMethodInterceptor.java:309)
 at org.alfresco.repo.node.index.AVMRemoteSnapshotTracker.processStores(AVMRemoteSnapshotTracker.java:87)
 at org.alfresco.repo.node.index.AVMRemoteSnapshotTracker.reindexImpl(AVMRemoteSnapshotTracker.java:58)
 at org.alfresco.repo.node.index.AbstractReindexComponent$1.execute(AbstractReindexComponent.java:300)
 at org.alfresco.repo.transaction.RetryingTransactionHelper.doInTransaction(RetryingTransactionHelper.java:386)
 at org.alfresco.repo.transaction.RetryingTransactionHelper.doInTransaction(RetryingTransactionHelper.java:276)
 at org.alfresco.repo.node.index.AbstractReindexComponent.reindex(AbstractReindexComponent.java:306)
 ... 3 more
Caused by: java.io.FileNotFoundException: /CMSIndex/lucene-indexes/avm/sitestore/4ba77f5d-ce9c-4bc1-b162-2fb05fb9d1de/segments_1 (Too many open files)
 at java.io.RandomAccessFile.open(Native Method)
 at java.io.RandomAccessFile.<init>(RandomAccessFile.java:216)
 at org.apache.lucene.store.FSDirectory$FSIndexOutput.<init>(FSDirectory.java:682)
 at org.apache.lucene.store.FSDirectory.createOutput(FSDirectory.java:443)
 at org.apache.lucene.index.SegmentInfos.write(SegmentInfos.java:296)
 at org.apache.lucene.index.SegmentInfos.prepareCommit(SegmentInfos.java:761)
 at org.apache.lucene.index.SegmentInfos.commit(SegmentInfos.java:825)
 at org.apache.lucene.index.IndexWriter.init(IndexWriter.java:1086)
 at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:626)
 at org.alfresco.repo.search.impl.lucene.index.IndexInfo.makeDeltaIndexWriter(IndexInfo.java:964)
 at org.alfresco.repo.search.impl.lucene.index.IndexInfo.getDeltaIndexWriter(IndexInfo.java:1007)
 at org.alfresco.repo.search.impl.lucene.AbstractLuceneBase.getDeltaWriter(AbstractLuceneBase.java:223)
 at org.alfresco.repo.search.impl.lucene.AbstractLuceneIndexerImpl.flushPending(AbstractLuceneIndexerImpl.java:928)
 ... 14 more
12:10:55,019 ERROR [org.quartz.core.ErrorLogger] Job (DEFAULT.org.springframework.scheduling.quartz.JobDetailBean#5570984c threw an exception.
org.quartz.SchedulerException: Job threw an unhandled exception. [See nested exception: org.alfresco.error.AlfrescoRuntimeException: 0017151442 Reindex failure for org.alfresco.repo.node.index.AVMRemoteSnapshotTracker]
 at org.quartz.core.JobRunShell.run(JobRunShell.java:227)
 at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:549)
Caused by: org.alfresco.error.AlfrescoRuntimeException: 0017151442 Reindex failure for org.alfresco.repo.node.index.AVMRemoteSnapshotTracker
 at org.alfresco.repo.node.index.AbstractReindexComponent.reindex(AbstractReindexComponent.java:315)
 at org.alfresco.repo.node.index.IndexRecoveryJob.execute(IndexRecoveryJob.java:59)
 at org.quartz.core.JobRunShell.run(JobRunShell.java:216)
 ... 1 more
Caused by: org.alfresco.repo.search.impl.lucene.LuceneIndexException: 0017151441 Failed to flush index
 at org.alfresco.repo.search.impl.lucene.AbstractLuceneIndexerImpl.flushPending(AbstractLuceneIndexerImpl.java:934)
 at org.alfresco.repo.search.impl.lucene.AVMLuceneIndexerImpl.getLastAsynchronousSnapshot(AVMLuceneIndexerImpl.java:2013)
 at org.alfresco.repo.search.impl.lucene.AVMLuceneIndexerImpl.getLastAsynchronousSnapshot(AVMLuceneIndexerImpl.java:1991)
 at org.alfresco.repo.search.impl.lucene.AVMLuceneIndexerImpl.getLastIndexedSnapshot(AVMLuceneIndexerImpl.java:1867)
 at org.alfresco.repo.search.AVMSnapShotTriggeredIndexingMethodInterceptor.getLastIndexedSnapshot(AVMSnapShotTriggeredIndexingMethodInterceptor.java:316)
 at org.alfresco.repo.search.AVMSnapShotTriggeredIndexingMethodInterceptor.getLastIndexedSnapshot(AVMSnapShotTriggeredIndexingMethodInterceptor.java:309)
 at org.alfresco.repo.node.index.AVMRemoteSnapshotTracker.processStores(AVMRemoteSnapshotTracker.java:87)
 at org.alfresco.repo.node.index.AVMRemoteSnapshotTracker.reindexImpl(AVMRemoteSnapshotTracker.java:58)
 at org.alfresco.repo.node.index.AbstractReindexComponent$1.execute(AbstractReindexComponent.java:300)
 at org.alfresco.repo.transaction.RetryingTransactionHelper.doInTransaction(RetryingTransactionHelper.java:386)
 at org.alfresco.repo.transaction.RetryingTransactionHelper.doInTransaction(RetryingTransactionHelper.java:276)
 at org.alfresco.repo.node.index.AbstractReindexComponent.reindex(AbstractReindexComponent.java:306)
 ... 3 more
Caused by: java.io.FileNotFoundException: /CMSIndex/lucene-indexes/avm/sitestore/4ba77f5d-ce9c-4bc1-b162-2fb05fb9d1de/segments_1 (Too many open files)
 at java.io.RandomAccessFile.open(Native Method)
 at java.io.RandomAccessFile.<init>(RandomAccessFile.java:216)
 at org.apache.lucene.store.FSDirectory$FSIndexOutput.<init>(FSDirectory.java:682)
 at org.apache.lucene.store.FSDirectory.createOutput(FSDirectory.java:443)
 at org.apache.lucene.index.SegmentInfos.write(SegmentInfos.java:296)
 at org.apache.lucene.index.SegmentInfos.prepareCommit(SegmentInfos.java:761)
 at org.apache.lucene.index.SegmentInfos.commit(SegmentInfos.java:825)
 at org.apache.lucene.index.IndexWriter.init(IndexWriter.java:1086)
 at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:626)
 at org.alfresco.repo.search.impl.lucene.index.IndexInfo.makeDeltaIndexWriter(IndexInfo.java:964)
 at org.alfresco.repo.search.impl.lucene.index.IndexInfo.getDeltaIndexWriter(IndexInfo.java:1007)
 at org.alfresco.repo.search.impl.lucene.AbstractLuceneBase.getDeltaWriter(AbstractLuceneBase.java:223)
 at org.alfresco.repo.search.impl.lucene.AbstractLuceneIndexerImpl.flushPending(AbstractLuceneIndexerImpl.java:928)
 ... 14 more
12:10:55,692 ERROR [org.alfresco.repo.search.impl.lucene.index.IndexInfo] Failed building filter reader beneath d2e6b2e4-a5e0-4541-9c25-3248e3bcf24a
java.io.FileNotFoundException: /CMSIndex/lucene-indexes/workspace/SpacesStore/d2e6b2e4-a5e0-4541-9c25-3248e3bcf24a/IndexInfoDeletions (Too many open files)
 at java.io.FileInputStream.open(Native Method)
 at java.io.FileInputStream.<init>(FileInputStream.java:120)
 at org.alfresco.repo.search.impl.lucene.index.IndexInfo.getDeletions(IndexInfo.java:1098)
 at org.alfresco.repo.search.impl.lucene.index.IndexInfo.createMainIndexReader(IndexInfo.java:2099)
 at org.alfresco.repo.search.impl.lucene.index.IndexInfo.getMainIndexReferenceCountingReadOnlyIndexReader(IndexInfo.java:1227)
 at org.alfresco.repo.search.impl.lucene.AbstractLuceneBase.getSearcher(AbstractLuceneBase.java:168)
 at org.alfresco.repo.search.impl.lucene.ADMLuceneSearcherImpl.query(ADMLuceneSearcherImpl.java:287)
 at org.alfresco.repo.search.impl.lucene.ADMLuceneSearcherImpl.query(ADMLuceneSearcherImpl.java:242)
 at org.alfresco.repo.search.SearcherComponent.query(SearcherComponent.java:67)
 at org.alfresco.repo.search.AbstractSearcherComponent.query(AbstractSearcherComponent.java:53)
 at sun.reflect.GeneratedMethodAccessor706.invoke(Unknown Source)
 at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
 at java.lang.reflect.Method.invoke(Method.java:597)
 at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:307)
 at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
 at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
 at net.sf.acegisecurity.intercept.method.aopalliance.MethodSecurityInterceptor.invoke(MethodSecurityInterceptor.java:80)
 at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
 at org.alfresco.repo.security.permissions.impl.ExceptionTranslatorMethodInterceptor.invoke(ExceptionTranslatorMethodInterceptor.java:44)
 at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
 at org.alfresco.repo.audit.AuditMethodInterceptor.invoke(AuditMethodInterceptor.java:146)
 at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
 at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:107)
 at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
 at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:202)
 at com.sun.proxy.$Proxy34.query(Unknown Source)
 at org.alfresco.repo.web.scripts.RepoStore$3$1.execute(RepoStore.java:397)
 at org.alfresco.repo.web.scripts.RepoStore$3$1.execute(RepoStore.java:391)
 at org.alfresco.repo.transaction.RetryingTransactionHelper.doInTransaction(RetryingTransactionHelper.java:386)
 at org.alfresco.repo.web.scripts.RepoStore$3.doWork(RepoStore.java:390)
 at org.alfresco.repo.web.scripts.RepoStore$3.doWork(RepoStore.java:387)
 at org.alfresco.repo.security.authentication.AuthenticationUtil.runAs(AuthenticationUtil.java:508)
 at org.alfresco.repo.web.scripts.RepoStore.getDocumentPaths(RepoStore.java:386)
 at org.springframework.extensions.webscripts.DeclarativeRegistry.initWebScripts(DeclarativeRegistry.java:223)
 at org.springframework.extensions.webscripts.DeclarativeRegistry.reset(DeclarativeRegistry.java:178)
 at org.alfresco.repo.web.scripts.RepositoryContainer.getRegistry(RepositoryContainer.java:496)
 at org.springframework.extensions.webscripts.AbstractRuntime.executeScript(AbstractRuntime.java:143)
 at org.springframework.extensions.webscripts.servlet.WebScriptServlet.service(WebScriptServlet.java:118)
 at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
 at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
 at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
 at org.alfresco.web.app.servlet.GlobalLocalizationFilter.doFilter(GlobalLocalizationFilter.java:58)
 at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
 at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
 at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
 at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
 at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
 at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
 at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
 at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
 at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:859)
 at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588)
 at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489)
 at java.lang.Thread.run(Thread.java:662)
12:10:55,693 ERROR [org.alfresco.repo.search.impl.lucene.AbstractLuceneBase] Error
java.io.FileNotFoundException: /CMSIndex/lucene-indexes/workspace/SpacesStore/d2e6b2e4-a5e0-4541-9c25-3248e3bcf24a/IndexInfoDeletions (Too many open files)
 at java.io.FileInputStream.open(Native Method)
 at java.io.FileInputStream.<init>(FileInputStream.java:120)
 at org.alfresco.repo.search.impl.lucene.index.IndexInfo.getDeletions(IndexInfo.java:1098)
 at org.alfresco.repo.search.impl.lucene.index.IndexInfo.createMainIndexReader(IndexInfo.java:2099)
 at org.alfresco.repo.search.impl.lucene.index.IndexInfo.getMainIndexReferenceCountingReadOnlyIndexReader(IndexInfo.java:1227)
 at org.alfresco.repo.search.impl.lucene.AbstractLuceneBase.getSearcher(AbstractLuceneBase.java:168)
 at org.alfresco.repo.search.impl.lucene.ADMLuceneSearcherImpl.query(ADMLuceneSearcherImpl.java:287)
 at org.alfresco.repo.search.impl.lucene.ADMLuceneSearcherImpl.query(ADMLuceneSearcherImpl.java:242)
 at org.alfresco.repo.search.SearcherComponent.query(SearcherComponent.java:67)
 at org.alfresco.repo.search.AbstractSearcherComponent.query(AbstractSearcherComponent.java:53)
 at sun.reflect.GeneratedMethodAccessor706.invoke(Unknown Source)
 at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
 at java.lang.reflect.Method.invoke(Method.java:597)
 at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:307)
 at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
 at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
 at net.sf.acegisecurity.intercept.method.aopalliance.MethodSecurityInterceptor.invoke(MethodSecurityInterceptor.java:80)
 at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
 at org.alfresco.repo.security.permissions.impl.ExceptionTranslatorMethodInterceptor.invoke(ExceptionTranslatorMethodInterceptor.java:44)
 at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
 at org.alfresco.repo.audit.AuditMethodInterceptor.invoke(AuditMethodInterceptor.java:146)
 at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
 at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:107)
 at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
 at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:202)
 at com.sun.proxy.$Proxy34.query(Unknown Source)
 at org.alfresco.repo.web.scripts.RepoStore$3$1.execute(RepoStore.java:397)
 at org.alfresco.repo.web.scripts.RepoStore$3$1.execute(RepoStore.java:391)
 at org.alfresco.repo.transaction.RetryingTransactionHelper.doInTransaction(RetryingTransactionHelper.java:386)
 at org.alfresco.repo.web.scripts.RepoStore$3.doWork(RepoStore.java:390)
 at org.alfresco.repo.web.scripts.RepoStore$3.doWork(RepoStore.java:387)
 at org.alfresco.repo.security.authentication.AuthenticationUtil.runAs(AuthenticationUtil.java:508)
 at org.alfresco.repo.web.scripts.RepoStore.getDocumentPaths(RepoStore.java:386)
 at org.springframework.extensions.webscripts.DeclarativeRegistry.initWebScripts(DeclarativeRegistry.java:223)
 at org.springframework.extensions.webscripts.DeclarativeRegistry.reset(DeclarativeRegistry.java:178)
 at org.alfresco.repo.web.scripts.RepositoryContainer.getRegistry(RepositoryContainer.java:496)
 at org.springframework.extensions.webscripts.AbstractRuntime.executeScript(AbstractRuntime.java:143)
 at org.springframework.extensions.webscripts.servlet.WebScriptServlet.service(WebScriptServlet.java:118)
 at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
 at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
 at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
 at org.alfresco.web.app.servlet.GlobalLocalizationFilter.doFilter(GlobalLocalizationFilter.java:58)
 at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
 at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
 at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
 at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
 at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
 at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
 at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
 at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
 at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:859)
 at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588)
 at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489)
 at java.lang.Thread.run(Thread.java:662)
12:10:55,693 ERROR [org.springframework.extensions.webscripts.AbstractRuntime] Exception from executeScript - redirecting to status template error: 0017151443 Failed to open IndexSarcher for /CMSIndex/lucene-indexes/workspace/SpacesStore/
org.alfresco.repo.search.impl.lucene.LuceneIndexException: 0017151443 Failed to open IndexSarcher for /CMSIndex/lucene-indexes/workspace/SpacesStore/
 at org.alfresco.repo.search.impl.lucene.AbstractLuceneBase.getSearcher(AbstractLuceneBase.java:184)
 at org.alfresco.repo.search.impl.lucene.ADMLuceneSearcherImpl.query(ADMLuceneSearcherImpl.java:287)
 at org.alfresco.repo.search.impl.lucene.ADMLuceneSearcherImpl.query(ADMLuceneSearcherImpl.java:242)
 at org.alfresco.repo.search.SearcherComponent.query(SearcherComponent.java:67)
 at org.alfresco.repo.search.AbstractSearcherComponent.query(AbstractSearcherComponent.java:53)
 at sun.reflect.GeneratedMethodAccessor706.invoke(Unknown Source)
 at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
 at java.lang.reflect.Method.invoke(Method.java:597)
 at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:307)
 at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
 at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
 at net.sf.acegisecurity.intercept.method.aopalliance.MethodSecurityInterceptor.invoke(MethodSecurityInterceptor.java:80)
 at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
 at org.alfresco.repo.security.permissions.impl.ExceptionTranslatorMethodInterceptor.invoke(ExceptionTranslatorMethodInterceptor.java:44)
 at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
 at org.alfresco.repo.audit.AuditMethodInterceptor.invoke(AuditMethodInterceptor.java:146)
 at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
 at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:107)
 at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
 at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:202)
 at com.sun.proxy.$Proxy34.query(Unknown Source)
 at org.alfresco.repo.web.scripts.RepoStore$3$1.execute(RepoStore.java:397)
 at org.alfresco.repo.web.scripts.RepoStore$3$1.execute(RepoStore.java:391)
 at org.alfresco.repo.transaction.RetryingTransactionHelper.doInTransaction(RetryingTransactionHelper.java:386)
 at org.alfresco.repo.web.scripts.RepoStore$3.doWork(RepoStore.java:390)
 at org.alfresco.repo.web.scripts.RepoStore$3.doWork(RepoStore.java:387)
 at org.alfresco.repo.security.authentication.AuthenticationUtil.runAs(AuthenticationUtil.java:508)
 at org.alfresco.repo.web.scripts.RepoStore.getDocumentPaths(RepoStore.java:386)
 at org.springframework.extensions.webscripts.DeclarativeRegistry.initWebScripts(DeclarativeRegistry.java:223)
 at org.springframework.extensions.webscripts.DeclarativeRegistry.reset(DeclarativeRegistry.java:178)
 at org.alfresco.repo.web.scripts.RepositoryContainer.getRegistry(RepositoryContainer.java:496)
 at org.springframework.extensions.webscripts.AbstractRuntime.executeScript(AbstractRuntime.java:143)
 at org.springframework.extensions.webscripts.servlet.WebScriptServlet.service(WebScriptServlet.java:118)
 at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
 at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
 at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
 at org.alfresco.web.app.servlet.GlobalLocalizationFilter.doFilter(GlobalLocalizationFilter.java:58)
 at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
 at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
 at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
 at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
 at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
 at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
 at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
 at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
 at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:859)
 at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588)
 at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489)
 at java.lang.Thread.run(Thread.java:662)
Caused by: java.io.FileNotFoundException: /CMSIndex/lucene-indexes/workspace/SpacesStore/d2e6b2e4-a5e0-4541-9c25-3248e3bcf24a/IndexInfoDeletions (Too many open files)
 at java.io.FileInputStream.open(Native Method)
 at java.io.FileInputStream.<init>(FileInputStream.java:120)
 at org.alfresco.repo.search.impl.lucene.index.IndexInfo.getDeletions(IndexInfo.java:1098)
 at org.alfresco.repo.search.impl.lucene.index.IndexInfo.createMainIndexReader(IndexInfo.java:2099)
 at org.alfresco.repo.search.impl.lucene.index.IndexInfo.getMainIndexReferenceCountingReadOnlyIndexReader(IndexInfo.java:1227)
 at org.alfresco.repo.search.impl.lucene.AbstractLuceneBase.getSearcher(AbstractLuceneBase.java:168)
 ... 48 more
12:11:00,023 ERROR [org.quartz.core.JobRunShell] Job DEFAULT.org.springframework.scheduling.quartz.JobDetailBean#5570984c threw an unhandled Exception:
org.alfresco.error.AlfrescoRuntimeException: 0017151445 Reindex failure for org.alfresco.repo.node.index.AVMRemoteSnapshotTracker
 at org.alfresco.repo.node.index.AbstractReindexComponent.reindex(AbstractReindexComponent.java:315)
 at org.alfresco.repo.node.index.IndexRecoveryJob.execute(IndexRecoveryJob.java:59)
 at org.quartz.core.JobRunShell.run(JobRunShell.java:216)
 at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:549)
Caused by: org.alfresco.repo.search.impl.lucene.LuceneIndexException: 0017151444 Failed to flush index
 at org.alfresco.repo.search.impl.lucene.AbstractLuceneIndexerImpl.flushPending(AbstractLuceneIndexerImpl.java:934)
 at org.alfresco.repo.search.impl.lucene.AVMLuceneIndexerImpl.getLastAsynchronousSnapshot(AVMLuceneIndexerImpl.java:2013)
 at org.alfresco.repo.search.impl.lucene.AVMLuceneIndexerImpl.getLastAsynchronousSnapshot(AVMLuceneIndexerImpl.java:1991)
 at org.alfresco.repo.search.impl.lucene.AVMLuceneIndexerImpl.getLastIndexedSnapshot(AVMLuceneIndexerImpl.java:1867)
 at org.alfresco.repo.search.AVMSnapShotTriggeredIndexingMethodInterceptor.getLastIndexedSnapshot(AVMSnapShotTriggeredIndexingMethodInterceptor.java:316)
 at org.alfresco.repo.search.AVMSnapShotTriggeredIndexingMethodInterceptor.getLastIndexedSnapshot(AVMSnapShotTriggeredIndexingMethodInterceptor.java:309)
 at org.alfresco.repo.node.index.AVMRemoteSnapshotTracker.processStores(AVMRemoteSnapshotTracker.java:87)
 at org.alfresco.repo.node.index.AVMRemoteSnapshotTracker.reindexImpl(AVMRemoteSnapshotTracker.java:58)
 at org.alfresco.repo.node.index.AbstractReindexComponent$1.execute(AbstractReindexComponent.java:300)
 at org.alfresco.repo.transaction.RetryingTransactionHelper.doInTransaction(RetryingTransactionHelper.java:386)
 at org.alfresco.repo.transaction.RetryingTransactionHelper.doInTransaction(RetryingTransactionHelper.java:276)
 at org.alfresco.repo.node.index.AbstractReindexComponent.reindex(AbstractReindexComponent.java:306)
 ... 3 more
Caused by: java.io.FileNotFoundException: /CMSIndex/lucene-indexes/avm/sitestore/75ca677a-2c28-479e-abdd-766087d8ae77/segments_1 (Too many open files)
 at java.io.RandomAccessFile.open(Native Method)
 at java.io.RandomAccessFile.<init>(RandomAccessFile.java:216)
 at org.apache.lucene.store.FSDirectory$FSIndexOutput.<init>(FSDirectory.java:682)
 at org.apache.lucene.store.FSDirectory.createOutput(FSDirectory.java:443)
 at org.apache.lucene.index.SegmentInfos.write(SegmentInfos.java:296)
 at org.apache.lucene.index.SegmentInfos.prepareCommit(SegmentInfos.java:761)
 at org.apache.lucene.index.SegmentInfos.commit(SegmentInfos.java:825)
 at org.apache.lucene.index.IndexWriter.init(IndexWriter.java:1086)
 at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:626)
 at org.alfresco.repo.search.impl.lucene.index.IndexInfo.makeDeltaIndexWriter(IndexInfo.java:964)
 at org.alfresco.repo.search.impl.lucene.index.IndexInfo.getDeltaIndexWriter(IndexInfo.java:1007)
 at org.alfresco.repo.search.impl.lucene.AbstractLuceneBase.getDeltaWriter(AbstractLuceneBase.java:223)
 at org.alfresco.repo.search.impl.lucene.AbstractLuceneIndexerImpl.flushPending(AbstractLuceneIndexerImpl.java:928)
 ... 14 more
12:11:00,025 ERROR [org.quartz.core.ErrorLogger] Job (DEFAULT.org.springframework.scheduling.quartz.JobDetailBean#5570984c threw an exception.
org.quartz.SchedulerException: Job threw an unhandled exception. [See nested exception: org.alfresco.error.AlfrescoRuntimeException: 0017151445 Reindex failure for org.alfresco.repo.node.index.AVMRemoteSnapshotTracker]
 at org.quartz.core.JobRunShell.run(JobRunShell.java:227)
 at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:549)
Caused by: org.alfresco.error.AlfrescoRuntimeException: 0017151445 Reindex failure for org.alfresco.repo.node.index.AVMRemoteSnapshotTracker
 at org.alfresco.repo.node.index.AbstractReindexComponent.reindex(AbstractReindexComponent.java:315)
 at org.alfresco.repo.node.index.IndexRecoveryJob.execute(IndexRecoveryJob.java:59)
 at org.quartz.core.JobRunShell.run(JobRunShell.java:216)
 ... 1 more
Caused by: org.alfresco.repo.search.impl.lucene.LuceneIndexException: 0017151444 Failed to flush index
 at org.alfresco.repo.search.impl.lucene.AbstractLuceneIndexerImpl.flushPending(AbstractLuceneIndexerImpl.java:934)
 at org.alfresco.repo.search.impl.lucene.AVMLuceneIndexerImpl.getLastAsynchronousSnapshot(AVMLuceneIndexerImpl.java:2013)
 at org.alfresco.repo.search.impl.lucene.AVMLuceneIndexerImpl.getLastAsynchronousSnapshot(AVMLuceneIndexerImpl.java:1991)
 at org.alfresco.repo.search.impl.lucene.AVMLuceneIndexerImpl.getLastIndexedSnapshot(AVMLuceneIndexerImpl.java:1867)
 at org.alfresco.repo.search.AVMSnapShotTriggeredIndexingMethodInterceptor.getLastIndexedSnapshot(AVMSnapShotTriggeredIndexingMethodInterceptor.java:316)
 at org.alfresco.repo.search.AVMSnapShotTriggeredIndexingMethodInterceptor.getLastIndexedSnapshot(AVMSnapShotTriggeredIndexingMethodInterceptor.java:309)
 at org.alfresco.repo.node.index.AVMRemoteSnapshotTracker.processStores(AVMRemoteSnapshotTracker.java:87)
 at org.alfresco.repo.node.index.AVMRemoteSnapshotTracker.reindexImpl(AVMRemoteSnapshotTracker.java:58)
 at org.alfresco.repo.node.index.AbstractReindexComponent$1.execute(AbstractReindexComponent.java:300)
 at org.alfresco.repo.transaction.RetryingTransactionHelper.doInTransaction(RetryingTransactionHelper.java:386)
 at org.alfresco.repo.transaction.RetryingTransactionHelper.doInTransaction(RetryingTransactionHelper.java:276)
 at org.alfresco.repo.node.index.AbstractReindexComponent.reindex(AbstractReindexComponent.java:306)
 ... 3 more
Caused by: java.io.FileNotFoundException: /CMSIndex/lucene-indexes/avm/sitestore/75ca677a-2c28-479e-abdd-766087d8ae77/segments_1 (Too many open files)
 at java.io.RandomAccessFile.open(Native Method)
 at java.io.RandomAccessFile.<init>(RandomAccessFile.java:216)
 at org.apache.lucene.store.FSDirectory$FSIndexOutput.<init>(FSDirectory.java:682)
 at org.apache.lucene.store.FSDirectory.createOutput(FSDirectory.java:443)
 at org.apache.lucene.index.SegmentInfos.write(SegmentInfos.java:296)
 at org.apache.lucene.index.SegmentInfos.prepareCommit(SegmentInfos.java:761)
 at org.apache.lucene.index.SegmentInfos.commit(SegmentInfos.java:825)
 at org.apache.lucene.index.IndexWriter.init(IndexWriter.java:1086)
 at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:626)
 at org.alfresco.repo.search.impl.lucene.index.IndexInfo.makeDeltaIndexWriter(IndexInfo.java:964)
 at org.alfresco.repo.search.impl.lucene.index.IndexInfo.getDeltaIndexWriter(IndexInfo.java:1007)
 at org.alfresco.repo.search.impl.lucene.AbstractLuceneBase.getDeltaWriter(AbstractLuceneBase.java:223)
 at org.alfresco.repo.search.impl.lucene.AbstractLuceneIndexerImpl.flushPending(AbstractLuceneIndexerImpl.java:928)
 ... 14 more

3 Replies
afaust
Master

Re: Too many open file Error - Alfresco community 4.0

Since you are still using Lucene, you need to be extra careful in your customisations about how you are doing queries. You especially have to make sure that you ALWAYS close your result sets when using the SearchService#query() operation (using try-finally blocks in Java code). Not closing result sets will lead to significant file duplication and was one of the most common causes for running into "too many open files" issues in old Lucene search subsystem.

Also, you can set your file handle limit to "unlimited"...

sanketmohite
Active Member

Re: Too many open file Error - Alfresco community 4.0

Alex, Thanks a lot for your quick response. It is highly appreciated.

Just to give you background, we are using alfresco as storage system and contents search feature is not at all used

So is it possible to set some configuration to stop generating Lucene Index?

Also Few questions which I raised previously

1. Is there any way by which I can  identify which files are being open when this "Too many open files" error is thrown?

2. Can we prevent the re-indexing which might be causing this error?

afaust
Master

Re: Too many open file Error - Alfresco community 4.0

In order to "disable" indexing, simply configure SOLR to be used without actually setting up a SOLR server.

The list of open files can be checked via the ls utility.