Problems with solr when migratin from Alfresco 5.2 to 6.2

cancel
Showing results for 
Search instead for 
Did you mean: 
iohann95
Active Member II

Problems with solr when migratin from Alfresco 5.2 to 6.2

I'm having serious troubles trying to migrate my data from alfresco 5.2 to 6.2.

Everything works but when the reindex is happening, all sort of errors happen, the one that are blocking me this time is:

solr6_1                  | 2021-08-26 03:15:01.896 WARN  (qtp868964689-12) [   ] o.a.s.h.a.LukeRequestHandler Error getting file length for [segments_1x]
solr6_1                  | java.nio.file.NoSuchFileException: /opt/alfresco-search-services/data/archive/index/segments_1x
solr6_1                  |      at java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:92)
solr6_1                  |      at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111)
solr6_1                  |      at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:116)
solr6_1                  |      at java.base/sun.nio.fs.UnixFileAttributeViews$Basic.readAttributes(UnixFileAttributeViews.java:55)
solr6_1                  |      at java.base/sun.nio.fs.UnixFileSystemProvider.readAttributes(UnixFileSystemProvider.java:149)
solr6_1                  |      at java.base/sun.nio.fs.LinuxFileSystemProvider.readAttributes(LinuxFileSystemProvider.java:99)
solr6_1                  |      at java.base/java.nio.file.Files.readAttributes(Files.java:1764)
solr6_1                  |      at java.base/java.nio.file.Files.size(Files.java:2381)
solr6_1                  |      at org.apache.lucene.store.FSDirectory.fileLength(FSDirectory.java:243)
solr6_1                  |      at org.apache.lucene.store.NRTCachingDirectory.fileLength(NRTCachingDirectory.java:128)
solr6_1                  |      at org.apache.solr.handler.admin.LukeRequestHandler.getFileLength(LukeRequestHandler.java:615)
solr6_1                  |      at org.apache.solr.handler.admin.LukeRequestHandler.getIndexInfo(LukeRequestHandler.java:588)
solr6_1                  |      at org.apache.solr.handler.admin.CoreAdminOperation.getCoreStatus(CoreAdminOperation.java:348)
solr6_1                  |      at org.apache.solr.handler.admin.StatusOp.execute(StatusOp.java:48)
solr6_1                  |      at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:384)
solr6_1                  |      at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:388)
solr6_1                  |      at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:174)
solr6_1                  |      at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:173)
solr6_1                  |      at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:748)
solr6_1                  |      at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:729)
solr6_1                  |      at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:510)
solr6_1                  |      at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:361)
solr6_1                  |      at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:305)
solr6_1                  |      at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
solr6_1                  |      at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:583)
solr6_1                  |      at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
solr6_1                  |      at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
solr6_1                  |      at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226)
solr6_1                  |      at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
solr6_1                  |      at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:513)
solr6_1                  |      at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
solr6_1                  |      at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
solr6_1                  |      at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
solr6_1                  |      at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:213)
solr6_1                  |      at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:119)
solr6_1                  |      at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
solr6_1                  |      at org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:335)
solr6_1                  |      at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
solr6_1                  |      at org.eclipse.jetty.server.Server.handle(Server.java:539)
solr6_1                  |      at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:333)
solr6_1                  |      at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
solr6_1                  |      at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283)
solr6_1                  |      at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108)
solr6_1                  |      at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
solr6_1                  |      at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
solr6_1                  |      at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
solr6_1                  |      at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
solr6_1                  |      at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
solr6_1                  |      at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
solr6_1                  |      at java.base/java.lang.Thread.run(Thread.java:829)

After that error the reindex seems to stop.

Thats a strange error to happen since I didn't brought the index from the old version since it's not compatible.

 

I'm using the alfresco community docker installer:

https://github.com/Alfresco/alfresco-docker-installer

11 Replies
iohann95
Active Member II

Re: Problems with solr when migratin from Alfresco 5.2 to 6.2

Sorry for the double post but now i'm also getting this error several times:

solr6_1                  | 2021-08-26 03:34:31.427 ERROR (org.alfresco.solr.AlfrescoCoreAdminHandler@63a5e46c_Worker-2) [   ] o.a.s.h.RequestHandlerBase java.lang.NullPointerException
solr6_1                  |      at org.apache.solr.update.UpdateLog.lookup(UpdateLog.java:968)
solr6_1                  |      at org.apache.solr.handler.component.RealTimeGetComponent.process(RealTimeGetComponent.java:219)
solr6_1                  |      at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:296)
solr6_1                  |      at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:173)
solr6_1                  |      at org.alfresco.solr.SolrInformationServer.executeQueryRequest(SolrInformationServer.java:3387)
solr6_1                  |      at org.alfresco.solr.SolrInformationServer.getTrackerInitialState(SolrInformationServer.java:1415)
solr6_1                  |      at org.alfresco.solr.tracker.AbstractTracker.getTrackerState(AbstractTracker.java:332)
solr6_1                  |      at org.alfresco.solr.tracker.AbstractTracker.track(AbstractTracker.java:227)
solr6_1                  |      at org.alfresco.solr.tracker.ActivatableTracker.track(ActivatableTracker.java:97)
solr6_1                  |      at org.alfresco.solr.tracker.TrackerJob.execute(TrackerJob.java:60)
solr6_1                  |      at org.quartz.core.JobRunShell.run(JobRunShell.java:202)
solr6_1                  |      at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:573)
solr6_1                  |
solr6_1                  | 2021-08-26 03:34:31.429 ERROR (org.alfresco.solr.AlfrescoCoreAdminHandler@63a5e46c_Worker-2) [   ] o.a.s.SolrInformationServer Got no response from a tracker initial state request.
solr6_1                  | 2021-08-26 03:34:31.437 ERROR (org.alfresco.solr.AlfrescoCoreAdminHandler@63a5e46c_Worker-12) [   ] o.a.s.h.RequestHandlerBase java.lang.NullPointerException
solr6_1                  |      at org.apache.solr.update.UpdateLog.lookup(UpdateLog.java:968)
solr6_1                  |      at org.apache.solr.handler.component.RealTimeGetComponent.process(RealTimeGetComponent.java:219)
solr6_1                  |      at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:296)
solr6_1                  |      at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:173)
solr6_1                  |      at org.alfresco.solr.SolrInformationServer.executeQueryRequest(SolrInformationServer.java:3387)
solr6_1                  |      at org.alfresco.solr.SolrInformationServer.getTrackerInitialState(SolrInformationServer.java:1415)
solr6_1                  |      at org.alfresco.solr.tracker.ShardStatePublisher.getTrackerState(ShardStatePublisher.java:123)
solr6_1                  |      at org.alfresco.solr.tracker.AbstractTracker.track(AbstractTracker.java:227)
solr6_1                  |      at org.alfresco.solr.tracker.TrackerJob.execute(TrackerJob.java:60)
solr6_1                  |      at org.quartz.core.JobRunShell.run(JobRunShell.java:202)
solr6_1                  |      at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:573)
solr6_1                  |
angelborroy
Alfresco Employee

Re: Problems with solr when migratin from Alfresco 5.2 to 6.2

How are you deploying Alfresco?

Hyland Developer Evangelist
iohann95
Active Member II

Re: Problems with solr when migratin from Alfresco 5.2 to 6.2

I'm using the community docker installer: https://github.com/Alfresco/alfresco-docker-installer

 

And also the alfresco s3 adapter (version 1.2.1)

 

Regards,

Iohann

angelborroy
Alfresco Employee

Re: Problems with solr when migratin from Alfresco 5.2 to 6.2

That should be a problem with Docker Volumes binding for SOLR. Review that section in project documentation in order to be sure that everything is working fine.

Hyland Developer Evangelist
iohann95
Active Member II

Re: Problems with solr when migratin from Alfresco 5.2 to 6.2

I don't know... solr is able to reach the folder. The index sometimes grows to about 2gb but then it  hangs. Looking at Solr Admin it seems to always hang at aroung 130k docs in the index.

 

I even installed another machine with the same configurations, but without data migration, and it works just fine...

angelborroy
Alfresco Employee

Re: Problems with solr when migratin from Alfresco 5.2 to 6.2

How many RAM is assigned to your SOLR Docker service?

Hyland Developer Evangelist
angelborroy
Alfresco Employee

Re: Problems with solr when migratin from Alfresco 5.2 to 6.2

Did you try increasing RAM for this service?

This may be a cause of your system running consistently but failing from 130k docs.

Hyland Developer Evangelist
iohann95
Active Member II

Re: Problems with solr when migratin from Alfresco 5.2 to 6.2

I try a few hours ago:

    solr6:
        build:
          context: ./search
          args:
            SEARCH_TAG: $SEARCH_CE_TAG
            SOLR_HOSTNAME: solr6
            ALFRESCO_HOSTNAME: alfresco
            ALFRESCO_COMMS: none
            CROSS_LOCALE: "true"
        mem_limit: 4096m
        environment:
            #Solr needs to know how to register itself with Alfresco
            SOLR_ALFRESCO_HOST: "alfresco"
            SOLR_ALFRESCO_PORT: "8080"
            #Alfresco needs to know how to call solr
            SOLR_SOLR_HOST: "solr6"
            SOLR_SOLR_PORT: "8983"
            #Create the default alfresco and archive cores
            SOLR_CREATE_ALFRESCO_DEFAULTS: "alfresco,archive"
            SOLR_JAVA_MEM: "-Xms3584m -Xmx3584m"
            SOLR_OPTS: "
                -XX:NewSize=2048m
                -XX:MaxNewSize=2048m
            "
        volumes: 
            - ./data/solr-data:/opt/alfresco-search-services/data  

Same result... in solr admin, "archive" core shows around 134k documents indexed. "alfresco" core, shows around 9k

 

001.png002.png

just a note: my previous message is missing: i didn't deleted it.

iohann95
Active Member II

Re: Problems with solr when migratin from Alfresco 5.2 to 6.2

Solr used 100gb while indexing and took 2 days crashing sometimes, but eventually it commited the whole tlog and now the index is much smaller.

 

Now it's working fine...