Solr optimization doubts

[gabrielmarinb]

Hi:

We are planning Solr index optimization and we ask four your guidance and experience regarding this. As we have a large repository with more than 250 M files and 600K uploads and 800K deletions daily we consider this action as an important manintenance action to safegauard solr performance.

As this is is the first time we do this, we have some doubts:

• Is it better to start optimization by using Optimize Now button or making an API call to Solr?
• Can I upload files to Alfresco during optimization? Will they be indexed? or after optimization task?
• Can you provide guidance on how long does it take in my case with almost 900GB of indexes to finish optimization?
• Where can I see optimization progress or errors? Will Solr logging page be available during optimization execution? What message should I expect once optimization has succesfully finished?

I really apreciate your guidance and help.

Regards 

Gabiel Marin

[angelborroy · Accepted solution]

Are you aware of what hitting that "Optimize Now" button implies?

Please, check this article before moving forward: https://lucidworks.com/post/solr-segment-merge-frees-wasted-space-caused-by-deleted-documents/

Additional information is provided in Alfresco Official documentation:

https://docs.alfresco.com/search-services/latest/config/performance/#optimize-solr-index

[gabrielmarinb]

Thanks a lot for the info @angelborroy . I see that optimizing is like a "point of no return" action. Anyway after considering the information I see this action as recommended in my case where I daily upload lots of new documents and delete lots of old documents.

Also I am aware that I will need to keep optimizing regularly. I also already have double the index size allocated before performing this task

Am I missing something?

[gabrielmarinb]

Finnaly we performed optimization using maxSegments=90 considering we had more than 200 (.../solr/alfresco/update?optimize=true&maxSegments=90)

Optimization took about 7 hours and finished succesfully freeing about 15% of index space.

Thanks a lot for your guidance