Lucene index merge failure

cancel
Showing results for 
Search instead for 
Did you mean: 
darionis
Active Member

Lucene index merge failure

Hi all,

Alfresco community 4.2 with Lucene search has been working for years but today it stopped working with the following message:

Index merger failed with java.io.IOException: There is not enough space on the disk

Index size is 60 GB and there was 360 GB free disk space which was all taken when index merge started. Problem was solved by restarting Alfresco, and disk space was released so I still have 360GB of free disk space.It looks like thaere was no harm since search still works.

I am not aware what "index merge" is and all i found is "Index Merging Performance" article on Alfresco hub but i still don't understand:

  • how often index merging is perfomed
  • what triggers index merging
  • does it really need so much disk space

Any answer to these questions will be higly appreciated, Thank you!

 

 

2 Replies
afaust
Master

Re: Lucene index merge failure

Indexes are kept in a tiered structure, in what sort of amounts to "generations" of data, e.g. long term stable index entries are bunched in larger, older generations, newer entries in small, young ones, This is done to keep performance of tracking / updates as high as possible. Every so often, when some generation grows over a calculated threshold, it needs to be merged into the next older generation. So the interval is directly dependent on the number of already existing entries, the frequency and pattern of updates, and can't be specified in absolute terms like "every x days".

Another thing - apart from merges - that can lock up significant disk space in the old Lucene system, is the result of programming errors in Alfresco customisations. When search results are not properly closed in Java code it can leave index files alive even after they should have been merged, which also won't be cleaned up until Alfresco is stopped (blocked by active file handles). In my experience, this is actually the most likely cause of high disk usage - not index merges. Unfortunately, this may be hard to fix when those incorrect code usages are in 3rd party customisations.

darionis
Active Member

Re: Lucene index merge failure

Thanks a lot, this is very interesting, I will need to check if search results are properly closed.