How to allocate dedicated memory to solr

cancel
Showing results for 
Search instead for 
Did you mean: 
hardik_thakkar
Active Member II

How to allocate dedicated memory to solr

Hi Team,

I have 32 GB RAM available on server(alfresco and solr on same instance) and i have allotted dedicated 24 GB in java opts in tomcat/bin/setenv.sh and tomcat/scripts/ctl.sh file.

Due to insufficient memory solr stops working, when stopping server it has generated hs_err_pid1-3-9.log file.

Error file contains below logs:

# There is insufficient memory for the Java Runtime Environment to continue.
# Native memory allocation (mmap) failed to map 24984682496 bytes for committing reserved memory.
# Possible reasons:
#   The system is out of physical RAM or swap space
#   In 32 bit mode, the process size limit was hit
# Possible solutions:
#   Reduce memory load on the system
#   Increase physical memory or swap space
#   Check if swap backing store is full
#   Use 64 bit Java on a 64 bit OS
#   Decrease Java heap size (-Xmx/-Xms)
#   Decrease number of Java threads
#   Decrease Java thread stack sizes (-Xss)
#   Set larger code cache with -XX:ReservedCodeCacheSize=
# This output file may be truncated or incomplete.
#
#  Out of Memory Error (os_linux.cpp:2640), pid=10309, tid=0x00007f541a1b8700
#
# JRE version:  (8.0_131-b11) (build )
# Java VM: Java HotSpot(TM) 64-Bit Server VM (25.131-b11 mixed mode linux-amd64 compressed oops)
# Failed to write core dump. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again

Can we allocate dedicated memory to solr? If yes, where to configure the same?

I had gone through solrnodes memory  ducumentation, as per calculation only solr requires 27 GB memory.

I have 1 million documents in repository.

After every server restart, solr starts working and memory utilization continues more than 90% and after sometimes due to insufficient memory solr stops working.

Thanks,
Hardik

5 Replies
abhinavmishra14
Moderator
Moderator

Re: How to allocate dedicated memory to solr

Solrnodes-memory-calculation  would provide how much memory is needed to solr depending on various aspects. As you have already gone through this doc, 27GB is required memory that you have calculated. If calculation is correct then you would require that much memory.

It would be wise to separate the solr and alfresco on different nodes. 

Chekcout these posts: 

https://hub.alfresco.com/t5/alfresco-content-services-forum/java-heap-space-issue-on-solr4/td-p/5735...

https://www.ziaconsulting.com/developer-help/alfresco-solr-memory/

Docs:

https://docs.alfresco.com/5.2/concepts/jvm-tuning.html

https://docs.alfresco.com/5.2/concepts/jvm-settings.html

Also see if these are an option for you:

Minimize the memory requirements for Solr nodes

  • Reduce the cache sizes and check the cache hit rate.
  • Disable ACL checks using alfresco.doPermissionChecks=false
  • Disable archive indexing, if you are not using it.
  • Check the number of empty transactions. If there are many empty transactions, purge the transactions from Alfresco using the action=FIX action.
  • Find the exact number of nodes in the store (N), exact number of transactions in the repository (T), number of ACLs (A) and related ACL transactions in the repository (X).
  • Since everything scales to the number of documents in the index, add the Index control aspect to the documents you do not want in the index.
~Abhinav
(ACSCE, AWS SAA-C02, GAIQ)
hardik_thakkar
Active Member II

Re: How to allocate dedicated memory to solr

Hi @abhinavmishra14 ,

I have allocated 40 GB to JAVA_OPTS, but still solr stops working after sometime.

I have gone through links shared by you, but didn't find any configuration to allocate dedicated memory to solr.

I have enabled metadata indexing only and that solves my issue.

Thanks,

Hardik

Claudie
Member II

Re: How to allocate dedicated memory to solr

  1. Open solr. in. cmd in any text editor (path: bin/solr. ...
  2. Add the set SOLR_JAVA_MEM=-Xms1g -Xmx1g ( it will set the minimum and maximum Java heap memory by 1 GB. however you will only able the see 981.38 MB in solr dashboard)
  3. save the solr. in.
hardik_thakkar
Active Member II

Re: How to allocate dedicated memory to solr

Hi @Claudie 

Solr.in.cmd file is available for solr6, right?

I'm using solr4 integrated with alfresco community version 5.2.

Is there any configuration available for solr4 and alfresco 5.2 version?

Thanks,

Hardik

Destinee
Member II

Re: How to allocate dedicated memory to solr

This is an attempt to give basic information only. For a better understanding of the issues involved, read the included links, look for other resources, and ask well thought out questions via Solr support resources.

This guide does not contain info specific to Solr installs that are using HDFS for index storage. If anyone has any info about how to achieve effective index data caching with HDFS, please share that information with the solr-user mailing list or the #solr IRC channel on freenode so it can be incorporated here.

 

 

 

myaarpmedicare