I have Alfresco 7.2 Community installed in Docker using the install script from Angel Borroy running on an Ubuntu server with very generous hardware although the disc is relatively slow. The repository holds several million documents from which about 450,000 need to be deleted. I have the node UUIDs to be deleted in an external Oracle table.
I created an external Java tool that does this running on a separate machine with this logic:
- create a SQL connection
- create a CMIS connection
- read 1,000 nodes (UUIDs)
- delete the nodes in a loop with CMIS
- mark the UUID as processed in the external table
I wait 15 seconds before processing the next batch of 1,000 nodes. I can do about 6,000 nodes per hour.
The problem is that at first the throughput is very good but after about a day processing it gets slower and slower pretty fast (the degradation is not linear) until errors show up in alfresco.log that eventually lead to the system being unresponsive. I restart Alfresco to continue. At first it goes for almost 1 day but sucessive restarts tend to process less and less nodes before becoming unresponsive which makes e think Alfresco is doing some of its own processing in the background. Since I believe this is not a heavy load, there must be sometjhing else wrong.
The errors are usually related to:
2022-12-08 10:11:33,749 ERROR [org.springframework.extensions.webscripts.AbstractRuntime] [http-nio-8080-exec-48] Exception from executeScript: 11080404 Wrapped Exception (with status template): nested exception is org.apache.ibatis.exceptions.PersistenceException: ### Error querying database. Cause: org.springframework.dao.ConcurrencyFailureException: No node row exists: ID: 16309826 DB row: null### Cause: org.springframework.dao.ConcurrencyFailureException: No node row exists: ID: 16309826 DB row: null
org.springframework.extensions.webscripts.WebScriptException: 11080404 Wrapped Exception (with status template): nested exception is org.apache.ibatis.exceptions.PersistenceException:
2022-12-08 11:34:49,161 ERROR [org.alfresco.repo.activities.feed.cleanup.FeedCleaner] [DefaultScheduler_Worker-5] Exception during cleanup of feeds
org.mybatis.spring.MyBatisSystemException: nested exception is org.apache.ibatis.exceptions.PersistenceException:
### Error querying database. Cause: org.springframework.jdbc.CannotGetJdbcConnectionException: Failed to obtain JDBC Connection; nested exception is org.postgresql.util.PSQLException: The connection attempt failed.
### The error may exist in alfresco/ibatis/#resource.dialect#/activities-common-SqlMap.xml
### The error may involve alfresco.activities.select_activity_feed_entries_max_id
### The error occurred while executing a query
### Cause: org.springframework.jdbc.CannotGetJdbcConnectionException: Failed to obtain JDBC Connection; nested exception is org.postgresql.util.PSQLException: The connection attempt failed.
this goes on for a while until a heap space OutOfMemory stops everything:
Caused by: java.lang.OutOfMemoryError: Java heap space
This happens when heap space gets to the maximum. I can set more space but Alfresco just uses it up with the same result. Nobody else and no other process is using this Alfresco and it's the only application on this server.
I also have lots of cache full but since it's a warning I think it's not the cause:
2022-12-08 09:57:40,318 WARN [org.alfresco.repo.cache.TransactionalCache.org.alfresco.cache.node.nodesTransactionalCache] [http-nio-8080-exec-39] Transactional update cache 'org.alfresco.cache.node.nodesTransactionalCache' is full (125000).
So I have to reboot Alfresco at first in less than a day and then more frequently.
My qestion is what could be causing this and what can I check to see how to fix it.
Thanks.
