Using Amazon S3 AND File-system based store for content storage

[milochanzy]

Hi,

We want to use S3 as a content store ALONG with the existing File Content Store. So S3 content store will be used for specific document types as they may have different use case. The Alfresco documentation is a bit confusing hence raising it on the Forum. The confusion is as follows:

Multi-Store Support:
As per Alfresco documentation it supports multiple content store via content store selector. Mapping a document type can place the content to respective content store.

AWS S3 Connector:
As per Alfresco documentation "Using an Alfresco Module Package (AMP), the connector supplies a new content store which replaces the default file system-based implementation for the standard and deleted content stores."

So if anyone can help if content store can be selected based on document type at runtime. Is there any documentation on how to configure the same?

Thanks,
Milochanzy!

[milochanzy]

Hi Team,

Any feedback is much appreciated. Alfresco by default provides an Aggregator Content Store of S3 connector, can we use Content Store Selector in place? Such that based on type of document, content store is used.

Thanks,
Milochanzy.

[afaust]

Technically speaking, yes - you can use the content store selector or an aggregating content store with a store such as the S3 connector AND a file-system based store to have flexible storage / fallback.

Now, the problem in this instance will be that Alfresco typically tends to go for a naiive / brutal "trivial use cases only" default configuration in their modules / Enterprise addon-functionalities. E.g. just as the S3 connector documentation you linked to states, it replaces the default implementation (configuration) - so if you wanted to create a custom setup where both the default  and the S3 stores are active, and both are fronted by a store selector or aggregating store, you'd have to writer custom Spring XML to wire the Alfresco store implementations together.

Configuring Alfresco content stores can be quite a pain, especially since any non-trivial scenario involves Spring XML, and goes beyond what you can do in alfresco-global.properties. For that reason I created https://github.com/Acosix/alfresco-simple-content-stores - an addon which provides a flexible set of content store templates which can be wired up using only alfresco-global.properties. Now, this module currently does not support a S3 content store, but a 3rd-party module such as https://github.com/douglascrp/alfresco-cloud-store could be easily adapted to provide the necessary store template to become compatible. I doubt though that Alfresco will ever adapt their official S3 connector for interoperability, nor allow me access to their source code / default configuration to potentially fix any compatibility issues on my end.

[jpotts]

One of my clients used a traditional file-based content store together with the S3 connector. We used the store selector to determine whether the node was stored in S3 or on the file system. We also successfully moved nodes back-and-forth between the two.

The content store selector uses a property on an aspect to determine which store the content should be stored in. So, if you want to make a selection based on type, you'd just use a rule or a behavior to inspect the type and then set the aspect accordingly.

[milochanzy]

Hi Axel,

Thanks a ton for the great insight. Can we choose to have File System as default content store and use S3 as alternate store where documents get pushed only if we select a particular document type (from content model by specifying the store name)? Can you share you feedback on such a scenario?

Thanks,
Milochanzy!

[milochanzy]

Hi @jpotts,

Thanks a ton for your reponse, much appreciated! I was looking for confirmation on the exact set-up you suggested in your response. Can you please share if there are any special consideration to be made for implementing it? Say latency, back-up, resiliency etc.

Thanks,
Milan.