Moving from SMB to WebDAV

cancel
Showing results for 
Search instead for 
Did you mean: 

Moving from SMB to WebDAV

resplin
Intermediate
5 8 24K

This long blog post explains how we are determining the future of shared network drive support in Alfresco Content Services. Specifically, we are considering dropping support for using the SMB protocol and instead increasing our investment in WebDAV for these use cases. We want to explain our reasoning and seek your feedback.

Analysis of the Problem

Alfresco has long worked to make the power of enterprise applications available to users who don't want to understand all the details of an ECM or BPM system. Early in the product's life, Alfresco added to the Content Repository the ability to be accessed as a shared network drive so that knowledge workers could receive the benefits of ECM while continuing their habit of "throwing everything in the Z: drive". We ended up implementing this capability three separate times: the CIFS dialect of SMBv1 in our JLAN module, standards compliant WebDAV, and the Windows specific WebDAV implemented in our AOS module. But the broad adoption of this capability has made it worthwhile.

Microsoft recently announced the end-of-life of SMBv1, including the CIFS dialect implemented by Alfresco. This protocol was the main exploit vector for the recent round of cryptolocker attacks. We cannot recommend our customers use this protocol, so we have been evaluating other options. You can see some of our conversations on this topic in the thread Re: SMB2 / SMB3 server support‌.

That conversations mentions a number of ways to implement SMBv3 which we investigated: upgrading our current implementation, using 3rd party open source libraries (there aren't any mature implementations of the server), implementing a storage back-end for Samba, and using proprietary libraries. Recognizing that the effort involved in implementing SMBv3 would slow down our progress on the other priorities described on our Content Repository Roadmap 2017‌, we also looked at alternatives ways to meet the same use cases.

WebDAV is an obvious choice to replace SMB, as our implementation is mature and it is widely used by our customers. It is also more robust than SMB when used on high latency networks such as when deploying the Content Repository in a cloud environment like AWS, which is an increasingly common use case. In many ways, WebDAV is a better fit for ECM use cases than SMB, which is intended to be used by high performance filesystems. Customers who attempt to use ACS as a file server are sometimes disappointed as a content repository makes a different set of trade-offs from a file server; it has many more capabilities but lower total throughput. Specifically, a file server uses SMB to allow mid-file access and high performance operations by exposing raw file handles to client applications, but this is not possible when the content is encrypted, is stored in an object store like S3 or Centerra, or is stored in a cheap high-latency infrequent access storage tier.

Many of the use cases where customers have expressed a preference for SMB over WebDAV require high frequency mid-file access. These use cases are not suitable for pulling directly from an content repository because they don't allow it to perform ECM functions. Instead, customers should synchronize the desired content to the client machine and back to the repository when the file is finished being used. Our proprietary https://community.alfresco.com/community/ecm/blog/2016/12/02/desktop-sync-100-ga-for-windows?sr=sear...‌ offers this capability, and is one of many solutions provided by both Alfresco partners and the open source community that can be used for this purpose.

Though our analysis suggests that WebDAV is an adequate replacement for SMBv1 in most use cases, we wanted to hear from a larger set of customers.

Research

We sent a survey to 150 customers who have previously indicated that they use either the CIFS or WebDAV shared network drives, and 52 responded. Important findings included:

  1. WebDAV is already used more widely than CIFS.
  2. The ACS Windows Explorer shortcuts available through CIFS are not as widely used as expected.
  3. Concerns with SSO access to shared network drives. NTLMv1 is also insecure, and the survey showed that Kerberos is much more widely used.

We specifically asked customers why they don't use WebDAV in every circumstance, and a few key reasons surfaced:

  1. Concerns about performance: WebDAV does not perform as well as CIFS on a local network. Part of this performance is due to the mid-file access that CIFS can provide, but we believe there is room for optimization in our implementation which will help address this concern.
  2. Concerns about compatibility: Some applications, such as Adobe products, struggle when accessing large files over WebDAV. One reason why CIFS performs better for these applications is the direct file handles for mid-file access that we discussed earlier. The second reason is that our CIFS implementation intelligently handles the file shuffling these applications do during write operations. We plan to port this shuffling from our CIFS implementation to our WebDAV libraries.
  3. Multiple customers raised a concern that the Windows 255 character limit impacts WebDAV folders. Our plan is to use repository shortcuts to make it easy to mount deep folders on short paths.
  4. The largest file that can be shared with WebDAV is 4GB. Desktop Sync is a better way to work with such files, as working with a 4GB file over WebDAV would require many round-trips of the full file.

For those who are interested, here are the detailed results from the survey. Note that the questions are usually multi-select, so results do not add to 100%. Also, there was an "Other" option where respondents could enter additional text, which accounts for the last few results in many questions. I apologize for the truncation in the answers.

Truncated options: Shared network drive, Custom application, An official Alfresco Connector, Publication through a web portal or public web site, Other: From Jive or Liferay Portlets.

Truncated options: Engineering Designs, Graphic Design, Other: all types of research files.

The Future

As a result of this analysis and research, we intend to take the following actions:

  • It is expected that Alfresco Content Services 5.2 will be the last release with a CIFS implementation. Along with retiring CIFS, we will be retiring NTLM and the ACS Windows Explorer shortcuts. Instead we will recommend the use of WebDAV and Kerberos.
  • We will compensate for the identified shortcomings in WebDAV by:
    • Implementing smart file shuffling with WebDAV to increase compatibility with commonly used applications.
    • Making it easy to deep-link into the Alfresco Content Repository over WebDAV to avoid issues with path length.
    • Focusing on improving WebDAV performance at scale.
    • Continuing to improve the performance of Desktop Sync when used with very large files.
  • We are also considering SAML support for shared network drives, though that work is not currently scheduled and won't be available in the next release.
  • If there is sufficient customer demand for an SMBv3 implementation, we will reconsider that development effort. In order to lower the cost of development, it is likely that we would leverage a proprietary 3rd party library. As such, any future SMBv3 functionality is not expected to be part of our open source offerings.

I look forward to discussing the implications of this change in the comments below.

8 Comments
angelborroy
Alfresco Employee

Great job , however I'm very disappointed about different concerns:

  • Probably you didn't asked to any factory or there is no factory in your client list. Many assembly line terminals have only CIFS protocol support. So you are discarding all this sector with one decision. And I'm pretty sure that there are also other environments where CIFS is a must.
  • Alfresco is (was?) Open Source. Why to choose a proprietary SMBv3 library? Obviously you can argue trusting on support, performance and some other vague topics. There are few Java Open Source SMBv3 libraries out there and you can even contribute to improve them! Or is Alfresco different now from the last years?
  • Alfresco is spending a huge amount of money in developing things around the platform, but it has no money to consolidate a core feature? It looks weird for me.
  • WebDAV is not an alternative for CIFS and it will not become an alternative in the future. The list of patches you referred before is just only an starting point. In the end your support service will spend ten more times money than developing properly a SMBv3 support for Alfresco.

This is not important for me as developer, but probably your business people will be scared when Alfresco ranking decrease in next Garner report (hint: take a look at what is scored in Integration & Interoperability critical capability).

Anyway, I know that this is a taken decision and you are only translating it to us. 

When I look at Alfresco today, I don't recognise that freshness from the founders...

resplin
Intermediate

Thank you for the feedback Angel. Some things to consider:

  • You make a good point about assembly line terminals. That did not come up in the survey, probably due to it being a small percentage of our total users. A related concern that is probably more common is document scanners / multi-function printers that can save to an SMB directory but not to WebDAV. Hopefully these older devices phase out quickly.
  • Alfresco has always followed an open core model, and this decision is inline with principles we have used in the past (https://community.alfresco.com/community/ecm/blog/2016/03/29/applying-the-open-core-model-in-alfresc...). Our bias was to provide an up-to-date open source SMB capability, but we could not find an open source library that implements the server portion of the protocol. If you know of one, we would love to examine it.
  • We looked at building such a library ourselves, as we have done previously. It is a large project and based on the above analysis doesn't provide significant value to most of our users. As we looked at why people preferred SMB, the majority of responses were related to application compatibility (that we think we can address), and performance in use cases that don't really fit ECM. When asked to select from a menu of features, up-to-date SMB support doesn't rank highly.
  • Our core ECM capability is rather mature which led us to focus the 5.0 release family on providing solutions and integrations on top of it. However, over the next few releases you will see some significant changes to the core. See the Content Repository Roadmap 2017‌.
  • We think this is the right approach based on our analysis, but we are just starting the process of reaching out to impacted customers and will continue to listen to feedback and incorporate it into our plans.

We appreciate the conversation.

angelborroy
Alfresco Employee

I've been investigating alternatives to deal with this problem. Probably, I'm missing something, but what is the difference between this proprietary software (Visuality Systems) and this other open source (smbj)? Both look like SMBv3 clients...

s_palyukh
Established Member

Angel Borroy, the problem that there is no java based, stable implementation of server-side with open sources on the Internet. SMB client isn't a problem...

I don't like that Alfresco going to stop supporting some cool features like SMB, but if be honest, the most part of our clients don't use SMB, and others who use can be switched to WebDAV without a lot of efforts and problems...

angelborroy
Alfresco Employee

Just trying to figure out why Alfresco preferred a proprietary solution instead an open source one. The only reason should be that the first one were a SMB server. This is why I'm asking.

angelborroy
Alfresco Employee

Another interesting approach from Adobe: https://github.com/adobe/node-smb-server (100% JavaScript)

  • SMB2 is currently work in progress
  • extensible via backend SPI
s_palyukh
Established Member

Interesting approach, but the support of SMB3 isn't implemented yet, only SMB2 in progress...  I think when there will be some opensource smb3 server available on the Internet, then our community implement integration or maybe Alfresco will return support of this protocol, of course, if it will be needed

resplin
Intermediate

Sergey Palyukh‌ is correct. The library from Visuality Systems is the only Java implementation we could find of a modern version of the SMB server protocol. We would prefer an open source implementation if we could find one.

We had not previously reviewed the node-smb-server. That's interesting. Thank you for sharing Angel Borroy‌.

Our immediate plan is to focus on WebDAV and our other roadmap priorities, but we will continue to evaluate the effort involved with modern SMB support.