Working with Amazon S3 IAM User Profiles and SME Enterprise File Share and Sync

Amazon S3 FolderAlthough Storage Made Easy supports sophisticated permissions for Amazon S3, AWS provides the ability to setup specific profiles using Amazon IAM (Identity and Account Management).

If you have  existing users setup using Amazon S3 profiles via IAM, in which they have been set specific permissions to access a bucket, for use with SME EFSS you will need to generate a keypair for each user. Continue reading →

Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather

New Feature Added: Real Time Document Collaboration

Real Time Document Editing

We have added a new feature to the Storage Made Easy platform, real time document collaboration. This is available for team account or Enterprise File Share and Sync on-premise users. It enables real time document editing between team members when online editing using the web document editor. It enables multiple users to open a document at the same time, and enables with everyone to contribute and/or review the document in real time.

This editing facility is available for any document in any cloud that is mapped to work with SME, be it public or private storage (for EFSS on-site users).

The below video shows the feature in action.

Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather

WebDav the interoperable protocol for file and document access

WebDav is an acronym for Web Distributed Authoring and Versioning and can also be referred to as just plain old DAV.

WebDav is an extension of the HTTP protocol that was originally designed by Jim Whitehead from the University of California at Santa Cruz in 1996 when he was working at the World Wide Web consortium and it later became an Internet Engineering Task Force (IETF) standard.

WebDav was built as an interoperable standard to support remote collaborative authoring of Web sites and individual documents, as well as remote access to document based systems.

Today it It is the most popular network file-system protocol for use across the Internet, and although it has been integrated as a interoperable layer into many existing product implementations it is also notably missing as an interoperable API standard from many, such as DropBox, Google Drive, Amazon S3 and more.

The Storage Made Easy WebDav Gateway

SME provide a way to access any mapped cloud by secure WebDav irrespective of whether the underlying Cloud Supports the WebDav protocol natively. As WebDav is so well supported in many desktop and mobile Apps this means that Cloud Data can easily be integrated and accessible without having to move it to access the features of a particular Application that is WebDav enabled.

Connecting to WebDav Servers and Windows Shares

SME can also be configured to connect to servers that support the WebDav protocol. This use of WebDav from a SME perspective is using WebDav as a back end cloud to store data rather than exposing existing clouds to be accessible using the WebDav protocol.

Many existing NAS or SAN devices such as those as the NetGear ReadyNAS and the Synology devices range already provide WebDav as an access protocol to access data. Also existing web servers such as Apache can also be configured to use WebDav using the Mod Dav extension.

Many users of SME want to expose windows file shares and make them directly available through the SME service to all devices. The most appropriate and secure way to do this is not to expose such shares directly but to configure Microsoft Internet Information Server to expose these shares over WebDav.

Advantages of WebDav for Windows File Sharing

This has the following advantages:

Seamless integration with the IIS Manager

A secondary protocol provides a security DMZ with regards to direct access to windows shares

IIS WebDAV can be enabled at the site level, allowing IT administrators to restrict WebDAV access to specific sites on a server.

IIS WebDAV supports per-URL authoring rules, allowing administrators to specify custom WebDAV security settings on a per-URL basis. This fine-grained control gives administrators the ability to maintain one set of security settings for normal HTTP requests and a separate set of security settings for WebDAV.

IIS WebDAV supports both shared and exclusive locks to prevent lost updates due to overwrites

WebDAV supports secure connection as well. By enabling HTTPS over all WebDAV connections, security is fortified. SSL certificates can also be installed to increases security measures

Why WebDav as a Cloud Connector ?

WebDAV is an optimized protocol for document access over http. It is proven as being latency independent and is efficient over wide area networks especially in contrast to file protocols such as NFS and CIFS.

Using secure WebDAV ensures the data is encrypted during transmission and due to the optimizations that data is stored efficiently and quickly .

Why Not The Common Internet File System (CIFS)

CIFS is the standard way that windows users share files across corporate intranets and the Internet with a secure VPN connection.

To expose such shares directly to the internet or to other none windows PC’s it is needed to use a bridging technology. Samba is often used as such as technology. With Samba, the ports 139/tcp and 445/tcp are exposed over a public IP Address. Once this is done such shares are accessible.

The drawbacks of this are:

– The CIFS protocol used by Windows file sharing does not provide data encryption

The protocol itself is quite chatty.

No level of security indirection

CIFS is is an optimized protocol for access to data over a network that has been extended by VPN and has been used in this context by many companies for a long time. The disadvantage of this is that all devices have to support , be setup, and work with the VPN. preventing access by some devices and Apps and making Adhoc ‘on the fly’ access difficult.

Securing WebDav Servers

It is beyond the scope of this blog post to go into great detail on the steps required to secure WebDav servers but Microsoft has a very good guide on how to secure the IIS WebDav Service. This can be accessed at:

In addition to this you should note the following best practices:

Folder Permissions: Use non-anonymous authentication. Modify the NTFS permissions on the folder to only allow the access necessary to the users who require such access

Prevent File Execution: If you are only using WebDAV as a file store and not using it to display web pages, then execute permissions should be removed from that site or folder.

Apache WebDav servers can be configured to use LDAP authentication and also two factor authentication and any deployments should consider implementing these.

Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather

Thoughts on Amazon’s new onsite Storage Gateway announcement

Amazon Web Services has announced that it now offers a new storage gateway appliance (virtual machine image) that can be placed on a customers site. What benefit is this ? It really provides an easy way to integrate local storage or systems with the facility to replicate data to the Amazon Cloud. For example you could add the technology to an existing data center so that it resided between servers and storage  so that you could easily start replicating data to Amazon S3.

Note,however,these are actually stored as EBS Volumes. So although users can access data stored in this fashion locally from the gateway, if they wish to access this data directly through AWS they would need to start an EC2 instance and attached the EBS volume. . This in and of itself makes it easier to then integrate S3 stored data with other AWS services (if this is important to you). Note that this is not ‘replacing’ what you already have (ie. “great, I can just use the Cloud”), it is in addition to what you already have.

Firstly lets look at what the requirements are to host the Gateway.  These are:

  • VMware ESXi hypervisor (v4.1) on a physical machine with at least 7.5GB of RAM
  • Four (4) virtual processors assigned to the appliance VM along with 75GB of disk space for the Open Virtual Alliance (OVA) image installation and data.
  • A “proper” sized network connection to Amazon.
  •  iSCSI initiators on either Windows server 2008, Windows 7 or Red Hat Enterprise Linux

(Also note that the Gateway beta is optimised for block write sizes which are more than 4Kb.  AWS warns that using smaller I/O sizes are likely to cause overhead which can result in storage space that is effectively ‘lost’. This means that prior to installation there needs to be a check made on the file systems / volumes to ensure they can use the larger allocation sizes).

Firstly we’d like to point out that it’s great to see Amazon adding their own on-premise Cloud Gateway. It’s great to see them competing with the likes of  EMC, TwinStrata, and Nasuni. It would have been nice to see NFS or CIFS supported as interfaces, as from our own interactions with customers, we believe that is what customers really want to see, but maybe we can expect to see that added as the Gateway offering matures.

(Differences between iSCSI & NFS: iSCSI and NFS both allow storage access over an IP networking infrastructure. The difference is that iSCSI enables block storage transfer whereas NFS and CIFS transfers files.)

Many customers may find that they already have the capabilities for which they would use the Gateway, such as snapshots, backup and archiving, which is a pretty old, mature and I would expect a little more cost effective mechanism of achieving similar goals. However with that said we can see many use case where, with our own Cloud File Server Appliance where customers will really embrace the Gateway.

So where does the AWS Cloud Gateway end and the SME Cloud Appliance begin ? Well, the first things to understand about the SME Cloud Appliance is that it acts at a layer ‘above’ the storage. It provides a mechanism to unify disparate data sources into one file tree, add unified user access management and permissions, add unified governance and e-compliance, has focus on enabling companies to manage ‘Cloud Sprawl’, and leverages the ability for companies to let staff “bring your own device” (BYOD). In short, as I often say when asked to comment about Storage in general, the response is “it’s all about the App”. Storage in and of itself is not a single source in companies and secondly having things stored is no good unless you have unified, search, logic, control and anytime anywhere access that supports all desktops and all devices. This is what we essentially are bringing to the table with our Cloud File Server Appliance.

To take advantage of the Amazon Cloud Gateway what would be required is for us to connect to the local iSCSI stored data within the Gateway and this is something we will be looking to do in the short term.

For further information see the Amazon Cloud Gateway Storage FAQ’s. Also note that Amazon are also doing a free webcast on 23rd February.


Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather