Thoughts on Amazon’s new onsite Storage Gateway announcement

Amazon Web Services has announced that it now offers a new storage gateway appliance (virtual machine image) that can be placed on a customers site. What benefit is this ? It really provides an easy way to integrate local storage or systems with the facility to replicate data to the Amazon Cloud. For example you could add the technology to an existing data center so that it resided between servers and storage  so that you could easily start replicating data to Amazon S3.

Note,however,these are actually stored as EBS Volumes. So although users can access data stored in this fashion locally from the gateway, if they wish to access this data directly through AWS they would need to start an EC2 instance and attached the EBS volume. . This in and of itself makes it easier to then integrate S3 stored data with other AWS services (if this is important to you). Note that this is not ‘replacing’ what you already have (ie. “great, I can just use the Cloud”), it is in addition to what you already have.

Firstly lets look at what the requirements are to host the Gateway.  These are:

  • VMware ESXi hypervisor (v4.1) on a physical machine with at least 7.5GB of RAM
  • Four (4) virtual processors assigned to the appliance VM along with 75GB of disk space for the Open Virtual Alliance (OVA) image installation and data.
  • A “proper” sized network connection to Amazon.
  •  iSCSI initiators on either Windows server 2008, Windows 7 or Red Hat Enterprise Linux

(Also note that the Gateway beta is optimised for block write sizes which are more than 4Kb.  AWS warns that using smaller I/O sizes are likely to cause overhead which can result in storage space that is effectively ‘lost’. This means that prior to installation there needs to be a check made on the file systems / volumes to ensure they can use the larger allocation sizes).

Firstly we’d like to point out that it’s great to see Amazon adding their own on-premise Cloud Gateway. It’s great to see them competing with the likes of  EMC, TwinStrata, and Nasuni. It would have been nice to see NFS or CIFS supported as interfaces, as from our own interactions with customers, we believe that is what customers really want to see, but maybe we can expect to see that added as the Gateway offering matures.

(Differences between iSCSI & NFS: iSCSI and NFS both allow storage access over an IP networking infrastructure. The difference is that iSCSI enables block storage transfer whereas NFS and CIFS transfers files.)

Many customers may find that they already have the capabilities for which they would use the Gateway, such as snapshots, backup and archiving, which is a pretty old, mature and I would expect a little more cost effective mechanism of achieving similar goals. However with that said we can see many use case where, with our own Cloud File Server Appliance where customers will really embrace the Gateway.

So where does the AWS Cloud Gateway end and the SME Cloud Appliance begin ? Well, the first things to understand about the SME Cloud Appliance is that it acts at a layer ‘above’ the storage. It provides a mechanism to unify disparate data sources into one file tree, add unified user access management and permissions, add unified governance and e-compliance, has focus on enabling companies to manage ‘Cloud Sprawl’, and leverages the ability for companies to let staff “bring your own device” (BYOD). In short, as I often say when asked to comment about Storage in general, the response is “it’s all about the App”. Storage in and of itself is not a single source in companies and secondly having things stored is no good unless you have unified, search, logic, control and anytime anywhere access that supports all desktops and all devices. This is what we essentially are bringing to the table with our Cloud File Server Appliance.

To take advantage of the Amazon Cloud Gateway what would be required is for us to connect to the local iSCSI stored data within the Gateway and this is something we will be looking to do in the short term.

For further information see the Amazon Cloud Gateway Storage FAQ’s. Also note that Amazon are also doing a free webcast on 23rd February.

 

Facebooktwitterredditpinterestlinkedinmailby feather

OpenStack now supported for SME Open Cloud SaaS Platform and Cloud Appliance

We are really pleased to announce that we have added OpenStack Swift object Storage support to the SME Open Cloud Platform. Swift is a sub project of OpenStack and provides a highly scalable redundant unstructured data store. Swift is 5 separate services, object, container, account, auth and proxy. Although each of these can be scaled separately, in practice they run together.

Never heard of Swift? it’s the underlying distributable object store that supports RackSpace Cloud Files. It’s akin to Amazon’s S3 implementation but unlike implementations such as Eucalyptus, which clone S3 API’s, but are not sponsored by Amazon, openStack and Swift has RackSpace firmly onboard, and have proven scale.

As Swift is used by Rackspace Cloud Files. Swift RackSpace claim it is production-ready code that is scalable to massive levels (100-petabyte clusters and 100000 requests per second). Swift sacrifices C for A and P from a CAP theorem perspective. Although most operations happen synchronously consistency is sacrificed in failure scenarios.

From our perspective we have seen ISP’s and larger SMB users of our on-premise Cloud Gateway appliance expressing interest in SME supporting this, and we supply this as VMWARE Appliance (OR XEN, KVM) or as a dedicated hardware appliance for smaller companies who wish to embrace their own private Cloud infrastructure.

As with our S3 API endpoint support SME will overlay a more traditional file store on top of Swift layered with the business functionality we provide in our  Cloud File Server, which includes virtual drives and clients for Mac, Windows and Linux, and feature rich mobile clients for iPad, iPhone, Android and BlackBerry, as well as value added features to Swift such as Webdav and FTP support.

Setting up Swift with SME is easy. First you need to add a new Cloud Provider and then the Cloud Wizard will be invoked. The first step is to enter your OpenStack details:

When entering the endpoint URL you should be sure to include the Port. An example URL is: http://<IP Address>:11000/v1.0.

Next you will need to choose which containers you want to work with and which should be the default container for any uploads to smart folders.

Once you have done this you will be ready to start the meta-sync which pulls in and caches all the information about containers and files.

If you have any issues connecting please refer to this advanced post on using SME with OpenStack 1.60 and SWAuth.

Once complete you will be able to access/amange your OpenStack files from the SME Web clients,  as well as using a Cloud Drive on Windows, Mac or Linux, and mobile clients for Android, iOS, and BlackBerry, and  the plethora of other tools and clients that SME provides. We’v e posted some screenshots below of this.

Web File Manager

iOS OpenStack

Android OpenStack

Firefox Plug-In OpenStack

Chrome OpenStack Plug-In

Mac Cloud Drive OpenStack

The OpenStack Swift API’s also get embedded for use within our own feature rich multi-cloud API framework in which we add many business driven features.. You can find details about that on our developer page

Facebooktwitterredditpinterestlinkedinmailby feather