Using the SME Cloud Platform with OpenStack 1.60 and Keystone and SWAuth authentication Services

One of the Object Storage Clouds we supported early was OpenStack Swift, which continues to go from strength to strength.

We thought it was time to give a refresh update of how to setup SMEStorage SaaS with OpenStack Swift so that companies / users can easily test / work with their OpenStack Cloud using SMEStorage website and access clients prior to requesting access to our on-presmise Appliance with OpenStack support.

First a refresher on OpenStack authentication:

There are 3 modes of OpenStack authentication:

1. DevAuth: Mostly for development

2. SwAuth: swauth is purely swift authentication

3. Keystone: Keystone is for all OpenStack services and can be configured with just MySQl or it can be integrated with other Identity Management system

SME Supports swauth and Keystone.

For SWAuth

When using swauth we use the standard documented swift methods for authentication and getting the storage url ie as per:

curl -v -H 'X-Storage-User: <tenantName: userName>' -H 'X-Storage-Pass: password' http://IP Address:8080/auth/v1.0

When entering details into the Provider details page for OpenStack on SMEStorage you need to give the tenant name:username and the password. SME also requires a URL endpoint which should be in the format: http://IP Address:8080/auth/v1.0

We expect the following in the response headers

X-Storage-Url: http://storage server IP:PORT/v1/AUTH_xxxxxxxxxx
X-Storage-Token: AUTH_xxxxxxxxxx

We will use the URL and token to connect to OpenStack Swift storage
 

For Keystone:

Ideally Keystone should work as swauth but we have never seen this working correctly. To that end we access keystone making this http request:

curl -d '{"auth": {"tenantName": "username", "passwordCredentials":{"username": "thisisusername", "password": "your password"}}}' -H "Content-type: application/json" http://IP Address:5001/v2.0/tokens

We expect the response in the following JSON format and will take the public URL for swift service and the token to access your swift storage.

 "access": {
       "serviceCatalog": [
           {
               "endpoints": [
                   {
                       "adminURL": "http://IP:PORT/",
                       "internalURL": "http://IP:PORT/v1/AUTH_xxxxxxxxxx",
                       "publicURL": "http://IP:PORT/v1/AUTH_xxxxxxxxxx",
                       "region": "MyRegion"
                   }
               ],
               "endpoints_links": [],
               "name": "swift",
               "type": "object-store"
           },
       ..........
       ],
       .........
       "token": {
           "expires": "2012-09-05T15:34:18Z",
           "id": "TOKEN WE WILL USE",
           }
       }
       }
   }
}

When entering the provider details into SME we require the tenant name:username, password and keystone authorisation server ie. http://IP Address:5001/v2.0/tokens

If you  are unable to connect to the your OpenStack storage please send us the input from the above command and we will check.

Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather

Automating OpenStack Swift backup to Amazon S3

One of the interesting things you can do when you have added an OpenStack Swift Provider to an Account is to automate the backing up off data to another Cloud, something we call ‘Cloud Safe’. This means if the OpenStack Swift data is not available then you can access the data from the backup Cloud.

We’ll step through an example of how to automate the backing up of data from OpenStack Swift to Amazon S3.

First you’ll need an account that supports such a backup. For this you’ll need either a Personal Lifetime Cloud or an Organisation Cloud File Server Account, or you’ll need to be using the SMEStorage Open Cloud Platform Appliance.

Once you have mapped OpenStack to your Account you can proceed to the Cloud Dashboard to assign a backup Cloud Provider for OpenStack:

If you click on the “Add Cloud Provider” dialog you will be asked to choose whether you wish to add a primary cloud provider or a backup cloud provider:

Ensure you Select the backup provider radio button and also Amazon S3 as the cloud to be used for backup. Once you have done this you will be taken to a wizard to enter your Amazon S3 Details:

Once you have stepped through the wizard and completed adding your S3 account a sync task will be generated that will synchronise the existing data that resides on OpenStack with the Amazon S3 Cloud you just added. You can check on the status of this Cloud sync task from the Cloud Sync task link in the right sidebar after login from SMEStorage.com.

Once this task has completed your data is sync’d to S3. If you upload any new data a sync task will be created to also transfer this to S3. If you delete any data then it will be deleted from your primary cloud but the default action is not to remove it from your backup cloud, in the event you later wish to find such removed data.

This mechanism of “Cloud Safe” or Cloud-to-Cloud backup will actually work with any combination of Clouds that are mapped to your account.

Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather

Achieving S3 API compatibility with OpenStack Swift using SME

One of the things that is close to our hearts at SMEStorage is interoperability and, wherever possible, enabling open solutions. As outlined in our previous post, we recently added support for OpenStack Swift object storage. Of course there are other Object Storage frameworks, such as Amazon S3, but this is closed source, unlike Swift. There are however open source implementations of the Amazon S3 API’s including Walrus from Eucalyptus. There are also many tools, solutions, exiting code and frameworks that use S3 compatible API’s, and no doubt there will be occassions when it would be useful to enable users to connect to SWift using Amazon S3 API’s.

As SMEStorage has a compatible S3 API abstraction it turns out that is very easy. Let’s demonstrate how easy below:


After launching the free version of CloudBerry explorer Choose File-Walrus Accounts, and then choose to add an account.

The endpoint for the s3 compatible interface with SMEStorage is s3.storagemadeeasy.com. The username is the SMEStorage Account username, and the secret key can be found for each user on the “My DashBoard” page under a section called “Tech Info“.

Once we have entered the details we can test the connection:

and finally we can connect over the S3 API to work with our OpenStack Swift files:

As you can see this was easy and takes less than a couple of minutes to access and work with OpenStack Swift files using S3. We expect it to be very useful for ISP’s and SMB’s who want to deploy their own private cloud solution using OpenStack and enables interoperability and prevents lock-in.

Update: Note that in the Swift Diablo release that OpenStack also supports the ability to turn on Amazon S3 API’s.

Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather

OpenStack now supported for SME Open Cloud SaaS Platform and Cloud Appliance

We are really pleased to announce that we have added OpenStack Swift object Storage support to the SME Open Cloud Platform. Swift is a sub project of OpenStack and provides a highly scalable redundant unstructured data store. Swift is 5 separate services, object, container, account, auth and proxy. Although each of these can be scaled separately, in practice they run together.

Never heard of Swift? it’s the underlying distributable object store that supports RackSpace Cloud Files. It’s akin to Amazon’s S3 implementation but unlike implementations such as Eucalyptus, which clone S3 API’s, but are not sponsored by Amazon, openStack and Swift has RackSpace firmly onboard, and have proven scale.

As Swift is used by Rackspace Cloud Files. Swift RackSpace claim it is production-ready code that is scalable to massive levels (100-petabyte clusters and 100000 requests per second). Swift sacrifices C for A and P from a CAP theorem perspective. Although most operations happen synchronously consistency is sacrificed in failure scenarios.

From our perspective we have seen ISP’s and larger SMB users of our on-premise Cloud Gateway appliance expressing interest in SME supporting this, and we supply this as VMWARE Appliance (OR XEN, KVM) or as a dedicated hardware appliance for smaller companies who wish to embrace their own private Cloud infrastructure.

As with our S3 API endpoint support SME will overlay a more traditional file store on top of Swift layered with the business functionality we provide in our  Cloud File Server, which includes virtual drives and clients for Mac, Windows and Linux, and feature rich mobile clients for iPad, iPhone, Android and BlackBerry, as well as value added features to Swift such as Webdav and FTP support.

Setting up Swift with SME is easy. First you need to add a new Cloud Provider and then the Cloud Wizard will be invoked. The first step is to enter your OpenStack details:

When entering the endpoint URL you should be sure to include the Port. An example URL is: http://<IP Address>:11000/v1.0.

Next you will need to choose which containers you want to work with and which should be the default container for any uploads to smart folders.

Once you have done this you will be ready to start the meta-sync which pulls in and caches all the information about containers and files.

If you have any issues connecting please refer to this advanced post on using SME with OpenStack 1.60 and SWAuth.

Once complete you will be able to access/amange your OpenStack files from the SME Web clients,  as well as using a Cloud Drive on Windows, Mac or Linux, and mobile clients for Android, iOS, and BlackBerry, and  the plethora of other tools and clients that SME provides. We’v e posted some screenshots below of this.

Web File Manager

iOS OpenStack

Android OpenStack

Firefox Plug-In OpenStack

Chrome OpenStack Plug-In

Mac Cloud Drive OpenStack

The OpenStack Swift API’s also get embedded for use within our own feature rich multi-cloud API framework in which we add many business driven features.. You can find details about that on our developer page

Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather