Testing Cloud Storage Reliability – using the Cloud to test the Cloud

We often get asked about how we deal with SLA’s so our Operational team thought a blog post exposing some of these processes would be a good place to start !

For front end web testing the team use selenium (selenium-webdriver) but over this they use Watir, which makes it possible to write test with the Ruby language.

The below shows automated tests using all the variety of different options in the SME Web File Manager.

Watr Selenium cloud testing

We also however deal with many Cloud Storage endpoints and for us to be able to understand the reliability of these endpoints we also run automated tests against these every 10 minutes. These test reports are sent to operational administrators and if tests fail we send an email to a SaaS service called Page Duty provides SaaS IT on-call schedule management, alerting and incident tracking.

The below report shows one of these test, in this case the adding of Cloud Providers from different storage services and syncing of meta-data.

SME Cloud Testing

For monitoring the site availability, we use a combination of tools, both Pingdom and Server Density.

Pingdom is used to assess site availability and timing. Server Density is used to monitor the server cluster metrics from our US sites (Atlanta and Phoenix) and our European sites (Amsterdam and United Kingdom).

An example of metrics from two of our servers using Server Density are show below:

Server Density metrics

The key thing about the server metrics is that Alerts can be set against them and depending on the level of escalation required these two can be configured to be either sent by ServerDensity over SMS or sent from ServerDensity to Pager Duty to call an Admin if urgent.

The other thing that Server Density is used for us to give us a feel for quality of service against the network bandwidth. The way we achieve this is that we have separate $20 Linode instances configured that upload and download large and small files to our servers using both the API and our protocol adaptors. This enables us to assess the quality of connection over the various interfaces and again alerts are set if our own SLA for this is breached so that we can investigate further with our hosting provider.

Below is a graphical look at what this Quality of Connection Service looks like in Server Density.

bandwidth quality of service

Pingdom enables us to monitor response times (ping latency) from different locations and also monitor our uptime. Again, outages are sent via SMS to PagerDuty which calls the on duty SME engineer.

Below is an uptime report for our SME EU Server cluster. Of the 11 reported outages, 10 where planned and the majority of the downtime was planned and was done out of hours and related to upgrading our core EU server infrastructure. If we take this into account then for the last year SME EU is running at 99.996% uptime.

SME Pingdom report

Other automation that we use is the use of Chef to build our Cloud Appliance, both on IaaS infrastructure, for internal use and, if the customer wants a bare metal install rather than having the SME hypervisor delivered as a software Appliance.

We hope this has been a useful overview and welcome any feedback.

Facebooktwitterredditpinterestlinkedinmailby feather

Storage Made Easy: Reflections on 2012 and looking forward through 2013

Now that 2012 has drawn to a close and we are in mid January we thought it would be useful to reflect on what we have seen through our own business as trends and pointers, look at some of the things we have done over the course of the last year and finish by looking at what Storage Made Easy brings into 2013.

Key Trends:

A key trend for our business was a switch from online personal and SMB clients to more direct channel and corporate clients using our hybrid Appliance. The work we are doing with Huddle on the channel side and Xtime and Finser on the direct side are great examples of this trend. From July our business moved from a model of online Personal / SMB SaaS to Direct Corporate / Channel sales utilising our hybrid on-premise cloud appliance. Pre July the revenue split was 90:10 online SaaS and today the model is 10:90 weighted towards the direct business/channel revenue stream. We see this as a key indication that :

a. Larger companies want to ‘own’ their own data and are focused on making their storage integrate with what they already have such as their existing identity management systems (in many case Active Directory) and existing structured data systems.

b. Companies want to stop staff using un-authorised services and have come to the conclusion that they need to put structures in place to not only prevent this but to offer an alternative.

b. Companies are focused on the data legislation and governance of structured data so that they can track all file events, a mandatory requirement for certain verticals, such as government and healthcare for example.

c. The ability to search across disparate data sets easily and also on the move is becoming more and more important for companies as they try to make productive use of their core corporate asset, their data !

Our Focus for 2013:

Our Focus for 2013 is going to be as follows:

1. Simplifying our pricing proposition. The result of this will be visible over the next few weeks. We have tried to be as flexible as possible with what we provide to users but we find that this can result in users being paralysed by the paradox of choice so we will be making these much easier to understand.

2. Concentrating on formally launching our on-premise cloud Appliance. Having done a limited release launch in 2012, we will be shortly making available the ability to download the Appliance direct from our site.We have spent a fair bit of time re-designing the installation procedure with regards to networking, https keys etc,  to make it easy to install and get going. Below is an overview of the Architecture of the SME Appliance.

3. Enabling easy IaaS deployments so that users and companies can easily deploy their own personal or company cloud onto IaaS infrastructures. We already do this for Linode  but we will be expanding this to Amazon EC2 (and maybe one other provider we cannot discuss yet) and making the whole process an easier click through process.

We had a great growth year in 2012 and we look forward to continuing that in 2013.



Facebooktwitterredditpinterestlinkedinmailby feather