Installing a Private Enterprise File Share and Sync Appliance on Windows Azure Compute Services

This blog post is a technical post outlining the steps needed to deploy the Storage Made Easy Cloud Control Gateway and Enterprise File Share and Sync solution on the Azure IaaS compute infrastructure.

Storage Made Easy provides a private enterprise file share and sync solution that can not only be used with Azure Blob Storage Data but which can also be used as a cloud security point to secure other storage points or sync and share solutions, such as Office365 and SharePoint. We call this Cloud Control and you can read more about it here.

Azure is a good choice for companies looking to utilize Public Cloud as it has very strong security and compliance framework:

– ISO 27001/27002
– SOC 1/SSAE 16/ISAE 3402 and SOC 2
– Cloud Security Alliance CCM
– FedRAMP
– FISMA
– FBI CJIS (Azure Government)
– PCI DSS Level 1
– United Kingdom G-Cloud
– Australian Government IRAP
– Singapore MTCS Standard
– HIPAA
– EU Model Clauses
– Food and Drug Administration 21 CFR Part 11
– FERPA
– FIPS 140-2
– CCCPPF
– MLPS

What you will need

You will need to provide Azure resources which we’ll refer to with the following names:

Note: The storage account must be unique in the Azure region.

Note: In most script commands, you can use upper case letters and they will be converted to lower case. However, URLs may not be converted and all objects in Azure are in lower case – so use lower case in URLs.

Tools to Work with Azure

• You will need Windows Azure PowerShell, described in:http://azure.microsoft.com/en-us/documentation/articles/install-configure-powershell/Follow the link titled “Microsoft Web Platform Installer”.
  Note: We use PowerShell because currently it is the only way to get a ReservedIP (which is an IP address which doesn’t change on every reboot).
• To use PowerShell, you need a Microsoft desktop. [We have found it easiest to do this via an Windows 2012 Server VM in Azure, because it automatically validates your login with the Azure Active Directory].
• To start PowerShell, run “Microsoft Azure PowerShell”.

As well as logging in to Active Directory, you need to set up your Azure subscription to do anything with Azure. Do this as follows:

         Get-AzurePublishSettingsFile

This starts a web page for you to sign into your Azure account. Once that is done, you can download your “publishsettings” file, e.g. to the file “Downloads\…publishsettings”.

         Import-AzurePublishSettingsFile  Downloads\...publishsettings

This loads the “…publishsettings” file into your PowerShell session.
Note: If you don’t do this, most commands given below will fail.

The “publishsettings” commands only need to be done once if you have a single command. If you want to check use:

         Get-AzureSubscription

Azure Authentication

The Azure tools require you to be authenticated with Azure’s ActiveDirectory.

[Again, One way of quickly doing this if you are not using a windows machine is to start up a Windows Server 2012 VM in Azure and run the Azure tools from there. When you use Remote Desktop, the password you enter to connect to the Windows machine also authenticates you in Azure.]

Import The Appliance Virtual Disk (VHD)

The SME appliance is provided as a blob in the SME Azure storage blob area. You must copy it to your own Azure storage account.

We call the storage account you will use “mystorageaccount” for the purpose of these instructions, but be warned: the name must be unique across all of Azure, and “mystorageaccount” is already taken, so you will have to choose a new account name and remember to substitue it in the instructions.

So, in the Azure Web Pages, create your storage account (referred to as ‘mystorageaccount’).

Then create a storage container in the storage account (referred to as ‘mycontainer’).
In the mystorageaccount page in the Azure Web pages, copy the “StorageAccountKey” by clicking onthe RHS of the mystorageaccount line, then on and then copy the primary access key. This becomes the value of the “StorageAccountKey” parameter below (replace “Fun4oP1q…” with key you copied).

Execute these two PowerShell commands to copy the VHD blob to your account, to create the “SME261_20150216.vhd” virtual hard disk as a blob in your account.:

   $destContext = New-AzureStorageContext `
           –StorageAccountName "mystorageaccount" `
           -StorageAccountKey  "Fun4oP1q....=="

   $blob = Start-AzureStorageBlobCopy `

-SrcUri "https://smestoragepublic.blob.core.windows.net/sme-appliances/SME261_20150216.vhd" ` -DestContainer "mystoragecontainer" `
-DestBlob "SME261_20150216.vhd" `
-DestContext $destContext

As well as the “StorageAccount key”, you must change the “StorageAccountName” parameter in the first command and “DestContainer” in the second command.

The commands are split over multiple lines: the backtick (`) is the escape character: when used at the end of a line, the command interpreter joins the lines. If there is any whitespace after the backtick, it won’t work: it must immediately precede the new-line character.

The backtick, if misplaced, can lead to strange errors. As a result, what we do now is to copy these multi-line commands and then join the lines into one, eliminating the (`) characters.

Increase the size of the VM (if needed)

The size of the virtual machine as shipped is 20GB.

If you need more space, either immediately or when the machine gets larger, you can increase the VM size, up to the maximum allowed by Azure (currently 127GB), as follows:

1. Shut down the VM if it is running.
2. Export the VM Image to a file – this is a fixed-size VHD.
3. Resize the VHD file, using either Hyper-V, or PowerShell ‘s“Resize-VHD” command (see Musumeci’s comment in this blog: http://azure.microsoft.com/en- us/documentation/articles/virtual-machines-create-upload-vhd-windows-server/).
4. Copy the new VHD (“newVHD”) to Azure storage and attach it to the existing VM (“OldVM”) following http://azure.microsoft.com/en-us/documentation/articles/virtual-machines-linux- how-to-attach-disk/#attachexisting.
   Note: Do not use the instructions in the article for initializing newVHD – this will erase the copy of the existing data, which we need later!
5. Do steps 5-9 of http://geekpeek.net/increase-root-partition-size-lvm-centos/ to increase the newVHD filesystem seen by Centos.
6. Import newVHD as a blob into Azure, create a VM image from it and create the VM from it. Choose the size of instance to run the VM that is appropriate to the newVHD size (e.g. >=A3 for >60GB VHD).

Create A VM Image:

You now need to create a ‘VM Image’, here called “myVMImage”.

The VM Image ties together the VHD and various configuration parameters that will guide the instantiatation of the VM. Only one VM Image can link to a VHD.

The PowerShell command is:

Add-AzureVMImage –OS Linux -ImageName myVMImage -MediaLocation ` "https://mystorageaccount.blob.core.windows.net/mycontainer/SME261_20150216.vhd" ` -RecommendedVMSize Medium

Change the ImageName parameter if you want.

Acquire a Reserved IP Address

We recommend that you acquire a Reserved IP address.

Getting a Reserved IP address takes a bit more setup than just getting a standard IP address (‘VIP’ in Azure), but the big benefit is that the IP address remains with your account as long as you want it, whereas the VIP changes whenever you shut down and restart the VM. So, with a Reserved IP Address, you only need to set up your DNS records once.

This link describes the details: http://azure.microsoft.com/blog/2014/05/14/reserved-ip-addresses. Type in the following command – changing the location to suit you, and the “ReservedIPName” parameter if you want:

    $ip = New-AzureReservedIP –ReservedIPName "SmeReservedIP" –Location "West Europe"

We use “West Europe” in the examples below. If your Azure account is in another location, change all Location paramters in the remaining instructions. You can get a list of the locations with:

    Get-AzureLocation

You can use a physical IP address to access the SME VM. To see the physical IP address, you’ll have to do this:

    Get-AzureReservedIP "SmeReservedIP"

Create a Service:

These instructions create a “standalone” VM – there is just one VM in the “service”. (More complex setups have multiple VMs or web servers.)

We will use the service when we create the VM later. Creating it is very simple:

    New-AzureService "smeApp" –Location "West Europe"

The “Location” parameter must be the same as the location of the reserved IP. We use “smeApp” for as the name of the service: as we will see, the service name becomes the name of the VM. It is more useful to use the Appliance as the name, so we use smeApp here.

Create, Start and Stop the VM

Now we use the Service and ReservedIP in creating the IP address. This action implicitly binds the ReservedIP to the service; if you use another service and want a reserved IP, you will need to create another one.

   $vmConfig = New-AzureVMConfig -Name "SmeAppVmConfig" -InstanceSize Medium `
                -ImageName "smeapp-2.6.1-20150216" -HostCaching ReadWrite`
    $vm = $vmConfig | New-AzureVM -ServiceName "smeApp" `
                –ReservedIPName "SmeReservedIP"
    Stop-AzureVM -ServiceName "smeApp" –Name smeApp

The reason we have to stop the VM is for the next step, where we remove the automatically allocated endpoint and create the ones we expect.

Update Endpoints and VM, start VM

Now we adjust the PowerShell object “$vm” created above and then save its changes to Azure.

The SSH endpoint is attached to a public port other than 80, so we need to adjust that, and then add the HTTP endpoints. The “Get-AzureEndpoint” just dumps the status of the VM’s endpoints.

$vm | Remove-AzureEndpoint SSH
$vm | Add-AzureEndpoint -Name "SSH" -Protocol "tcp" -PublicPort 22 -LocalPort 22
$vm | Add-AzureEndpoint -Name "HTTP" -Protocol "tcp" -PublicPort 80 -LocalPort 80
$vm | Add-AzureEndpoint -Name "HTTPS" -Protocol "tcp" -PublicPort 443 -LocalPort 443
$vm | Get-AzureEndpoint -Name "HTTP2" -Protocol "tcp" -PublicPort 8080 -LocalPort 8080
$vm | Update-AzureVM

“$vm” is an object in PowerShell. Some commands require the definition of a parent “object”:

For example, the $vm is the parent of the “AzureEndPoint”. Piping the $vm object into these commands is easier than specifying the name in the ‘Add-AzureEndpoint’, ‘Get-AzureEndpoint’ and ‘Update-AzureVM’ commands.

And finally, we can start the VM:

Start-AzureVM -ServiceName smeApp –Name smeApp

Configuring the SME Server:

Now follow the setup instructions at:

https://storagemadeeasy.com/wiki/cloudappliance/applinstallation/

These notes give you additional guidance on setting up the DNS etc. for Azure if you are not an experienced Linux networking engineer.

SME Server Hostnames Settings

The SME appliance has a built in protocol gateway that can serve different content using this protocols, even if the underlying storage does not support these protocols natively.

• –  The ‘domain name’
• –  S3
• –  WebDAV
• –  FTPS

sme.yourdomain.com s3.yourdomain.com webdavsme.yourdomain.com ftp.yourdomain.com

For some Microsoft applications to work properly, the WebDAV URL must be “webdav” followed by the basic domain – and that is without an intervening “.”, so not “webdav.sme.yourdomain.com”.

You need to enter the “Domain Name” for all deployments. Enter WebDAV and S3 if you plan to use them (and you can redo this section later if you want to start using WebDAV or S3).

SME Server Network Settings

If you use a standalone VM in Azure as recommended, you do not need to set anything here:

SSL Certificate Settings

If you will be using the SME appliance for testing, then you can realistically use a self-signed certificate, as outlined in the installation guide. If you will be providing a public service to users who may want to be reassured of your server’s identity, then you should buy a certificate from a Certificate Authority (CA).

If you do use a self-signed certificate, this is considered a “weak” certificate. This means that the first time users access the SME appliance they will probably have to create an exception to allow the browser to proceed to the site, but subsequent accesses will proceed without any questions.

The examples above show how to construct the domain names so they can be covered by a single wildcard certificate, with a Common Name (CN) of “*.yourdomain.com”. This would not work if, for example, you had the S3 domain as “s3.sme.yourdomain.com”.

The first box in the page shown above is the certificate: open the “.crt” file in a text editor, copy everything including the header (“—–BEGIN CERTIFICATE—–“) and trailer lines and paste into the box. For the second box, the private key, do the same thing but using the “.key” file.

For a self-signed certificate, you leave the last box – the CA Chain – blank.

Make sure to go back to the ‘Overview’ page to check that the Hostname and Certificate information show up propertly there. Then hit “Apply” and reboot the server.

Once you have installed the certificate, you will have to set up A records to point to the three domains. To test out the installation of the certificate before setting up the A records, you can point your desktop machine to the server by adding a line to /etc/hosts on Linux or on Windows “C:\WINDOWS\system32\drivers\etc\hosts”. Add a line like this to the end of the hosts file:

    104.42.141.183  sme.yourdomain.com  s3.yourdomain.com webdavsme.yourdomain.com

(On Windows, you will probably have to alter the file’s properties via: Properties/Security/Edit Button/Your UserId then ‘Allow’ permissions – modify,read,write).

Enterprise WebDAV:

Some organisations use virtual drives in Microsoft Operating Systems for day-to-day work. Although SME provides an installable network drive, Cloud folders can also be mounted as virtual drives using a common script, using this syntax:

net use x: "https://<<WebDavURL>>/<<providerFolder>>" <<password>> /USER:<<account>>

The user and password are the StorageMadeEasy credentials for the user. For example:

net use x: "https://webdavsme.yourdomain.com/Google Drive" secret /USER:CORP_ACCOUNT

Using WebDav in this scenario can also be done in a Kerberized manner, end-to-end.

Summary:

We have stepped through a complete end-to-end guide of how to install the Storage Made Easy combined Cloud Control Gateway and Enterprise File Share and Sync solution onto the Azure Compute Services Fabric. If you wish access to the SME blob file to try this yourself then please contact us.