What’s new for Microsoft Azure Storage at TechEd 2014

At TechEd, we introduced a number of exciting changes in addition to release of a new service which is first of its kind amongst all cloud service providers. If you missed the talk on Azure Storage service at TechEd, you can view it here. Here is a brief list of things we spoke about:

A new service called Azure Files
Increase in storage capacity limits to 500 TB per storage account for existing and new accounts.
Increase in storage bandwidth limits to 10 Gibps Ingress and 30 Gibps Egress per storage account located only in U.S regions for existing and new accounts.
General availability of our Import/Export Service
A new REST version 2014-02-14 that includes Azure Files and a new Shared Access Signature (SAS) feature that allows users to control the REST protocol version used to process the request independent of the SAS token version.

We are also releasing an updated Microsoft Azure Storage Client Library 4.0 here that supports the features listed above and can be used to exercise the new features. In addition, a new version of the Storage Emulator 3.2 can be found in here that supports the new Storage version for Azure Blob, Azure Table and Azure Queue. Note that the emulator does not support Azure Files.

Introducing Microsoft Azure Files

Microsoft Azure Files provides a file share in the cloud via SMB 2.1 protocol. This file share can be used by applications in the cloud to share files between VM instances within a same region. In addition, it provides a REST interface for the file share to be accessed from anywhere at the same time the files are being accessed by VMs via SMB. More details can be found at in this blob and this MSDN page.

Increase in storage limits

All storage accounts in all regions now get 500 TB capacity per storage account.

In addition we have increased the bandwidth limits for storage accounts located in US regions to the following:

For Geo Redundant Storage (GRS) Accounts

Ingress=10 Gibps
Egress=20 Gibps

For Local Redundant Storage (LRS) Accounts

Ingress=20 Gibps
Egress=30 Gibps

Bandwidth per storage accounts in other regions remains unchanged. More details in here.

General Availability of our Import/Export service

Our import/export service is exiting preview and now available in Europe and Asia Pacific regions too. Customers need not apply for tokens and can now import/export data from/to storage accounts by shipping disks. More details can be found at this blog post.

Changes in Shared Access Signatures (SAS)

Before we go into more details about the change, we will briefly describe the versioning scheme in Azure Storage and what Shared Access Signature (SAS) is.

Shared Access Signature

Shared Access Signatures (SAS) are pre-authenticated tokens with which clients can provide access to intended storage resources with specific permissions and without having to share their secret keys. This authentication model is very useful for scenarios like mobile applications, services that require access to storage resources but cannot store secret keys because of security or compliance. Owner of storage resource can control the following parameters while creating a SAS token:

The resources that user can access using the token
The start and end time for access
The permissions (like read/write etc.) allowed

Anyone who has the SAS token can access the resources allowed with the provided permissions on the resource(s). More information can be found in this MSDN page.

Applications typically use a single REST version to perform their REST calls to Azure Storage. The storage service introduces a new version whenever new service features are added or breaking changes (syntax or semantics) in the protocol occur. Applications can control the REST version via the x-ms-version header. However for SAS, the query parameter sv is used as the SAS authentication version and as the request processing version. It is worth noting that sv is determined by the application generating the SAS token. This could be a problem when clients using SAS token to access storage resources may want a different version than the one specified in sv.

Example:https://myaccount.blob.core.windows.net/mycontainer?restype=container&comp=list&sv=2013-08-11&si=readpolicy&sig=a39 %2BYozJhGp6miujGymjRpN8tsrQfLo9Z3i8IRyIpnQ%3d

Changes in 2014-02-14 version

As mentioned before, SAS token specifies an sv query parameter which decides the version used for interpreting the SAS parameters and the REST protocol. This introduces a problem for applications in which a service generating the SAS token and client service consuming the SAS token version use different versions of storage REST APIs.

To resolve this issue, we have decoupled the versioning of a SAS token from the REST protocol version. Starting in version 2014-02-14 (i.e. sv=2014-02-14), we have introduced a new query parameter api-version that can be appended by client applications dependent on SAS tokens to specify which protocol version that storage service should use to process the request.

Example:https://myaccount.blob.core.windows.net/mycontainer?restype=container&comp=list&sv=2014-02-14&si=readpolicy&sig=a39 %2BYozJhGp6miujGymjRpN8tsrQfLo9Z3i8IRyIpnQ%3d&api-version="2012-02-12"

Hence these are the different parameters that control versioning in a URL to a SAS protected resource:

SignedVersion (sv) query parameter within a SAS token that defines the SAS version to use for authentication and authorization and interpret any SAS parameter that impacts authentication or authorization. This is what SAS generators would set.
API version (api-version) query parameter to define the REST protocol version to use. Client applications using SAS token should set this value to lock to a given storage service version.

More information can be found here in MSDN.

Examples:

Let’s use the ListBlobs API as an example to see how the version is interpreted. ListBlobs protocol changed starting with 2013-08-11 in which the Uri of the blob was removed from the protocol body. Client library for versions before 2013-08-11 expected this Uri to be present.

Using SAS version 2014-02-14, here are two possible scenario that can occur, controlled by the consumer of the SAS token.

List Blobs using sv=2014-02-14 version and no api-version override.

https://myaccount.blob.core.windows.net/mycontainer?restype=container&comp=list&sv=2014-02-14&si=readpolicy&sig=a39 %2BYozJhGp6miujGymjRpN8tsrQfLo9Z3i8IRyIpnQ%3d

In here, the service will authenticate and authorize access using 2014-02-14 SAS version and will also execute the API using the same 2014-02-14 for the protocol.

List Blobs using sv=2014-02-14 version and api-version override.

https://myaccount.blob.core.windows.net/mycontainer?restype=container&comp=list&sv=2014-02-14&si=readpolicy&sig=a39 %2BYozJhGp6miujGymjRpN8tsrQfLo9Z3i8IRyIpnQ%3d&api-version=2013-08-11

Here, the service will authenticate and authorize using 2014-02-14 version and execute the API using 2013-08-11 of the protocol. Hence clients using Storage client library versioned 2013-08-11 will work.

Best Practices

For generating SAS tokens, the recommendation would be to use the Storage Client library 4.0 or later for generating the SAS tokens and return the token as is to clients like the below code snippet:

CloudStorageAccount account = CloudStorageAccount.Parse(cxnString);
CloudBlobClient blobClient = account.CreateCloudBlobClient();
CloudBlobContainer container = blobClient.GetContainerReference("demo");
CloudBlockBlob blob = container.GetBlockBlobReference("phot.jpg");
string sasToken = blob.GetSharedAccessSignature(
new SharedAccessBlobPolicy()
     {
          Permissions = SharedAccessBlobPermissions.Read | SharedAccessBlobPermissions.Write,
          SharedAccessExpiryTime = DateTime.UtcNow.AddDays(1)
     });

// return the token as is or if it is Uri needs to be handed out then 
// return blob.Uri.AbsoluteUri + sasToken;
return sasToken;

For SAS Consumers, the recommendations are as follow:

For applications using Storage client library 2.0 and above: client libraries will just work when client applications are using SAS tokens that have sv set to 2014-02-14 or later.
For applications building their own REST protocol implementation: When a token with sv is set to version 2014-02-14 or later, you can append api-version query parameter and set it to the needed version.

Please do use this version and provide us feedback via comments on this blob, MSDN forum or stackoverflow.

Microsoft Azure Storage Team