PartitionKey or RowKey containing the percent ‘%’ character causes some Windows Azure Tables APIs to fail

Description and Symptoms

We have identified an issue that would affect services using Windows Azure Tables whenever the percent character ‘%’ appears as part of the PartitionKey or RowKey.

The affected APIs are GET entity, Merge Entity, Update Entity, Delete Entity, Insert Or Merge Entity and Insert Or Replace Entity APIs. If any of these APIs are invoked with a PartitionKey or RowKey that contains the ‘%’ character, the user could erroneously receive a 404 Not Found or 400 Bad Request error code. In addition, in the case of upsert (Insert Or Merge Entity and Insert Or Replace APIs), the request might succeed but the stored string might not be what the user intended it to be.

Note that Insert Entity, Entity Group Transactions and Query Entities APIs are not affected since the PartitionKey and RowKey is not part of the URL path segment.

Root Cause

The Windows Azure Table Service is double decoding the URL path segment when processing a request which is resulting in an erroneous interpretation of the string whenever the ‘%’ character appears. Note that the query string portion of the URL is not affected by this issue nor is any URL that appears as part of the HTTP body. Therefore, any other property filters used in a query will be unaffected by this issue – only PartitionKey and RowKey are affected.

Here is an example of how this issue occurs: Inserting an entity with PartitionKey = “Metric%25” and RowKey = “Count” would succeed, since PartitionKey, RowKey and custom values are part of the request payload and not the URL path segment. Now, when you intend to retrieve this existing entity, the Get Entity HTTP URL will look like:

http://foo.table.core.windows.net/Metrics(PartitionKey='Metric%2525',RowKey='Count')

However due to the double decoding bug, the PartitionKey is getting interpreted as “Metric%” on the server side which is not what the user intended. In this case, a 404 Not Found is returned.

Workarounds

If you did not currently commit any entities where ‘%’ is used as part of the PartitionKey or RowKey we suggest that you consider the following:

1. Avoid using ‘%’ as part of your PartitionKey and RowKey and consider replacing it with another character, for example ‘-‘.
2. Consider using URL safe Base64 encoding for your PartitionKey and RowKey values.

Note: Do not double encode your PartitionKey and RowKey values as a workaround, since this would not be compatible with future Windows Azure Tables releases when a fix is applied on the server side.

In case you already have inserted entities where ‘%’ appears as part of the PartitionKey or RowKey, we suggest the following workarounds:

1. For Get Entity:
   • Use the Entity Group Transaction with an inner GET Entity command (refer to the example in the subsequent section)
   • Use the Query Entities API by relying on the $Filter when retrieving a single entity. While this is not possible for users of the Windows Azure Storage Client library or the WCF Data Services Client library, this workaround is available to users who have control over the wire protocol. As an example, consider the following URL syntax when querying for the same entity mentioned in the “Root Cause” section above:
     http://foo.table.core.windows.net/Metrics()?$filter=(PartitionKey%20eq%20'Metric%2525')%20and%20(RowKey%20eq%20'Count')
2. For Update Entity, Merge Entity, Delete Entity, Insert Or Merge Entity and Insert Or Replace Entity APIs, use the Entity Group Transaction with the inner operation that you wish to perform. (refer the example in the subsequent section)

Windows Storage Client Library Workaround Code Example

Consider the case where the user has already inserted an entity with PartitionKey = “Metric%25” and RowKey = “Count”. The following code shows how to use the Windows Azure Storage Client Library in order to retrieve and update that entity. The code uses the Entity Group Transaction workaround mentioned in the previous section. Note that both the Get Entity and Update Entity operations are performed as a batch operation.

// Creating a Table Service Context
TableServiceContext tableServiceContext = new TableServiceContext(tableClient.BaseUri.ToString(), tableClient.Credentials);
 
// Create a single point query
DataServiceQuery<MetricEntity> getEntityQuery = (DataServiceQuery<MetricEntity>) 
     from entity in tableServiceContext.CreateQuery<MetricEntity>(customersTableName)
     where entity.PartitionKey == "Metric%25" && entity.RowKey == "Count"
     select entity;
 
// Create an entity group transaction with an inner Get Entity request
DataServiceResponse batchResponse = tableServiceContext.ExecuteBatch(getEntityQuery);
            
// There is only one response as part of this batch
QueryOperationResponse response = (QueryOperationResponse) batchResponse.First();
 
if (response.StatusCode == (int) HttpStatusCode.OK)
{
    IEnumerator queryResponse = response.GetEnumerator();
    queryResponse.MoveNext();
    // Read this single entity
    MetricEntity  singleEntity = (MetricEntity)queryResponse.Current;
 
    // Updating the entity
    singleEntity.MetricValue = 100;
    tableServiceContext.UpdateObject(singleEntity);
    
    // Make sure to save with the Batch option
    tableServiceContext.SaveChanges(SaveChangesOptions.Batch);
}

Java Storage Client Workaround Code Example

As the issue discussed above is related to the service, the same behavior will exhibit when performing single entity operations using the Storage Client Library for Java. However, it is also possible to use Entity Group Transaction to work around this issue. The latest version that can be used to implement the proposed workaround can be found in here.

// Define a batch operation.
TableBatchOperation batchOperation = new TableBatchOperation();

// Retrieve the entity
batchOperation.retrieve("Metric%25", "Count", MetricEntity.class);

// Submit the operation to the table service.
tableClient.execute("foo", batchOperation);

For more on working with Tables via the Java Storage Client see: http://blogs.msdn.com/b/windowsazurestorage/archive/2012/03/05/windows-azure-storage-client-for-java-tables-deep-dive.aspx

Long Term Fix

We will be fixing this issue as part of a version change in a future release. We will update this post with the storage version that contains the fix.

We apologize for any inconvenience this may have caused.

Jean Ghanem