In this post we are looking at the Azure Stack Consistent Storage. We will look at how the tenant can consume the storage resources and how a service administrator can provision storage services for tenants to consume them. If we look at the storage landscape, we have to today in the private cloud we see below in the picture four areas:
- Private Cloud – SAN and NAS storage
- Private Cloud with Microsoft SDS – Microsoft Azure Stack Storage
- Hybrid Cloud Storage – StorSimple with Azure storage
- Public Cloud Storage – Azure storage
In this blogpost we are going to zoom in at the second option. Private Cloud with Microsoft SDS aka Microsoft Azure Stack Storage. So what is Azure-consistent Storage look like? It is:
- Azure-consistent blob, table and account management services for private and hosted cloud deployments
Software Defined Storage (SDS)
- Extends Microsoft Software-defined Storage (SDS) stack with virtualized Azure-consistent Storage Services, SOFS, Storage Spaced-based data and services
- Storage cloud services for Microsoft Azure Stack
- Azure Storage cmdlets, APIs and ARM templates work with no changes.
Azure-Consistent Storage builds on existing Microsoft Software-defined Storage (SDS) technologies. This contains Storage Pools, Spaces, ReFS, Cluster Shared Volumes, Scale-out File Server, SMB 3.1.1, SMB Direct and introduced some new features as well like Storage Spaces Direct(S2D).
If we look at the role of the Storage Tenant Administrator. He creates a subscription; within this subscription the tenant administrator creates a resource group. In the resource group you can create a Storage Account. In the Storage Account the tenant then can create containers with blob storage (Block / Page) and/or Table storage. The tenant administrator also manages the security for the storage account. You can share temporary permissions (token-based) or longer-term (key-based) access. This is exactly the same concept as you experience in Azure today.
In Azure Stack you can distinguish the user roles in 3 levels:
Storage Tenant Administrator
- Consumes storage services from a service provider
- Develops cloud storage applications
Storage Service Provider
- Provides the storage services to customers on shared infrastructure
If we look at the Storage Service Provider, we can divide them in 2 separate roles:
Storage Fabric Administrator
- Manages traditional storage fabric life cycle
Storage Service Administrator
- Manages storage cloud service life cycle
If we look at the persona perspective, we see that the storage tenant can consume PaaS with Azure block blobs and tables and run IaaS with Azure page blobs and SMB. The storage developers develop to the same Azure APIs their applications. For a Service Provider they offer a rich service administrator experience using web-based tools and PowerShell.
Let’s look closer at the Storage Service Administrator role and how they can leverage these service at large scale. If we look at the required infrastructure needed you will see that you can leverage your current investment in storage by using a SOFS cluster connected to traditional SAN/NAS storage. This is great to start for current service providers to deploy Azure-consistent Storage. Other options are a traditional SOFS with JBOD, the new flavor of SOFS using Storage Spaces Direct (S2D) or even you can implement Hyper-converged where you have a Compute and Storage cluster using Storage Spaces Direct (S2D).
The storage fabric administrator uses existing storage fabric management tools as an example VMM, PS/DSC to provision Azure-consistent Storage. The storage cloud service administrator then deploys & manages storage cloud services and manages the entire lifecycle of blob, table and management services.
To provision storage services the storage cloud service administrator creates storage “offers” for tenants to consume. An offer is a package of cloud services that tenants can then obtain.
Before tenants can start consuming storage as a service we also need to provision the storage resource provider in a region. We look at this in more detail in an upcoming section. A cloud service administrator creates desired number of MAS “Regions”, one storage Resource Provider per region. Then he creates and add storage fabric resources (example file shares) into the “Farm” for that region. A farm is a collection of all the storage resources. It can be multiple Scale Out File Servers and File Shares. Then the administrator include the storage as a cloud service in a plan and set quotas on the storage service. You might want to add more cloud services in the the offer. And that’s it. The offer is now avaialble for tenants.
Now let’s see what resources are needed to provision this Azure-consistent storage service. We start from left to right were we have a bunch of virtual machines, File shares, storage replication, SOFS clusters, performant networking and load balancers. From these resource we create Azure Service Fabric Clusters and create the front-end, Table and other virtualized services. We have the blob service installed on the SOFS cluster. Then the Storage Resource Provider management services that we will integrate in Azure Stack. And on the right side we have Storage cloud services in production.
That’s in a nutshell. Now let’s look closer at this architecture and look at fabric components in a logical order. We see here below multiple front-ends load balanced by a hard or software load balancer. You can dynamically scale these front-ends without service interruption. A layer below you see the high performance networking for the SMB3. Below that we have a SOFS cluster and installed on the SOFS cluster the blob backend. This can also be scaled linear by adding more SOFS nodes. It is even possible to add multiple SOFS clusters to this deployment.
Now let’s add on top of the fabric components the tenant management components. If we look at a region we see here a Provider Region. In this region other cloud services may exist like compute, network, web, SQL etc. It’s important to remember that each region can have 1 resource provider for each cloud service. So for the storage cloud service we see a farm for the Storage Resource Provider. This manages storage account life cycle in that specific region. It also exposes the Storage management API for tenant to consume.
Through the same Storage Resource Provider farm the Storage Service administration is performed. In the portal the right privileges are assigned to Storage Service Administrators to perform their tasks. As mentioned before you can manage storage with the portal, PowerShell or through tools using the management API from the storage resource provider
In the next blog post we will continue looking at the Azure-consistent Storage solution and will look at the Storage account types used and look in more detail how blob and table storage is working under the hood!