So I am working on various solutions to monitor our deployed Azure Stacks. My first choice, of course, is System Center Operations Manager (SCOM) since I had many hours planning, deploying, configuring, and fixing SCOM for years. However, I have stepped away from SCOM and when I come back to work with it I feel a bit of a newbie at times until the gears in my head start to slowly turn and bring back some of the knowledge I may still have. With my experience with SCOM, I never had to monitor any Cloud solutions. Everything was always on premise so monitoring anything in Azure Stack and Azure, to be honest, is new to me.
I have already deployed the Azure Stack Management Pack for SCOM a while back and talked a little about it in a previous blog: “Azure Stack and Operations Manager – Goes Together Like Peanut Butter and Chocolate.” It is a great management pack and was very easy to configure. However, this management pack only monitors the Azure Stack Infrastructure.
When we say Azure Stack Infrastructure we are not including the actual physical hardware provided by your OEM. That is also something that should be monitored but won’t be monitored with the Azure Stack MP. What the Azure Stack MP monitors is the internals of Azure Stack such as the Resource providers (not SQL, MySQL, and App Service), scale unit health, region health, update statuses, etc. I needed a way to monitor all my infrastructure that was deployed to support the App Service, SQL, and MySQL RP’s. Since these are not monitored currently as part of Azure Stack Infrastructure we would need to monitor them at the tenant level.
In a blog series that I am also currently writing I go into how I deployed the App Service Resource Provider in a Highly Available deployment. With App Service, we are allowed to deploy those VM’s within the Default Provider Subscription. However, with SQL RP those hosting servers if deployed on Stack need to be deployed in a tenant subscription and not in the Default Provider Subscription.
Monitoring Azure Stack Tenant Workloads using SCOM
Overall this is a three-step process. You need to import the correct Management Pack into SCOM, then you need to configure the Subscription that you want to monitor, then you need to add the Monitors for that Subscription. Most of it is fairly straightforward like any other management pack like this one. The only problem I struggled with was getting the information for the second step when you had to change the default settings that Azure Public Cloud uses for your Azure Stack. So the rest of this blog is broken up into 4 sections:
- Importing the Microsoft Azure Management Pack
- Configuring Subscriptions for the Azure Management Pack
- Adding Microsoft Monitoring
- Final Thoughts
Importing Microsoft Azure Management Pack
So now I am off on my adventure once again with SCOM and importing management packs, etc. The first thing I did was make sure I knew exactly what management pack I needed. If you go to systemcentercore.com you will find a good number of management packs for Azure. Some are a bit old and not necessary. Some I will not need at this time. So I imported the Microsoft Azure management pack from the catalog within the SCOM console.
Configure Subscriptions for Azure Monitoring
Once the management pack has been imported then you need to configure a subscription to be monitored. Within the Administration workspace look for the Microsoft Azure settings. Click on that and over in the window to the right you should see the Microsoft Azure Overview. If you were just adding an Azure Cloud subscription the next few steps are pretty basic. You would just click on Add Subscription, keep the default settings for the authentication mode, and on the Configure Azure endpoints, you would click next. This is where it gets different for Azure Stack. This is also where you are going to need to collect some information as well.
Endpoint Configuration Information
There are a few items we need to discover as we move on configuring this management pack to monitor our tenants on Azure Stack. Most of the information needed will be different then what was the default for Azure Public Cloud.
- Management Service URI
- Management API version
- Storage Service DNS Suffix
- Storage service API version
- Authority URI
- ARM Management URI
- Graph URI
- Audience URI
I will eliminate the fields we don’t need to change. Authority URI will remain the same at https://login.windows.net as well as the Graph URI will stay the same at https://graph.windows.net. The remaining fields will be custom to your Stack and also custom to your tenant as well.
Management Service URI
This will be your (Tenant) Azure Resource Manager. For an ASDK this would be something like https://management.local.azurestack.external. In my case, it would look something like https://management.dfw.azurestack.[Domain].com
Storage Service DNS Suffix
The storage Service suffix for my case was dfw.azurestack.[Domain].com.
ARM Management URI
I used the same as the Management Service URI. Once again I could be wrong but it worked for me. Now for the rest, it took some deeper diving to get.
Get API Information
For me, this was pretty all new as well. If you do a search you get plenty of information on API’s for Azure, Azure Stack, etc. However, trying to actually narrow it down to the correct version was a bit confusing. What is even more confusing is the fact that there are various profiles one can use. For instance, there is a Hybrid API Profile, the latest API Profile, and a standard API profile. The following Microsoft Documentation is where I started looking. It talks about the current Resource Provider API’s supported by profiles in Azure Stack. I am going to stick with the profile we use when we install Azure Stack PowerShell which is 2017-03-09 for my API’s that I will use. https://docs.microsoft.com/en-us/azure/azure-stack/user/azure-stack-profiles-azure-resource-manager-versions#details-for-the-2017-03-09-profile
Discovering Storage API.
Update: I found another way to get the API information that is a little easier than PowerShell and CLI. From within you Azure Stack Tenant Portal, go to Resource Explorer, drill down on Microsoft. Storage, look for the resourceType “Storageaccounts“, and you should see the apiVersions listed.
The Resource Provider API documentation will show that the current API version for Storage Accounts is 2016-01-01. You can also use a little PowerShell to find out if this is a supported API version or not. First, run the following PowerShell command that will list all the Resource Types for the Microsoft.Storage Provider.
Get-AzureRmResourceProvider -ProviderNamespace Microsoft.Storage | fl ResourceTypes
Next, run the following PowerShell command to get the API version of the specific resource type we are looking for. In this case, I am looking for the resource type “StorageAccounts”.
((Get-AzureRmResourceProvider -ProviderNamespace Microsoft.Storage).ResourceTypes | Where-Object ResourceTypeName -eq storageaccounts).ApiVersions
So I am assuming that we will either use 2016-01-01 or 2015-06-15. Since the Microsoft Documentation shows that our current API Profile supports 2016-01-01 I am going to use this for my Storage Services API when configuring my Azure MP.
Discovering Management API.
Update: Just like the Storage API, you can do the same for the Management API. From within you Azure Stack Tenant Portal, go to Resource Explorer, drill down on Microsoft.Resources, look for the resourceType “providers”, and you should see the apiVersions listed. (Now, I am not 100% on the “providers” part. I have reached out to a Microsoft contact to verify I am giving you correct information. However, it matches what PowerShell provides.)
Like the storage API, I wasn’t really sure what API to use for the management API version. Out of the box the management pack uses 2016-03-01 but this is for Azure Public Cloud and most likely isn’t the correct API for Azure Stack. Again, I could be incorrect so if I am I hope someone calls me out on this. So within the same Microsoft Documentation used for the Storage API, there is a Microsoft.Resource Provider with an API version of 2016-02-01. I have decided to use this API for the Management API version within the Azure management pack.
Audience URI
So, I had no clue how to get the audience URI to be honest. There is probably an easier way to do this but this is the way I did it so I am passing this on to you. My research brought up a query that allowed me to find out my audience URI for my current subscription. The command looked simple enough.
curl ‘https://management.local.azurestack.external/metadata/endpoints?api-version=2015-01-01’
This would produce a result that gave me almost all the information I needed. However, the data was cut off and I couldn’t see the complete audience URI. So I dumped the results into a JSON file which gave me a single line of information that I could read and use.
$source = ‘https://management.dfw.azurestack.[domain].com/metadata/endpoints?api-version=2016-02-01’ $destination = “c:\temp\json\newfile.json” curl $source -OutFile $destination
At the end of the results I found an “audiences” result. I then grabbed that information and used it for this field.
Adding a Subscription
Now we have all the information we need to fill out the Configure Azure endpoints configuration section we can now enter the information.
On the Select SPN Configuration option window I allow it to auto-create the SPN.
Next, you will get prompted to sign on to your account. You will then be given your service principle information along with a KEY you should copy and keep in a safe spot. If you have already added a couple subscriptions that information isn’t provided. The very last screen should show all available subscriptions that you can add. Click next to the subscription you want to monitor and click Next. Followed by selecting the Server Pool you want to use and then Add Subscription to add it to SCOM.
You should now have your subscription listed in Configured subscriptions in the Microsoft Azure Overview Window.
Adding Microsoft Azure Monitoring
Now the last step is to go into the Authoring work space and add monitoring for the Subscription we just configured.
The first thing I want to do is click on Add Monitoring Wizard. I then selected the Microsoft Azure Monitoring.
The next step would be to name the Monitor along with selecting a destination management pack.
On the next screen, you will have a chance to pick the subscription this monitoring will be assigned to. You also have a choice to do a full scan of your Azure Stack tenant environment at this time as well.
On the service types window you can select what services you want to monitor. It will show what was discovered and how many instances of each.
On the Excluded list you can create filters to exclude certain items.
Click next on the Metrics window.
Verify everything is good on the Summary window.
Once the monitors have been added you can now see them in the list under Microsoft Azure Monitoring.
At this point, you can go to the monitoring workspace and scroll through the different views of the Azure Management Pack.
Optional Configurations
Just like most management packs, there are a lot of configurations that could be done for tuning, etc. There are things that can be done on the Azure/Azure Stack side as well. I highly recommend you download the management pack guide and read the Optional Configuration section. This includes how to enable Azure diagnostics on Virtual Machines, Web Roles, and Workers, and much more. https://www.microsoft.com/en-us/download/details.aspx?id=50013
Final Thoughts
As I was working with another task before I published this blog I came across a handy PowerShell command to get the API versions a lot easier than above. The way I did it works as well but this one is puts everything in a nice list and is formatted very well.
Get-AzureRmResourceProvider | Select ProviderNamespace -Expand ResourceTypes | Select * -Expand ApiVersions | `
Select ProviderNamespace, ResourceTypeName, @{Name=“ApiVersion”; Expression={$_}}
So as you can see as I work with Azure Stack on a daily basis I learn new things every day. Most of the time there are more ways to skin a cat as the saying goes. Same with working with Azure and Azure Stack. There is more than one way to get the information you are looking for. My guess, do what works for you if it is working and giving you the information you need. For some reason when trying to do a search for monitoring Azure Stack Tenants I couldn’t find much. Maybe it was Bing or Google but there doesn’t seem to be a lot of tenant monitoring information on Azure Stack out there yet. I am hoping this helps someone a little. At least I hope it didn’t confuse them more than it may have confused me.