Creating an Application Deployment Gallery Item – SQL Gallery item

By Azure Resource Manager, Compute, DSC, Microsoft Azure Stack, Powershell, SQL, VMExtensions, , , , , , , , , , , , , ,

Welcome to the second blog post (of 2) on creating a complex application deployment gallery item.

Now we got all fundamentals in place, we need a deployment template to thigh everything together into a deployable artefact and expose that to the portal via a Gallery Item.

Microsoft made a template available on GitHub to get you started with creating this complex gallery item: https://github.com/Azure/azure-quickstart-templates/tree/master/marketplace-samples/simple-windows-vm

For this example, we keep things simple just to prove what can be done. A lot more flexibility can be achieved by utilizing nested templates in combination with option controls for example.

Let’s create the Gallery Item!

Directory structure

For this case, we’ll use the directory structure as displayed in the image.

The directory structure is in no means mandatory as files are referenced using relative paths.

Deployment Template

The Gallery Item in its core is a UI wrapper around a deployment template. The deployment template will be stored in the DeploymentTemplates folder and will be called “mainTemplate.json”. Current experience tells that the filename of this artefact cannot be any different, if you don’t use this name, things just won’t work. The template is declarative and speaks largely for itself so I won’t go into detail about it.

Please note the parameter mapping to the DSC extension where the SQLPID and username / password are send to the secure settings so their values won’t be visible at the resource group deployment history when the deployment is done.

CreateUIDefinition

The UI in the portal is generated from the CreateUIDefinition.json. The portal exposes a plural off controls which can be leveraged. The main goal is to gather the user input for the deployment template to consume. The main outline of the CreateUIDefinition.json looks like this:

It is possible to add a schema reference during editing to help you with intellisense like completions, the schema is currently the best documentation out there to know what is possible, so take a look here if you are interested in more than documented here).

First let’s view the CreateUIDefinition and then explain the controls that are used and how they are rendered. The CreateUIDefinition.json is also stored inside the DeploymentTemplates folder.

The CreateUIDefinition rendering can be previewed before you decide to package it up by having the portal gather it externally. Use the following code to do this (I assume you already are connect with your Azure Stack environment using a Service Admin account, if you are not, see here how this is done. I also assume you created the “tenantartifacts” storage account in the previous blog post):

This will upload the json to a storage account and construct a uri which tells the portal to load the json and display the content.

Now let’s look at how this json is build up.

Basics

Basics is a mandatory blade which always exist. Even when you decide not to populate it with controls, it is still used for things like Resource Group creation, location and subscription selection.

In our case, we decided to populate the basics blade with input parameters:

Which is rendered as:

The input parameters:

    • vmName: uses the Microsoft.Common.Textbox type. Input is required and the value is constrained to a pattern match with a defined regex (3 to 15 characters). The input is used for ComputerName and VMName.
Between 3 to 15 chars:
Under 3 chars:
Over 15 chars:
    • adminUsername: uses the Microsoft.Compute.UserNameTextBox type. It is targeted for a Windows deployment and used for the Administrator user and potentially SQL login account (if the SQL authentication mode is set to SQL). The input is checked for illegal entries by portal native defined patterns.
Cannot contain illegal characters:
Between 1 and 15 chars:
Cannot be a single digit:
Can be a number over 1 digit:
Can be a number followed by a letter:
Can be a normal username:
    • adminPassword: uses the Microsoft.Compute.CredentialsCombo type. Length and complexity is guarded by the portal, input is validated but can be hidden by setting hideConfirmation to true.
Between 12 and 123 characters
Complexity requirements:
Valid length:
Valid length, invalid confirmation:
hideConfirmation true:

Basics can contain elements of the following types:

Steps

Multiple steps are defined and each form a blade.

Resource Customization step

Right after the Basics, the Resource Customization step is defined.

Which is rendered as:

The input parameters:

    • vmSize: uses the Microsoft.Compute.SizeSelector type. It is constrained to a certain amount of allowed sized and has a couple of recommended sizes. The osPlatform, offer, publisher and sku properties are used for billing purposes (cost prediction) not (yet) available in Azure Stack.
Default size is first in recommended sizes:
The recommended sizes:
All allowed sizes:
    • sqlDataDiskSize: uses the Microsoft.Common.Textbox type. Input is required and the value is constrained to a pattern match with a defined regex (numbers only). The input is used to specify the Data Disk size in GB.
Numbers allowed:
Letters not allowed:
Must contain value:
  • sqlLogDiskSize: uses the Microsoft.Common.Textbox type. Input is required and the value is constrained to a pattern match with a defined regex (numbers only). The input is used to specify the Log Disk size in GB.

SQL Settings

The SQL Settings is the last defined step.

Which is rendered as:

This time, there is a context sensitive submenu which only appears when the SQL Features contain the SQLENGINE feature.

The SQL Engine settings are bound within a section called “SQL Engine”. You can see the label of the section appear above the elements that belong to the section. The section has the visible attribute defined which is linked to the sqlFeatures parameter outcome. “[contains(steps(‘sqlSettings’).sqlFeatures, ‘SQLENGINE’)]” defines if the sqlFeatures outcome contains the value SQLENGINE.

The input parameters:

    • sqlProductKey: uses the Microsoft.Common.Textbox type. No input is required and the default value is an empty string. If the user wants to bring in their SQL license key, they can do so via this input or later on after installation is finished (you cannot downgrade a trail installation to standard as it defaults to enterprise feature set).
    • sqlFeatures: uses the Microsoft.Common.DropDown type. There is no default set but a selection is required. The DropDown box has a set of predefined items from which the user can pick. The user interacts with the labels and the values are used internally.
Required:
Dropdown listing:
Selection:
    • sqlInstanceName: uses the Microsoft.Common.Textbox type. Input is required and the value is constrained to a pattern match with a defined regex (valid Instance names only, https://technet.microsoft.com/en-us/library/ms143531(v=sql.90).aspx). The input is used to specify the SQL Instance Name. Defaults to MSSQLSERVER, which is also referred to as the default instance.
Valid instance name:
Tooltip hints:
Value required:
First letter not being underscore or a letter:
Over 16 characters:
Keyword:
Valid complex name:
Illegal characters:
    • sqlInstancePort: uses the Microsoft.Common.Textbox type. Input is required and the value is constrained to a pattern match with a defined regex (numbers only). The input is used to specify the Port number on which the SQL instance listens. Besides the TCP listening port, there will also be a Windows firewall rule and NSG allow rule created for this port.
    • sqlAuthMode: uses the Microsoft.Common.OptionsGroup type. A user selects the desired authentication scheme used by the SQL Server. SQL or Windows with default to SQL. If SQL is selected, a SQL login will be created using the provided administrator username and password.
SQL selected:
Windows selected:
Tooltip:

Outputs

Finally, the CreateUIDefinition contains an Outputs section.

This does not generate any UI but instead passed all user defined input to the deployment template. When you have developer-tools opened in your browser, you can see the outputs send to the console stream if you are using the CreateUIDefinition preview functionality:

If you have selected only to install Integration Services, you will notice all outputs belonging to the SQLENGINE section are not present. They are also missing from the summary blade:

Images

Now we have the hard part done, we need some pictures so the resulting gallery items will be quickly discoverable.

Name Pixels Image
Wide 255×115
Large 115×115
Medium 90×90
Small 40×40
Screenshot0 533×324

UIDefinition

The UIDefinition.json is a bit of a mystery still. The schema uri doesn’t lead anywhere and there is no documentation (yet). The following UIDefinition works though and it should be saved in directly under the root folder. From the json we can see that it contains portal instructions on what blade type and extension to use.

Manifest

Finally, we need to reference everything in a manifest file.

It is possible to add localized data from resjon files into this manifest by using the ms-resource references. In this case I chose to ignore most of this possibility to avoid addition of unnecessary complexity. At a bare minimum, though, the AzureGalleryPackager checks for longSummery and description to come from this (default / fall-back) resjon file.

Before we can create a gallery item we need to add a strings folder to our gallery item and add a resources.resjon file containing the following:

All metadata concerned with the gallery item is contained within the manifest. Also, the references to the images and deployment templates are relatively defined.

The gallery item will show up in its own category which is defined in the manifest as well. In this case, “@bgelens”.

Warning:

It is possible you created the manifest using a different schema. The AzureGalleryPackager has support for 3 sets of schema (2014-09-01, 2015-04-01 and 2015-10-01). I had to use ILSpy to peak into the AzureGalleryPackager dlls to figure this out. Depending on the schema, different keys and constructs are expected. In this case the latest version is used which expects the images construct for example. The older versions expect an “icons” construct instead. Also, the 2014 version doesn’t support the custom artefact type which is necessary for what we try to achieve here (in other words, the 2014 version is not compatible with the intend).

Creating the azpkg

Now we edited the files, it’s time to create the azpkg we need to add the Gallery item to the portal. Download the Azure Gallery Packager, unblock it and extract it somewhere you can find it. Let’s move the folder of our gallery item to the root of C. Now open a command prompt or PowerShell console and navigate to the packager and run: .\AzureGalleryPackager.exe -m C:\Sql2016SP1onServer2016Core\Manifest.json -o C:\MyMarketPlaceItems

Note you should create the target folder before as the packager will otherwise fail.

You should now have a marketplace item in azpkg format. If the packager did not work out, you probably have made some sort of typo somewhere or it just doesn’t like the paths you provide.

Adding the Gallery Item to Azure Stack

Earlier on, a storage account was created which is going to be used again. If you did not create the storage account, please do so now by following the instructions from uploading the OS Image.

Run the following script to add the azpkg to Azure Stack (I assume you already setup the Azure Stack connection in PowerShell).

You should see a message with StatusCode Ok (if you get created, just run it again to be sure).

Now we login to the portal, wait a bit, refresh a couple of times and eventually we get:

Deploying the Gallery Item

Now we have added the Gallery Item we can deploy it.

We fill out the Basics with the Administrator user and password plus the VM name. Also, we create a new resource group with this deployment. At the Resource Customization step we define we want a 200GB SQL Data disk and a 100GB Log disk.

Next, at the SQL Settings step, we define we want both the SQL Engine and IS, name the instance and provide a non-default port, 10001. Finally, we hit the summer where the portal validates the input. The tenant user can download the deployment template for quick redeployment.

The portal adds a Buy step where the tenant user agrees with the purchase. At this time, we cannot disable this behavior. After the purchase is done, deployment is started.

The tenant user can follow the deployment as it is happening. In this screenshot, we can see that currently the VM is being deployed and all prerequisite steps where successful.

Now the deployment is finished, we can validate that we get what we expected.

First let’s look at the NSG to see if the custom SQL port has been opened.

As you can see, the 10001 port is opened. Let’s check the DSC extension status.

Finally, let’s look up the public IP address and open a connection with the deployed SQL instance.

The IP address is 192.168.102.27.

    

Testing in production

Developing and iterating on the gallery items is probably something you do not want to do in public. Having a secondary Azure Stack available for this development can be costly so luckily we can easily develop in production without tenants seeing what we do.

First, as we already explained before, you can preview the CreateUIDefinition by uploading it to a public location and have the portal pointed towards it.

Next, it is also possible to add a filter to the Gallery Item manifest which allows you to hide the Gallery Item from general sight.

When you use this approach, you can find the Gallery Item by appending a query key to the azurestack uri. E.g: https://portal.azurestack.local?microsoft_azure_marketplace_ItemHideKey=MyNewGI

Resources

I found a GitHub repository explaining more of the available UI elements and how they are rendered. If what you need is not in here and not obvious through the schema, take a look here. Also, the Gallery Item GitHub repository for Elasticsearch has been a great resource as well as the Azure Portal documentation.

Conclusion

That’s it! I hope this helps you with creating your own Gallery Items. This was a pretty simple one, but imagine this example to be extended with a number of VMs parameter making use of the copy functionality and having settable options like, enable contained database support and others. You could use this as an easy way to deploy your SQL Resource Provider SQL hosts with.

You could also make more complex multi-tier application Gallery Items like SQL AlwaysOn or a RDS farm for example. Finally, we won’t be held back anymore. Gone are the days with the VM Role limitation of Azure Pack!

Remember, this has been tested on TP2 and because of that, in no means I can guarantee it will work on TP3 and on. But if this happens, be sure I’ll start my journey all over again 🙂

You can find the source files I used, together with the DSC resources, scripts and azpkg on my GitHub repo here!

Spread the word. Share this post!