Steve Spencer's Blog

Blogging on Azure Stuff

Using Azure DevOps to Restart a Web App

Recently I had an issue with one of the web sites I was supporting and it seemed to be falling over each night and it was difficult to work out what was wrong. Whilst I was working it out I need a mechanism to restart the website over night so that I could then take the time to figure out exactly what was wrong. There were a number of ways to achieve this but the simplest one for me was to use an Azure DevOps Release pipeline triggered on a schedule. The Azure DevOps “Azure App Service Manage” task allows me to achieve this.


To get started create a new release pipeline with an Empty job


Click on the “1 job, 0 task” link and then click on the “+” in Agent Job.


Enter “App service” in the search box and select “Azure App Service Manage” from the list of tasks that appear and click “Add”.


The task will default to Swap Slots but you can change this. Select your Azure Subscription and click “Authorize” if you haven’t already authorised your Azure Subscription.


Clicking “Authorize” will take you through the sign in process where you will need to enter your username and password for the Azure subscription that contains your app service. Once authorised select the Action you want to perform. Currently the list contains:


Select “Restart App Service” then pick your app service from the “App Service name” list, Also change the display name as it defaults to “Swap Slots:”


Your pipeline is now configured to restart your app service.


You now need to trigger this. Click on the pipeline tab


Then click on the Schedule button


Enable the trigger and select your desired schedule, edit the release pipelines title and click save. Your web site will now restart based upon the schedule you picked.

You can restart multiple web apps with a single release pipeline


You are able to chain each website or do it in parallel by changing the pre-deployment conditions:


To chain them select “after stage” and in parallel select “after release” for each stage.

When the schedule is run a new release is created and the web apps will restart and you will be able to see the status of each attempt in the same way you do with your standard releases.

Adding Security Policies To Azure API Management

The Azure API Management service allows you to publish your APIs both internally and externally and to control who and what can access them. Out of the box you will get a standard API key for each of you users who sign up to the API, but this is often not enough meet the security requirements for you or your partners. API Management allows you to add a more fine grained security model you each of your APIs and this can be done using the policy feature. Policies are used for more than just security and there are numerous policies that allow you to change the behaviour of your API through configuration. Documentation for the types of policies can be found here. Sample policy examples can be found here.

Two policies that I am going to discuss here will allow you to restrict access to your API through IP Whitelisting and through validating JWT claims. I will also discuss how you can put different controls onto your API for different partners.

Policies can be set at different levels and the documentation will highlight the areas where they are applicable. For security policies I am going to talk about protecting at the API level and at the product level. Adding a policy at the API level will be applicable to all subscribers to the API whereas adding the policy at the product level will be applicable to all subscribers to the product. A product can contain multiple APIs and and API can be in multiple products. So we can add in protection at either level depending upon what your exact requirements are. The policies are the same but their impact will depend upon where they are applied.


Lets start with API level policies. To add or edit policies then you need to navigate to your API in the Azure Management portal. Then click on the API option, then click on the API you wish to protect


The easiest way to add a policy is to click the Add Policy link in the inbound section.


Click Filter IP Addresses and Add IP Filter


This form allows you to add ranges or single IP addresses to both allow or deny.When you have finished click Save.

You will now see the policy in the policy editor view. If you are happier to add this in manually or want to copy this and version control the config then you can access this via the Code Editor menu on the Inbound processing policies box



Appling this policy on the API means that only IP addresses within this range can access this specific API and can be useful to ensure that this specific API is blocked from being accessed regardless of the product has been subscribed to. Its also useful if you want to block access from specific IP addresses. However, you may have different partners who have different security arrangements or that you want to give different permissions to . To allow for this you will need to add the policy at the product level.

To edit the policy at the product level, click Products, pick the product you want to secure.


In this example I have a new More Secure API that I’ve created and there’s an access control section which allows you to pick the users who have access to this API


So I’ve immediately blocked access to this API to guest users and we can add user authentication to  the API if we want, such as OAuth 2.0 and OpenID connect.

However, this post is talking about adding security policies and if we want to allow only specific IP addresses to access this API we can edit the policy at the Product level. To access the policy definition click Policies


You’ll notice that this is just the editor view and the easiest way is to add the policy at the API level using the wizard and copy the config to here. Products are a mechanism to allow you to group and protect APIs which means that from a management point of view you could create a product for each of your partners making it easier to maintain the security details for each and make it easier to disable access and remove only the security policies that apply to the specific partner. Managing this at the API level means that you will end up with a large number of security policies relating to a large number of partners making it difficult to manage. Security polices at the Product level are more important when you want to do some specific protection like checking claims in a signed JWT. The Product level policy allows you to have different signing keys for each product meaning that you can have different signing keys for each of your partners (assuming one product per partner).


This policy requires a JWT signed with the key eW91ci0yNTYtYml0LXNlY3JldA== and that also has the claim admin=true. If there is an error then 401 is returned with the message “You have failed the security checks please contact your administrator”

To summarise, we can add policies at both API and product level. Product level polices allow us to create a new product for each of our partners and then add specific security policies to the product tailored to our specific partners needs. The product level policy makes it easier to manage the security policies at a partner level but we can allso add global security policies at the API level such as blocking access from certain IP address ranges. Policies can do a lot more than security so check out the links at the start of the post for further information

My Video Library

Introduction to Azure Machine Learning Studio – Video walk through recorded March 2019

Introduction to Azure Log Analytics – Recorded at Dot Net Sheffield November 2018

Azure Machine Learning for Developers – Recorded at Dot Net Sheffield November 2018

Introduction to Microsoft Flow – Video walk through recorded September 2018

Easy Integration with Flow and Logic Apps – Recorded at Dot Net Sheffield August 2018

Add Existing Logs to Azure Log Analytics – Video walk through recorded June 2018

Visual Studio Team Service Load Testing – Video walk through recorded April 2018

Introduction to Azure Service Fabric – Recorded at Dot Net Sheffield March 2017

Building Scalable and Resilient Web Apps with Microsoft Azure – Recorded at Dot Net Sheffield March 2017

Service Fabric for the Microservices Win, baby! – Recorded at Microsoft’s UK TechDays Online Sept 2016

Adding existing logs to Log Analytics

I created a video to walk you through how to add existing logs to Log Analytics. There have been some changes to the way you do this.

The location of the settings to configure this has now move to Log Analytics in the Azure Portal. Previously, this was in the Operations Management Suite (OMS).

Logon to your Azure Portal ( and click through to your log analytics workspace.  Then click on Advanced Settings


The Advanced Settings page will allow you to configure your data sources and where your logs will be pulled from. The rest of the video is the same.


Using Azure Logic Apps to Import CSV to SQL Server

When Logic Apps first came out I wrote a blog post explaining how to convert a CSV file into XML.A lot of this is still relevant, especially the integration account and the schemas and maps that are in my github repo. This post will show how Logic Apps are now even simpler to use with flat file decoding and also show how to insert the CSV data into a SQL server. The SQL part of the blog was adapted from this post:

Logic Apps has evolved since I last wrote about this topic and you now no longer need to create a function to transform our csv to xml.


The Transform XML connector is used now with the same maps we used in the previous post

In order to add the individual rows to the database there are a number of things you need to do. We will use an XML schema mapping in a stored procedure to extract the data from the transformed xml.

In your SQL database you will need to add a stored procedure, table and an XML schema, The SQL Scripts to create the table, stored procedure and xml schema have been added to the github repo. The stored procedure takes the xml file that has been transformed and uses the xml schema to extract the firstname, middlename and surname from the xml and then store the data in the employees table. In the logic app you need to add a SQL server connector and configure the connection to your Azure SQL database and also add in the stored procedure with the parameter as the output from the Transform XML.


The only other thing I needed to do to get this working was to remove the first row of the csv file as it contained the header fields and I didn’t want that inserted into the database.


The “length” expression is:  length(variables('csvdata'))


The “indexOf” expression is: indexOf(variables('csvdata'),'\r\n')

However if you add this in the editor the back slash will be delimited and you will end up with \\r\\n which will not work. To fix this you will need to click the View Code button, search for the \r\n and remove the extra back  slash

The “substring” expression is: substring(variables('csvdata'),add(variables('firstnewlineposition'),2),sub(variables('csvlength'),add(variables('firstnewlineposition'),2)))

The trigger for my Logic App was when a new file was added to OneDrive, so click the run button and then drop a file into the configured OneDrive location and the csv entries should be added to your database.

Cloud Load Testing Behind a Firewall with Visual Studio Team Services

Recently I’ve been looking at how we can load test one of our services so that we are able to understand the load our partners can put onto our systems before we start to have any issues. We used the Cloud-based Load Testing (CLT) service of Visual Studio Team Services (VSTS). I created a short video showing you how to easily setup a url based load test. The next stage of our load testing was to load test the service that our partners provide, Their service required IP whitelisting to connect which meant the CLT service would not be able to connect. Luckily for us the CLT service allows you to deploy agents into your own infrastructure  to carry out the load testing and they are controlled by the same CLT service that we used to load test our own service. This blog post will show you how to install the agent and configure the load test for this scenario.

The load test agent is installed using a PowerShell script which can be obtained from here. Open the PowerShell as administrator and don’t forget to unblock the script


To enable you to run the script and to configure the agents to talk to the CLT service, you will need to create a PAT token in VSTS.

Login and click on your user icon, then select Security


Select Personal Access Tokens, then Add


Fill in the form and click Create Token at the bottom of the page


This creates your token and this is the only time you will be able to access the token


You need to make sure that you copy it now as it you will not be able to access it after you have left the page,

Gong back to PowerShell, run the following command

.\ManageVSTSCloudLoadAgent.ps1 -TeamServicesAccountName StevesVSTS -PATToken 37abawavsmsgj6hpakltwhdjt4jrqsmup2jx62hlcxbju2l2tbja -ConfigureAgent -AgentGroupName StevesInternalTest

This will take a few minutes to run. If you didn’t run PowerShell as an Administrator you might see errors. When it has run the VSTSLoadAgentService should be installed and running


Now need to configure a load test. Following on from my video, you can create a url test


I ran a test web app in Visual Studio using localhost as the address


When the test has been created select the Settings tab, click “Use self-provisioned agents” and select your agent from the list and add the number of agents you want to use. You could install the agents on a number of machines in your environment using the same script, then you will be able to add more than 1 agent if required. As I only installed the one agent I can only select 1. You can see how many agents the CTL service can see


If there are less than you think you will need to check to make sure the service was installed without error and that it is running 

Save the load test and run it.


Whilst the test was running the performance meters in Visual Studio showed that the web page was being loaded.

When the test is complete you should see that it has not cost you any VUM (Virtual User Minutes) as the load tests are running on your own agents


The Cloud Load Test service allows you to load test both publicly accessible and private websites and services. As long as the servers running the load test agents have outbound access to the internet for HTTPS then we are able to load test private sites and services and the load test does not cost anything apart from the cost of the infrastructure that the agents are running on locally.

Azure Key Vault Logging and Events with Log Analytics

Following on from my previous blog post ( which explains how to set up Azure key vault with logging enabled, this post explains how to access the details of these logs and also to create an alert so you can see if someone is accessing the key vault from an unknown ip address (for example)

Open the Azure portal and navigate to the Resource Groups section and pick the resource group that we configured last time which contains the key vault and log analytics resources


Click your log analytics item, to open Log Analytics.

You can then select Log Search


This screen allows you to create your own query or select from existing ones.


Selecting “All Collected Logs” will show you the logs for the last day. I’ve highlighted the areas where you can change the time period, see the query and also click on Advanced Analytics to give a richer environment for analysing your logs.


If you want to query just for the Key Vault Audit logs then you can use the following query:

search * | where Category=="AuditEvent"


This will default to a list view, but clicking the Table button will format the data in an easier to read table.


You can sort and filter on the column headers. This can also be achieved using the order by clause as follows:

search * |where Category=="AuditEvent"  | order by TimeGenerated desc

A blog post discussing the query language can be found here

We are interested in all calls where someone has tried to access a Secret from the key vault. For that we are looking for an AuditEvent with an OperationName of SecretGet. If we also want to restrict the columns we retrieve then you can use “project” e.g.

search * | where Category=="AuditEvent"  and OperationName == "SecretGet"
| order by TimeGenerated desc
| project TimeGenerated, OperationName, CallerIPAddress, ResultSignature, requestUri_s


Now we are familiar with writing queries we can look at alerting. I’d like to set up an alert when the key vault is access from an IP Address other than the one where my application is running. This can be done as follows:

search * | where Category=="AuditEvent" and CallerIPAddress != ""

This ip address is actually the Azure Portal and is shown when you view the resource group that contains the key vault.I’m using this ip address so that I will actually get an alert (at the wrong time) when my application runs

Click New Alert Rule


The following screen should appear


The Alert Target should be the Log Analytics we’ve been using and the Target Criteria (when clicked) should show the query we’ve just written


We need to configure the rule for when this alert should be triggered. I’m interested when at least 1 attempt has been made in the last 5 minutes to access the Key Vault from an unknown location, so I set the threshold to be zero and click Done. We’ve now configured the logic to determine when the event is fired. Now we need to say what we want to happen when it fires.Firstly we need to give the alert a name and description


Now we need to configure how we are alerted. For this you need to create an action group. An action group allows you to define a collection of activities that will happen when the alert is fired. Click New Action Group


Action Types can be any of the following:


An action group can have multiple actions and you can select both email and SMS in a single action.Once you have created your Action Group you need to select in then click “Create alert rule”


Your alert is now set up and running. You can view/edit alerts by selecting Monitor in the Azure Portal


then click Alerts (preview), you will be able to see the alerts that have fired.


Click Manage Rules to edit the alert.

When the alert is fired I will get an email containing the details of the alert.

Log analytics is a powerful tool and whilst this series of posts has been related to auditing of Key Vault we can use log analytics for a wide variety of log sources such as Application Insights. We can also use the same mechanism for alerting to these other log sources,

The next post is a video that shows you how to connect existing log files to log analytics

Setting up Azure Key Vault with Audit logging

Azure Key Vault is a good way to share secrets with your partners in a way that allows you to have control over the access to each of the assets in Azure. We also need to know who is accessing the resources and from where so that we can monitor for suspicious activity. This post will talk through setting up the key vault and then configuring logging to keep track of the audit information for your certificates, keys and secrets. For each application that you want to access your resources you will need to create some credentials that the application can use.

To allow an application to access key vault an App Registration needs to be added to Azure Active Directory (AAD). This effectively sets up a username and password that the application can use for credentials.

Open the azure portal ( and navigate to Active Directory.

Click "App registrations"


Then "New application registration"


Name needs to be unique within your AD, select Web API/API and enter sign-on url. If you not building a website then enter anything in here. It might be useful to use a url related to your existing domain with application name appended. It doesn’t need to be a valid url. The click “Create”

Once created copy the Application ID as this is equivalent to a username to be used when calling the Key Vault in code. You now need to create the password.

Click Settings then Keys




Enter a name in the description field and select a duration, then click Save. The new key value will be displayed. You will need to copy this as it will not be visible again once you leave this page. This will be used as the password.


Now create the Key Vault. To do that it is a good idea to put it in a specific resource group, especially if you are creating a set of resources that the key vault is going to access or if you are going to setup third party access. Once the Resource Group has been created, select it and add a Key Vault. When the Create Key Vault panel appears, click Access Policies, click "Add new"


Pick the application you just created in AAD and select Get in Secret permissions, Save then go back to the main Key Vault pane and click Create

You have just given the application we created earlier access to just retrieving secrets. As you can see from the access policy you can give the application permissions to access a combination of Keys, Secrets and Certificates with the minimum access of Get. The Key Vault security is at the vault level and you cannot protect individual secrets at the user level. By granting only Get access on the Secret the application will not be able to list the Secrets available and will only be able to retrieve secrets it knows the names of.

Now the Key vault is set up and can be accessed, we want to know who is accessing the vault and from where. Out of the box this is not enabled and requires additional configuration and resources to allow us to be able to retrieve this audit information. This is achieved by enabling diagnostic logs in the Key Vault.

Before you can enable this you need to create a new storage account in this resource group to store the logs, then add Application Insights to the resource group


Once these have been provisioned, navigate to the Key Vault you just created & click Diagnostic logs


Click "Turn on diagnostics"


Select “Archive to Storage Account” and Pick the storage account you’ve just created

Select “Send to Log Analytics” and Create a new OMS workspace in your resource group


Once created select this for Log Analytics


select the AuditEvent log and click Save. 

Now any changes to the Key Vault plus any access from your application will be logged and visible via log analytics. There’s a 10 – 15 minute delay between accessing the Key Vault and the log appearing.

To Add a Secret to the vault, Navigate to the vault, click Secrets then Add


Select Manual from the Upload options, enter a name and the secret


Remember the name you gave the Secret as you will need this in your code when accessing the key vault. This secret will now have a unique identifier that you will use. The one I’ve just created is:

You should see in the logs this secret being created and also when it gets accessed.

Accessing the KeyVault in C# can be seen here:

The application in the example uses settings as defined below:

ClientID is the Application ID we created in the application registration in AD

ClientSecret is the key you created (that you had to save as it wasn’t visible again) as part of creating the application registration in AD.

Each Key, Secret and Certificate has a unique url which is used as the SecretURI e.g.

You now have your key vault set up with audit logging and are able to access it. My next blog post will talk you through how to access the logs and also how to set up alerting

Adding Rigour to an AzureML Web Service Deployment

As a developer I want an automated mechanism to build, test and deploy everything I do, so when my team and I came to implementing something with AzureML it was the first thing I was challenged on. We have a group of analysts who want to use AzureML to build part of our system without having to translate their requirements into a language that we developers can use to build a system in code. I’d been playing around with AzureML and deployed a few services as part of a proof of concept but hadn’t looked at how we could automated the deployment process and add some control. We didn’t want to have a mechanism that would allow the analysts to build and deploy a new model to production, without some way to check whether what they have built was fit for purpose. After a bit of research I found that we could export both the experiment and the web service as JSON from AzureML. Exporting both the experiment and the web service definitions would allow us to version control the source. We could also import these definitions to allow us to move the experiment and web service to different subscriptions. Exporting and importing the experiment was relatively straight forward using PowerShell.

Export-AmlExperiementGraph & Import-AmlExperiementGraph

The full PowerShell to export and import the experiment is:

Get-AmlWorkspace -ConfigFile .\config-source.json
$exp = Get-AmlExperiment | where Description -eq '<ExperrimentName>' -ConfigFile .\config-source.json
Export-AmlExperimentGraph -ExperimentId $exp.ExperimentId -OutputFile 'C:\experiment.json' -ConfigFile .\config-source.json

Get-AmlWorkspace -ConfigFile .\config-dest.json
Import-AmlExperimentGraph -InputFile 'C:\experiment.json' -ConfigFile .\config-dest.json

Note: This relies on two config files, one to identify the source workspace and the other to identify the destination workspace. These are in the following formats:

"Location": "West Europe",
"WorkspaceId": "<WorkspaceID",
"AuthorizationToken": "<AuthorizationToken>"

You can find the workspace id and authorization tokens by opening your AzureML workspace then clicking settings.



Exporting the experiment as JSON allows you to add it to source control and version the experiment.

Importing the experiment into a new Workspace will allow you to open the workspace and view/edit the experiment and also to manually deploy the web service. This on its own will give you some control over the web service and allow you to control how and when it gets deployed and will stop the analytics team from accidentally deploying something to production and potentially breaking the system.

It is also possible to export a deployed web service and then import it into a new subscription.

When we came to trying to export the web service we ran into a few issues. Firstly there seemed to be a number of ways to export the web service definition and they seemed to produce different JSON.

We settled for the last link but also took some missing information from the first link. The process for exporting is a little more complicated.

Firstly, the web service definition needs to be exported as a JSON file. The web service import uses resource manager and requires a different mechanism to login to the experiment export/import. I used:

Login-AzureRmAccount –SubscriptionId <My Subscription ID>

This use the interactive login so if you want to automate this then you need to use the service principle 

To export the web service:

$webService = Get-AzureRmMlWebService -Name "Source Service Name" -ResourceGroupName "Source Resource Group Name" 
Export-AzureRmMlWebService -WebService $webService -OutputFile 'C:\wsexport.json' 

The JSON then needs editing to add in the new storage account and commitment plan.

The JSON can then be imported into the new subscription using New-Azure​RmMl​Web​Service 

Changing the JSON to add in the commitment plan id seemed to cause problems and I kept getting the error “Commitment Plan ID must be provided”. This error was confusing as I was including the commitment plan id in the configuration and I thought that I had it in the correct place. If you open the JSON that you export and find the storage account node then you will need to overwrite this with:

     "name": "<StorageAccountName>",  
     "key": "<StorageAccountKey>" },  
"commitmentPlan": { 
      "id": "/subscriptions/0<Subscription-ID>/resourceGroups/<ResourceGroupName>/providers/Microsoft.MachineLearning/commitmentPlans/<CommitmentPlanName>"}, 

My issue was that I had the commitment id wrong rather than in the wrong place and I found the correct id using:


Once I had this configured correctly the import worked without any errors using:

New-AzureRmMlWebService -Force -ResourceGroupName "New Resource Group Name" -Name "Web Service Name" -Location "West Europe" -DefinitionFile "C:\wsexport.json" 

As we’ve used PowerShell to automate the export and import we could then easily script the config file edits and wire this in to an automated test and deploy process. We can write tests that check the web service parameters have not been changed by the analytics team and that the return data is the correct format, so we can ensure that the deployed service will at least function correctly. It’s more difficult to check whether the actual machine learning process is correct, but we will know when the interfaces are broken. We have also version controlled both the experiment and web service JSON so we can easily roll back if necessary. We decided that we needed both so that the experiment didn’t need to be copied to a new workspace and then automate a web service deploy, we just needed the web service in the new subscription, but we wanted the version control for the experiment too.

Using Bots For Form Filling

After watching James Mann’s talk on the bot framework at DDD I decided to look at whether the Bot framework can offer an alternative to web pages for filling in forms. The Microsoft Bot Framework is a framework that helps you build bots that can run in a variety of messaging apps such as Skype, Skype For Business, Facebook and Slack. There are a number of ways to help you build the bot and you can interface with the Cognitive Services to add intelligence to your bot using services such as LUIS.

The simplest mechanism for retrieving data from a user is to use FormFlow within the BotBuilder SDK. This streamlines the creation of a bot to collect data from a user. In order to start building Bots you will need Visual Studio 2017 installed along with the  Bot Builder extensions. This quick start will help you get the prerequisites sorted. The FormFlow example creates a Bot that will walk you through ordering a sandwich and along with the advanced example you can add optional ingredients along with some simple terms you can add to make it easier to pick multiple items from a list.

In order to debug and test your Bot there is a local emulator that allows you to run your Bots in a local test messaging app. You will need to download and install this emulator prior to testing.

I followed the examples through and found that there is a compilation error with the sample code in the FormBuilder section:


This conversation on stackoverflow seems to resolve the issue.


When you run the Bot in the Visual Studio debugger, it creates a web service and you need to use the url of this service in the Bot emulator which is run separately.


I need to take the URL http://localhost:3979/, add /api/messages and use it to configure the Bot emulator


Now click connect. Your Bot is now ready to test. To start the Bot you need to type a message. It doesn’t matter what you type, but “Hello” is probably a polite way to start Smile


To interact with the Bot you can either enter the number of the sandwich you want or type something and the Bot will try and make sense of what you type. If you type some text that does not appear in the list like “chickn” then the Bot wont understand, but typing “chicken” will give you all the matching answers to reduce your options. you also need to type whole words. so Chick wont match whereas Chicken will.


The FormBuilder example allows you to add validation and also custom code. The custom code allows you to ask for specific toppings and also say “everything except olives pickles”


There is also validation included in the FormBuilder example and it allows you to check to make sure that the address starts with a number



Optional stages can be added. For example if you pick Foot Long then you get a free cookie or drink


FormFlow is a good way to start with your Bot if you want to capture user input. The next stage for me is to work out how this can be applied to a more complex form filling scenario where there are dynamic lists which change based upon the input answers from previous questions.