Rico Suter's blog

Introducing Namotion.Interceptor for .NET

2026-02-19T00:00:00+00:00

Namotion.Interceptor is a new .NET library for reactive object models. It is under active development and APIs may still change between versions.

It is designed for any .NET application where a domain model needs to be reactive: industrial automation, IoT, real-time UIs, digital twins, MVVM desktop apps, or anything where local property changes should propagate automatically to derived state, observers, or external systems. The design hinges on C# 13 partial properties, a recent language feature that lets a source generator emit real getters and setters for your properties without proxies, weaving, or base-class requirements. From the start, the project also optimizes hard for low allocations and high throughput, since interceptor libraries tend to sit on the hot path of high-frequency systems.

In practice, mark a class with [InterceptorSubject], declare your properties as partial, and the source generator handles the rest. You get automatic change events, derived properties, lifecycle callbacks, and INotifyPropertyChanged support, all generated at compile-time.

[InterceptorSubject]
public partial class Person
{
    public partial string FirstName { get; set; }
    public partial string LastName { get; set; }

    [Derived]
    public string FullName => $"{FirstName} {LastName}";
}

var context = InterceptorSubjectContext
    .Create()
    .WithFullPropertyTracking();

var person = new Person(context);
context.GetPropertyChangeObservable().Subscribe(c =>
    Console.WriteLine($"{c.Property.Name}: {c.GetNewValue<object?>()}"));

person.FirstName = "John";
// FirstName: John
// FullName: John

The library is structured as six packages, each building on the previous so you can adopt only what you need:

Core provides [InterceptorSubject], the source generator, and the read/write interceptor pipeline that everything else plugs into.
Tracking adds observable and queue-based change streams, automatic recalculation of [Derived] properties, lifecycle attach/detach events, and atomic transactions.
Validation rejects invalid writes using standard [Required], [MaxLength], [Range] attributes or custom validators.
Registry keeps a runtime catalog of every subject and property, enables strongly-typed metadata via property attributes, dynamic property creation, and stable subject IDs.
Connectors synchronize the object graph with external systems. Bidirectional MQTT, OPC UA, and WebSocket integrations are included, along with infrastructure for custom connectors.
Integrations expose subjects through .NET host frameworks: ASP.NET Core (REST + JSON), Blazor (auto re-rendering with <TrackingScope>), HotChocolate (GraphQL subscriptions), and MCP (so AI agents can browse and manipulate the object graph).

For example, wiring a subject to an MQTT broker is roughly two steps:

[InterceptorSubject]
public partial class Sensor
{
    [Path("mqtt", "temperature")]
    public partial decimal Temperature { get; set; }
}

builder.Services.AddMqttSubjectClient<Sensor>(
    brokerHost: "mqtt.example.com",
    sourceName: "mqtt",
    brokerPort: 1883,
    topicPrefix: "sensors/room1");

Local writes to sensor.Temperature are now published to the broker, and inbound MQTT messages update the property and trigger change notifications. The same pattern works for OPC UA, WebSocket, or your own custom connector.

A typical use case is an IoT or industrial scenario where the same C# domain model is the source of truth for a Blazor dashboard, the contract for an OPC UA client talking to a PLC, and the wire format for an MQTT bridge, without writing protocol glue code by hand.

It also powers HomeBlaze v2, a complete rewrite of my home automation platform for .NET and Blazor, now built on Namotion.Interceptor for all device modeling and synchronization. The v2 codebase currently lives alongside the library in the same repository while the two mature together, and will eventually be split out into its own repo (replacing the HomeBlaze v1 codebase there). HomeBlaze is also the primary driver behind many of the library’s design decisions.

Feedback, issues, and contributions are very welcome. Source code and documentation can be found on GitHub

Hosting an Nginx Reverse Proxy in an Azure Web App for Containers (App Service)

2022-12-10T00:00:00+00:00

The following steps describe how to set up an Azure Web App for Container to act as a Nginx reverse proxy to an other application. I used this to redirect all HTTP calls from a public domain into an internal Azure VM. The benefit is that this way you can still use App Service specific features like certificate handling or the AD-based authentication; and also SSL termination is handled by the proxy (the application runs only on HTTP).

Create Web App for Containers

In the Azure Portal create a new Web App for Containers resource:

Choose “Docker Container” as Publish method and “Linux” as Operating System:

Setup Nginx container

In the newly created Web App for Containers go to the Deployment Center and choose “nginx:latest” as Docker image:

Map storage with configuration

Next, create a new Azure Storage and a new storage container (e.g. nginx). In this container create a new nginx.config file:

events {}
http {
    server {
        location / {
            proxy_pass http://yourtargethost/;

            proxy_set_header X-Forwarded-Host mynginxsample.azurewebsites.net;
            proxy_set_header X-Forwarded-Proto https;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;

            proxy_pass_request_headers on;
            proxy_set_header Upgrade $http_upgrade;
            proxy_set_header Connection "Upgrade";
        }
    }
}

mynginxsample.azurewebsites.net is the public host address the external clients will use; http://yourtargethost/ is the host where the requests should be redirected to. The following headers are needed so that the target application knows the original host name:

proxy_set_header X-Forwarded-Host mynginxsample.azurewebsites.net;
proxy_set_header X-Forwarded-Proto https;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;

To support web socket connection pass through, use the following options (required for e.g. SignalR):

proxy_pass_request_headers on;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "Upgrade";

In the App Service, go to Configuration and create a new Path Mapping with the Mount path /etc/nginx:

Optional: Enable forwarded header processing in ASP.NET Core

In ASP.NET Core you need to enable the processing of forwarded headers (X-Forwarded-*), so that the application uses the correct public host names. To enable the feature, add the following configuration code:

public void ConfigureServices(IServiceCollection services)
{
    services.Configure<ForwardedHeadersOptions>(options =>
    {
        options.ForwardedHeaders = ForwardedHeaders.All;

        // allow all (for security reasons, this should only be used when 
        // only the proxy can access the application directly)
        options.KnownNetworks.Add(new IPNetwork(IPAddress.Parse("0.0.0.0"), 0));
    });

    ...
}

Then enable the configuration in the HTTP pipeline with:

public void Configure(IApplicationBuilder app, IHostEnvironment env, ILogger<Startup> logger)
{
    app.UseForwardedHeaders();

    ...
}

The code above should only be executed when the application is actually running behind a proxy (i.e. consider putting it behind a configuration flag). For more information see Configure ASP.NET Core to work with proxy servers and load balancers.

Restart the app

Now restart the app so that the configuration is picked up and you should have the Nginx proxy running.

How to set up an ASP.NET Core project with multiple API versions, OpenAPI generation and Azure API Management

2021-02-25T00:00:00+00:00

In this article I’ll describe how to set up an ASP.NET Core web API project with multiple versions, automatic generation of multiple OpenAPI specifications with NSwag and uploading them to Azure API Management.

Our sample service will expose two different API versions under the routes /api/v1 and /api/v2. So conceptually there is a service version (e.g. v1.6) and multiple API versions (e.g. v1.8 and v2.1). Only the major API version is exposed in the URL. The URL only contains and changes for major versions because minor versions are not breaking (see semver) and thus we don’t need to introduce and expose a new version.

Two remarks:

For simplicity you could use the latest API version (v2.1) as the service version.
To maintain backwards compatibility with older consumers we need to serve multiple versions simultaneously - at least two to be able to do gradual migrations without service interruptions.

In our setup, each major version will produce an OpenAPI specification which is served in the Swagger UI for testing and also generated at build time as local OpenAPI JSON specification files. This file generation is very useful to review the public API changes in Pull Requests (when committed to the repository) and to upload to Azure API Management in the CD pipeline later when the service is deployed.

There is one caveat though: By default the paths in the specifications are relative to the root URL of the service, i.e. in both specs the operation paths start with /api/v.... The problem is that Azure API Management does not support two APIs with the same external URL and then route a request to the correct API based on the operations in the specs, i.e it cannot choose the correct API based on the full URL. That’s why we need to remove the path prefixes in the specs (e.g. remove /api/v1) and use this prefix as the base path of the Azure API Management API base URL.

To clarify this:

The service is served under https://myurl.com (here myurl.com is served by Azure API Management)
The API v1 is served under https://myurl.com/api/v1 (using the route /api/v1 in the ASP.NET Core application internally)
The Azure API Management API backend URL must be https://myurl.com/api/v1 (and not https://myurl.com) so that a request can clearly be correlated to a single APIM API and backend server URL
And thus the API v1 specification which is uploaded to APIM must not contain operations with /api/v1

Setup the ASP.NET Core application

All the sample code can be found in this repository.

First we will create a new project where we enable API versioning in the Startup.cs with versioning in the path:

public void ConfigureServices(IServiceCollection services)
{
    ...

    services
        .AddApiVersioning(options =>
        {
            options.ReportApiVersions = true;
            options.AssumeDefaultVersionWhenUnspecified = false;
            options.DefaultApiVersion = new ApiVersion(1, 0);
            options.ApiVersionReader = new UrlSegmentApiVersionReader();
        })
        .AddVersionedApiExplorer(options =>
        {
            options.DefaultApiVersion = new ApiVersion(1, 0);
            options.GroupNameFormat = "'v'VVV";
            options.SubstituteApiVersionInUrl = true;
        });

After versioning is set up we also register two OpenAPI documents - one for each major version (using NSwag):

public void ConfigureServices(IServiceCollection services)
{
    ...

    var versions = new[]
    {
        // Here you can control the minor version of each supported major version
        new Version(2, 1),
        new Version(1, 8)
    };

    foreach (var version in versions)
    {
        services.AddOpenApiDocument(options =>
        {
            options.Title = "Product Service";
            options.Description = "Manages products and their metadata.";

            options.DocumentName = "v" + version.Major;
            options.ApiGroupNames = new string[] { "v" + version.Major };
            options.Version = version.Major + "." + version.Minor;

            // Patch document for Azure API Management
            options.AllowReferencesWithProperties = true;
            options.PostProcess = document =>
            {
                var prefix = "/api/v" + version.Major;
                foreach (var pair in document.Paths.ToArray())
                {
                    document.Paths.Remove(pair.Key);
                    document.Paths[pair.Key.Substring(prefix.Length)] = pair.Value;
                }
            };
        });
    }
}

Note: To avoid problems with $ref handling in Azure API Management we need to enable allowReferencesWithProperties so that $refs are inlined and not described with oneOf structures. If you do not need to import the specs into Azure API Management I recommend to not patch the operation paths and server URL and leave them as is (relative to the hosted URL).

Because we patched the operation paths for Azure API Management, we need to add the removed prefix to the server URL so that the served Swagger UI still works:

public void Configure(IApplicationBuilder app, IWebHostEnvironment env)
{
    ...
    
    app.UseSwaggerUi3();
    app.UseOpenApi(options =>
    {
        options.PostProcess = (document, request) =>
        {
            // Patch server URL for Swagger UI
            var prefix = "/api/v" + document.Info.Version.Split('.')[0];
            document.Servers.First().Url += prefix;
        };
    });

    ...

If you now start the project and navigate to /swagger you should be able to switch between the versions in the dropdown:

As this is working we will now add the NSwag.MSBuild package to the project to generate the specs at build time as files. For that we need to create two NSwag configuration files, nswag_v1.json and nswag_v2.json (only v1 is shown):

{
  ...
  "documentGenerator": {
    "aspNetCoreToOpenApi": {
      "project": "ProductService.csproj",
      "documentName": "v1",
      "output": "openapi_v1.json",
      ...
    }
  }
}

In the ProductService.csproj file we now need to run NSwag so that the OpenAPI JSON files are automatically generated on build:

<Project Sdk="Microsoft.NET.Sdk.Web">
  ...
  <Target Name="NSwag" AfterTargets="Build">
    <Exec Command="$(NSwagExe_Core31) run nswag_v1.json /variables:Configuration=$(Configuration)" />
    <Exec Command="$(NSwagExe_Core31) run nswag_v2.json /variables:Configuration=$(Configuration)" />
  </Target>
</Project>

After building the solution you should see the two specs generated:

In your CI build you can now add these two specs to the CI artifacts and then in your CD pipeline (release) you can pick them up and upload to Azure API Management.

Upload the specs to Azure API Management

Assuming you have one Azure API Management instance per environment, we can now run the following script in the CD pipeline to create an API Version Set with the v1 and v2 APIs:

$ApiMgmtContext = New-AzApiManagementContext -ResourceGroupName "Namotion" -ServiceName "NamotionApim"

$VersionSet = New-AzApiManagementApiVersionSet `
    -Context $ApiMgmtContext `
    -Name "Product Service API" `
    -Description "The Product Service API." `
    -Scheme Segment

Import-AzApiManagementApi `
    -Context $ApiMgmtContext `
    -ApiVersionSetId $VersionSet.ApiVersionSetId `
    -ApiId "product-server-api-v1" `
    -Path "products" `
    -ServiceUrl "https://internal.products.myurl.com/api" `
    -ApiVersion "v1" `
    -SpecificationFormat "OpenApiJson" `
    -SpecificationPath "./openapi_v1.json"

Import-AzApiManagementApi `
    -Context $ApiMgmtContext `
    -ApiVersionSetId $VersionSet.ApiVersionSetId `
    -ApiId "product-server-api-v2" `
    -Path "products" `
    -ServiceUrl "https://internal.products.myurl.com/api" `
    -ApiVersion "v2" `
    -SpecificationFormat "OpenApiJson" `
    -SpecificationPath "./openapi_v2.json"

Note: If you have a single Azure API Management instance for multiple environments, then you need to add the environment name to the version set’s -Name (e.g. Product Service API (Dev)) and API’s -ApiId parameters (e.g. product-server-api-v2-dev).

As you can see in this sample, you should deploy your service to https://internal.products.myurl.com so that the API v1 is hosted on https://internal.products.myurl.com/api/v1. We add the /api to the ServiceUrl so that the API Version name only contains v1.

After running this script you should see the following API hierarchy in the Azure API Management instance:

Track the API versions in Apimundo

Apimundo is an architecture and service API documentation system which can automatically track, document and compare web API interfaces with OpenAPI specs. After creating a new OpenAPI Endpoint, you can easily register multiple specification endpoints and add the prefix like this:

After the system automatically indexes your specs (might take some minutes) you can see a single list of API versions where each major version can progress individually with new minor versions:

The final result can be played with here (Demo).

Azure Event Hubs: Improve scalability with batch parallelization

2020-05-05T00:00:00+00:00

In one of my projects we are ingesting lots of device events with Azure Event Hubs. In the given scenario it is important to process the events from one device in sequence. In our initial implementation we just read a batch of events from the partition and process the events within the batch in sequence. However at some point we reached a limit and we couldn’t use all CPU resources of the machines we used - mainly because half the work of one event is I/O bound.

If you process each partition stricly sequentially then your maximum parallelization is the number of partitions.

So what can we do? Of course the simplest solution is to increase the number of partitions in the Event Hub - but what if you already reached the maximum number of partitions (e.g. 32) or you cannot change the partition count because it is a running system?

A general solution to this works as follows: The partition consumers reads a batch of events and then processes this batch as fast and efficient as possible, maybe not strictly sequentially.

How can we improve this processing in our scenario? Instead of processing all events in sequence we process the events in the batch in parallel but in sequential groups by device. This way we can parallelize within the batch and at the same time ensure that events from one device are processed in sequence.

The implementation of this is quite simple:

Group the events by partition key and put the groups into a queue
Spawn multiple threads and process the groups in parallel but sequentially within the groups

This functionality is implemented in the open-source library Namotion.Messaging (in the library events are the same as messages):

public static async Task ProcessByPartitionKeyAsync<TMessage, TPartitionKey>(
    this IEnumerable<Message> messages,
    Func<Message, TMessage> transform,
    Func<TMessage, TPartitionKey> partitionKeySelector,
    Func<IEnumerable<TMessage>, CancellationToken, Task> processPartitionMessages,
    int partitionParallelization = 4,
    CancellationToken cancellationToken = default)
{
    var deserializedMessages = transform != null ? messages
        .Select(m => transform(m))
        .AsParallel()
            .AsOrdered()
        .ToArray() : messages.Cast<TMessage>().ToArray();

    var batchPartitionsQueue = new ConcurrentQueue<TMessage[]>(
        deserializedMessages
            .GroupBy(partitionKeySelector)
            .Select(g => g.ToArray())
            .ToArray());

    var tasks = Enumerable
        .Range(0, Math.Min(deserializedMessages.Length, partitionParallelization))
        .Select(i => Task.Run(async () =>
        {
            while (batchPartitionsQueue.TryDequeue(out var batchPartition))
            {
                await processPartitionMessages(batchPartition, cancellationToken);
            }
        }, cancellationToken));

    await Task.WhenAll(tasks);
}

MessageEnumerableExtensions.cs

These extension methods can now be easily used in a event receiver background service:

public class MyBackgroundService : BackgroundService
{
    private const int PartitionParallelization = 4;
    private readonly IMessageReceiver receiver;

    public MyBackgroundService(IMessageReceiver receiver)
    {
        this.receiver = receiver;
    }

    protected override async Task ExecuteAsync(CancellationToken cancellationToken)
    {
        await receiver.ListenWithRetryAsync(ProcessMessagesAsync, logger, cancellationToken);
    }

    private async Task ProcessMessagesAsync(IEnumerable<Message> allMessages, CancellationToken cancellationToken)
    {
        await allMessages.ProcessByPartitionKeyAsync(
            m => DeserializeMessage(m.Content),
            m => m.DeviceId,
            async (messages, ct) =>
            {
                foreach (var message in messages)
                {
                    ... // TODO: Process messages
                }
            }, PartitionParallelization, cancellationToken);

        await receiver.ConfirmAsync(allMessages, cancellationToken);
    }

    private DeviceMessage DeserializeMessage(Message message)
    {
        return ...; // TODO: Deserialize message
    }
}

Azure DevOps: My versioning flow to publish .NET packages from a GitHub repository

2019-08-15T00:00:00+00:00

The starting point of this guide is a GitHub repository with some SDK-style .NET projects for which you want to enable CI/CD builds to automatically run tests, build packages and push them to a preview or NuGet.org package feed.

I created this setup and versioning flow for my new library Namotion.Reflection and will describe it in this article. You can copy the pipeline YAML file and directly use it in your own repository.

The preview and release flow

The repository is using GitHub flow and thus the lastest releasable version is on the master branch.
Package version numbers are managed in the in the repository in the .csproj files.
Builds on master create preview packages with the version x.y.(z+1)-build.xxx so that the version is always greater than the latest published version (the current version x.y.z in the repository) and always lower then the next published version (the next released version will be at least version x.y.(z+1) but without preview tag and thus greater than x.y.(z+1)-build.xxx).
When we release a new version, then we update the versions of the changed packages on master (commit name Release vx.y.z) and merge into the release branch to trigger a non-preview build and push the packages to nuget.org.
(Another option would be to not merge to the release branch but tag the commit Release vx.y.z with vx.y.z and trigger a release this way.)

1. Enable package build in your SDK-style .NET projects

First we need to enable package builds for all projects which should produce packages.

For this, open your solution in Visual Studio and enable “Generate NuGet package on build” on all projects you want to automatically deploy to the package feeds:

Alternatively you can also enable this in the .csproj file directly by adding a GeneratePackageOnBuild tag:

<Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup>
    <TargetFrameworks>netstandard1.0;netstandard2.0;net40;net45</TargetFrameworks>
    <Version>0.1.0</Version>
    <GeneratePackageOnBuild>true</GeneratePackageOnBuild>
    <GenerateDocumentationFile>true</GenerateDocumentationFile>
  </PropertyGroup>
  ...

Also consider adding the GenerateDocumentationFile attribute so that the XML documentation is included in the package.

2. Add an azure-pipelines.yml file to your repository

The next step is to add an azure-pipelines.yml file which is later picked up by the Azure DevOps build pipeline to build, test and publish the NuGet packages - the complete YAML file can be found here. The head of the build file looks like this:

trigger:
  branches:
      include:
      - master
      - release
pr:
- master

pool:
  vmImage: 'windows-2019'

variables:
  BuildConfiguration: Release
  Projects: '**/*.csproj'

steps:
...

The build will trigger on the master and release branches and on all PRs merging into master.

The specified vmImage is windows-2019 (with Visual Studio) because the Namotion.Reflection projects use multiple target frameworks and one of them is the full .NET Framework (netfx, Windows-only). If your projects only use .NET Core or .NET Standard target frameworks you should be able to build on Ubuntu with ubuntu-latest image. You can find a list of all available vmImages here.

Now, let’s look at the required pipeline steps.

Install required SDKs and tools

In the first steps we install the required SDKs and tools which are used later:

.NET Core 2.2 is needed to build the solution and to use the DNT tool
The DNT (DotNetTools) CLI tool is used to patch the project versions later

- task: UseDotNet@2
  displayName: 'Install .NET Core SDK'
  inputs:
    packageType: 'sdk'
    version: '2.2.203'
- task: CmdLine@2
  displayName: 'Install DNT'
  condition: and(succeeded(), eq(variables['Build.SourceBranch'], 'refs/heads/master'))
  inputs:
    script: 'npm i -g dotnettools'

Patch project versions

If the build is running on the master branch, we increase the project patch version (0.1.0 => 0.1.1) and add a a build suffix (0.1.1 => 0.1.1-build.20190518.35). This is needed so that the package versions in the preview NuGet feed are unique even without changing the versions in the .csproj files.

- task: UseDotNet@2
  displayName: 'Install .NET Core SDK'
  inputs:
    packageType: 'sdk'
    version: '2.2.203'
- task: CmdLine@2
  displayName: 'Install DNT'
  condition: and(succeeded(), eq(variables['Build.SourceBranch'], 'refs/heads/master'))
  inputs:
    script: 'npm i -g dotnettools'
- task: CmdLine@2
  displayName: 'Update project version patch'
  condition: and(succeeded(), eq(variables['Build.SourceBranch'], 'refs/heads/master'))
  inputs:
    script: 'dnt bump-versions patch'
    failOnStderr: true
- task: CmdLine@2
  displayName: 'Patch project version preview'
  condition: and(succeeded(), eq(variables['Build.SourceBranch'], 'refs/heads/master'))
  inputs:
    script: 'dnt bump-versions preview "$(Build.BuildNumber)"'
    failOnStderr: true

Build and test

In the following steps we build, test and create NuGet packages.

- task: DotNetCoreCLI@2
  displayName: 'Build solution'
  inputs:
    command: 'build'
    projects: '$(Projects)'
    arguments: '--configuration $(BuildConfiguration)'
    feedsToUse: 'select'
    versioningScheme: 'off'
- task: DotNetCoreCLI@2
  displayName: 'Run tests'
  inputs:
    command: 'test'
    projects: '$(Projects)'
    arguments: '--configuration $(BuildConfiguration) --collect "Code Coverage"'
    publishTestResults: true
    feedsToUse: 'select'
    versioningScheme: 'off'

The build step automatically creates packages because we enabled “Generate NuGet package on build” before. The --collect "Code Coverage" parameter in the test step enables test code coverage publishing but this might only work if the build agent has Visual Studio installed (i.e. a Windows build agent). To customize the code coverage execution, you can specify a .runsettings file (sample).

Pack and publish artifacts

The last steps copy the NuGet packages and publish them as build artifacts which are later picked up by a release pipeline and pushed to a package feed.

- task: CopyFiles@2
  displayName: 'Copy packages'
  condition: and(succeeded(), ne(variables['Build.Reason'], 'PullRequest'))
  inputs:
    Contents: '**/*.nupkg'
    TargetFolder: '$(Build.ArtifactStagingDirectory)'
    flattenFolders: true
- task: PublishBuildArtifacts@1
  displayName: 'Publish artifacts'
  condition: and(succeeded(), ne(variables['Build.Reason'], 'PullRequest'))
  inputs:
    PathtoPublish: '$(Build.ArtifactStagingDirectory)'
    ArtifactName: 'drop'
    publishLocation: 'Container'

3. Setup the build pipeline in Azure DevOps

To actually use this build definition, we need to create an Azure DevOps project and then a new build pipeline. In your Azure DevOps dashboard, click on ”+ Create project”:

After creating the project, go to Pipelines / Builds, create a new build pipeline and select GitHub (yaml):

In the next step, select your GitHub repository and authenticate it:

The “Review” step should automatically select your YAML build script from the repository and after clicking on “Run” the pipeline is created and the first build is started:

4. Setup the release pipelines for the CI and release branch

Now that the build pipeline is set up, we will create two release pipelines:

Namotion.Reflection - Master: Publish preview packages from the master branch to a preview NuGet feed.
Namotion.Reflection - Release: Publish release packages from the release branch to the public NuGet.org feed.

Create release pipeline to publish to preview NuGet feed

First go to Artifacts and create a new NuGet feed where preview packages should be pushed to:
Then go to Pipelines / Releases, create a new release pipeline and select Empty job (no template).
Use Push to preview feed as stage name.
Click on “+ Add an artifact” and select the build pipeline in the dialog.
Click on the artifact trigger symbol and enable a continuous deployment trigger on the master branch:

Click on tasks and add a new .NET Core task with the following properties:
- Command: nuget push
- Path to NuGet package(s) to publish: $(System.DefaultWorkingDirectory)/**/*.nupkg
- Target feed location: Use “This organization/collection” and select your preview NuGet feed

Create a release pipeline to NuGet.org feed

In order to push packages to the public NuGet.org feed, we need to setup a new service connection in Azure Devops:
- Go to “Project settings” (bottom right)
- Select “Service Connections”
- Select “+ New service connection” and select “NuGet”
- Use “ApiKey” and fill in the missing fields:
  - Connection name: NuGet.org (must match the name in the build step)
  - Feed URL: https://api.nuget.org/v3/index.json
  - ApiKey: <your NuGet.org push API key>
  - For more information see Create a service connection.

Create a new release pipeline in the same way as before, but use continuous deployment trigger on the release branch:

Click on tasks and add a new .NET Core task with the following properties:
- Command: nuget push
- Path to NuGet package(s) to publish: $(System.DefaultWorkingDirectory)/**/*.nupkg
- Target feed location
  - External NuGet server
  - NuGet server: Select “NuGet.org” which is service connection which has been created before

4. Test the pipelines

Now you should be able to queue a new build in the build pipeline. This build is then picked up by the preview release pipeline which pushes the packages to the preview NuGet feed:

5. Publish new package versions

To publish new package versions, you first need to update package versions in the .csproj. This can be easily accomplished with the DNT (DotNetTools) tool:

dnt bump-versions patch

Commit these changes to the master branch and merge master into the release branch to trigger the release pipeline which pushes the packages to NuGet.org.

Dependency Injection in .NET: A way to work around missing named registrations

2019-08-02T00:00:00+00:00

The default .NET dependency injection container Microsoft.Extensions.DependencyInjection does not provide a way to register named services. That means you cannot register an interface multiple times and then resolve a specific instance with a name or key. This article shows a technique to solve this by adding a generic marker parameter to the service interface so that it can be registered multiple times and resolved with the marker type.

Let’s start with a sample: In the code below we have two service classes, OrderService and ProductService, both of them want to publish a message to a message queue through the help of an IMessagePublisher service interface. The first, non-working implementation would look like this:

public interface IMessagePublisher
{
    void Publish(Message message);
}

public class OrderService
{
    // Error: The DI container does not know what registration he needs to inject here
    public OrderService(IMessagePublisher orderPublisher) { ... }

    public Create() { orderPublisher.Publish(new OrderCreatedMessage()); }
}

public class ProductService
{
    // Error
    public OrderService(IMessagePublisher productPublisher) { ... }

    public Create() { productPublisher.Publish(new ProductCreatedMessage()); }
}

// Register two publishers with the same interface (MyMessagePublisher is an actual implementation of IMessagePublisher)
services.AddSingleton<IMessagePublisher>(new MyMessagePublisher("MyOrderCreatedQueue"));
services.AddSingleton<IMessagePublisher>(new MyMessagePublisher("MyProductCreatedQueue"));

The previous code won’t work because the IMessagePublisher interface has been registered multiple times and the resolver does not know what instance to resolve to. The problem we have here is that the default .NET dependency container does not support named service registrations. Something like this is not possible with the default container:

public class OrderService
{
    public OrderService([Name("OrderPublisher")] IMessagePublisher orderPublisher) 
    { ... }
}

services.AddNamedSingleton<IMessagePublisher>(
    "OrderPublisher", new MyMessagePublisher("MyOrderCreatedQueue"));

Most advanced dependency injection container libraries support a form of named registrations. However, the problem with them is that you will always have some sort of dependency to the injection library; in the previous sample a dependency to the NameAttribute - this is a leaky abstraction and should be avoided.

One solution to this problem is to introduce the same interface but with an additional generic parameter. Then implement an interceptor class to wrap the original instance and “enrich” it with a generic type:

// Define interface with a generic parameter
public interface IMessagePublisher<T> : IMessagePublisher { }

public interface IMessagePublisher
{
    void Publish(Message message);
}

// Implement a publisher interceptor class which adds the generic parameter to an existing 
// publisher instance and which just calls the methods of the intercepted instance.
public class MessagePublisher<T> : IMessagePublisher<T>
{
    private IMessagePublisher _publisher;

    public MessagePublisher(IMessagePublisher publisher)
    {
        _publisher = publisher;
    }

    public virtual void Publish(Message message)
    {
        _publisher.Publish(message);
    }
}

// Now we can register specific publishers
services.AddSingleton<IMessagePublisher<OrderCreatedMessage>>(
    new MessagePublisher<OrderCreatedMessage>(
        new MyMessagePublisher("MyOrderCreatedQueue")));
services.AddSingleton<IMessagePublisher<ProductCreatedMessage>>(
    new MessagePublisher<ProductCreatedMessage>(
        new MyMessagePublisher("MyProductCreatedQueue")));

// In the services, we can now inject specific publishers
public class OrderService
{
    public OrderService(IMessagePublisher<OrderCreatedMessage> orderPublisher) 
    { ... }
}

public class ProductService
{
    public OrderService(IMessagePublisher<ProductCreatedMessage> productPublisher) 
    { ... }
}

As you can see, with the new interfaces we are able to register two publishers and inject the desired ones into the services. With the new MessagePublisher<T> interceptor class you can also enhance existing publisher implementations with additional logic by implementing methods with new aspects:

public class LoggingMessagePublisher<T> : MessagePublisher<T>
{
    private readonly ILogger _logger;

    public LoggingMessagePublisher(ILogger logger, IMessagePublisher publisher) 
        : base(publisher)
    {
        _logger = logger;
    }

    public virtual void Publish(Message message)
    {
        _logger.LogInformation("Publishing a message.");
        base.Publish(message);
    }
}

IMessagePublisher<ProductCreatedMessage> publisher = 
    new LoggingMessagePublisher<ProductCreatedMessage>(
        logger, new MyMessagePublisher("MyOrderCreatedQueue"));

To simplify applying these interceptors you can also implement some extension methods:

public static class MessagePublisherExtensions
{
    public static IMessagePublisher<T> WithMessageType<T>(this IMessagePublisher publisher)
    {
        return new MessagePublisher<T>(publisher);
    }

    public static IMessagePublisher<T> WithLogging<T>(this IMessagePublisher<T> publisher)
    {
        return new LoggingMessagePublisher<T>(publisher);
    }

    public static IMessagePublisher WithLogging(this IMessagePublisher publisher)
    {
        // Here we use object because we don't care about the type
        return new LoggingMessagePublisher<object>(publisher);
    }
}

IMessagePublisher<ProductCreatedMessage> publisher = 
    new MyMessagePublisher("MyOrderCreatedQueue")
        .WithLogging(logger)
        .WithMessageType<ProductCreatedMessage>();

In simpler cases, you can also leave out the IMessagePublisher interface and only use a IMessagePublisher<T> interface.

What do you think of this technique to circumvent the missing named registration support?

This technique is heavily used in my abstraction projects Namotion.Messaging and Namotion.Storage.

Logging with ILogger in .NET: Recommendations and best practices

2019-05-06T00:00:00+00:00

This article describes recommendations and best practices for using the ILogger based logging system which has been introduced with .NET Core but is also available in all .NET Standard 2.0 supporting .NET frameworks.

Introduction

The logging interfaces provided by the Microsoft.Extensions.Logging.Abstractions NuGet package provide common logging abstractions with implementations for various logging backends and sinks.

A Logging backend is the place where logs are written to, e.g. files, Application Insights (AI), Seq, Kibana, etc. In the Serilog logging library they are called sinks.

ILogger vs ILoggerProvider vs ILoggerFactory

ILogger

The responsibility of the ILogger interface is to write a log message of a given log level and create logging scopes. The interface itself only exposes some generic log methods which are then used by “external” extension methods like LogInformation or LogError.

ILoggerProvider

A logger provider is an actual logging sink implementation, e.g. console, Application Insights, files or Serilog (an adapter to the Serilog logging abstraction). The ILoggerProvider’s only responsibility is to create ILogger instances which log to an actual sink. A logger instance which is created by a logger provider will only log to the associated logger provider.

ILoggerFactory

The ILoggerFactory logger factory instance is the boostrapper of the logging system: It is used to attach logger providers and create logger instances - either typed (ILogger<T>) or untyped (ILogger). These logger instances will log to all registered logger providers.

Log statements

Use structured logs

It is recommended to always use semantic/structured logs so that the logging backend receives the string with placeholders and its values separately. It is then able to just replace them in its UI on demand. This way each log statement preserves all associated properties and a template hash (in AI “MessageTemplate”) which allows the backend to apply advanced filtering or search for all logs of a given type.

logger.LogWarning("The person {PersonId} could not be found.", personId);

The previous statement will log the following properties to Application Insights:

Message: The person 5 could not be found.
MessageTemplate: The person {PersonId} could not be found.
PersonId: 5

Advantages of structured logs:

Properties are stored as seperate custom properties of the log entry and you can filter based on them in the logging backend
A message template/message hash is stored so that you can easily query for all logs of a given log statement type
Serialization of properties only happens if the log is actually written (not the case with string interpolation)

Disadvantages:

The order of the parameters in the C# log statement have to be correct and this cannot be statically checked by the compiler in the same way as it’s done in interpolated strings

Always pass exception as first parameter

To log an exception, always pass the exception object as first argument:

logger.LogWarning(exception, "An exception occured")

Otherwise it is treated as custom property and if it has no placeholder in the message it will not end up in the log. Also the formatting and storage is exception specific if you use the correct method overload.

Always use placeholders and avoid string interpolation

To ensure that a message template is correctly logged and properties are transferred as custom properties, you always need to log properties with placeholders in the correct order, e.g.

logger.LogWarning("The person {PersonId} could not be found.", personId);

Never use string interpolation because the replacement will be done in your application and the logging backend has no longer access to the message template and individual custom properties:

logger.LogWarning($"The person {personId} could not be found.");

Another downside of string interpolation is that objects are always serialized into text in your app even if the log is not actually written because there is a log level filter configured.

Do not use dots in property names

Avoid using dots in placeholder property names (e.g Device.ID) because some ILogger implementations (e.g. Serilog) do not support this. The main reason for this is that some logging backends cannot handle these property names and thus Serilog has to implement the lowest common denominator.

Scopes

Use scopes to add custom properties to multiple log entries

Use scopes to transparently add custom properties to all logs in a given execution context.

using (logger.BeginScope(
    new Dictionary<string, object> { {"PersonId", 5 } }))
{
    logger.LogInformation("Hello");
    logger.LogInformation("World");
}

Now both log statements will be logged with an additional PersonId property. Most scope implementations even work correctly in parallel code because they use async contexts.

You should create scopes for logical contexts, regions or per unit of work; for example:

Per HTTP request
Per event queue message
Per database transaction

In each of these scopes you should set the following properties if available:

Correlation ID (use the X-Request-ID HTTP header to read it in HTTP request scopes)
- Don’t forget to pass the correlation ID when calling other HTTP endpoints
- In Application Insights the property should be called operation_id
Transaction ID

Consider adding a scope identifier to filter logs by scope

In order to be able to show all logs of a given custom scope, I implemented a simple extension method which automatically adds a scope identifier and uses value tuples to specify properties for all log entries in the scope:

public static IDisposable BeginNamedScope(this ILogger logger,
    string name, params ValueTuple<string, object>[] properties)
{
    var dictionary = properties.ToDictionary(p => p.Item1, p => p.Item2);
    dictionary[name + ".Scope"] = Guid.NewGuid();
    return logger.BeginScope(dictionary);
}

This exention method can be used like this:

foreach (var message in messages)
{
    using (logger.BeginNamedScope("Message", ("Message.Id", message.Id), ("Message.Length", message.Length)))
    {
        logger.LogInformation("Hello");
        logger.LogInformation("World");
    }
}

Both log entries will now have a Message.Scope property with the same value: Using this property you are now able to show all logs of a given scope in your logging backend.

Use scopes to add additional custom properties to a single entry

If you want to add additional properties to a log statement which should not be part of the message template, use a “short lived” scope:

using (logger.BeginPropertyScope(("UserId", "CurrentUserId")))
{
    logger.LogTrace("The message has been processed.");
}

The used extension method to use value tuples looks like this:

public static IDisposable BeginPropertyScope(this ILogger logger,
    params ValueTuple<string, object>[] properties)
{
    var dictionary = properties.ToDictionary(p => p.Item1, p => p.Item2);
    return logger.BeginScope(dictionary);
}

Organization

Use a list of conceptual property names

You should build up a list of constant log entry property names (usually set with scopes) for your domain so that always the same name is used for them. For example always use UserId instead of Message.UserId and Request.UserId so that it’s easier to filter by user id over all log entries.

Log levels

It is recommended to think about what available log level to use. With this differentiation you can automatically create alerts, reports and issues.

For more information about when to use which log level, head over to the official documentation.

Use correct log levels

Here is a summary of the available .NET log levels and their meaning:

Trace/Verbose: Logs that contain the most detailed messages. These messages may contain sensitive application data. These messages are disabled by default and should never be enabled in a production environment.
Debug: Logs that are used for interactive investigation during development. These logs should primarily contain information useful for debugging and have no long-term value.
Information: Logs that track the general flow of the application. These logs should have long-term value.
Warning: Logs that highlight an abnormal or unexpected event in the application flow, but do not otherwise cause the application execution to stop.
Error: Logs that highlight when the current flow of execution is stopped due to a failure. These should indicate a failure in the current activity, not an application-wide failure.
Critical: Logs that describe an unrecoverable application or system crash, or a catastrophic failure that requires immediate attention.

Log exceptions with full details

Exceptions should always be logged as exceptions (i.e. not only the exception message) so that the stacktrace and all other information is retained. Use the correct log level to distinguish crictial, erroneous and informational exceptions.

Logger implementation

Use concrete implementations only in the application root

Use concrete implementations (e.g. Serilog types, etc.) only in the root of the application (e.g. Startup.cs of your ASP.NET Core app). All services and “logger consumers” should only use the ILogger, ILogger<T> or ILoggerFactory interface via (constructor) dependency injection. This way everything except the application bootstrapper is independent of the actual logger implementation and changing the logging sink is a no-brainer.

Consider using Serilog for a stable ILogger implementation

The ILogger (and ILoggerFactory) are quite generic and open interfaces with a lot of room about how to actually implement them. To show the problem, let’s have a look at the BeginScope method:

IDisposable BeginScope<TState>(TState state)

As you can see, the state can be any object. The actual implementation has to deal with that somehow but each implementation might differ on how it handles them. If you pass a Dictionary<string, object> to Serilog’s ILogger implementation, then it adds all these key-value tuples as custom properties to all log statements in the scope.

That is why I recommend to use Serilog as an intermedate layer/abstraction between ILogger and the actual sink (e.g. Application Insights) and not directly use the Application Insights ILogger implementation. The reason for that is that the AI implementation might not support all features (e.g. scopes are not supported) and changing a sink will not change the behavior and feature set of the logger implementation.

Sample code to use Serilog and ILogger in an ASP.NET Core web app (UseSerilog() can be found in Serilog.AspNetCore):

public class Program
{
    public static void Main(string[] args)
    {
        Log.Logger = new LoggerConfiguration()
            .WriteTo.Seq("http://localhost:5341")
            .WriteTo.Console()
            .CreateLogger();

        CreateWebHostBuilder(args)
            .UseSerilog()
            .Build()
            .Run();
    }

    public static IWebHostBuilder CreateWebHostBuilder(string[] args) =>
        WebHost.CreateDefaultBuilder(args)
            .UseStartup<Startup>();
}

Sample code to use Serilog and ILogger in a console application (AddSerilog() can be found in Serilog.Extensions.Logging):

public class Program
{
    public static void Main(string[] args)
    {
        Log.Logger = new LoggerConfiguration()
            .WriteTo.Seq("http://localhost:5341")
            .WriteTo.Console()
            .CreateLogger();

        new HostBuilder()
            .ConfigureServices((hostContext, services) =>
            {
                services.AddHostedService<MyHostedService>();
            })
            .ConfigureLogging((hostContext, logging) =>
            {
                logging.AddSerilog();
            })
            .Build()
            .Run();
    }
}

Setup for Application Insights without dependency injection

Here you can see a simple setup code to initialize the Serilog logger with an Application Insights sink and then add this logger provider to the logger factory:

Packages:

Microsoft.Extensions.Logging
Microsoft.Extensions.Logging.Abstractions
Serilog.Extensions.Logging
Serilog.Sinks.Seq

TelemetryConfiguration.Active.InstrumentationKey = instrumentationKey;

var serilogLogger = new LoggerConfiguration()
    .MinimumLevel.Debug()
    .MinimumLevel.Override("Microsoft", LogEventLevel.Information)
    .Enrich.FromLogContext()
    .WriteTo.ApplicationInsightsTraces(TelemetryConfiguration.Active)
    .CreateLogger();

var loggerFactory = (ILoggerFactory)new LoggerFactory();
loggerFactory.AddSerilog(serilogLogger);

Now you have a logger factory with which you can create new untyped loggers:

var logger = loggerFactory.CreateLogger();

… or typed loggers:

var logger = loggerFactory.CreateLogger<MyService>();

In this case, the type is used to add the SourceContext custom property in Application Insights for filtering.

Inject ILogger or ILogger<T>, avoid injecting ILoggerFactory

Prefer injecting the ILogger interface and if necessary the ILogger<T> interface. Ideally your class’ constructor requests an ILogger and the IoC framework creates an instance of an ILogger<T> where T is the requesting class.

Avoid injecting ILoggerFactory because it would violate the SRP (single responsibilty principle) if a service creates child instances (there are exceptions where you need to do this).

Libraries and logging

Try to avoid logging in libraries

Most libraries should not use logging but use regular control flow like return values, callbacks, events or exceptions. Consider using System.Diagnostics.Trace for debug logging.

Use logging via dependency injection in library code

.NET libraries which need to log, should only use the ILogger interface to communicate with the logging infrastructure. They should use it via (constructor) injection. Avoid using a static property/singleton because it would force all callers to use the same logging infrastructure.

Assume you have a library which has a MyHttpClient class to communicate with a backend. The signature of its constructor should look like this:

public class MyHttpClient
{
    private ILogger _logger;

    public MyHttpClient(string url, ILogger logger)
    {
        _logger = logger ?? NullLogger.Instance;
    }

    ...
}

Only use logging abstractions in libraries

In libraries you should only use the described logging abstractions and no concrete types because they are set up in the application root. This all implies that a .NET library should only have a dependency to Microsoft.Extensions.Logging.Abstractions for logging.

For more information about logging in libraries, I recommend reading Good citizenship - logging from .NET libraries

Testing

Use the null logger to ignore logs in tests

Use NullLogger.Instance as null object in tests to avoid lots of null checks just for testing.

Use Seq for local logging

I recommend to use the Seq as a local development logging backend. You can easily run it via Docker and write local development logs to it. This way you can better access and browse logs than with plain text files.

Just start a local Seq image:

docker run -d --restart unless-stopped --name seq -e ACCEPT_EULA=Y -p 5341:80 datalust/seq:latest

And add Seq as a Serilog logging sink:

var serilogLogger = new LoggerConfiguration()
    .WriteTo.Seq("http://localhost:5341")
    .CreateLogger();

var loggerFactory = (ILoggerFactory)new LoggerFactory();
loggerFactory.AddSerilog(serilogLogger);

var logger = loggerFactory.CreateLogger();

Now you can browse to http://localhost:5341 to access the local logging UI.

Technology Summary: Azure Blob Storage

2019-04-29T00:00:00+00:00

Category: Object storage (binary, unstructured)
Docker image: Not available
Local test replacement: Local file system

Microsoft Azure Blob Storage is used to store chunks of binary data to access from cloud services and applications. The technology is mainly used as replacement for a shared file system which is not available in the cloud.

You can store static files that are frequently used by a website, such as images, CSS files, and PDF files or store uploaded files instead of putting them into a more expensive, structured storage.

When to use

When you need to store files but don’t have a shared file system (e.g. applications in the cloud)

Constraints

Blob storage does not provide querying capability; a blob can only be accessed by its path/key/name (partial access is possible)

Concepts

The following list shows the fundamental concepts of the Blob Storage technology:

Storage account:
- The Azure resource of a Blob Storage is called storage account, provides a namespace in Azure and is required to manage and access containers and blobs.
Container:
- Each instance may have multiple blob containers. Containers are like root directories where Blobs can be stored.
Blob (Binary Large Objects):
- Binary data, e.g. file content
- Each blob has a unique name within its container. This name may contain slashes so that the blobs can be displayed as if they were stored in an hierarchical directory structure.
- There are three Blob types:
  - Block Blobs: Block blobs let you upload large blobs efficiently
  - Append Blobs: An append blob is comprised of blocks and is optimized for append operations
  - Page Blobs: Page blobs are a collection of 512-byte pages optimized for random read and write operations

Security

The authorization level can be defined per container:

Private: Only someone with the credentials for the storage account can access the container and the Blobs in it.
Blob: The container is private but the Blobs are publicly accessible
Container: The container is public and the Blobs are public: If someone knows the container name, they can access all of the Blobs, and they can get a list and iterate through them.

There are three ways to access stored blobs:

Direct access: If the blob is public, it can be accessed via its HTTP URL which is in the following form:
```
  https://mystoragename.blob.core.windows.net/mycontainername/myblobname
```
Pass-through to user (front-end proxy service): A cloud application with full access to the blob storage loads the blob data and passes it through to an authorized client. In an ASP.NET MVC appliation, you create an authorized controller action which has the storage credentials needed to access the file, and have the application require authentication before letting the requestor download the file.
Access with an SAS token
- A service authenticates a client as needed and then generates an SAS (Signed Access Signature) token in the backend and provides it to the client.
- This token can have an expiry date and is added to the private blob URL so that it can be accessed directly.

Data replication

To avoid data loss, you can choose between the following replication modes:

Locally redundant storage (LRS): This means three copies of your blobs are stored in a single facility in a single region
Zone redundant storage (ZRS): It replicates your data across 2 to 3 facilities, either within a single region or across two regions
Geo-Redundant Storage (GRS): This replicates your data three times in your chosen data center, and then replicates it three times in a secondary data center that is far away.
Read-Access Geo-Redundant Storage (RA-GRS): This is geo-redundant storage plus the ability to read the data in the secondary data center.

Usage

The common usage scenarios are:

Access over an HTTP RESTful API
Access using a client library, for example in .NET (see sample below)
Access with an Azure storage management application like Cerulean

The usage of the .NET client libary (Microsoft.Azure.Storage.Blob) is very simple:

var connectionString = "myconnectionstring";
var containerName = "mycontainername";
var blobName = "myblobname";

if (CloudStorageAccount.TryParse(connectionString, out var storageAccount))
{
    CloudBlobClient blobClient = storageAccount.CreateCloudBlobClient();
    CloudBlobContainer blobContainer = blobClient.GetContainerReference(containerName);

    // Create the container if it does not exist yet
    if (await blobContainer.ExistsAsync() == false)
    {
        await blobContainer.CreateAsync();
    }

    // Retrieve blob reference
    CloudBlockBlob blob = blobContainer.GetBlockBlobReference(blobName);

    // Read blob to a string
    using (var stream = new MemoryStream())
    {
        // Here you can also directly stream into the 
        // HTTP response stream to avoid memory allocations: 
        // await blob.DownloadToStreamAsync(HttpContext.Response.Body);        

        await blob.DownloadToStreamAsync(stream); 
        
        stream.Position = 0;
        using (var streamReader = new StreamReader(stream))
        {
            var text = await streamReader.ReadToEndAsync();
            // TODO: Use text
        }
    }
}

Pricing

The pricing is calculated from the following factors:

The amount of space the blobs take up
The number of operations performed
The amount of data transferred
The selected data redundancy option

More information

Alternatives

Amazon S3

Azure DevOps: How to update the Azure Function default Host Key in a PowerShell task

2019-01-21T00:00:00+00:00

For a recent project I dynamically create new Azure Functions in an Azure DevOps release pipeline. One of the requirements was to automatically update each default Host Key to a given value so that it’s easier to access the newly created HTTP functions.

Because there is no easy out-of-the-box API in Azure CLI or Azure PowerShell, I wanted to share the final solution here.

To update the default Azure Function Host Key in an Azure PowerShell build/release task, just follow these steps:

1. Create a new “Azure PowerShell” task in your Azure DevOps build or release pipeline

2. Choose an “Azure Subscription” which has privileges to access the resource

3. Under “Azure PowerShell Version” use the “Latest installed version”

4. Use “Inline Script” and insert the following script:

$functionName = "my-azure-function";
$resourceGroup = "my-azure-function-resource-group";
$functionHostKey = "my-new-function-host-key";

$publishingCredentials = Invoke-AzureRmResourceAction -ResourceGroupName $resourceGroup -ResourceType "Microsoft.Web/sites/config" -ResourceName "$functionName/publishingcredentials" -Action list -ApiVersion 2015-08-01 -Force
$authorization = [Convert]::ToBase64String([Text.Encoding]::ASCII.GetBytes(("{0}:{1}" -f $publishingCredentials.Properties.PublishingUserName, $publishingCredentials.Properties.PublishingPassword)))
$accessToken = Invoke-RestMethod -Uri "https://$functionName.scm.azurewebsites.net/api/functions/admin/token" -Headers @{Authorization=("Basic {0}" -f $authorization)} -Method GET

$data = @{
"name" = "default"
"value" = "$functionHostKey"
} | ConvertTo-Json;

$response = Invoke-RestMethod -Method PUT -Headers @{Authorization = ("Bearer {0}" -f $accessToken)} -ContentType "application/json" -Uri "https://$functionName.azurewebsites.net/admin/host/keys/default" -body $data

Update the variables $functionName, $resourceGroup and $functionHostKey to your liking - you can also use build variables, e.g. $(Build.BuildNumber):

As you can see, this script retrieves the Kudu credentials from your App Service to retrieve an access token. With this access token it then calls the Azure Function’s key management API to update the default host key.

Azure DevOps: Create a Web App for Containers CI/Release pipeline for an ASP.NET Core app

2018-10-17T00:00:00+00:00

In this tutorial we will:

Step 1: Build an ASP.NET Core app and push the sources to a Git repository
Step 2: Setup the required Azure resources (Web App for Containers, etc.)
Step 3: Setup an Azure DevOps CI build which builds a Docker image and pushes it to a private Azure Container registry (ACR)
Step 4: Create an Azure DevOps release pipeline which deploys the new Docker image to a Linux based Web App for Containers (App Service)

Important: The tutorial is using the name DockerDemo for the demo project and Azure resources. It is recommended to use a new name for your sample app, because the name DockerDemo is now in use and cannot be used for new Azure resources.

Step 1: Create an ASP.NET Core project

In this step we will implement an ASP.NET Core sample app, add the Azure DevOps YAML build scripts and push everything to a Git repository. The sample app and build script can ge found on GitHub/RSuter/DockerDemo. You can skip this part of the tutorial and directly use this public repository.

To manually create the app and build script, follow these steps:

Create a new directory for your new repository
Create a new ASP.NET Core project (with Linux Docker support) in the src directory

In the root of the repository, add the YAML build script azure-pipelines.yml:

pool:
  vmImage: 'Ubuntu 16.04'

variables:
  imageName: 'docker-demo:$(Build.BuildId)'

steps:
- script: docker build -f DockerDemo/Dockerfile -t $(dockerId).azurecr.io/$(imageName) .
  workingDirectory: src/
  displayName: 'Docker Build'

- script: |
    docker login -u $(dockerId) -p $pswd $(dockerId).azurecr.io
    docker push $(dockerId).azurecr.io/$(imageName)
  env:
    pswd: $(dockerPassword)
  workingDirectory: src/
  displayName: 'Docker Push'

This Azure DevOps YAML build script will build the Dockerfile in src/DockerDemo and push it to the Container registry tagged with current build id. Because the CI build triggers a release with the same build id, the build pipeline is able to deploy this exact version. It is common practice to build a docker image only once and deploy it to all the different environments.

Your file structure and solution should now look as follows:
Commit all your files and push the repository to your Azure DevOps project’s repository.

Step 2: Create the Azure Resources

In the second step, we will setup all required Azure resources: A Container registry, a Web App for Containers and its service plan:

Browse to the Azure Portal
Create a new resource group with the name DockerDemo
Create a new Azure Container Registry called DockerDemoContainerRegistry
- In the “Access keys” section, enable admin user so that a registry password is generated
Create a new Web App for Containers (Linux) resource with the name DockerDemoAppService and a new App Service Plan called DockerDemoAppServicePlan

After these steps, you should end up with a resource group with the following resources:

Step 3: Create the build definition

Now, we will setup the build definition in Azure DevOps:

Browse to your Azure DevOps project, navigate to Pipelines/Build and create a new build pipeline
Select your repository: The UI should automatically select your azure-pipelines.yml file:
After creating the build definition, add the following build variables so that the script can successfully run:
- dockerId: The name of the “Container registry” (without azurecr.io, e.g. dockerdemocontainerregistry)
- dockerPassword: The registry password which can be found in the Azure Portal by opening the “Container registry” resource and retrieving the password under “Access keys”
Now, you can run a build against your repository and the Container registry should be populated with a new image called docker-demo with the tag set to the current build id:

Step 4: Create the release pipeline

In this step, we will create a new release pipeline which is triggered by a successful build.

Limitation: At the moment, the Azure App Service task only supports deploying a single Docker image to the App Service. I think more tasks to deploy “image groups” via Docker Compose or Kubernetes will be supported soon.

Create a new release pipeline
Select the artifact from your build definition and enable the continuous deployment trigger:
Create a new “Azure App Service deployment”
Click on the first stage and create a new “Azure App Service deployment”
Prepare the newly created stage/environment:
- Select a subscription and the app service name
- Set the “App type” to “Linux app”
- In “Image Source” select “Container Registry”
- Enter the registry address and select the image docker-demo
Edit the “Deploy Azure App Service” task:
- Use $(Build.BuildId) as “Tag” so that the correct Docker image version from the associated CI build is deployed

Now, if you commit a change to your Git repository, a new CI build is triggered, a new image version is pushed and eventually a new release is created. This release will update the used image tag in the App Service which will trigger the actual image deployment.