Token based authentication is a fairly common way of authenticating a user for an HTTP application. However, handling this in a load balanced environment has always involved extra caring. In this post, I will show you how this is handled in ASP.NET Core by demonstrating it with HAProxy and Redis through the help of Docker.
@ 11-28-2016
by Tugberk Ugurlu

Token based authentication is a fairly common way of authenticating a user for an HTTP application. ASP.NET and its frameworks had support for implementing this out of the box without much effort with different type of authentication approaches such as cookie based authentication, bearer token authentication, etc. ASP.NET Core is a no exception to this and it got even better (which we will see in a while).

However, handling this in a load balanced environment has always involved extra caring as all of the nodes should be able to read the valid authentication token even if that token has been written by another node. Old-school ASP.NET solution to this is to keep the Machine Key in sync with all the nodes. Machine key, for those who are not familiar with it, is used to encrypt and decrypt the authentication tokens under ASP.NET and each machine by default has its own unique one. However, you can override this and put your own one in place per application through a setting inside the Web.config file. This approach had its own problems and with ASP.NET Core, all data protection APIs have been revamped which cleared a room for big improvements in this area such as key expiration and rolling, key encryption at rest, etc. One of those improvements is the ability to store keys in different storage systems, which is what I am going to touch on in this post.

The Problem

Imagine a case where we have an ASP.NET Core application which uses cookie based authentication and stores their user data in MongoDB, which has been implemented using ASP.NET Core Identity and its MongoDB provider.


This setup is all fine and our application should function perfectly. However, if we put this application behind HAProxy and scale it up to two nodes, we will start seeing problems like below:

System.Security.Cryptography.CryptographicException: The key {3470d9c3-e59d-4cd8-8668-56ba709e759d} was not found in the key ring.
   at Microsoft.AspNetCore.DataProtection.KeyManagement.KeyRingBasedDataProtector.UnprotectCore(Byte[] protectedData, Boolean allowOperationsOnRevokedKeys, UnprotectStatus& status)
   at Microsoft.AspNetCore.DataProtection.KeyManagement.KeyRingBasedDataProtector.DangerousUnprotect(Byte[] protectedData, Boolean ignoreRevocationErrors, Boolean& requiresMigration, Boolean& wasRevoked)
   at Microsoft.AspNetCore.DataProtection.KeyManagement.KeyRingBasedDataProtector.Unprotect(Byte[] protectedData)
   at Microsoft.AspNetCore.Antiforgery.Internal.DefaultAntiforgeryTokenSerializer.Deserialize(String serializedToken)

Let’s look at the below diagram to understand why we are having this problem:


By default, ASP.NET Core Data Protection is wired up to store its keys under the file system. If you have your application running under multiple nodes as shown in above diagram, each node will have its own keys to protect and unprotect the sensitive information like authentication cookie data. As you can guess, this behaviour is problematic with the above structure since one node cannot read the protected data which the other node protected.

The Solution

As I mentioned before, one of the extensibility points of ASP.NET Core Data Protection stack is the storage of the data protection keys. This place can be a central place where all the nodes of our web application can reach out to. Let’s look at the below diagram to understand what we mean by this:


Here, we have Redis as our Data Protection key storage. Redis is a good choice here as it’s a well-suited for key-value storage and that’s what we need. With this setup, it will be possible for both nodes of our application to read protected data regardless of which node has written it.

Wiring up Redis Data Protection Key Storage

With ASP.NET Core 1.0.0, we had to write the implementation by ourselves to make ASP.NET Core to store Data Protection keys on Redis but with 1.1.0 release, the team has simultaneously shipped a NuGet package which makes it really easy to wire this up: Microsoft.AspNetCore.DataProtection.Redis. This package easily allows us to swap the data protection storage destination to be Redis. We can do this while we are configuring services as part of ConfigureServices:

public void ConfigureServices(IServiceCollection services)
    // sad but a giant hack :(
    var redisHost = Configuration.GetValue<string>("Redis:Host");
    var redisPort = Configuration.GetValue<int>("Redis:Port");
    var redisIpAddress = Dns.GetHostEntryAsync(redisHost).Result.AddressList.Last();
    var redis = ConnectionMultiplexer.Connect($"{redisIpAddress}:{redisPort}");

    services.AddDataProtection().PersistKeysToRedis(redis, "DataProtection-Keys");

    // ...

I have wired it up exactly like this in my sample application in order to show you a working example. It’s an example taken from ASP.NET Identity repository but slightly changed to make it work with MongoDB Identity store provider.

Note here that configuration values above are specific to my implementation and it doesn’t have to be that way. See these lines inside my Docker Compose file and these inside my Startup class to understand how it’s being passed and hooked up.

The sample application can be run on Docker through Docker Compose and it will get a few things up and running:

  • Two nodes of the application
  • A MongoDB instance
  • A Redis instance


You can see my docker-compose.yml file to understand how I hooked things together:

    build: .
    dockerfile: mongo.dockerfile
    container_name: haproxy_redis_auth_mongodb
      - "27017:27017"

    build: .
    dockerfile: redis.dockerfile
    container_name: haproxy_redis_auth_redis
      - "6379:6379"

    build: .
    dockerfile: app.dockerfile
    container_name: haproxy_redis_auth_webapp1
      - ASPNETCORE_server.urls=
      - WebApp_MongoDb__ConnectionString=mongodb://mongo:27017
      - WebApp_Redis__Host=redis
      - WebApp_Redis__Port=6379
      - mongo
      - redis

    build: .
    dockerfile: app.dockerfile
    container_name: haproxy_redis_auth_webapp2
      - ASPNETCORE_server.urls=
      - WebApp_MongoDb__ConnectionString=mongodb://mongo:27017
      - WebApp_Redis__Host=redis
      - WebApp_Redis__Port=6379
      - mongo
      - redis

    build: .
    dockerfile: haproxy.dockerfile
    container_name: app_lb
      - "5000:80"
      - webapp1
      - webapp2

HAProxy is also configured to balance the load between two application nodes as you can see inside the haproxy.cfg file, which we copy under the relevant path inside our dockerfile:

  log local0
  log local1 notice

  log global
  mode http
  option httplog
  option dontlognull
  timeout connect 5000
  timeout client 10000
  timeout server 10000

frontend balancer
  mode http
  default_backend app_nodes

backend app_nodes
  mode http
  balance roundrobin
  option forwardfor
  http-request set-header X-Forwarded-Port %[dst_port]
  http-request set-header Connection keep-alive
  http-request add-header X-Forwarded-Proto https if { ssl_fc }
  option httpchk GET / HTTP/1.1\r\nHost:localhost
  server webapp1 webapp1:6000 check
  server webapp2 webapp2:6000 check

All of these are some details on how I wired up the sample to work. If we now look closely at the header of the web page, you should see the server name written inside the parenthesis. If you refresh enough, you will see that part alternating between two server names:



This confirms that our load is balanced between the two application nodes. The rest of the demo is actually very boring. It should just work as you expect it to work. Go to “Register” page and register for an account, log out and log back in. All of those interactions should just work. If we look inside the Redis instance, we should also see that Data Protection key has been written there:

docker run -it --link haproxy_redis_auth_redis:redis --rm redis redis-cli -h redis -p 6379
LRANGE DataProtection-Keys 0 10


Conclusion and Going Further

I believe that I was able to show you what you need to care about in terms of authentication when you scale our your application nodes to multiple servers. However, do not take my sample as is and apply to your production application :) There are a few important things that suck on my sample, like the fact that my application nodes talk to Redis in an unencrypted fashion. You may want to consider exposing Redis over a proxy which supports encryption.

The other important bit with my implementation is that all of the nodes of my application act as Data Protection key generators. Even if I haven’t seen much problems with this in practice so far, you may want to restrict only one node to be responsible for key generation. You can achieve this by calling DisableAutomaticKeyGeneration like below during the configuration stage on your secondary nodes:

public void ConfigureServices(IServiceCollection services)

I would suggest determining whether a node is primary or not through a configuration value so that you can override this through an environment variable for example.

React and Redux has been my go-to UI tools for a while now and I wanted share the fundemental benefits of adopting this type of architecture on your web applications. This is the first part of the blog post series I will publish on this.
@ 10-22-2016
by Tugberk Ugurlu

For the last a few months, I have been using React and Redux. I successfully converted a small application from Aurelia to React + Redux combination. Actually, it's a lie that that application is small. I told you it's small because it was less than halfway done at the time I started converting. That Web application is meant to be complex and has complicated data needs. Besides that application, we started building a completely new web application on top of React and Redux: SQL Clone Web Client. We haven’t shıpped ıt yet but we are approaching there.

This is an interesting move from someone like me who hated and thought React was bad.

However, I must admit that it was a premature thinking. I had my reasons but those reasons blocked me to look further to see the actual benefıt and power of React. In this post, I am going to hopefully answers the "why" question behind React and I will take it further to mix it with Redux.

I also suggest you to have a look at the short video of Cory House answering the "Why React" question.


If you don't know already, React is a library from Facebook to help you create user interfaces. It's important to make the distinction here that React is not a framework like Angular or Aurelia. You can see React as the V in MVC. However, this doesn't mean that you cannot create full-blown applications with React. There are a few other tiny libraries which you can plug into React like react-router, react-redux and they close the React's gap to build scalable Web applications.

React takes a declarative and component based approach on creating user interfaces. For example, the below is the well-known "Hello World" example of React:

var HelloMessage = React.createClass({
  render: function() {
      return <div>Hello {}</div>;

ReactDOM.render(<HelloMessage name="John" />, mountNode);

mountNode is just an element here which you can get a hold of through document.getElementById or similar APIs to that.

This will print "Hello John" inside the mountNode. At this point, it's OK to be puzzled with all the buzz about React because this is not something which blows your mind and if you are seeing this for the first, it's also OK to go "WTAF" here because you have something HTML{ish} inside your JavaScript code. This is neither HTML nor part of Javascript. That is JSX and it needs transpiling. As stated, you go "WTF" over this (well, at least I did) because it might go against the "separation of concerns" thinking you have been holding onto. I have been there and done this. I stopped looking at React at that point and convinced myself that it's not something that would scale in terms of maintenance.

Do you have the same reaction? If so, let me tell something which I wish someone would told me when I had this reaction and I think deserves to be emphasized over and over again. The above "Hello World" example is not the place where you should start learning React. It should be "Thinking in React" documentation that you should start with. It's such a good documentation explaining what the mindset you should be in while you are working with React. However, you might be dealing with another challenge in our mind now while this direction can put you into a different situation:

This kind of has a point. I believe the amount of boilerplate code which you write with React to get your application running is higher than any of its opponents. However, small applications is probably not going to highlight the value of React that well. You see the value it brings when you think in terms of the whole application. Let me tell you a few things about this.

To me, most valuable thing about an evolvable application (doesn't have to be a web application, any sort of it) is to be able to reason about its behaviour at any point during its evolution. Whenever, you see some unexpected behaviour, you should able to say "this is happening because this part in my application is doing something wrong". This is also valid for expected behaviours as well as you will be contributing to its evaluation by adding new features to it and you should know exactly where you need to apply the changes to add those features.

The other point I want to make here is based on amazing Brian Holt's talk at Dev Day: "React: Learn Once Write Anywhere". If my memory is not failing me, he stated a very important fact about a React application: its constant complexity level. I accept that a typical React application starts complex but that complexity mostly stays the same during the entire evolution of the web application unless you go outside the boundaries of the structure. However, if you go with a simple approach at the begging, your complexity will grow as you grow your web application codebase. I have witnessed this several times and I will tell you that it’s not fun to apply changes after a certain point.

If you are struggling with the React getting started process, there is an approach which I see very efficient that will ease your process. You can start working without thinking about the separation of your application into smaller components. With this way, you can write your components as big, fat components. Once you have two or three like that, you will start seeing patterns which will make you think about isolations. That's where you can start drawing mockups like this one. I went this way at the start where I was really fuzzy about React but now, it's a set mindset for me that I start with a mockup.

Final point I want to touch on is about stability and problem solving, I think one of the biggest values of React is its stability and adoption. I can assure you that the most of your Web UI problems have an answer in React and it is also highly likely that someone has already written up a blog post, put a sample somewhere or answered a Stackoverflow question on that. Also, Facebook and some other big companies rely on React. So, the evolution of React is a fairly safe progress in this fast evolving JavaScript ecosystem.


In this post, I just wanted cover the React part of the story and I should admit that it's not much. However, my aim here is to convince you to give React a chance. I can understand that you may have questions about how to handle the data at this stage (e.g. how to react to the changes that happens behind the background, etc.). This is probably a good time for you to look at what Flux application architecture is all about and have a go with one of the state management libraries like Redux. This is the exact topic that I will touch on the second part of this blog post series, which will also highlight the practical strength of React along with Redux.

Further Resources

I want to share a few thoughts that I have been keeping to myself on showing progress for long-running asyncronous operations on a system where individual events can be sent during ongoing operations.
@ 07-30-2016
by Tugberk Ugurlu

With today's modern applications, we end up with lots of asynchronous operations happening and we somehow have to let the user know what is happening. This, of course, depends on the requirements and business needs.

Let's look at the Amazon case. When you buy something on Amazon, Amazon tells you that your order has been taken. However, it doesn't really mean that you have actually purchased it as the process of taking the payment has not been kicked off yet. The payment process probably goes through a few stages and at the end, it has a few outcome options like success, faılure, etc. Amazon only gives the user the change to know about the outcome of this process which is done through e-mail. This is pretty straightforward to implement (famous last words?) as it doesn't require things like real-time process notifications, etc..

On the other hand, creating a VM on Microsoft Azure has different needs. When you kick the VM creation operation through an Azure client like Azure Portal, it can give you real-time notifications on the operation status to show you in what stage the operation is (e.g. provisioning, starting up, etc.).

In this post, I will look at the later option and try to make a few points to prove that the implementation is a bit tricky. However, we will see a potential solution at the end and that will hopefully be helpful to you as well.

Problematic Flow

Let's look at a problematic flow:

  • User starts an operation by sending a POST/PUT/PATCH request to an HTTP endpoint.
  • Server sends the 202 (Accepted) response and includes a some sort of operation identifier on the response body.
  • Client subscribes to a notification channel based on this operation identifier.
  • Client starts receiving events on that notification channel and it decides on how to display them on the user interface (UI).
  • There are special event types which give an indication that the operation has ended (e.g. completed, failed, abandoned, etc.).
    • The client needs to know about these event types and handle them specially.
    • Whenever the client receives an event which corresponds to operation end event, it should unsubscribe from the notification channel for that specific operation.

We have a few problems with the above flow:

  1. Between the operation start and the client subscribing to a notification channel, there could be events that potentially could happen and the client is going to miss those.
  2. If the user disconnects from the notification channel for a while (for various reasons), the client will miss the events that could happen during that disconnect period.
  3. Similar to above situation, the user might entirely shut its client and opens it up again at some point later to see the progress. In that case, we lost the events that has happened before subscribing to the notification channel on the new session.

Possible Solution

The solution to all of the above problems boils down to persisting the events and opening up an HTTP endpoint to pull all the past events per each operation. Let's assume that we are performing payment transactions and want to give transparent progress on this process. Our events endpoint would look similar to below:

GET /payments/eefcb363/events

          "id": "c12c432c",
          "type": "ProcessingStarted",
          "message": "The payment 'eefcb363' has been started for processing by worker 'h8723h7d'.",
          "happenedAt": "2016-07-30T11:05:26.222Z"
          "details": {
               "paymentId": "eefcb363",
               "workerId": "h8723h7d"

          "id": "6bbb1d50",
          "type": "FraudCheckStarted",
          "message": "The given credit card details are being checked against fraud for payment 'eefcb363' by worker 'h8723h7d'.",
          "happenedAt": "2016-07-30T11:05:28.779Z"
          "details": {
               "paymentId": "eefcb363",
               "workerId": "h8723h7d"

          "id": "f9e09a83",
          "type": "ProgressUpdated",
          "message": "40% of the payment 'eefcb363' has been processed by worker 'h8723h7d'.",
          "happenedAt": "2016-07-30T11:05:29.892Z"
          "details": {
               "paymentId": "eefcb363",
               "workerId": "h8723h7d",
               "percentage": 40

This endpoint allows the client to pull the events that have already happened, and the client can mix these up with the events that it receives through the notification channel where the events are pushed in the same format.

With this approach, suggested flow for the client is to follow the below steps in order to get 100% correctness on the progress and events:

  • User starts an operation by sending a POST/PUT/PATCH request to an HTTP endpoint.
  • Server sends the 202 (Accepted) response and includes a some sort of operation identifier (which is 'eefcb363' in our case) on response body.
  • Client subscribes to a notification channel based on that operation identifier.
  • Client starts receiving events on that channel and puts them on a buffer list.
  • Client then sends a GET request based on the operation id to get all the events which have happened so far and puts them into the buffer list (only the ones that are not already in there).
  • Client can now start displaying the events from the buffer to the user (optionally in chronological order) and keeps receiving events through the notification channel; updates the UI simultaneously.

Obviously, at any display stage, the client needs to honor that there are special event types which give an indication that the operation has ended. The client needs to know about these types and handle them specially (e.g. show the completed view, etc.). Besides this, whenever the client receives an event which corresponds to operation end event, it should unsubscribe from the notification channel for that specific operation.

However, the situation can be a bit easier if you only care about percentage progress. If that's case, you may still want to persist all events but you only care about the latest event on the client. This makes your life easier and if you decide that your want to be more transparent about showing the progress, you can as you already have all the data for this. It is just a matter of exposing them with the above approach.


Under any circumstances, I would suggest you to persist operation events (you can even go further with event sourcing and make this a natural process). However, your use case may not require extensive and transparent progress reporting through the client. If that's the case, it will certainly make your implementation a lot simpler. However, you can change your mind later and go with a different progress reporting approach since you have been already persisting all the events.

Finally, please share your thoughts on this if you see a better way of handling this.

I had the privilege to attend NDC Oslo 2016 as a speaker this year. It was a fabulous experience. With this post, I want to share my talk and the list of sessions that grabbed my attention.
@ 06-25-2016
by Tugberk Ugurlu

As I blogged earlier, I had the privilege to attend NDC Oslo 2016 as a speaker this year. It was a fabulous experience and I took a different approach at NDC Oslo this year and rather than improving my existing knowledge, I decided to take different perspective on problems. There were really a few areas I wanted to get more information on from the World class experts:

  • Machine Learning
  • Functional Programming
  • Containerization
  • Soft Skills

Of course, this is NDC and you have people like Mark Rendle and Rob Conery. So, it was inevitable that I ended up at sessions which were just fun :) Nevertheless, it was pretty useful 3 days for me at the conference and it feels like I achieved my end goal.


2016-06-08 09.04.02

One other amazing aspect of NDC Conferences is that all of the talks are being recorded and nearly all of those recordings are now up on Vimeo for public consumption :)

Here is the list of sessions I had a chance to attend in person:

All of them were really helpful and gave me a different perspective on the topics. Aside from those sessions, here are the talks that I really wanted to attend but I couldn't + I will definitely watch:

The schedule was full of amazing talks as you can see and there are also some other talks that seem interesting but I will probably not have time to look at:

My Session

At the conference I gave a talk on Zero Downtime Deployments. I tried to give useful and practical guidance based on my true experiences. I hope it was useful for everyone attended the talk. The recording video of my talk is also available now.

Getting Into the Zero Downtime Deployment World - Tugberk Ugurlu from NDC Conferences on Vimeo.

The slides of the talk is also available under my Speakerdeck account.

Finally, you can find the demo sample I used in my talk to simulate a zero-downtime deployment process under my GitHub account. It also has great instructions on how to run the sample. So, definitely give it a go.


Next week, I am off to Oslo for one of my favorite conferences: NDC Oslo and this time is a little bit more special as I am one of the speakers this year, talking about zero-downtime deployments.
@ 06-05-2016
by Tugberk Ugurlu

Next week, I am off to Oslo for one of my favorite conferences: NDC Oslo. I have been to NDC Oslo before in 2014 but this time is a little bit more special as I am one of the speakers this year. Seeing the list of awesome speakers at conference makes me so excited and nervous at the same time :)

As for my topic, I am going to be talking about zero-downtime deployments. I will tell you about some patterns, practices and techniques that make it this challenging task easier, such as semantic versioning and blue/green deployments. We’ll also walk through an end-to-end demo of how a high traffic web application can survive the challenge of deployments. If you are going to be there, I hope you will also join me at my talk.


I can see lots of people from my Twitter timeline already tweeting about NDC Oslo. So, I am really sure that this is going to be an exciting event. I hope to see lots of friends and make new friends while I am there. See you in Oslo :)

Hi 🙋🏻‍♂️ I'm Tugberk Ugurlu.
Coder 👨🏻‍💻, Speaker 🗣, Author 📚, Microsoft MVP 🕸, Blogger 💻, Software Engineering at Deliveroo 🍕🍜🌯, F1 fan 🏎🚀, Loves travelling 🛫🛬
Lives in Cambridge, UK 🏡