Windows Azure Management Client Libraries for .NET and It Supports Portable Class Library

One of the missing pieces of the Windows Azure story is within our reach now! A few days ago Azure folks have released Windows Azure .NET Management Client Libraries
2013-10-30 06:17
Tugberk Ugurlu


One of the missing pieces of the Windows Azure story is within our reach now! Awesome Azure folks have released Windows Azure .NET Management Client Libraries which also support portable class library (PCL). It is now possible to build custom Windows Azure management applications in .NET ecosystem with possible minimum effort. Brady Gaster has a very detailed blog post about the first release of these libraries if you would like to get started with it really quickly. In this post, I'll look at the these libraries in a web developer point of view.

Management libraries are available as NuGet packages and they are at the pre-release stage for now (ignore the HdInsight package below):

image

Some packages are there for the infrastructural purposes such as tracing, exception handling, etc. and other packages such as Storage and Compute has .NET clients to manage specific services which is good and bad at the same time. When we take a first look at the libraries, we can see a few great things:

  • All operations are based on the HttpClient. This gives us the opportunity to be very flexible (if the client infrastructure allows us) by injecting our inner handler to process the requests.
  • All client operations are asynchronous from top to bottom.
  • It supports Portable Class Library.

Getting started is also very easy. The following code is all I needed to do to query the storage accounts I have inside my subscription:

static void Main(string[] args)
{
	const string CertThumbprint = "your-cert-thumbprint";
	const string SubscriptionId = "your-subscription-id";
	X509Certificate2 cert = Utils.FindX509Certificate(CertThumbprint);
	SubscriptionCloudCredentials creds = 
		new CertificateCloudCredentials(SubscriptionId, cert);

	StorageManagementClient storageClient = 
		CloudContext.Clients.CreateStorageManagementClient(creds);

	StorageServiceListResponse response = storageClient.StorageAccounts.List();
	foreach (StorageServiceListResponse.StorageService storageService in response)
	{
		Console.WriteLine(storageService.Uri);
	}

	Console.ReadLine();
}

The client certificate authentication is performed to authenticate our requests. Notice that how easy it was to get a hold of a storage client. By using the CloudContext class, I was able to create the storage client by using the CreateStorageManagementClient extension method which lives under the Microsoft.WindowsAzure.Management.Storage assembly.

How HttpClient is Used Underneath

If we look under the carpet, we will see that each time I call the CreateStorageManagementClient, we'll use a different instance of the HttpClient which is not we would want. However, I assume that StorageManagementClient's public members are thread safe which means that you can use one instance of that class throughout our web applications but I'm not sure. Nevertheless, I would prefer an approach which is similar to the following one:

// single instance that I would hold onto throughout my application lifecycle. I would most
// probably handle this instance through my IoC container.
using (CloudContext cloudContex = new CloudContext(creds))
{
    // These StorageManagementClient instances are either the same reference or 
    // use the same HttpClient underneath.
    StorageManagementClient storageClient = 
        cloudContex.CreateStorageManagementClient();
}

On the other hand, we need to perform a similar operation to create a compute management client to manage our could service and virtual machines: call the CreateComputeManagementClient extension method on the CloudContext.Clients. Here, we have the same behavior in terms of HttpClient instance.

A Sample Usage in an ASP.NET MVC Application

I created a very simple ASP.NET MVC application which only lists storage services, hosted services and web spaces. I used an IoC container (Autofac) to inject the clients through the controller constructor. Below is the registration code I have in my application.

protected void Application_Start()
{
    // Lines removed for brevity...

    // Get a hold of the credentials.
    const string CertThumbprint = "your-cert-thumbprint";
    const string SubscriptionId = "your-subscription-id";
    X509Certificate2 cert = FindX509Certificate(CertThumbprint);
    SubscriptionCloudCredentials creds =
        new CertificateCloudCredentials(SubscriptionId, cert);

    // Get the ContainerBuilder ready and register the MVC controllers.
    ContainerBuilder builder = new ContainerBuilder();
    builder.RegisterControllers(Assembly.GetExecutingAssembly());

    // Register the clients
    builder.Register(c => CloudContext.Clients.CreateComputeManagementClient(creds))
        .As<IComputeManagementClient>().InstancePerHttpRequest();
    builder.Register(c => CloudContext.Clients.CreateStorageManagementClient(creds))
        .As<IStorageManagementClient>().InstancePerHttpRequest();
    builder.Register(c => CloudContext.Clients.CreateWebSiteManagementClient(creds))
        .As<IWebSiteManagementClient>().InstancePerHttpRequest();

    // Set the dependency resolver.
    AutofacDependencyResolver dependencyResolver = 
        new AutofacDependencyResolver(builder.Build());

    DependencyResolver.SetResolver(dependencyResolver);
}

Noticed that I registered the management client classes as per-request instance. This option will create separate client instances per each request. As the underlying architecture creates separate HttpClient instances per each client creating, I'll not be using the same HttpClient throughout my application lifecycle. I'm also not sure whether the client classes are thread safe. That's why I went for the per-request option.

My controller is even simpler.

public class HomeController : Controller
{
    private readonly IComputeManagementClient _computeClient;
    private readonly IStorageManagementClient _storageClient;
    private readonly IWebSiteManagementClient _webSiteClient;

    public HomeController(
        IComputeManagementClient computeClient, 
        IStorageManagementClient storageClient, 
        IWebSiteManagementClient webSiteClient)
    {
        _computeClient = computeClient;
        _storageClient = storageClient;
        _webSiteClient = webSiteClient;
    }

    public async Task<ActionResult> Index()
    {
        Task<StorageServiceListResponse> storageServiceResponseTask = 
            _storageClient.StorageAccounts.ListAsync();
        Task<HostedServiceListResponse> hostedServiceResponseTask = 
            _computeClient.HostedServices.ListAsync();
        Task<WebSpacesListResponse> webSpaceResponseTask = 
            _webSiteClient.WebSpaces.ListAsync();

        await Task.WhenAll(storageServiceResponseTask, 
            hostedServiceResponseTask, webSpaceResponseTask);

        return View(new HomeViewModel 
        { 
            StorageServices = storageServiceResponseTask.Result,
            HostedServices = hostedServiceResponseTask.Result,
            WebSpaces = webSpaceResponseTask.Result
        });
    }
}

public class HomeViewModel
{
    public IEnumerable<StorageServiceListResponse.StorageService> StorageServices { get; set; }
    public IEnumerable<HostedServiceListResponse.HostedService> HostedServices { get; set; }
    public IEnumerable<WebSpacesListResponse.WebSpace> WebSpaces { get; set; }
}

Notice that I used Task.WhenAll to fire all the asynchronous work in parallel which is best of both worlds. If you also look at the client classes, you will see that operations are divided into logical groups. For example, here is the IComputeManagementClient interface:

public interface IComputeManagementClient
{
    Uri BaseUri { get; }
    SubscriptionCloudCredentials Credentials { get; }

    IDeploymentOperations Deployments { get; }
    IHostedServiceOperations HostedServices { get; }
    IOperatingSystemOperations OperatingSystems { get; }
    IServiceCertificateOperations ServiceCertificates { get; }
    IVirtualMachineDiskOperations VirtualMachineDisks { get; }
    IVirtualMachineImageOperations VirtualMachineImages { get; }
    IVirtualMachineOperations VirtualMachines { get; }

    Task<Models.ComputeOperationStatusResponse> GetOperationStatusAsync(
        string requestId, 
        CancellationToken cancellationToken);
}

HostedServices, OperatingSystems, VirtualMachines and so on. Each logical group of operations are separated as class properties.

Getting Inside the HTTP Pipeline

The current implementation of the libraries also allow us to get into the HTTP pipeline really easily. However, the way we do it today is a little ugly but it depends on the mentioned HttpClient usage underneath. Nevertheless, this extensibility is really promising. Every management client class inherits the ServiceClient<T> class and that class has a method called WithHandler. Every available client provides a method with the same name by calling the underlying WithHandler method from ServiceClient<T>. By calling this method with a DelegatingHandler instance, we can get into the HTTP pipeline and have a chance to manipulate the request processing. For example, we can inject an handler which has a request retry logic if certain error cases are met. Windows Azure Management Library provides an abstract class for a retry handler which has the basic functionality: LinearRetryHandler. As this is an abstract class, we can inherit this class to create our own retry handler. If we need, we can always override the ShouldRetry method but for our case, we don't need it.

public class CustomRetryHandler : LinearRetryHandler
{
}

Now, we can use our retry handler to create the management clients:

RetryHandler retryHandler = new CustomRetryHandler();
IComputeManagementClient client = 
    CloudContext.Clients.CreateComputeManagementClient(creds).WithHandler(retryHandler)

Now, if a request to a Windows Azure Management API fails with some of the certain status codes, the handler will retry the request.

There are More…

There are more features that are worth mentioning but it will make a very lengthy blog post :) Especially the unit testing and tracing are the topics that I'm looking forward to blogging about.

Short Introduction Video for OWIN and Project Katana

I've recorded a short video which covers the brief introduction of OWIN and Project Katana. That short video will give you an idea about OWIN and Project Katana.
2013-10-21 13:29
Tugberk Ugurlu


I've recorded a short video which covers the brief introduction of OWIN and Project Katana. That short video will give you an idea about OWIN and Project Katana and why we should care about it.

An Introduction to OWIN and Project Katana from Tugberk Ugurlu on Vimeo.

Let me know whether this recording is any help to you. Your feedback is always more than welcome. Enjoy!

Replace the Default Server of OwinHost.exe with Nowin in Visual Studio 2013

This post will show you how to you can replace the default server of OwinHost.exe with Nowin in Visual Studio 2013
2013-09-27 11:14
Tugberk Ugurlu


Wow, I cannot believe I'm writing this post :) It's really exciting to see that .NET web stack has evolved this much. First, we had a web framework shipping out of band: ASP.NET MVC. Then, we were given a chance to play with hosting agnostic web frameworks such as ASP.NET Web API and ASP.NET SignalR (I know, their names are still confusing and give an impression that those frameworks are still bound to ASP.NET but they are actually not). Recently, we have been embracing the idea of separating the application, server and the host from each other by making our applications OWIN compliant.

By this way, we can easily build our pipeline on our own and switch the underlying host or server easily. Visual Studio 2013 even has a really nice extensibility point to switch the underlying host and still have the F5 experience. OwinHost.exe is one of those hosts that we can use as an alternative to IIS. Today, we can even take this further and completely replace the underlying server (which is HttpListener by default with OwinHost.exe) by preserving the host. There is an OWIN compliant server implementation called Nowin developed by a very clever guy, Boris Letocha. This server uses a raw .NET socket to listen to the HTTP requests coming in through the wire and will respond to them. By looking at the repository's readme file, I can say that this component is not production ready but will work out just fine for demonstration purposes.

I created a mini sample application to show you this cool feature. You can find it on my GitHub OwinSamples repository, too. It only contains the following startup class and initially have two NuGet packages installed: Owin and OwinHost.

using AppFunc = Func<IDictionary<string, object>, Task>;

public class Startup
{
    public void Configuration(IAppBuilder app)
    {
        app.Use(new Func<AppFunc, AppFunc>(ignoreNext => Invoke));
    }

    public async Task Invoke(IDictionary<string, object> env)
    {
        // retrieve the Request Data from the environment
        string path = env["owin.RequestPath"] as string;

        if (path.Equals("/", StringComparison.OrdinalIgnoreCase))
        {
            // Prepare the message
            const string Message = "Hello World!";
            byte[] bytes = Encoding.UTF8.GetBytes(Message);

            // retrieve the Response Data from the environment
            Stream responseBody = env["owin.ResponseBody"] as Stream;
            IDictionary<string, string[]> responseHeaders = 
                env["owin.ResponseHeaders"] as IDictionary<string, string[]>;

            // write the headers, response body
            responseHeaders["Content-Type"] = new[] { "text/plain" };
            await responseBody.WriteAsync(bytes, 0, bytes.Length);
        }
    }
}

I am not sure if you have noticed this but Visual Studio 2013 even has an item template for creating an OWIN startup class:

image

This seems nice but it secretly installs the Microsoft.Owin package, which is unnecessary if you ask me. Owin NuGet package should be enough IMHO.

I also applied the steps explained in my "OwinHost.exe on Visual Studio 2013" post to get my application running on top of OwinHost and here is the result:

image

Here, my host is OwinHost.exe and it uses the Microsoft.Owin.Host.HttpListener as the server by default. At the application level, we don't need to know or care about which server we are on but most of the OWIN server implementations expose their names through the server.Capabilities dictionary:

image

What we want to accomplish here is to keep the host as it is and only replace the server component that it uses underneath. As our host (OwinHost.exe) is OWIN compliant, it can work with any other type of OWIN compliant server implementations and one of them is Nowin. Installing Nowin over NuGet into your project is the first step that you need to do.

image

However, this's not enough by itself as OwinHost.exe has no idea that we want to use Nowin as our server. Luckily, we can manipulate the arguments we pass onto OwinHost.exe. You can configure these arguments through the Web pane inside the Visual Studio Project Properties window. Besides that, OwinHost.exe accepts a few command line switches and one of them is the –s (or –-server) switch to load the specified server factory type or assembly. These are all we needed.

image

After saving the changes we have made, we can run the application and get the same result on top of a completely different server implementation:

image

Also with the same way as we did earlier, we can see that the switch has been made and Nowin is in use:

image

Being able to do all of this is a very big deal; especially if you think that we have been tied to IIS for very long time. I'm so happy to see .NET web stack moving towards this flexible direction.

Pro ASP.NET Web API Book is Shipped and Available on Amazon

Today, I am very proud to say that Pro ASP.NET Web API Book is now shipped and available on Amazon for paperback sales :)
2013-09-25 06:28
Tugberk Ugurlu


You may be aware that I have been working on a book for while now: Pro ASP.NET Web API from Apress. Alexander Zeitler, Ali Kheyrollahi and myself have been hard at work lately to complete this high quality asset for ASP.NET Web API lovers. Today, I am very proud to say that Pro ASP.NET Web API is now available for sale on Amazon as paperback! I would like to thank everybody who put his/her time into this book to make it a really good resource.

Here is a bit information about the book structure:

In Part I, you'll get up to speed on Web API's modern HTTP programming model, asynchronous programming in .NET framework and basic HTTP information. Part II takes you through building a real application so you can see straight away how to put this new technology into practice. The second half of the book features dedicated chapters on topics like routing, controllers, validation and tracing, and we will have chapters on performance, dependency injection and an all-important look at unit testing to help you prepare your application for the real world.

Please leave your comments on Amazon as they are really important for us. Hope you will enjoy the book :)

Resources

Logging in the OWIN World with Microsoft.Owin: Introduction

Microsoft implementation of OWIN (Microsoft.Owin or Katana for now) has a great logging infrastructure and this post will walk you through the basics of this component.
2013-09-18 16:14
Tugberk Ugurlu


Microsoft implementation of OWIN (called Microsoft.Owin or Katana for now) has a great infrastructure for logging under Microsoft.Owin.Logging namespace inside the Microsoft.Owin assembly. Same as all other Microsoft.Owin components, the logging infrastructure is built on top of interfaces and there are two of them: ILoggerFactory and ILogger. The default implementation is using .NET Framework tracing components heavily such as System.Diagnostics.TraceSource and System.Diagnostics.TraceSwitch. You can learn about these components and how to instrument your applications with them through the .NET Development Guide's Tracing section on MSDN.

The responsibility of the logger factory is to construct a new logger when needed. You can see how ILoggerFactory interface looks like below:

namespace Microsoft.Owin.Logging
{
    /// <summary>
    /// Used to create logger instances of the given name.
    /// </summary>
    public interface ILoggerFactory
    {
        /// <summary>
        /// Creates a new ILogger instance of the given name.
        /// </summary>
        /// <param name="name"></param>
        /// <returns></returns>
        ILogger Create(string name);
    }
}

As you can see, the Create method returns an ILogger implementation. The ILogger implementation is solely responsible for writing the logs through its one and only method: WriteCore.

namespace Microsoft.Owin.Logging
{
    /// <summary>
    /// A generic interface for logging.
    /// </summary>
    public interface ILogger
    {
        /// <summary>
        /// Aggregates most logging patterns to a single method.  This must be compatible with the Func representation in the OWIN environment.
        /// 
        /// To check IsEnabled call WriteCore with only TraceEventType and check the return value, no event will be written.
        /// </summary>
        /// <param name="eventType"></param>
        /// <param name="eventId"></param>
        /// <param name="state"></param>
        /// <param name="exception"></param>
        /// <param name="formatter"></param>
        /// <returns></returns>
        bool WriteCore(TraceEventType eventType, int eventId, object state, 
            Exception exception, Func<object, Exception, string> formatter);
    }
}

For these interfaces, there are two implementations provided by the Microsoft.Owin assembly for logging: DiagnosticsLoggerFactory (publicly exposed) and DiagnosticsLogger (internally exposed). However, these are the types provided by the Microsoft's OWIN implementation and don't belong to core .NET framework. As it's discouraged to put non-framework types inside the IAppBuilder properties dictionary and request's environment, a Func is put inside the IAppBuilder properties dictionary with a key named "server.LoggerFactory". Here is the signature of that Func.

using TraceFactoryDelegate = 
    Func
    <
        string, 
        Func
        <
            TraceEventType, 
            int, 
            object, 
            Exception, 
            Func
            <
                object, 
                Exception, 
                string
            >, 
            bool
        >
    >;

Through this delegate, you are expected to create a Func object based on the logger name and this Func will be used to write logs. The currently provided extension methods to work with the Katana’s logging infrastructure hides these delegates from you. Here are a few extension methods under Microsoft.Owin.Logging namespace for the IAppBuilder interface:

public static void SetLoggerFactory(
    this IAppBuilder app, ILoggerFactory loggerFactory);
    
public static ILoggerFactory GetLoggerFactory(
    this IAppBuilder app);
    
public static ILogger CreateLogger(
    this IAppBuilder app, string name);

All these extension methods work on the TraceFactoryDelegate Func instance shown above but it’s all hidden from us as mentioned.

So, how we can take advantage of this provided components? It’s quite easy actually. You can create separate loggers for each of your components using one of the CreateLogger extension method overloads for the IAppBuilder interface and use that logger to write log messages. The following is just a little sample to give you a hint:

public class Startup
{
    public void Configuration(IAppBuilder app)
    {
        Log1(app);
        app.Use<MyCustomMiddleware>(app);
    }

    private void Log1(IAppBuilder app) 
    {
        ILogger logger = app.CreateLogger<Startup>();
        logger.WriteError("App is starting up");
        logger.WriteCritical("App is starting up");
        logger.WriteWarning("App is starting up");
        logger.WriteVerbose("App is starting up");
        logger.WriteInformation("App is starting up");

        int foo = 1;
        int bar = 0;

        try
        {
            int fb = foo / bar;
        }
        catch (Exception ex)
        {
            logger.WriteError("Error on calculation", ex);
        }
    }
}

public class MyCustomMiddleware : OwinMiddleware
{
    private readonly ILogger _logger;

    public MyCustomMiddleware(
        OwinMiddleware next, IAppBuilder app) : base(next)
    {
        _logger = app.CreateLogger<MyCustomMiddleware>();
    }

    public override Task Invoke(IOwinContext context)
    {
        _logger.WriteVerbose(
            string.Format("{0} {1}: {2}", 
            context.Request.Scheme, 
            context.Request.Method, 
            context.Request.Path));

        context.Response.Headers.Add(
            "Content-Type", new[] { "text/plain" });
            
        return context.Response.WriteAsync(
            "Logging sample is runnig!");
    }
}

There are a few things I would like to touch on here:

  • I used generic CreateLogger method to create loggers. This will create loggers based on the name of the type I’m passing in. In view of the default logger implementation provided by the Katana, this means that we will create TraceSource instances named as the full type name.
  • You can see that I receive an IAppBuilder implementation instance through the constructor of my middleware and I used that instance to create a logger specific to my middleware. I can hold onto that logger throughout the AppDomain lifetime as all members of my logger are thread safe.
  • Instead of using the WriteCore method to write logs, I used several extension methods such as WriteVerbose to write specific logs.

This is not enough by default as we didn’t configure what kind of tracing data we are interested in and how we would like to output them. We need to configure Trace Switches and Trace Listeners properly to instrument our application. Microsoft.Owin is root switch and if we enable it, we will see all messages we write through our loggers. The following configuration will enable the Microsoft.Owin switch:

<configuration>
  <system.diagnostics>
    <switches>
      <add name="Microsoft.Owin" value="Verbose" />
    </switches>
  </system.diagnostics>
</configuration>

When we run our application on Debug mode, we can see that the Output window will show our log data:

image

We can take it further and configure different listeners for different switches. The following configuration will enable TextWriterTraceListener for LoggingSample.MyCustomMiddleware SourceSwitch:

<configuration>
  <system.diagnostics>
    <switches>
      <add name="Microsoft.Owin" value="Verbose" />
    </switches>

    <sharedListeners>
      <add name="console" type="System.Diagnostics.ConsoleTraceListener" />
    </sharedListeners>

    <trace autoflush="true" />
    
    <sources>
      <source name="Microsoft.Owin">
        <listeners>
          <add name="console" />
        </listeners>
      </source>

      <source name="LoggingSample.MyCustomMiddleware">
        <listeners>
          <add name="file" 
               type="System.Diagnostics.TextWriterTraceListener" 
               initializeData="traces-MyCustomMiddleware.log" />
        </listeners>
      </source>
    </sources>
  </system.diagnostics>
</configuration>

Now we can see that our middleware’s logs will only be written into traces-MyCustomMiddleware.log file:

image

I think this post should give you a pretty good understanding of how Katana’s logging infrastructure has been set up and works under the hood. I plain on writing a few more posts on this logging infrastructure. So, stay tuned. Also, as I get used to the concepts of OWIN, I started to think that this logging infrastructure is more fitting as an OWIN extension (just like the SendFile extension) rather than being tied to Katana.

The sample I used here also available on GitHub inside my OwinSamples repository.