Sunday, August 11, 2013

Problems with WSDL of WCF web services behind load balancer

If you have a WCF web service, you can get its WSDL by appending ?wsdl to the URL:
http://server/web/Service.svc?wsdl
Typically, the generated WSDL is not complete. The types are loaded separately from the server:
<xsd:import schemaLocation="http://server/web/Service.svc?xsd=xsd0" />
For the type import, the current machine is used. Normally this isn't a problem. But if you use a load balancer, you end up with the following requests:
http://loadbalancer/web/Service.svc?wsdl

<xsd:import schemaLocation="http://node1/web/Service.svc?xsd=xsd0" />
This will not work, when node1 is not accessible directly.
Fortunately, you can force WCF to use the Loadbalancer also in the WSDL. You only have to add one line to the serviceBehavior in the Web.config:
<behaviors>
<serviceBehaviors>
<behavior name="MyBehavior">
<useRequestHeadersForMetadataAddress />
...
</behavior>
</serviceBehaviors>
</behaviors>

.net programs on 32 and/or 64 bit machines

Generally, a .net program can run on a 32 bit machine as well as on a 64 bit machine. But sometimes it is necessary to run the program also on a 64 bit machine in 32 bit mode, the so-called WoW64.
WoW64 stands for "Windows on 64-bit Windows", and it contains all the 32-bit binary files required for compatibility, which run on top of the 64 bit Windows. So, yeah, it looks like a double copy of everything in System32 (which despite the directory name, are actually 64-bit binaries).
You will need WoW64 par example, if you want to call 32 bit ActiveX components. Visual Studio provides for this purpose the so-called platform target:
  • x86
    32 bit application, runs either on Win32 or on Win64 in WoW64
  • x64
    64 bit application, runs only on Win64 (not in WoW64)
  • Any CPU
    runs on Win32 as 32 bit application and on Win64 as 64 bit application
This info will be stored in the PE header. At application startup, Windows checks the settings and starts the application in the appropriate mode (or not). If you want to check later, for which platform the application was built, you can use the corflags tool in Visual Studio Command Prompt:
> corflags MyApp.exe
Microsoft (R) .NET Framework CorFlags Conversion Tool.  Version  4.0.30319.1
Copyright (c) Microsoft Corporation.  All rights reserved.

Version   : v4.0.30319
CLR Header: 2.5
PE        : PE32
CorFlags  : 11
ILONLY    : 1
32BIT     : 1
Signed    : 1

The interesting parts are PE and 32BIT. The values are a little bit strange and hard to remember:

Platform targetPE32BIT
x86PE321
x64PE32+0
Any CPUPE320

References

Saturday, August 10, 2013

Problem with asynchronous HttpClient methods

Recently, I wrote a client application which should send some log messages to a server. Since it was only for statistics, the log didn’t have the highest requirements on reliability. Additionally, it shouldn’t block my application. Therefore I decided to send the message asynchronously, and in case of an error only to write something to the local log file. I came up with

static void LogMessage(string message)
{
  Uri baseAddress = new Uri("http://localhost/");
  string requestUri = "uri";

  using (HttpClient client = new HttpClient { BaseAddress = baseAddress })
  {
    client.PostAsJsonAsync(requestUri, message, cancellationToken).ContinueWith(task =>
      {
        if (task.IsFaulted)
          Console.WriteLine(“Failed: “ + task.Exception);
        else if (task.IsCanceled)
          Console.WriteLine ("Canceled");
        else
        {
          HttpResponseMessage response = task.Result;
          if (response.IsSuccessStatusCode)
            Console.WriteLine("Succeeded");
          else
            response.Content.ReadAsStringAsync().ContinueWith(task2 => Console.WriteLine(“Failed with status " + response.StatusCode));
        }
      });
  }
}

This code should send the message asynchronously (PostAsJsonAsync). And afterwards it should check, if the sending was successfully or not (ContinueWith). My expectation was to see an error, since there is nothing is listening on the specified address. But I didn’t see anything. Moreover, it even didn’t send any requests. Even for my requirements, this was not enough.

After some research, I added trace switches to my config file:

<system.diagnostics>
  <switches>
    <add name="System.Net" value="Verbose"/>
    <add name="System.Net.Http" value="Verbose"/>
    <add name="System.Net.HttpListener" value="Verbose"/>
    <add name="System.Net.Sockets" value="Verbose"/>
    <add name="System.Net.Cache" value="Verbose"/>
  </switches>
</system.diagnostics>

With it, one of the last lines of my debug output was

System.Net Error: 0 : [0864] Exception in HttpWebRequest#54246671:: - The request was aborted: The request was canceled..

This led me to the evil: it was the disposing of HttpClient too early. At the end of the using block the client will be disposed. But at this time, the message has not been sent. This happens, since I do something asynchronous inside the using block without waiting for its end.

The solution to this problem is to reverse the order of using and asynchronous: if I start the using in an asynchronous way, everything is fine:

static void LogMessage(string message)
{
  Uri baseAddress = new Uri("http://localhost/");
  string requestUri = "uri";

  new TaskFactory().StartNew(() =>
    {
      using (HttpClient client = new HttpClient { BaseAddress = baseAddress })
      {
        Console.WriteLine("Sending message");
        HttpResponseMessage response = client.PostAsJsonAsync(requestUri, message).Result;

        Console.WriteLine("Evaluating response");
        if (response.IsSuccessStatusCode)
          Console.WriteLine("Succeeded");
        else
          Console.WriteLine("Failed with status " + response.StatusCode);
      }
    });
}

Here I start a new task, which does inside the using / Dispose stuff. And the task is finished only after the Dispose.

This took me to the next stage: what about the async / await pattern from .net 4.5? The implementation is quite similar to the one above - the main difference is that the method now has to return a Task.

static async Task LogMessage (string message)
{
  Uri baseAddress = new Uri("http://localhost/");
  string requestUri = "uri";

  using (HttpClient client = new HttpClient { BaseAddress = baseAddress })
  {
    Console.WriteLine("Sending message");
    HttpResponseMessage response = await client.PostAsJsonAsync(requestUri, message);

    Console.WriteLine("Evaluating response");
    if (response.IsSuccessStatusCode)
      Console.WriteLine("Succeeded");
    else
      Console.WriteLine("Failed with status " + response.StatusCode);
  }
}

Also this implementation starts a new thread for the response message handling. But it does it only if really needed. And it does it as late as possible.

A little more sophisticated logging shows some details (the 2nd column is the thread number). The implementation with an explicit Task calls LogMessage on the main thread 9. Afterwards in continues immediately on the same thread. Approx. 20 ms later the new thread 12 starts with the HTTP handling:

21:28:42.524    9       CallMethod      Calling LogMessageWithTask
21:28:42.526    9       CallMethod      Continuing after LogMessageWithTask
21:28:42.545    12      LogMessageWithTask      Sending message
21:28:45.682    12      LogMessageWithTask      Evaluating response
21:28:45.682    12      LogMessageWithTask      Failed with status NotFound

With async / await it is a little bit different: also the request will be sent on the main thread. The response handling will be done later also in a second thread:

21:28:51.638    9       CallMethod      Calling LogMessageAsyncAwait
21:28:51.666    9       LogMessageAsyncAwait    Sending message
21:28:51.701    9       CallMethod      Continuing after LogMessageAsyncAwait
21:28:52.144    16      LogMessageAsyncAwait    Evaluating response
21:28:52.144    16      LogMessageAsyncAwait    Failed with status NotFound

However, the coding is more precise. And the request will be sent without the delay for creating the new Task.



You can find the source code at GitHub: https://github.com/Ritzlgrmft/HttpClientDispose.

Thursday, June 20, 2013

AJAX calls to different servers (CORS)

Developing modern HTML applications, sometimes you have the need to send an AJAX request data to another server. Unfortunately, this could be a security issue. Therefore all browsers implement the Same origin policy, which prevents such calls.
But what, if you really need it? The rescue is Cross-origin resource sharing (CORS). The principle is quite easy: the browser sends with the AJAX request an additional HTTP header:
Origin: http://www.foo.com
The server can analyze the header. If it decides to fulfill the request, it adds another header:
Access-Control-Allow-Origin: http://www.foo.com
If the server decides to trust all clients, it can also return
Access-Control-Allow-Origin: *
But if you send the AJAX request from a page with origin http://www.foo.com, while the server replies with http://www.bar.com, you cannot read the response.

Browser support

All major browsers support CORS with XMLHttpRequest. All but IE8 or IE9. These IE versions use a slightly different approach. When you want CORS, you have to use another object instead: XDomainRequest. Fortunately at least its interface is similar to XMLHttpRequest.

JQuery

JQuery uses for AJAX calls always XMLHttpRequest. The request to use XDomainRequest when needed was rejected. Instead it is recommended to use a JQuery plugin. For this purpose I found 2 implementations:
Both worked for me, but with jQuery.iecors I had problems receiving errors. At least in my tests the error function was not called. Therefore I finally decided to use jQuery.XDomainRequest.

CORS and ASP.NET Web Api

For supporting CORS with ASP.NET Web Api, you need to add the Access-Control-Allow-Origin header to the response. One way is to implement a custom message handler for this purpose. Carlos Figueira blogged a good description, how to do this.

Tuesday, June 18, 2013

OWIN with static files, exception handling and logging

As I wrote already in Self-host ASP.NET Web API and SignalR together, the OWIN configuration is done in Startup.Configuration():
public void Configuration(IAppBuilder app)
{
  // Configure WebApi
  var config = new HttpConfiguration();
  config.Routes.MapHttpRoute("API Default", "api/{controller}/{id}", new { id = RouteParameter.Optional });
  app.UseWebApi(config);

  // Configure SignalR
  app.MapHubs();
}
It looks, as if there are some features get activated with app.UseXXX() (or app.MapHubs(), respectively). But this is not completely true. When we look into the implementation of these "feature activating methods", they finally call
public IAppBuilder Use(object middleware, params object[] args)
{
  this._middleware.Add(AppBuilder.ToMiddlewareFactory(middleware, args));
  return (IAppBuilder) this;
}
This method adds the features to a list. During the request processing, every feature in this list will be checked, if it can handle the request. Therefore the order of the "feature activating methods" can be important. Not in the example above, since WebApi and SignalR do not compete for the same requests.

Microsoft.Owin.Diagnostics

The package Microsoft.Owin.Diagnostics contains one useful feature catching and displaying exceptions. For sure this shouldn't happen, but when it would be nice to know about. But as always you should consider to use it only during development.

To switch it on, simply add the following line at the beginning of Startup.Configuration():
app.UseShowExceptions(); 

Another feature in Microsoft.Owin.Diagnostics is
app.UseTestPage(); 
This displays the message Welcome to Katana to the client. You should place it at the very end of Configuration.Startup(). Otherwise your other features wouldn't never reached.

But anyway, this feature is useful only in hello world status. Later I would prefer to get HTTP 404 instead of this message.

Microsoft.Owin.StaticFiles

Like most of the other OWIN packages, also Microsoft.Owin.StaticFiles is in prerelease status. But this package is special, you even cannot find it in NuGet. To install it, you need to enter the following command in the Package Manager Console:
Install-Package Microsoft.Owin.StaticFiles -Version 0.20-alpha-20220-88 -Pre 
But probably the package is hidden since it doesn't work really. It has problems when you request more than one file in parallel. The solution is to use one of the nightly Katana builds (e.g. 0.24.0-pre-20624-416). Probably this is even more alpha than the hidden version. But is works better. Obviously there was some improvement between version 0.20 and 0.24.
You can get the nightly builds from a separate feed: http://www.myget.org/f/Katana.
After adding the package, just add one line to Startup.Configuration():
app.UseStaticFiles("StaticFiles");
The parameter specifies, in which directory the static files will be searched. Since I named it StaticFiles, you have to add a folder with this name to your project. And for every file you add to this folder you have to set in its properties that it will be copied to the output directory:

When you start the project, you can fire up the browser and enter http://localhost:8080/test.htm (without specifying the StaticFiles folder), and you get simply the page back.

Logging OWIN requests

Sometimes it would be interesting to see the incoming requests in a trace. This can be achieved with a custom feature. The constructor is quite simple. It just stores the reference to the next feature:
private readonly Func<IDictionary<string, object>, Task> _next; 

public Logger(Func<IDictionary<string, object>, Task> next)
{
  if (next == null)
    throw new ArgumentNullException("next");
  _next = next;
}
The implementation isn't really complicated, too:
public Task Invoke(IDictionary<string, object> environment)
{
  string method = GetValueFromEnvironment(environment, OwinConstants.RequestMethod);
  string path = GetValueFromEnvironment(environment, OwinConstants.RequestPath);

  Console.WriteLine("Entry\t{0}\t{1}", method, path);

  Stopwatch stopWatch = Stopwatch.StartNew();
  return _next(environment).ContinueWith(t =>
  {
    Console.WriteLine("Exit\t{0}\t{1}\t{2}\t{3}\t{4}", method, path, stopWatch.ElapsedMilliseconds,
      GetValueFromEnvironment(environment, OwinConstants.ResponseStatusCode), 
      GetValueFromEnvironment(environment, OwinConstants.ResponseReasonPhrase));
    return t;
  });
}
First, it prints some data of the current request. The more interesting part is, that it then calls the succeeding features (return _next(environment)). And when the succeeding features were evaluated, it finally (ContinueWith) prints some response data. I added here also method and path, since otherwise it were difficult to find the corresponding entries. Maybe it would be even better to use some kind of unique id for this purpose. But in my projects, method and path are enough.
GetValueFromEnvironment is only a little helper, since in some cases the environment dictionary does not contains all values:
private static string GetValueFromEnvironment(IDictionary<string, object> environment, string key)
{
  object value;
  environment.TryGetValue(key, out value);
  return Convert.ToString(value, CultureInfo.InvariantCulture);
}
Since the Logger traces the beginning and the end of the processing, it should be activated quite at the beginning of Startup.Configuration():
app.Use(typeof(Logger));

Logging the Request Body

With POST requests, it can be very handy to log also the request body. For this, you have only to extend the Invoke method a little bit:
string requestBody;
Stream stream = (Stream)environment[OwinConstants.RequestBody];
using (StreamReader sr = new StreamReader(stream))
{
  requestBody = sr.ReadToEnd();
}
environment[OwinConstants.RequestBody] = new MemoryStream(Encoding.UTF8.GetBytes(requestBody));
The access to the request body is provided by a Stream. The only caveat with this stream is that it is not seekable. That means you can read it only once. And maybe the simple logging will not be enough for your requirements. Sometimes you will also process it afterwards...
Fortunately, this is no big issue: just replace the old stream with a new MemoryStream. For sure, this is not good idea with big request bodies. In such a case you will need a more sophisticated solution. But normally, it should be good enough. Moreover, you can use it only during development. In production you can disable it by configuration, par example.

Logging SignalR

With the Logger from above, you get a nice logging of the several SignalR requests (when it uses LongPolling instead of WebSockets):
10:51:18,181     11     Entry     GET     /signalr/negotiate
10:51:18,263     8      Exit      GET     /signalr/negotiate     82              
10:51:18,271     11     Entry     GET     /signalr/ping
10:51:18,275     8      Exit      GET     /signalr/ping          4               
10:51:18,540     11     Entry     GET     /signalr/connect
10:53:08,806     14     Exit      GET     /signalr/connect       110260          
10:53:08,826     11     Entry     GET     /signalr/poll
10:54:58,960     15     Exit      GET     /signalr/poll          110128          
10:54:58,969     11     Entry     GET     /signalr/poll
10:56:49,136     12     Exit      GET     /signalr/poll          110161          
The used Logger prints also timestamp and thread id (in the first 2 columns). It is easily to see, how SignalR waits for 110 seconds for an answer from the server (a notification). When it doesn't get one, it simply sends the next request.
You can also see that all requests (at least in this example) are handled by the same thread (11). But the response is created by other threads (8, 14, 15 and 12).

Summary

It is quite easy to add additional features to an OWIN host. And it is also not too hard to implement own features. Drawbacks of the whole stuff are (hopefully only at the moment):
  • the beta status of some packages
  • the lack of documentation
But for a real developer, the best documentation is the code, anyway.

You can find the source code at GitHub: https://github.com/Ritzlgrmft/OwinConfiguration.

Wednesday, May 29, 2013

Garbage Collector's sabotage of my message pump

I spent my last days with a boring problem. I implemented a Windows service listening to WM_DEVICECHANGE messages (which will be sent par example, when you plug-in a new USB device). For this I implemented a NativeWindow, whose only purpose was to implement the window procedure for some custom message processing:

internal sealed class DeviceChangeHandler : NativeWindow
{
  private const int WM_DEVICECHANGE = 0x219;
  private const int DBT_DEVNODES_CHANGED = 0x7;

  public DeviceChangeHandler()
  {
    CreateHandle(new CreateParams());
  }

  protected override void WndProc(ref Message msg)
  {
    if (msg.Msg == WM_DEVICECHANGE && 
        msg.WParam.ToInt32() == DBT_DEVNODES_CHANGED)
    {
      // device change detected
      ...
    }

    base.WndProc(ref msg);
  }
}

In my service itself, I started a new Thread with the following ThreadStart:

private void RunDeviceChangeHandler()
{
  // create window
  DeviceChangeHandler deviceChangeHandler = new DeviceChangeHandler();

  // run message pump
  Application.Run();
}  

When I debugged the code, it worked always. Also my release build worked - but unfortunately not always. Sometimes I got the messages, sometimes not. Testing was tedious, since Windows takes some time until sending the message, especially after disconnecting the device.

Finally I remembered a tool I used 10 years ago: Spy++, which is still part of Visual Studio. I saw that, when I didn't get messages, there was also no window. That explained to me, why I didn't get the messages. But the question was now: why was there no window?

Next I defined a caption for my window (via the CreateParams). And I implemented a loop in which I call every 5 seconds FindWindow with the just specified caption. Now I saw that FindWindow was able to find my window. But only for some iterations, sooner or later it returned only 0.

By chance, I added tracing to the finalizer of my NativeWindow implementation. Now I saw that the finalizer was called some time, and afterwards FindWindow returned only 0. My first idea was that there was an exception in Application.Run(). But this was to easy.

The problem is optimization done by the just-in-time compiler. Since the variable deviceChangeHandler is no longer referenced after Application.Run(), it is a candidate for garbage collection. This I could verify by a call of GC.Collect(). Afterwards the windows was always gone.

The solution is to make the variable ineligible for garbage collection. For this purpose exists the method GC.KeepAlive().
With my modified ThreadStart, everything worked as expected:

private void RunDeviceChangeHandler()
{
  // create window
  DeviceChangeHandler deviceChangeHandler = new DeviceChangeHandler();

  // run message pump
  Application.Run();

  // make deviceChangeHandler ineligible for garbage collection
  GC.KeepAlive(deviceChangeHandler);
}  
 
As with the most complex problems, the solution is only one line. The art is to insert it at the right place.

Wednesday, May 15, 2013

Pitfall with ASP.NET Web Api, OWIN and FxCop


As I wrote in Self-host ASP.NET Web API and SignalR together, you can easily combine ASP.NET Web Api and OWIN. My Startup class contained at least the Configuration method:
public void Configuration(IAppBuilder app)
{
  HttpConfiguration config = new HttpConfiguration();
  config.Routes.MapHttpRoute("API Default", "api/{controller}/{id}", new { id = RouteParameter.Optional });
  app.UseWebApi(config);
}
With this, everything worked fine.

(Unfortunately) I am a well-behaved man. Therefore I ran the code analysis (aka FxCop). It returned the warning
CA2000: Microsoft.Reliability: In method 'Startup.Configuration(IAppBuilder)', call System.IDisposable.Dispose on object 'config' before all references to it are out of scope.
Understandable for me. And since Microsoft suggests to fix it, I changed my method to:
public void Configuration(IAppBuilder app)
{
  using (HttpConfiguration config = new HttpConfiguration())
  {
    config.Routes.MapHttpRoute("API Default", "api/{controller}/{id}", new { id = RouteParameter.Optional });
    app.UseWebApi(config);
  }
}
Sadly, I didn’t test it immediately. Instead I fixed a lot of other FxCop warnings. And I improved my coding otherwise. Finally I forgot this special fix.

When I finally put all together, I got only HTTP 500 Internal Server Error, without any further information. A lot of people suggest in such a case to debug the server, but also there no exception was thrown. It simply didn’t work. I slimmed down my coding until I removed the using statement. And then it worked again!

So it’s better to keep the original code and to add only a SuppressMessage attribute:
[SuppressMessage("Microsoft.Reliability", "CA2000:Dispose objects before losing scope",
                 Justification = "HttpConfiguration must not be disposed, otherwise web api will not work")]
public void Configuration(IAppBuilder app)
{
  HttpConfiguration config = new HttpConfiguration();
  config.Routes.MapHttpRoute("API Default", "api/{controller}/{id}", new { id = RouteParameter.Optional });
  app.UseWebApi(config);
}