Browsed by
Category: WebAPI

Web API – message handlers – usage

Web API – message handlers – usage

message handlers

The delegating handler in WebAPI is the most basic mechanism to intercept HTTP message lifecycle. We can find a very clear and useful visualisation of on WebAPI poster. We can see that message handlers is the first place in HTTP request processing which is able to read of modify the message. It is many cases when we would need to place some code before request will be executed and after. But first, we will introduce how to write that kind of handlers.

Create your own Web API delegating handler

There are two kind of that handlers:

  • global
  • route scoped

On each case we have to define our handler that extend DelegatingHandler class. If we would like to write custom behaviour to whole message processing we can extend HttpMessageHandler class and override method SendAsync. In our case we want to just intercept standard message processing.

public class CustomHandler : DelegatingHandler
{
    protected async override Task<HttpResponseMessage> SendAsync(
        HttpRequestMessage request, CancellationToken cancellationToken)
    {
        Debug.WriteLine("Process request");
        // Call the inner handler.
        var response = await base.SendAsync(request, cancellationToken);
        Debug.WriteLine("Process response");
        return response;
    }
}

As you can see in this code, we have a place for actions before request processing and after. We can operate although on request and response. We can also completely modify the behaviour of WebAPI. In our example we just write some messages to log during the processing. I want to warn you in this point that we should be fully aware of performance issues in that kind of handlers. All actions which we decide to put into handler, will be executed for each request. That’s why we shouldn’t write too complex processing here. We shouldn’t but we can if we have a good reason.

Global

Message handlers can be configured as a global, which means that they will be executed for each action in system.

GlobalConfiguration.Configuration
    .MessageHandlers
    .Add(new DateObsessedHandler());

Route scoped

Also we can define them specific to route simply by defining additional parameter during route creation.

IHttpRoute route = config.Routes.CreateRoute(
    routeTemplate: "api/MyRoute",
    defaults: new HttpRouteValueDictionary("route"),
    constraints: null,
    dataTokens: null,
    parameters: null,
    handler: new CustomHandler());

config.Routes.Add("MyRoute", route);

This is all options to create a message handler. This is really simple, but powerful mechanism.

Usage

In this part we will see some examples how we can use message handlers for common problems. Below I show only a sample implementation of this examples. Further details can be found in related blog posts.

Message logging

The most basic use of message handlers is logging information about requests.

Description: http://weblogs.asp.net/fredriknormen/log-message-request-and-response-in-asp-net-webapi

public class MessageLoggingHandler : MessageHandler
{
    protected override async Task IncommingMessageAsync(string correlationId, string requestInfo, byte[] message)
    {
        await Task.Run(() =>
            Debug.WriteLine(string.Format("{0} - Request: {1}rn{2}", correlationId, requestInfo, Encoding.UTF8.GetString(message))));
    }


    protected override async Task OutgoingMessageAsync(string correlationId, string requestInfo, byte[] message)
    {
        await Task.Run(() =>
            Debug.WriteLine(string.Format("{0} - Response: {1}rn{2}", correlationId, requestInfo, Encoding.UTF8.GetString(message))));
    }
}

Authentication

Other type of handlers can be a part of authentication functionality. We can check Autorization header globally for all requests.

Description: https://weblog.west-wind.com/posts/2013/Apr/30/A-WebAPI-Basic-Authentication-MessageHandler

public class BasicAuthenticationHandler : DelegatingHandler
{
    private const string WWWAuthenticateHeader = "WWW-Authenticate";

    protected override Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, 
    CancellationToken cancellationToken)
    {
        var credentials = ParseAuthorizationHeader(request);

        if (credentials != null)
        {
            var identity = new BasicAuthenticationIdentity(credentials.Name, credentials.Password);
            var principal = new GenericPrincipal(identity, null);

            Thread.CurrentPrincipal = principal;
            if (HttpContext.Current != null)
                HttpContext.Current.User = principal;
        }

        return base.SendAsync(request, cancellationToken)
            .ContinueWith(task =>
            {
                var response = task.Result;
                if (credentials == null && response.StatusCode == HttpStatusCode.Unauthorized)
                Challenge(request, response);

                return response;
            });
    }

    /// <summary>
    /// Parses the Authorization header and creates user credentials
    /// </summary>
    /// <param name="actionContext"></param>
    protected virtual BasicAuthenticationIdentity ParseAuthorizationHeader(HttpRequestMessage request)
    {
        string authHeader = null;
        var auth = request.Headers.Authorization;
        if (auth != null && auth.Scheme == "Basic")
            authHeader = auth.Parameter;

        if (string.IsNullOrEmpty(authHeader))
            return null;

        authHeader = Encoding.Default.GetString(Convert.FromBase64String(authHeader));

        var tokens = authHeader.Split(':');
        if (tokens.Length < 2)
            return null;

        return new BasicAuthenticationIdentity(tokens[0], tokens[1]);
    }


    /// <summary>
    /// Send the Authentication Challenge request
    /// </summary>
    /// <param name="message"></param>
    /// <param name="actionContext"></param>
    void Challenge(HttpRequestMessage request, HttpResponseMessage response)
    {
        var host = request.RequestUri.DnsSafeHost;                    
        response.Headers.Add(WWWAuthenticateHeader, string.Format("Basic realm="{0}"", host));
    }
}

Checking API keys

Sometimes when we publish API as a public service, it can be useful to add API key functionality to restrict access to our API.

Description: http://www.asp.net/web-api/overview/advanced/http-message-handlers

public class ApiKeyHandler : DelegatingHandler
{
    public string Key { get; set; }

    public ApiKeyHandler(string key)
    {
        this.Key = key;
    }

    protected override Task<HttpResponseMessage> SendAsync(
        HttpRequestMessage request, CancellationToken cancellationToken)
    {
        if (!ValidateKey(request))
        {
            var response = new HttpResponseMessage(HttpStatusCode.Forbidden);
            var tsc = new TaskCompletionSource<HttpResponseMessage>();
            tsc.SetResult(response);    
            return tsc.Task;
        }
        return base.SendAsync(request, cancellationToken);
    }

    private bool ValidateKey(HttpRequestMessage message)
    {
        var query = message.RequestUri.ParseQueryString();
        string key = query["key"];
        return (key == Key);
    }
}

Requests rate limiting

Other functionality of API that suits well to message handlers is rate limiting. It is useful to protect API from to many requests made by single users, which may cause system delays.

Description: http://blog.maartenballiauw.be/post/2013/05/28/Throttling-ASPNET-Web-API-calls.aspx

public class ThrottlingHandler
    : DelegatingHandler
{
    protected override Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
    {
        var identifier = request.GetClientIpAddress();

        long currentRequests = 1;
        long maxRequestsPerHour = 60;

        if (HttpContext.Current.Cache[string.Format("throttling_{0}", identifier)] != null)
        {
            currentRequests = (long)System.Web.HttpContext.Current.Cache[string.Format("throttling_{0}", identifier)] + 1;
            HttpContext.Current.Cache[string.Format("throttling_{0}", identifier)] = currentRequests;
        }
        else
        {
            HttpContext.Current.Cache.Add(string.Format("throttling_{0}", identifier), currentRequests,
                null, Cache.NoAbsoluteExpiration, TimeSpan.FromHours(1),
                CacheItemPriority.Low, null);
        }

        Task<HttpResponseMessage> response = null;
        if (currentRequests > maxRequestsPerHour)
        {
            response = CreateResponse(request, HttpStatusCode.Conflict, "You are being throttled.");
        }
        else
        {
            response = base.SendAsync(request, cancellationToken);
        }

        return response;
    }

    protected Task<HttpResponseMessage> CreateResponse(HttpRequestMessage request, HttpStatusCode statusCode, string message)
    {
        var tsc = new TaskCompletionSource<HttpResponseMessage>();
        var response = request.CreateResponse(statusCode);
        response.ReasonPhrase = message;
        response.Content = new StringContent(message);
        tsc.SetResult(response);
        return tsc.Task;
    }
}

https://github.com/stefanprodan/WebApiThrottle

Additional resources:

http://www.strathweb.com/2012/05/implementing-message-handlers-to-track-your-asp-net-web-api-usage/
http://blog.karbyn.com/index.php/message-handlers-in-web-api/

Photo source

OWASP AppSensor – opis mechanizmu

OWASP AppSensor – opis mechanizmu

owasp-opis
Źródło

OWASP

OWASP (Open Web Application Security Project) jest to organizacja non-profit mająca na celu analizę zagadnień związanych z bezpieczeństwem aplikacji i opracowywanie rozwiązań, które byłyby użyteczne zarówno dla organizacji jak i dla pojedynczych twórców oprogramowania. Organizacja ta działa otwarcie, a wszystkie materiały są darmowe i ogólnodostępne. Nie jest ona także związna z żadnym dostawcą oprogramowania i pozwala na niezależność w tym zakresie. Jej główną siłą napędową jest społeczność ludzi zainteresowanych tematem bezpieczeństwa. Tworzą oni narzędzia wspomagające zabezpieczanie i testowanie zabezpieczeń systemów komputerowych. Znane projekty to:

  • OWASP Zed Attack Proxy – narzędzie do automatycznego testowania bezpieczeństwa systemów
  • OWASP Top Ten – lista największych zagrożeń dla aplikacji webowych wraz z metodami zapobiegania im
  • OWASP OWTF – narzędzie do automatyzacji testów penetracyjnych

Wybór AppSensor

Od pewnego czasu interesuję się tematem bezpieczeństwa i dzięki temu trafiłem na ogranizację OWASP. Jest ona nastawiona w dość dużym stopniu na środowisko Java. Tak więc pisząc w .NET mogę korzystać z gotowych narzędzi OWASP, zaś korzystanie z bibliotek jest już dla mnie niemożliwe. Istnieją czasem porty bibliotek na platformę .NET, jednak nie są one dopasowane do tego środowiska. Projekt AppSensor zainteresował mnie swoją prostotą i możliwościami, które są z nim związane. Dlatego też postanowiłem zająć się jego implementacją dopasowaną dla webowych mechanizmów .NET w szczególności do WebAPI, ale z możliwością samodzielnego dostosowania do dowolnego frameworku np. Nancy czy ServiceStack.

AppSensor

Pomysł biblioteki AppSensor polega na tym, że w środowisku produkcyjnym aplikacja cały czas przetwarza różnego rodzaju informacje i akcje. Dzięki umieszczeniu wewnątrz kodu aplikacji odpowiednich punktów detekcji podejrzanych zachowań, jesteśmy w stanie na bieżąco, niemalże w czasie rzeczywistym, reagować na pojawiające się zagrożenia. W klasycznym podejściu ataki wykrywamy i blokujemy głównie na wejściu do aplikacji. Umożliwia nam to wykrycie podejrzanych zachowań takich jak wysłanie niepoprawnych ciągów znaków w requestach HTTP, co może sygnalizować próbę użycia np. SQL Injection. Nie biorą one jednak pod uwagę kontekstu w jakim wykonywane są te zapytania. System wykrywania, znając kontekst biznesowy konkretnego zapytania, jest w stanie wykryć więcej nadużyć, z większą trafnością. Jest na przykład w stanie wykryć zbyt częste próby logowania do aplikacji lub próby naruszenia uprawnień.

Podsumowując możemy powiedzieć, że AppSensor obejmuje wykrywanie nadużyć i reagowanie na zagrożenia:

  • zależne od kontekstu,
  • specyficzne dla danej aplikacji,
  • dynamicznie,
  • reagując w czasie rzeczywistym,
  • z możliwością adaptacji zasad reakcji.

Zasada działania

Cały mechanizm jest zasadniczo prosty w użyciu i składa się z trzech kroków:

  1. Detekcja – rozpoznanie nietypowego zachowania.
  2. Identyfikacja próby ataku – ocenienie czy wykryte zachowanie jest poza zakresem dopuszczalnych akcji użytkownika za pomocą interfejsu lub API.
  3. Reakcja – zastosowanie środka informującego lub zapobiegającego dalszym nadużyciom.

Identyfikacja ataku bazuje na prostym założeniu, że w większości aplikacji użytkowych, zbiór akcji dopuszczalnych jest rozłączny ze zbiorem akcji będących zagrożniem.

application_use

Kluczem jest więc odpowiednie odseparowanie tych 2 rodzajów zachowań. W podstawowej wersji tego mechanizmu można skorzystać z gotowego zbioru najpopularniejszych punktów detekcji zagrozeń https://www.owasp.org/index.php/AppSensor_DetectionPoints.

Bardzo dobrze przemówiło do mnie porównanie aplikacji z systemem wczesnego wykrywania zagrożeń i bez niego do banku zabezpieczonego i zarządzanego jak niektóre projekty informatyczne.

bank_safe
bank_unsafe

Źródło: https://www.owasp.org/images/0/06/Defend_Yourself-Integrating_Real_Time_Defenses_into_Online_Applications-Michael_Coates.pdf

Change tracking optimization in Entity Framework

Change tracking optimization in Entity Framework

Entity Framework change tracking optimization

EF_readonly_optimization change tracking metaphor

Data changes tracking

Entity Framework gives us an opportunity to work with data without bothering to notify database about entity changes made during data processing. EF provides few methods of tracking this modifications.

Snapshot change tracking

By default in Entity Framework it is enabled snapshot change tracking mechanism. It works by saving entity states each time when it is loaded from the database. When for example SaveChanges method is called, Entity Framework scans all entities in current context and compares them with saved state.

Entity Framework performs Detect Changes automatically when the following methods are called:

  • DbSet.Find
  • DbSet.Local
  • DbSet.Remove
  • DbSet.Add
  • DbSet.Attach
  • DbContext.SaveChanges
  • DbContext.GetValidationErrors
  • DbContext.Entry
  • DbChangeTracker.Entries

Change tracking mechanism is wider described in following resources:

https://msdn.microsoft.com/en-us/data/jj556205.aspx

http://blog.oneunicorn.com/2012/03/10/secrets-of-detectchanges-part-1-what-does-detectchanges-do/

http://blog.oneunicorn.com/2012/03/11/secrets-of-detectchanges-part-2-when-is-detectchanges-called-automatically/

http://blog.oneunicorn.com/2012/03/12/secrets-of-detectchanges-part-3-switching-off-automatic-detectchanges/

http://blog.oneunicorn.com/2012/03/13/secrets-of-detectchanges-part-4-binary-properties-and-complex-types/

Disabling AutoDetectChanges

If property AutoDetectChangesEnabled in DbContext.Configuration is set, EF runs change tracking according to rules described above. So you can disable it by setting this property to false. But you should remember to reset this value to previous one after making your special operations. In general case this mechanism is very useful, especially to modify multiple objects. However, it is not necessary for read-only queries or bulk insert operations.

Rules to safe disabling

Each time when you want to disable it, you should check if your code operating on data meets two following conditions:

  1. Method from EF cannot change the EF context that DetectChanges method needs to be called.
  2. Each time when entities was changed outside of EF methods and this modifications should be saved, DetectCanges method should be executed manually.

For more information

https://ilmatte.wordpress.com/2013/01/01/entity-framework-code-first-always-disable-autodetectchanges-when-importing-data/

http://rpajak.com/2012/04/

Disable tracking for read-only

The first possibility to optimize time of database queries is to disable this tracking during read-only operations. In case on WebAPI it could means, that each action that doesn’t modify any data need not change tracking mechanism. It leads us to create custom attribute to disable this mechanism per HTTP request.

In the example below I use static IoC.Container to get instance of DbContext. You also have to remember to configure Dependency Injector container to create context in request scope, not in transient scope.

In my experiment on working system, method is executed around 20% faster than before adding this attribute to them.

	public class ReadonlyAttribute : ActionFilterAttribute
	{
	    private IDbContext _dbContext;
	
	    public override void OnActionExecuting(HttpActionContext actionContext)
	    {
	        _dbContext = IoC.Container.GetInstance<IDbContext>();
	        _dbContext.Configuration.AutoDetectChangesEnabled = false;
	        base.OnActionExecuting(actionContext);
	    }
	    public override void OnActionExecuted(HttpActionExecutedContext actionExecutedContext)
	    {
	        _dbContext = IoC.Container.GetInstance<IDbContext>();
	        _dbContext.Configuration.AutoDetectChangesEnabled = true;
	        base.OnActionExecuted(actionExecutedContext);
	    }
	
	}

You can use it just by adding this attribute to chosen methods:

    [HttpGet]
    [Readonly]
    public IQueryable<PostDto> Get()
    {
        return _postRepository.All();
    }

Bulk inserts

The other case when you could want to disable tracking is bulk inserts. In this case it is not necessary to disable this functionality for whole action, but you can do it for particular operation:

    using (var context = new DbContext())
    {
        try
        {
            context.Configuration.AutoDetectChangesEnabled = false;
            posts.ForEach(p => context.Posts.Add(p));
        }
        finally
        {
            context.Configuration.AutoDetectChangesEnabled = true;
        }
        context.SaveChanges();
    }

Additionally, I really like the solution described in post http://joshgallagher.info/2014/06/14/entity-framework-performance-tip-for-creating-entities/

It enables to encapsulate this operation in single and simple object:

	public sealed class NoChangeTracking : IDisposable
	{
	    private readonly DbContext _dbContext;
	    private readonly bool _initialAutoDetectChangesValue;
	
	    public NoChangeTracking(DbContext dbContext)
	    {
	        if (dbContext == null) throw new ArgumentNullException("dbContext");
	        _dbContext = dbContext;
	
	        _initialAutoDetectChangesValue = dbContext.Configuration.AutoDetectChangesEnabled;
	
	        SetChangeDetection(false);
	    }
	
	    public void Dispose()
	    {
	        SetChangeDetection(_initialAutoDetectChangesValue);
	    }
	
	    private void SetChangeDetection(bool setting)
	    {
	    	_dbContext.Configuration.AutoDetectChangesEnabled = setting;
	    }
	
	}

Then you can use this object simply

	using(new NoChangeTracking(context))
	{
		context.Posts.Add(new Post());
	}

Disabling for single entity

In some cases you can disable tracking for single entity source. It could be useful for dictionaries types which are not changed in normal code.

	var entities = context.PostCategory.AsNoTracking();

http://gasior.net.pl/quick-tip-12-stosuj-asnotracking-gdzie-sie-da/

http://blog.staticvoid.co.nz/2012/4/2/entity_framework_and_asnotracking

Handling images WebAPI (+examples)

Handling images WebAPI (+examples)

WebAPI is used to serve data from server. One time of data could be images. In this post I will show how to write WebAPI methods operating on images data.

So at first we must decide where we want to store our images. Let’s suppose, that we have database that store our application data. We have 2 possibilities: to store images in files or directly in database. This two approaches has following adventages:

  • files
    • easy to test
    • fast
    • requires additional backup strategy
  • database
    • scalable
    • could be joined with db objects (no name conflicts)
    • transactional integrity
    • easy to apply permissions

Based on this points, you can see that storing images in database could be preferable when image is part of entities in your system i.e. photo of product. So we focus on this type of storing. You can easily change.

Handling images

GET

[HttpGet]
[Route("{id}/image")]
public HttpResponseMessage GetImage(int id)
{
	var image = _service.GetImage(id);

	MemoryStream ms = new MemoryStream();
	HttpResponseMessage result = new HttpResponseMessage(HttpStatusCode.OK);
	result.Content = new ByteArrayContent(image);
	result.Content.Headers.ContentType = new MediaTypeHeaderValue("image/png");
	return result;
}

As you can see, there is a MemoryStream, that returns image data to response. Also we must add ContentType header to define type of our image.

DELETE

This method is simple and trival

[HttpDelete]
[Route("{id}/image")]
public void DeleteImage(int id)
{
	_service.Remove(id);
}

POST

This part is a bit tricky at first you need to read multipart stream and then join each of part to one byte array. This array contains final image data that could be saved to database.

[HttpPost]
[Route("{id}/image")]
public async Task<HttpResponseMessage> PostImage(int id)
{
	if (Request.Content.IsMimeMultipartContent())
	{
		string fullPath = HttpContext.Current.Server.MapPath("~/uploads");
		MultipartFormDataStreamProvider streamProvider = new MultipartFormDataStreamProvider(fullPath);
		var task = await Request.Content.ReadAsMultipartAsync(streamProvider).ContinueWith(t =>
		{
			if (t.IsFaulted || t.IsCanceled)
				throw new HttpResponseException(HttpStatusCode.InternalServerError);
			MemoryStream ms = new MemoryStream();
			foreach (var file in streamProvider.FileData)
			{
				var info = new FileInfo(file.LocalFileName);
				var data = File.ReadAllBytes(info.FullName);
				ms.Write(data, 0, data.Length);
			}
			_service.UpdateImage(ms.ToArray());
			return new HttpResponseMessage(HttpStatusCode.OK);
		});
		return task;
	}
	else
	{
		throw new HttpResponseException(Request.CreateResponse(HttpStatusCode.NotAcceptable, "Invalid Request!"));
	}
}