Jump to: navigation, search

Difference between revisions of "OpenStack-SDK-DotNet/DesignDetails"

(Putting it all together)
(HTTP layer)
Line 9: Line 9:
  
 
== HTTP layer ==
 
== HTTP layer ==
The HTTP layer is the lowest level in the API. It provides a simple abstraction for working with the HTTP protocol, and making and receiving requests and responses. While it may be considered redundant, since the .NET framework already offers several ways to do this, HttpClient for example. There are two reason why this layer is important:
+
The HTTP layer is the lowest level in the API. It provides a simple abstraction for making requests and receiving responses over the HTTP protocol. While it may be considered redundant, since the .NET framework already offers several ways to do this, there are two reason why this layer is important:
:*Unit tests can inject HTTP responses (including error states, and timeouts) without needing a live server.
+
:*Unit tests can validate requests and inject HTTP responses (including error states, and timeouts) without needing a remote server.
::By using interfaces for the abstraction, and dependency injection, unit tests can be written that inject specific HTTP responses into code that takes a dependency on this layer. This eliminates the need for an individual developer to have a live server to test against. Since remote calls are not being made, latency is greatly reduced, tests execute quickly, and the developer can continue doing what he/she does best, and not waste time on watching tests run. This does not eliminate the need for integration tests that are intended to be run against a live server. Those tests should, and will still be written, but do not need to be run as a part of a gated check-in, and should be run by the CI system on a periodic basis.  
+
::By using interfaces for the abstraction and dependency injection, unit tests can be written that inject specific HTTP responses into code that takes a dependency on this layer. This eliminates the need for an individual developer to have a remote server to test against. Since remote calls are not being made, latency is greatly reduced, tests execute quickly, and the developer can continue doing what he/she does best. Developers don't have to waste time on watching tests run or waiting for the remote server to be available. This does not eliminate the need for integration tests that are intended to be run against a live server. Those tests should, and will still be written, but do not need to be run as a part of a gated check-in, and should be run by the CI system on a periodic basis.  
 
:*Tight coupling and dependencies are removed.
 
:*Tight coupling and dependencies are removed.
::The .NET framework is always evolving. There are several ways (WebRequest, WebClient, HttpClient, etc) with the .NET framework alone to make a HTTP requests, not to mention third-party libraries. The HTTP layer is _not_ intended to be a replacement for these APIs. It is intended to represent a contract for the API to abstract itself from the implementation. With this layer in place, a developer that takes this layer as a dependency does not need to know or understand how the request is being carried out, if the current framework version supports the functionality required, etc. By using the HTTP layer, if/when the underlying implementation needs to change, none of the code that has taken a dependency will need to change (unless there is a breaking change in the HTTP specification, which is highly unlikely). This gives the API tremendous flexibility and allows the API to easily adapt to cross platform and framework scenarios.  
+
::The .NET framework is always evolving. There are several ways (WebRequest, WebClient, HttpClient, etc) with the .NET framework alone to make an HTTP request, not to mention third-party libraries. The HTTP layer is '''not''' intended to be a replacement for these APIs. It '''is''' intended to represent a contract for the API to abstract itself from the implementation. A developer that takes this layer as a dependency does not need to know or understand how the request is being carried out, if the current framework version supports the functionality required, etc. By using the HTTP layer, if/when the underlying implementation needs to change, none of the code that has taken a dependency will need to change (unless there is a breaking change in the HTTP specification, which is highly unlikely). This gives the API tremendous flexibility and allows the API to easily adapt to cross platform and framework scenarios.
  
 
== REST layer ==
 
== REST layer ==

Revision as of 17:49, 2 May 2014

Overview

At a high level the design of the API is separated into four main layers (Client, POCO/Model, REST, HTTP).

APILayeredDesign.png

Each layer is described in more detail below. By separating the API into these four levels, the API is resilient to underlying dependency changes, can be easily unit tested, and easily extended. While the end-user/consumer of the API will not see these design details, it's important for any one who would like to contribute to the project, or extend the API to be familiar with the design. Below you will find details of each layer, and how they interact.

HTTP layer

The HTTP layer is the lowest level in the API. It provides a simple abstraction for making requests and receiving responses over the HTTP protocol. While it may be considered redundant, since the .NET framework already offers several ways to do this, there are two reason why this layer is important:

  • Unit tests can validate requests and inject HTTP responses (including error states, and timeouts) without needing a remote server.
By using interfaces for the abstraction and dependency injection, unit tests can be written that inject specific HTTP responses into code that takes a dependency on this layer. This eliminates the need for an individual developer to have a remote server to test against. Since remote calls are not being made, latency is greatly reduced, tests execute quickly, and the developer can continue doing what he/she does best. Developers don't have to waste time on watching tests run or waiting for the remote server to be available. This does not eliminate the need for integration tests that are intended to be run against a live server. Those tests should, and will still be written, but do not need to be run as a part of a gated check-in, and should be run by the CI system on a periodic basis.
  • Tight coupling and dependencies are removed.
The .NET framework is always evolving. There are several ways (WebRequest, WebClient, HttpClient, etc) with the .NET framework alone to make an HTTP request, not to mention third-party libraries. The HTTP layer is not intended to be a replacement for these APIs. It is intended to represent a contract for the API to abstract itself from the implementation. A developer that takes this layer as a dependency does not need to know or understand how the request is being carried out, if the current framework version supports the functionality required, etc. By using the HTTP layer, if/when the underlying implementation needs to change, none of the code that has taken a dependency will need to change (unless there is a breaking change in the HTTP specification, which is highly unlikely). This gives the API tremendous flexibility and allows the API to easily adapt to cross platform and framework scenarios.

REST layer

The REST layers primary responsibility is to interact with the REST endpoints of a remote service. Each remote service should be represented by it's own client in this layer. The intention of this layer is to provide a client side representation of a remote services API surface area, and to also abstract away the details of how the REST calls are formatted and made. A REST client takes in information from the code above it, adds and formats it as needed, and sends the request off to the server using the HTTP layer. By consolidating the code that interacts with a remote REST endpoint, updates, changes, or new version of the remote API can easily be accommodated. This layer, like the HTTP, layer can be used to test any code that takes a dependency on it in isolation, greatly reducing the complexity and length of the test code.

POCO/Model layer

The POCO/Model layer is responsible for transforming user facing models, into and out of a format that that REST layer understands. This isolates the code that is being used to parse, serialize, or deserialize the response from the REST layer into a strongly typed C# object that makes logical sense to the end user. Changes, updates, new versions, or extensions to remote service can easily be accommodated and contained in this single layer. Like the REST and HTTP layers, this layer can be used to test code that depends on it in isolation, removing the need to depend on the this layer directly or any of the layers below it.

Client layer

The top most layer in the API is the Client layer. This layer is exposed to the end user directly and is responsible for establishing an end-user-centric surface area. This surface area can, and most of the time should, be different then the remote services. The client layer _should_ be simple and lightweight, and is _not_ a place to add high level orchestration or processing. The client layer's main goal is to provide a friendly, intuitive, and native felling interface for the end user. The client takes information from the end user, passes it to the POCO layer and gives the responses back to the end user in the form of POCO objects.

Putting it all together

All of the code listed below is also available in the CustomServiceClientExample project. In this example there is a very simple service that we want to add to the API. The service has one endpoint, that simply echos text back to the caller, and adds the url that was requested. For example, making a GET request to "http://httpbin.org/get?m=Text" would return JSON similar to:

 {
   "args": {
       "m": "Text"
     },
    "url": "http://httpbin.org/get?m=text"
 } 

A contributor to the API would first create a new REST client and a factory that can be used to create it for the API.

 public interface IEchoRestClient
 {
   Task<IHttpResponseAbstraction> Echo(string message);
 }
 public interface IEchoRestClientFactory
 {
   IEchoRestClient Create(IServiceLocator serviceLocator);
 }

This interface matches the remote services endpoint surface area, and will depend on and surface the response from the HTTP layer. An implementation of the REST client and its factory might look like:

 internal class EchoRestClient : IEchoRestClient
 {
   internal const string serviceEndpoint = "http://httpbin.org/get";
   internal IServiceLocator ServiceLocator;
   public EchoRestClient(IServiceLocator serviceLocator)
   {
     this.ServiceLocator = serviceLocator;
   }
   public async Task<IHttpResponseAbstraction> Echo(string message)
   {
     var client = this.ServiceLocator.Locate<IHttpAbstractionClientFactory>().Create(CancellationToken.None);
     client.Method = HttpMethod.Get;
     client.Uri = new Uri(serviceEndpoint +"?m=" +message);
     return await client.SendAsync();
   }
 }
 internal class EchoRestClientFactory : IEchoRestClientFactory
 {
   public IEchoRestClient Create(IServiceLocator serviceLocator)
   {
     return new EchoRestClient(serviceLocator);
   }
 }

First the class defines the service endpoint (in this example it's hard coded). Secondly, the object has a field for a service locator. The service locator allows the client to locate the HTTP abstraction layer, and any other dependencies it might need. Next, the client defines a constructor that accepts a service locator. Then, the client implements the interface's Echo method. In this method the object locates and creates an instance of the HttpAbstractionClient, then populates it with the desired HTTP method, and the target Uri. Lastly the method calls the clients SendAsync method, and awaits the response. Since a raw JSON payload from the service is not suitable to be consumed by the caller, a simple POCO/Model class can be defined to represent the remote services response.

 public class EchoResponse
 {
   public string Message { get; private set; }
   public string Url { get; private set; }
   public EchoResponse(string message, string url)
   {
     this.Message = message;
     this.Url = url;
   }
 }

A POCO client can then be created that can interact with the REST layer, and surface the new POCO object to the client layer.

 public interface IEchoPocoClient
 {
   Task<EchoResponse> Echo(string message);
 }
 public interface IEchoPocoClientFactory
 {
   IEchoPocoClient Create(IServiceLocator serviceLocator);
 }
 internal class EchoPocoClient : IEchoPocoClient
 {
   internal IServiceLocator ServiceLocator;
   public EchoPocoClient(IServiceLocator serviceLocator)
   {
     this.ServiceLocator = serviceLocator;
   }
   public async Task<EchoResponse> Echo(string message)
   {
     var restClient = this.ServiceLocator.Locate<IEchoRestClientFactory>().Create(this.ServiceLocator);
     var resp = await restClient.Echo(message);
     var payload = await resp.ReadContentAsStringAsync();
     var obj = JObject.Parse(payload);
     return new EchoResponse((string)obj["args"]["m"], (string)obj["url"]);
   }
 }
 internal class EchoPocoClientFactory : IEchoPocoClientFactory
 {
   public IEchoPocoClient Create(IServiceLocator serviceLocator)
   {
     return new EchoPocoClient(serviceLocator);
   }
 }

The POCO client simply locates the REST client, calls and awaits the Echo method, parses the response into an EchoResponse class, then returns the response back to the caller. At this point we have abstracted away the REST and HTTP layers from the client, and the end user. Since we used factories and interfaces, we can also substitute either layer at run time with test mocks, which would cut out the communication to and from a remote server. Once the POCO client has been implemented we can create a client that can consume it.

 public interface IEchoServiceClient
 {
   Task<EchoResponse> Echo(string message);
 }
 internal class EchoServiceClient : IEchoServiceClient
 {
   internal IServiceLocator ServiceLocator;
   public EchoServiceClient(IServiceLocator serviceLocator)
   {
     this.ServiceLocator = serviceLocator;
   }
   public async Task<EchoResponse> Echo(string message)
   {
     var pocoClient = this.ServiceLocator.Locate<IEchoPocoClientFactory>().Create(this.ServiceLocator);
     return await pocoClient.Echo(message);
   }
 }

In a real world scenario the client will be far more complicated, in this example is very simple. Because the remote service itself is so simple, the surface area of the client and the underlying service are almost identical. The client is intended to be consumed by an end-user that may not know, care, or understand the underlying service. The client should be end-user-centric, and not intended to force the user to use the same pattern as the underlying service. In some cases, like one illustrated here, it's acceptable.

At this point in the example, all four layers have been created, but something is missing. In the client, and poco client the implementation is using service location to locate and create the clients. In order for this to work, The client factories need to be registered with the service locator. To do this, either the existing service registrar needs to be updated or a new service registrar needs to be created for the assembly.

 public class EchoServiceRegistrar : IServiceLocationRegistrar
 {
   public void Register(IServiceLocationManager manager, IServiceLocator locator)
   {
     manager.RegisterServiceInstance(typeof(IEchoPocoClientFactory), new EchoPocoClientFactory());
     manager.RegisterServiceInstance(typeof(IEchoRestClientFactory), new EchoRestClientFactory());
   }
 }

The code above, is an entry point used by the service locator. When the assembly that contains the registrar above is register with the service locator, it will search the assembly for this class, and call the Register method. The code above registers the two factories created earlier in the example, and creates a new instance for both to be returned when the interface is located by the consumer.

If the service needs to be exposed to the end user directly then it needs to inherit from the IOpenStackServiceClient interface,

 public interface IEchoServiceClient : IOpenStackServiceClient
 {
     ....

A service definition then needs to be created.

 internal class EchoServiceClientDefinition : IOpenStackServiceClientDefinition
 {
   public string Name { get; private set; }
   public EchoServiceClientDefinition()
   {
     this.Name = typeof (EchoServiceClient).Name;
   }
   public IOpenStackServiceClient Create(ICredential credential, CancellationToken cancellationToken, IServiceLocator serviceLocator)
   {
     return new EchoServiceClient(credential, cancellationToken, serviceLocator);
   }
   public IEnumerable<string> ListSupportedVersions()
   {
     return new List<string>();
   }
   public bool IsSupported(ICredential credential)
   {
     return true;
   }
 }

Next, we need to update the EchoServiceClient to take in a credential and canellation token. This allows the client to receive authentication information, and for the end user to cancel any long running requests (though in this example we will not support either of these scenarios).

 internal class EchoServiceClient : IEchoServiceClient
 {
   internal IServiceLocator ServiceLocator;
   public EchoServiceClient(ICredential credential, CancellationToken token, IServiceLocator serviceLocator)
   {
     this.ServiceLocator = serviceLocator;
   }
   ....
 }

Finally, the service needs to be registered with the OpenStackServiceClientManager.

 public class EchoServiceRegistrar : IServiceLocationRegistrar
 {
   public void Register(IServiceLocationManager manager, IServiceLocator locator)
   {
     manager.RegisterServiceInstance(typeof(IEchoPocoClientFactory), new EchoPocoClientFactory());
     manager.RegisterServiceInstance(typeof(IEchoRestClientFactory), new EchoRestClientFactory());
     var serviceManager = locator.Locate<IOpenStackServiceClientManager>();
     serviceManager.RegisterServiceClient<EchoServiceClient>(new EchoServiceClientDefinition());
   }
 }

At this point the example "Echo" service has been fully integrated into the API. An end user can consume it by simply using the code below.

 public async Task<string> SomeEndUserMethod()
 {
   var authUri = new Uri("https://region.identity.host.com:12345/v2.0/tokens");
   var userName = "user name";
   var password = "password";
   var tenantId = "XXXXXXXXXXXXXX-Project";
   
   var credential = new OpenStackCredential(authUri, userName, password, tenantId);
   var client = OpenStackClientFactory.CreateClient(credential);
   await client.Connect();
   
   var echoServiceClient = client.CreateServiceClient<EchoServiceClient>();
   var resp = await echoServiceClient.Echo("Hello world!");
   return resp.Message;
 }