RESTful Web Conversations at DevTalk.net

RESTful Web Conversations

My previous post concerning a conversation system in WebSharper involved a purely client-side model, where all conversation information was stored on the client in the form of JavaScript declaration. In this post, I want to move the conversation data to a RESTful back-end which will use the new WCF Web API (courtesy of @gblock and friends) and will follow the HATEOAS paradigm.

The source code for this project is available online at http://bitbucket.org/nesteruk/RestfulWebConversation

The Nature of Conversations

Conversations are great for trying out RESTful services. They are, at the same time, stateless (the conversation structure doesn’t change often on the server), and stateful in the sense that the user’s position along a conversation line must be preserved somewhere. As a result, conversations are uniquely positioned to test out the new REST Web API that Microsoft is coming out with.

So let’s get started. The first thing I’m going to do is once again define the conversation entities for persisting them in a database. In this case, there is only one entity: a ConversationItem.

public class ConversationItem
{
  [BsonId] public int Id;
  public int PartyId;
  public string Speech;
  public List<int> EnableList;
  public List<int> DisableList;
}

Now, we can easily create a ConversationRepository class that would implement the typically CRUD operations – I typically like to lump Create and Update into a single operation though.

With this structure defined, let’s talk about the way we are going to serve the data via RESTful endpoints. My first attempt at creating this service was somewhat confusing: I couldn’t figure out what project template we’re meant to use together with the new WCF Web API. I mean, the libraries themselves are on NuGet, but what type of project should it be? And how should it be configured?

Not having any particular idea, I took the WCF REST for .Net 4 template off the gallery, but ended upthrowing away most of what got generated and adding in the WebAPI stuff.

DTOs

In order to get the data accross the wire, I added a pair of DTO classes. The first is a ConversationItemDTO, which is essentially a DTO (data transfer object) for the conversation items. It mirrors the original entity completely, but has a different type of ‘noise’ attached to it – first, it uses properties instead of fields; second, it has prescriptive attributes that determine its layout should XML serialization be used (I care mainly about JSON, but still…).

[XmlRoot("ConversationItem")]
public class ConversationItemDTO
{
  [XmlAttribute("Id")]
  public int Id { get; set; }
  [XmlAttribute("PartyId")]
  public int PartyId { get; set; }
  [XmlAttribute("Speech")]
  public string Speech { get; set; }
  [XmlAttribute("EnableList")]
  public List<int> EnableList { get; set; }
  [XmlAttribute("DisableList")]
  public List<int> DisableList { get; set; }
}

The second DTO entity is a list aggregation of the above:

[XmlRoot("ConversationItems")]
public class ConversationItemList
{
  [XmlArray]
  public List<ConversationItemDTO> Items { get; set; }
}

Finally, I have added an entity-to-DTO extension method for the conversation items. It seems fitting that the transposition method is contained in neither the original entity nor the DTO, though we ould have easily stuck it into the DTO if need be:

// booooring :)
public static ConversationItemDTO ToDTO(this ConversationItem item)
{
  var dto = new ConversationItemDTO();
  dto.Id = item.Id;
  dto.Speech = item.Speech;
  dto.PartyId = item.PartyId;
  dto.EnableList = item.EnableList;
  dto.DisableList = item.DisableList;
  return dto;
}

This may seem a bit trivial, but that’s because the HATEOAS paradigm is not in play yet. In fact, we are currently cheating by returning enable/disable lists instead of telling the client which options are valid at a particular location. Let’s try to fix this.

HATEOAS

Okay, so before we get to returning links to available conversation options (that’s what HATEOAS is all about, in this case), let’s do a few simple things. The first would be to return a set of initial conversation items on the default URL. Just like the RestCart sample, I have defined a resource class, which is a typical WCF service contract, but with a lot less noise:

[ServiceContract]
public class ConversationResource
{
  private readonly IConversationRepository repository;
  public ConversationResource(IConversationRepository repository)
  {
    this.repository = repository;
  }
  [WebGet(UriTemplate = "")]
  public ConversationItemDTO GetInitial()
  {
    var result = repository.FindOne(1).ToDTO();
    result.Responses = repository.FindMany(new[] {2}).Select(r => r.ToDTO()).ToList();
    return result;
  }
}

This is really as good as it gets – no huge attribute stacks, no complicated Web.config manipulation. This is all that is needed to publish an entity. In the above Get() method, you can see that I’m simply using the original Mongo repository, taking the first 4 items, converting them to DTOs and returning them wrapped in a list.

So far so good, but no hypermedia links in sight. Let’s try to add a few. In fact, we are going to get rid of the enabled/disabled lists and instead return the items relevant at this particular point. In other words, our DTO becomes something like the following:

[XmlRoot("ConversationItem")]
public class ConversationItemDTO
{
  [XmlAttribute("Id")]
  public int Id { get; set; }
  [XmlAttribute("PartyId")]
  public int PartyId { get; set; }
  [XmlAttribute("Speech")]
  public string Speech { get; set; }
  [XmlArray("Responses")]
  public List<ConversationItemDTO> Responses { get; set; }
}

That’s better, but still confusing: how do we incorporate client state (i.e., where the client is in the conversation thread) into this model? The answer is that we can simply pass an array of currently enabled items in a GET request. (Note: I’m serious about this being a GET and not a POST. We are not changing server data, just providing some hints for the server to compute the next set of links.)

With that in mind, we define the following stub for a method getting a conversation item:

[WebGet(UriTemplate = "{id}?enabledOptions={enabledOptions}")]
public ConversationItemDTO Get(int id, string enabledOptions = null)
{
  ...
}

You’ll notice that there is a string argument with a comma-separated list of currently enabled items. The id here refers to the conversation item the user chose. We start by checking that it’s valid and returning a 404 if it’s not:

// get the chosen option from repository
var userResponse = repository.FindOne(id);
if (userResponse == null)
{
  var ctx = WebOperationContext.Current;
  ctx.OutgoingResponse.SetStatusAsNotFound(
    string.Format("Could not find a client response with id={0}", id));
  return null;
}

Next, we build a HashSet<int> with the relevant items and exclude the disabled ones:

// compute the set of relevant items
var items = new HashSet<int>();
if (enabledOptions != null)
  items.UnionWith(enabledOptions.Split(',').Select(s => int.Parse(s)));
items.UnionWith(userResponse.EnableList);
items.ExceptWith(userResponse.DisableList);

Finally, we get the actual items from the server, split them into the server and client parts, and return the result.

// get all the items
var all = repository.FindMany(items).OrderBy(i => i.Id);
var grouped = all.GroupBy(i => i.PartyId)
  .ToDictionary(k => k.Key, v => v.ToList());

// get server and client items
var server = grouped[(int) Party.Server].First();
const int clientKey = (int) Party.Client;
var client = grouped.ContainsKey(clientKey) ? grouped[clientKey]
                                            : EmptyList<ConversationItem>.Instance;
// now - finally - build the response
var result = server.ToDTO();
result.Responses = client.Select(i => i.ToDTO()).ToList();
return result;

That’s all there is to it! Let’s discuss how it all works.

Using the Service

I will show the client-side usage of the service in the next post: probably when I get back to a computer that has a working WebSharper license on it. Meanwhile, let me explain how the service works.

When someone goes to http://myservice.com/, the service returns a representation of the default conversation items for the server (the outer ConversationItem) and the set of possible responses. This is completely in line with HATEOAS in the sense that we return a set of links to things that can be done.

When someone goes to http://myservice.com/X, we assume that the client chose conversation option with id=X. We calculate the server response and the next set of client responses and return them to the consumer.

When someone goes to http://myservice.com/X?enabledOptions=A,B,C, the server also includes the currently enabled items A, B and C in the set of possible responses. This is necessary because we are typically too lazy to include the full set of responses each time. Instead, we do things incrementally. I have not tested this with client-side jQuery invocation, but there should be no problem.

Conclusion

Building RESTful services is now somewhat easier with the Web API, but there are still several fundamental issues which are, overall, not addressed:

First, there are no helpers for projecting data collections. The creation of DTOs is tedious, and I’ve spent a few hours testing the service to make sure that things are returned correctly.
It is entirely unclear how to handle errors. Should we throw exceptions? Or just return 404 in the output stream and return null? It would be great to have some prescriptive guidance and a few mechanisms to ensure that if I’m projecting data, all the typical problems (such as accessing an entity with a non-existent ID) are handled without my involvment.
The WCF Web API has no built-in mechanism for creating state machines and wiring them into a HATEOAS paradigm. It would be great to have this – possibly in the WF4 updates that have come out in the .Net Framework 4 platform update.

Overall, though, I’m happy with the results I got from the Web API, and am also very happy with the way NuGet let me add a pile of references to the projects. In the next post, I’ll build a single-page-AJAX front end and will probably hit an altogether different set of issues.

DevTalk.net