Sunday, April 27, 2008 9:56 PM bart

Q: Is IQueryable the Right Choice for Me?

Introduction

Recently I delivered a session on Custom LINQ Providers - LINQ to Anything at the TechDays conferences in Ghent and Lisboa. The core of the session focuses on expression trees and translating those at runtime into a query language like CAML or LDAP (as the running samples). In this approach, IQueryable is just a minor implementation detail - or rather a vehicle to bring you closer to the query pattern established by the query operator methods. Last week, one of the attendees who got inspired by the session and started to write a query provider, asked me for some input on the best approach, referring to the little bullet indicated on my slide below:

image

 

IQueryable<T> or not?

So, do you really need IQueryable<T>? To say the least, IQueryable<T> is an interesting interface and so is IQueryProvider. Why? Well, first of all the IQueryable<T> interface itself doesn't have all of the query operators, instead it's being extended with those operators (and their implementation, wiring up expression trees) by the Queryable class's extension methods. You can somewhat think of it as an abstract base class; suffice to say it's the right balance in the tension between interface providers and implementers. In addition to this, the infrastructure provided by IQueryable is basically just a means to glue together pieces of an expression tree or to trigger execution of such an expression tree in IQueryProvider, e.g. in response to calling the First method.

Although providing much flexibility, the little caveat of the IQueryable approach lies in the fact that most providers don't need all of those methods, so most of the time you have to disappoint the user stating that something is not supported. When? At runtime, through NotSupportedException. And in lots of cases, there's no way to avoid this: think of a query predicate passed in to the Where method underneath the covers. If it contains a call to certain method (e.g. String.EndsWith) which can't be translated to the target language (e.g. CAML) you're out of luck and you won't find out till the code runs (the compiler doesn't call into a query provider, asking whether or not the generated expression tree will be supported at runtime).

However, on the higher level of query operators you can get some compile-time help by just partially implementing the query pattern. How does that work? Well, I did have the following slide, outlining query expressions are just another pattern provided by the language (just like foreach, using, lock, etc):

image

(The answer to the question above is click here.)

Right, the sample above is completely non-sense, but the compiler will be more than happy to translate it:

var res = new Homer().Where(x => x.HasSkateboard).Select(x => x.Sister).Father;

Take a look at the Where call, which resolves to the instance method Where on Homer, taking in a Func<Bart, bool>. In other words, the x in x => x.HasSkateboard is of type Bart, and HasSkateboard is a (not so far-fetched) property on Bart, returning a bool. Resolving the Where call: accomplished and returning an instance of Marge. The Select call can be inferred similarly, which is left to the reader.

 

Cooking you own query pattern

So, this is how you can create your own query pattern implementation by means of instance methods. A sample is shown below:

class Table<T> : IEnumerable<T>
{
   public Query<T> Where(Expression<Func<T, bool>> predicate) { ... }
   public ClosedQuery<R> Select<R>(Expression<Func<T, R>> projection) { ... }
   ...
}

class Query<T> : IEnumerable<T>
{
   public ClosedQuery<R> Select<R>(Expression<Func<T, R>> projection) { ... }
   ...
}

class ClosedQuery<T> : IEnumerable<T>
{
   ...
}

basically by providing an entry-point like Table<T> and further providing a fluent interface pattern that produces Query<T> objects in a chained manner. How you go ahead and expose the query operators on the source object and the query class purely depends on the composability of the query you want to provide, e.g. do you want a maximum of one Where call or do you allow more flexibility? Don't forget a Select method though since C# 3.0 requires the user to specify it; VB 9.0 doesn't but still generates the call which decouples the data source object from a resulting query object, no matter how trivial it is.

Notice in the sample above the use of ClosedQuery, which is a class that will keep all the expression tree information (which could be as easy as a few fields set by an internal constructor, containing the predicate and projection and whatever else you have captured using query operator methods) as well as a reference to the source the query is operating on. It doesn't contain further query operators though, to avoid multiple class to e.g. Select (which IQueryable<T> allows and which can be a bit challenging to implement - why?).

The core thing in the code above however is the type of the parameters to the query operator methods: Expression<T>, since you want an expression tree for runtime parsing and translation. And for the parser itself, obviously you want to short-circuit things in the degenerate cases such as Table<T> and Query<T> to fallback on a common implementation - people can still write new Table<string>().Where(x => x.Length == 5) and iterate over it, which should just work.

 

Compiler errors

Assuming you have the code above, which you can implement trivially as throwing NotImplementedException or returning null, let's try to write the following query:

var res = from x in new Table<string>()
          where x.Length > 5
          select x;

This should compile just fine. But if we write:

var res = from x in new Table<string>()
          where x.Length > 5
          orderby x.Length descending
          select x;

we'll see this compile-timer error:

image

which clearly points at the problem by using the words "query pattern".

 

Conclusion

When do you switch to this approach instead of using IQueryable<T>? As usual, the answer is: it depends. I'd say, if you just implement Where and Select and maybe some others like Take and First, it might not be worth to go all the way with IQueryable<T>. When ordering comes into play, things get a little more tricky with OrderBy and ThenBy calls (with their descending variants). Also, if you want to support multiple calls to query operators, IQueryable<T> might be beneficial, think of cases where users write "client-side views":

var expensive = from x in new Table<Product>()
                where x.UnitPrice > 123
                select x;

var res = from x in expensive
          where x.Name.StartsWith("C")
          select x;

In fact, for LINQ to AD, I still have the pending work item on my list to switch over to a custom implementation of the query pattern rather than using IQueryable<T>, but since it started as a blog series entitled "The IQueryable Tales", I have a good historical excuse for its current structure. LINQ to SharePoint on the other hand supports quite a bit of query operators (Where, Select, OrderBy*, ThenBy*, GroupBy, Take, First, TakeWhile) which makes it a better candidate for an IQueryable<T>-based implementation.

Happy querying!

Del.icio.us | Digg It | Technorati | Blinklist | Furl | reddit | DotNetKicks

Filed under: ,

Comments

# W&ouml;chentliche Rundablage: ASP.NET MVC, Silverlight 2, jQuery, CSS, C#&#8230; | Code-Inside Blog

Pingback from  W&ouml;chentliche Rundablage: ASP.NET MVC, Silverlight 2, jQuery, CSS, C#&#8230; | Code-Inside Blog

# Weekly Links: ASP.NET MVC, Silverlight 2, jQuery, CSS, C#&#8230; | Code-Inside Blog International

Pingback from  Weekly Links: ASP.NET MVC, Silverlight 2, jQuery, CSS, C#&#8230; | Code-Inside Blog International

# Reflective Perspective - Chris Alcock &raquo; The Morning Brew #83

Pingback from  Reflective Perspective - Chris Alcock  &raquo; The Morning Brew #83

# Q: Is IQueryable the Right Choice for Me?

Tuesday, May 06, 2008 2:58 PM by DotNetKicks.com

You've been kicked (a good thing) - Trackback from DotNetKicks.com

# Dew Drop - May 14, 2008 | Alvin Ashcraft's Morning Dew

Wednesday, May 14, 2008 7:52 AM by Dew Drop - May 14, 2008 | Alvin Ashcraft's Morning Dew

Pingback from  Dew Drop - May 14, 2008 | Alvin Ashcraft's Morning Dew

# LINQ to MSI - Part 0 - Introduction

Friday, June 06, 2008 11:41 PM by B# .NET Blog

Introduction Lately I&#39;ve been delivering talks entitled &quot;LINQ to Anything&quot;, to be repeated

# Recent Faves Tagged With "groupby" : MyNetFaves

Thursday, May 28, 2009 3:20 AM by Recent Faves Tagged With "groupby" : MyNetFaves

Pingback from  Recent Faves Tagged With "groupby" : MyNetFaves