April 2007 - Posts

It's great to see the Silverlight story coming together. This morning at MIX07, Scott Guthrie demoed the cross-platform .NET API support in Silverlight together with dynamic language support (Python, Ruby), debugging across barriers using Visual Studio, etc. Jason Zander has his top 10 feature list posted today. Check out Scott's interview on Channel 9 too, as well as Tim Sneath's blog for all sorts of Windows Presentation * related content. And did you know you can get 4 GB hosting space for free on Microsoft Silverlight Streaming?

Have fun!

Del.icio.us | Digg It | Technorati | Blinklist | Furl | reddit | DotNetKicks

This post covers the answers to the C# 3.0 - Core Linq concepts questions. Alex and Ming provided a set of correct answers, so there's not much I can say (that's what quizzes are for after all) :-). Nevertheless, here's my story...

Let's refresh the questions first:

A piece of C# 3.0 code - Copy Code
1 // 2 // Put additional namespace imports here 3 // 4 5 class Program 6 { 7 static void Main() 8 { 9 Source<Customer> src = null; 10 var res = from s in src 11 orderby s.Name, s.Age descending 12 where s.Name == "Bart" 13 select s.Age; 14 } 15 } 16 17 class Customer 18 { 19 public string Name { get; set; } 20 public int Age { get; set; } 21 } 22 23 abstract class Source<T> 24 { 25 // 26 // Put code here 27 // 28 }

The goal is to extend the Source<T> class in order to make the query in Program::Main work "properly" (in a sense it does compile without errors). Notice we can't change line 23 to implement IQueryable<T> or so. Basically, what the compiler does as it encounters the query in line 10 to 13 is to rewrite it to the following:

Copy Code
1 var res = src.OrderBy(s => s.Name) 2 .ThenByDescending(s => s.Age) 3 .Where(s => s.Name == "Bart") 4 .Select(s => s.Age);

These => things are called lambda expressions and are simply a shorthand way to write anonymous delegates. The whole point about this model is that the compiler will try to resolve the method calls to OrderBy, ThenByDescending, Where and Select with the appropriate signature. It does so by starting to look for instance members (of course starting with OrderBy since subsequent calls depend on that method's return type). If it doesn't find instance members, it will start to look for extension methods that are brought in scope with a using statement. If we'd implement IEnumerable<T> from System.Linq on line 23, and added a using System.Linq statement to lines 1-3, we'd be using LINQ-to-Objects which has an implementation for all of these methods. In this case, we don't have such an implementation and we'll have to implement the methods on our own. Here are the signatures:

Copy Code
1 abstract class Source<T> 2 { 3 public abstract Source<T> OrderBy(Func<T, string> keySelector); 4 public abstract Source<T> ThenByDescending(Func<T, int> keySelector); 5 public abstract Source<T> Where(Func<T, bool> predicate); 6 public abstract Source<T> Select(Func<T, int> selector); 7 }

To make this work, you'll need to import System.Linq in order to have the Func* generic delegates in scope. For the generic parameter types, you could think of alternatives but we won't consider these. This implementation will cause the compiler to emit anonymous methods for the various method parameter delegates, i.e.:

Copy Code
1 var res = src.OrderBy(delegate (Customer s) { return s.Name; }) 2 .ThenByDescending(delegate (Customer s) { return s.Age; }) 3 .Where(delegate (Customer s) { return s.Name == "Bart"; }) 4 .Select(delegate (Customer s) { return s.Age; });

However, an alternative exists which won't translate into IL code directly: enter expression trees. The signatures this time look like this:

Copy Code
1 abstract class Source<T> 2 { 3 public abstract Source<T> OrderBy(Expression<Func<T, string>> keySelector); 4 public abstract Source<T> ThenByDescending(Expression<Func<T, int>> keySelector); 5 public abstract Source<T> Where(Expression<Func<T, bool>> predicate); 6 public abstract Source<T> Select(Expression<Func<T, int>> selector); 7 }

You'll need to import System.Linq.Expressions to make this work. This time, the compiler finds the appropriate instance methods in Source<T> but with Expression<TT> parameters, which results in the generation of expression tree code. This code would drive use too far from home, but feel free to inspect it via ILDASM. Basically, it allows the implementation of Source<T> to parse the expression tree at runtime to translate it into some kind of domain specify query language (e.g. SQL, LDAP, CAML). To implement a more flexible custom query provider, you can implement IQueryable<T> as explained in other posts.

Enjoy!

Del.icio.us | Digg It | Technorati | Blinklist | Furl | reddit | DotNetKicks

A couple of weeks ago I announced the availability of LINQ-SQO v0.9 (Release Candidate). Today the project has reached another release stage: LINQ-SQO v0.9.1. This release is in conformance with the Orcas Beta 1 LINQ-to-Objects implementation and can be downloaded here (see releases). Major features are:

  • Fixes for a few bugs
  • Implementation of Average, Min, Max and Sum operations on floating numbers
  • Additional method overloads for various Enumerable extension methods using IEqualityCOmparer<T> parameters
  • Extra overloads for Aggregate and SelectMany methods
  • New naming convention for generic parameters (TSource, TResult, TElement, TKey, etc instead of T, R, E, K, etc)

The unit test project still has to be extended to reflect these changes. The future roadmap of the project will depend on the release schedule of Orcas and will follow major Orcas milestones (v0.9.x releases where x indicates the major milestone, i.e. x = 1 for Beta 1, x = 2 for Beta 2, etc).

FYI: An update for the LINQ-to-SharePoint project is coming soon in the next few weeks. It will cover a set of bug fixes, support for other field types (including Lookup fields), a revamped entity model (working towards update support in a later stage), etc.

Enjoy!

Del.icio.us | Digg It | Technorati | Blinklist | Furl | reddit | DotNetKicks

It's been a long ten days since the first announcement of the LINQ to SharePoint project, with lots of implementation and testing work but tonight I'm very excited to announce the availability of the first public alpha of the LINQ to SharePoint project. A few highlights of this release:

  • Supports writing LINQ queries for SharePoint lists in both C# 3.0 and Visual Basic 9.0
  • The SpMetal entity type creation tool supports both C# 3.0 and Visual Basic 9.0
  • Talks to SharePoint over web services or though the SharePoint Object Model
  • Most CAML query constructs are available via LINQ

The current alpha release is built for the Orcas March 07 CTP. An update for Orcas Beta 1 will follow later.

 

For more information, visit the following pages:

Warning: This release is an alpha release; use it at your own risk. You shouldn't use the alpha release in any production environment whatsoever.

 

A little sample of a (very simple) LINQ to SharePoint query in C# and VB:

C# 3.0 LINQ-to-SharePoint query
1 var users = new SharePointDataSource<User>(new Uri("http://wss.mycompany.local")); 2 3 var res = from u in users 4 orderby u.Birthdate descending 5 where u.FirstName.StartsWith("B") && u.Age >= 24 && u.FavoriteFood == FavoriteFood.Pizza 6 select new { u.FullName, u.Age, Interest = u.AccountBalance * 0.07 }; 7 8 foreach (var u in res) 9 Console.WriteLine(u); 10

VB 9.0 LINQ-to-SharePoint query
1 Dim users As New SharePointDataSource(Of User)(New Uri("http://wss.mycompany.local")) 2 3 Dim res = From u In users _ 4 Order By u.Birthdate Descending _ 5 Where u.FirstName.StartsWith("B") And u.Age >= 24 And u.FavoriteFood.Value = FavoriteFood.Pizza _ 6 Select u.FullName, u.Age, Interest = u.AccountBalance.Value * 0.07 7 8 For Each u In res 9 Console.WriteLine(u) 10 Next

Much more complicated queries are supported though, allowing you to focus on the query itself rather than on the CAML plumbing. A list of a few complexities that LINQ-to-SharePoint can deal with:

  • Recursive parsing of query trees and translation of boolean negation into normal forms using De Morgan's laws.
  • Analysis of projection operations to create a list of required fields in the query to reduce bandwith.
  • Condition analysis and validation required for translation to supported CAML constructs.
  • Nullable type support language differences between VB and C#.
  • MultiChoice-to-enum mapping and support for fill-in choices.
  • Visual Basic's runtime libraries for string comparisons.

 

Please report any issues with the software via the project website. You feedback is highly appreciated!

Enjoy!

Del.icio.us | Digg It | Technorati | Blinklist | Furl | reddit | DotNetKicks

All C# quizzes on my blog so far were covering C# 2.0 language features. Today, we'll start with a first quiz on C# 3.0, which is rather a knowledge check than a quiz in the classic sense of the word. This quiz focuses on the core concepts in C# 3.0 that drive LINQ.

Consider the code fragment below:

Defect piece of LINQ code - Copy Code
1 // 2 // Put additional namespace imports here 3 // 4 5 class Program 6 { 7 static void Main() 8 { 9 Source<Customer> src = null; 10 var res = from s in src 11 orderby s.Name, s.Age descending 12 where s.Name == "Bart" 13 select s.Age; 14 } 15 } 16 17 class Customer 18 { 19 public string Name { get; set; } 20 public int Age { get; set; } 21 } 22 23 abstract class Source<T> 24 { 25 // 26 // Put code here 27 // 28 }

This piece of code doesn't compile with the following messages:

Error 1 The type arguments for method 'System.Linq.Queryable.OrderBy<TSource,TKey>(System.Linq.IQueryable<TSource>, System.Linq.Expressions.Expression<System.Linq.Func<TSource,TKey>>)' cannot be inferred from the usage. Try specifying the type arguments explicitly.

The goal of this quiz is to provide a minimal abstract definition of the Source<T> class in order to make the code compile. We won't execute the application though, we're just focusing on the compilation (and the result of the compilation).

A few rules:

  • You're only allowed to put abstract method signatures in regions marked by comments.
  • You shouldn't implement any interface (after all, the previous rule doesn't allow touching line 23).
  • Any method you add to Source<T> should return an object of the generic type Source`1.

Additional questions:

  1. Two different (categories of) solutions for implementation of Source<T> exist, explain.
  2. Will these two solutions mentioned in question 1 produce different IL code for Program::Mail? If so, explain why and point out the differences.
  3. How many different minimal solutions exist (renaming a parameter doesn't count for a different solution of course)?

Have fun!

Del.icio.us | Digg It | Technorati | Blinklist | Furl | reddit | DotNetKicks

Answers to the C# Quiz - Fun with System.Xml. Here's a quick refresh of the problem:

Copy Code
1 using System; 2 using System.Xml; 3 4 class Program 5 { 6 static void Main() 7 { 8 new Bar<int>().CreateBar<double>("Bart").CreateBar<long>("DeSmet"); 9 } 10 } 11 12 class Bar<T> 13 { 14 static XmlDocument doc = new XmlDocument(); 15 16 private XmlElement root; 17 18 public Bar() 19 { 20 root = doc.CreateElement("Root"); 21 } 22 23 private Bar(XmlElement root) { this.root = root; } 24 25 public Bar<S> CreateBar<S>(string foo) 26 { 27 XmlElement f = doc.CreateElement(foo); 28 root.AppendChild(f); 29 return new Bar<S>(root); 30 } 31 }

The comments on the original post are mostly correct. The problem with this fragment is that for every Bar<T> with a distinct T, another instance of the static XmlDocument "doc" will be created. This is due to the way generics in .NET work. What happens is this:

  • new Bar<int>() creates a <Root> tag inside the document "doc" which is owned by Bar<int>; it's on the first use of Bar<int> that the "doc" instance is created;
  • the CreateBar<double>("Bart") call on the Bar<int> instance will work fine, since it references the same document "doc"; it does return a Bar<double> instance however, pointing to the "root" element of the current instance (ctor line 23);
  • next, the CreateBar<long>("DeSmet") call happens on the type Bar<double> which has its own "doc" element (the constructor call of line 14 has been called for Bar<double> during line 29's constructor call - or before - because that's the first use of Bar<double>); here things fail due to the way DOM works concerning document contexts:
  • Unhandled Exception: System.ArgumentException: The node to be inserted is from adifferent document context.
    at System.Xml.XmlNode.AppendChild(XmlNode newChild)
    at Bar`1.CreateBar[S](String foo)
    at Program.Main()

A solution? One solution is to use a singleton:

Copy Code
1 internal static class SingletonDocument 2 { 3 static XmlDocument doc; 4 5 public static XmlDocument Instance 6 { 7 get 8 { 9 if (doc == null) 10 doc = new XmlDocument(); 11 return doc; 12 } 13 } 14 }

thus replacing line 14 of the original fragment with

Copy Code
static XmlDocument doc = SingletonDocument.Instance;

Note: The SingletonDocument shouldn't be declared as a child class of the Bar class, since you'd end up with distinct statics again in such a case (Bar<int>.SingletonDocument is different from Bar<double>.SingletonDocument for example).

An alternative is to change the CreateBar<S>(string foo) method like this:

Copy Code
1 public Bar<S> CreateBar<S>(string foo) 2 { 3 XmlElement f = doc.CreateElement(foo); 4 root.AppendChild(f); 5 if (typeof(S) != typeof(T)) 6 Bar<S>.doc = doc; 7 return new Bar<S>(root); 8 }

You can omit the type-check in line 5 if you'd like. The basic idea here is to "transfer" the existing instance every time a new Bar<S> is created. However, this trick only helps "from the inside"; it's easy to forget to "forward" the existing instance and it doesn't work "from the outside", that is: different distinct types created outside CreateBar<S> will have different doc instances. Therefore, the singleton seems like the best general-purpose solution (i.e. without changing the original code structure, just solving the problem of the different statics) to this problem. An additional question raised by the solution mentioned above is whether or not there's still value in using a static variable anyway:

Copy Code
1 class Bar<T> 2 { 3 private XmlDocument doc = new XmlDocument(); 4 5 private XmlElement root; 6 7 public Bar() 8 { 9 root = doc.CreateElement("Root"); 10 } 11 12 private Bar(XmlElement root) { this.root = root; } 13 14 public Bar<S> CreateBar<S>(string foo) 15 { 16 XmlElement f = doc.CreateElement(foo); 17 root.AppendChild(f); 18 Bar<S> res = new Bar<S>(root); 19 res.doc = doc; 20 return res; 21 } 22 }

The source of this quiz problem? LINQ's IQueryable<T> has a method called CreateQuery<TElement>(...) which results in an equivalent circumstance as the one illustrated in the quiz problem... In a larger context than a simple quiz such a bug might be a bit more subtle to spot.

Del.icio.us | Digg It | Technorati | Blinklist | Furl | reddit | DotNetKicks

Earlier this week, "The IQueryable Tales" were published on my blog, with great success. This series covered the end-to-end creation of a custom query provider for LINQ, focusing on the LINQ to LDAP case. For your reference, here are a few links to the episodes:

As the matter in fact, this implementation is primarily meant for "academic purposes", showing how to go ahead in creating your own query provider using IQueryable<T>. Other samples around the blogosphere include LINQ to Amazon and LINQ to NHibernate. The LINQ to LDAP provider was written in a quick-n-dirty fashion in about one afternoon but has serious restrictions because of the underlying query language (LDAP filters) being quite simplistic, limiting our expressiveness. Nevertheless, it's both a good sample annex tutorial and useful for people dealing with Active Directory or ADAM who don't like to mess around with LDAP filters themselves.

Today, I'm happy to announce another fun project of mine, of which an alpha is to be released on CodePlex in the next few weeks: LINQ to SharePoint. Again, it's meant as a (good) sample of implementing query providers and to serve developers when consuming SharePoint data from lists. Essentially, the implementation provides a translation of LINQ queries to CAML, together with a "list schema to entity mapping tool" called SpMetal.

CAML, which stands for Collaborative Application Markup Language, is the language used by SharePoint for various purposes, including querying. LINQ to SharePoint translates LINQ queries like the following:

1 DateTime startDate = new DateTime(2007, 01, 01); 2 string firstNameStartsWith = "B"; 3 int maximumAge1 = 24; 4 string lastNameContains = "Smet"; 5 int minimumAge2 = 50; 6 var res = from usr in users 7 where ( (usr.FirstName.StartsWith(firstNameStartsWith) && !(usr.Age > maximumAge1)) 8 || (usr.LastName.Contains(lastNameContains) && usr.Age >= minimumAge2)) 9 && usr.Member == true 10 && usr.MemberSince >= startDate 11 && usr.ShortBio != null 12 orderby usr.MemberSince descending 13 orderby usr.Age ascending 14 select new 15 { 16 Name = usr.FirstName + " " + usr.LastName, 17 usr.Age, 18 usr.Homepage, 19 usr.ShortBio, 20 usr.Activities, 21 MembershipInfo = new { usr.MembershipType, usr.MemberSince }, 22 FoodPreferences = new { usr.FavoriteFood, usr.FavoriteFoodOtherValue } 23 };

into the corresponding CAML query, like this:

<Query> <Where> <And> <And> <And> <Or> <And> <BeginsWith> <FieldRef Name="First_x0020_name" /> <Value Type="Text">B</Value> </BeginsWith> <Leq> <Value Type="Number">24</Value> <FieldRef Name="Age" /> </Leq> </And> <And> <Contains> <FieldRef Name="Last_x0020_name" /> <Value Type="Text">Smet</Value> </Contains> <Geq> <Value Type="Number">50</Value> <FieldRef Name="Age" /> </Geq> </And> </Or> <Eq> <Value Type="Boolean">1</Value> <FieldRef Name="Member" /> </Eq> </And> <Geq> <Value Type="DateTime">2007-01-01T00:00:00Z</Value> <FieldRef Name="Member_x0020_since" /> </Geq> </And> <IsNotNull> <FieldRef Name="Short_x0020_Bio" /> </IsNotNull> </And> </Where> <OrderBy> <FieldRef Name="Member_x0020_since" Ascending="FALSE" /> <FieldRef Name="Age" /> </OrderBy> </Query>

Finally it's sent to the server using either the SharePoint Object Model (SPList) or the SharePoint web services (lists.asmx). The query provider is "intelligent" on several fields:

  • The CAML Query schema