Friday, February 13, 2009 8:25 AM bart

Getting Started with Oslo – Introducing “M”

Introduction

Last year at the PDC, we announced the modeling platform “Oslo”. It’s one of those technologies I’m extremely thrilled about, for various reasons. The mission to reduce the impedence mismatch between humans and computers is a huge one and great progress has been made in the past by making things more declarative in nature. Think of COM with its IDL files, think of custom attribute support in the CLR, think of configuration files to modify the behavior of an application without the need to recompile, and I’m sure you can think of many others. However, in solving this problem, quite often sub-optimal solutions arise where generic tools like XML are used, obfuscation the domain being modeled. And that’s precisely one of the things Oslo aims to help with: modeling of problem domains, in a DSL-capable way, with tool support.

 

The three pillars

The Oslo project consists of three core pillars. First, there’s the “M” modeling language. During the design of Oslo, the team realized that the “classic” approach to modeling – based on boxes and lines, a graphical representation that is – wasn’t the most appealing approach to developers (and more generally, “domain experts”). People love the low barrier to entrance that text brings, and that’s precisely what “M” enables: textual expressiveness to model domains (e.g. textual DSLs). The second pillar is the one that provides graphical (and interactive) representations over those domains, using a tool called Quadrant (more about that later). And finally, models need to be stored, and that storage space is called the Repository. Samples of repositories are SQL databases and XAML files.

Oslo = M + Quadrant + Repository = Write + Interact + Store

Ultimately the model stored in the repository is turned to execution, much like custom attributes (used to provide a model, e.g. to define a web service using attributes for a service and methods) in .NET influence the runtime (e.g. provide runtime generated code, like SOAP).

In this post, we’ll walk through M and Repository a little bit, based on the “Oslo” SDK – January 2009 CTP. Basically, I want to give people a jumpstart with M with the bare minimum of information required. In subsequent posts, I’ll dive into the details of the M programming language and cover other topics like MGrammar.

 

IntelliPad – your new textual editor toy

Without much further ado, let’s start writing a model in “M”. As “M” models are textual representations, you can use any text editor of your choice. Actually “Oslo” comes with a new fully customizable editor (written in WPF) called “IntelliPad”:

image

Because of its highly customizable nature, the first thing you should do is selecting an editor mode in the top-right corner:

image

We’ll go for “M Mode” obviously:

image

Switching the editor to this mode causes an additional menu to become available, which we’ll explore later on:

image

 

Writing your first model

Alright, time to write code now. Let’s model a collection of programming languages to take an easy start. The first concept to get used to is the one of a module. Modules are like namespaces and provide a unit of encapsulation that allows to control visibility and scope. Modules can contain exports (of types and such, see further) and can be imported somewhere else. In addition, modules have an impact on the deployment of the model to the repository, something that can get mapped on concepts like database schemas. Let’s call our module: Geeks. M is a curly brace language, so we end up with:

module Geeks
{
}

Notice how keywords in IntelliPad (see subsequent screenshot, or believe my code text formatting above) are shown in bold. Keywords are first-class citizens in domain-specific modeling as we’ll see in a future post on MGrammar.

Next, we’ll add a type to our module. A type is the definition of a “shape” of data. Instances of the type (in C# lingo) are called values, so you can also state that a type is an expression that describes a set of values. However, a type by itself doesn’t define storage, something that will be arranged by an extent. We’ll get to this in a second, but let’s first define a type called Language, that represents a programming language:

module Geeks
{
    type Language
    {
        Name : Text;
        Year : Integer16;
        Creator : Text;
    }
}

Some explanation is required. The declaration of the type itself should be fairly straightforward: again curly braces, specifying a name, nothing special. Inside the type, we declare its named fields. For example, Name is of (built-in) type Text. The ‘:’ operator used to specify the type of a field is called the “ascription operator”. Similarly we define other fields called Year and Creator. In most models, the default types are insufficiently specific to express the intent of the model. For example, we might want to restrict the length of a text field or restrict the range of a number. This can be done by defining constraints.

module Geeks
{
    type Language
    {
        Name : Text where value.Count < 20;
        Year : Integer16;
        Creator : Text;
    }
}

In the above, we’ve restricted the length of the name of a programming language to 20 characters (names of programming languages tend to be cryptic, sometimes just one-letter “words” <g>). The value keyword is used to refer to the current element being tested. Also the type itself can be annotated, for example to specify a key (which maps on primary keys in T-SQL):

module Geeks
{
    type Language
    {
        Name : Text where value.Count < 20;
        Year : Integer16;
        Creator : Text;
    } where identity Name;
}

As you can imagine this identity declaration will be used to define relationships between objects, something that’s outside the scope of this first blog post. Cool. Now we have a type, we need to define an extent. Extents map to tables in SQL and contain values of some declared type (that ultimately has the role of schematizing the table, i.e. the DDL). To declare an extent, no special keyword is required. We simply state the name of the extend an use the ascription operator again to specify the type, (optionally) suffixed with a Kleene operator to state its multiplicity. For example, we want to have 0 or more language, so we end up with:

module Geeks
{
    type Language
    {
        Name : Text where value.Count < 20;
        Year : Integer16;
        Creator : Text;
    } where identity Name;

    Languages : Language*;
}

Other multiplicity annotations are + (1 or more), ? (0 or 1) and #min..max. Here’s what the result should look like so far:

image

 

Inspecting the model declaration

Now let’s see what this model declaration corresponds to on the repository level. In other words, what code will get generated to define the model in the store. The default repository technology is SQL, so the language this gets translated into is T-SQL for the “Oslo” repository. In the “M Mode” menu, select “Repository T-SQL Preview”. Now you’ll see a split view between declaration and compilation result:

image

I’ve scrolled to the table definition code (the create table DDL statement), so you can see how fields and constraints are translated. Notice I’ve made a mistake: my programming language name should be restricted to 20 characters, not 19. Change the code on the left (< to <=) and the right will follow immediately:

image

 

Pre-populating data

What about declaring some data to be added to the repository? While the above is much like schematizing data and adding some deployment aspects to it (like extents and modules), we still need some way to declare data. We can do this by altering our extent definition as follows:

module Geeks
{
    type Language
    {
        Name : Text where value.Count <= 20;
        Year : Integer16;
        Creator : Text;
    } where identity Name;

    Languages : Language* {
        { Name = "Lisp", Year = 1958, Creator = "McCarthy"},
        { Name = "C++", Year = 1983, Creator = "Stroustrup"},
        { Name = "Haskell", Year = 1990, Creator = "Peyton-Jones"},
        { Name = "C#", Year = 2001, Creator = "Hejlsberg"},
        { Name = "F#", Year = 2005, Creator = "Syme"},
        { Name = "M", Year = 2008, Creator = "Langworthy"}
    }
}

Looks a lot like JSON, and as you can see the signal-to-noise ratio is quite optimal. Just some curly braces, assignments and commas (no closing tags or constructor calls). I’ll talk about M’s type-system another time, but its structural typing (as opposed to nominal typing, typically used by object-oriented languages like C#) provides a more natural way of dealing with data (the “has a” relationship outweighs the “is a” relationship in importance).

Notice how the Repository T-SQL preview updates itself as you type. Scrolling to the bottom will reveal a bunch of insert DML statements:

image

It’s important to point out that “Oslo” is a modeling language, not some new kind of data access technology. That means you’ll use it to write your models in a concise manner (using text), visualize and interact with them (using Quadrant), but to access and manipulate the data at the end of the day you’ll rely on your favorite data access technology of choice. That will obviously depend on the store you’re deploying to, but assuming the default of SQL, such a data access technology might be ADO.NET, LINQ to SQL, Entity Framework, or whatever you like.

 

Compile

Time to compile our model and deploy it to the repository. To do this, “Oslo” comes with a series of command-line tools (expect to see more IDE-level things in the future). First, save the model to a file called “Geeks.m”. In order to invoke the required tools in a convenient way, we’ll need to set the environment path. Unfortunately, the Oslo SDK doesn’t install an “Oslo Command Prompt” that does this, so I decided to cook my own (inspired by the Visual Studio 2008 Command Prompt):

cls
@echo Setting environment for using Microsoft Oslo SDK 1.0 tools.
@set PATH=%PATH%;%programfiles%\Microsoft Oslo SDK 1.0\Bin

Just create a shortcut that invokes %comspec% /k <path to the script above> and you’re in business. The two tools we’re going to use, m.exe and mx.exe, should be on the path now:

image

The first step to get our model deployed is to compile it. For that we use the M Compiler, m.exe. Let’s give it a try:

>m geeks.m

image

Hmm, a SQL file. Quite interesting. Take a look what’s in it: some DDL statements to create a schema and a table, as well as the DML statements to insert our data in the created table. Not bad at all and directly usable to create a schema, table and populate it with initial data. But typically we want some intermediate step, much like an object file in C or a .dll assembly in the managed word, so that we can deploy multiple of those together. We can do this by specifying the /p flag (for package), specifying the output format to be an “image” (instead of the default of “script” which created the SQL file):

m /p:image geeks.m

image

Quite a big file (a factor 10 compared to the original). You might wonder what’s inside. Curiosity is good, so let’s open it up in Notepad:

image

Ha, our friend “PK”: after the late Phil Katz, creator of PKZIP. A zip-file that is. Let’s copy it to a file with extension .zip, extract it and take a look at the contents:

image

The [Content_Types].xml file isn’t really interesting, but the others are. First the .manifest. Guess what, XAML!

image

Even more, this particular instance of XAML is a standard: OPC or the Open Packaging Convention (specified in ECMA-376, Part 2). What’s more important to our discussion here are the contents of the package, in our case two files. The first is a “sqldom” file (a SQL “Document Object Model”), the second one is an “m-ast” (an M “Abstract Syntax Tree”). I’ll dive into both in a second. But first, I wanted to point out that the use of OPC makes M-packages available to anyone for consumption. The “Oslo” platform has helpers (MImage in Microsoft.M.Framework.dll) to read the contents of an .mx “M image file”, but you can use any ZIP library and XAML parser to access the contents.

On to the .sqldom file first. This is the relevant portion to the deployment aspect of the image file, in that the MX tool (see further) reads it to generate T-SQL for execution on the repository in order to create the model over there. Let’s show a few relevant portions of the file (it’s quite a giant despite our small model – remember how verbose XML is, and what our DSL mission is all about? ;-)). Notice again it’s a XAML file, with the top-level parts denoting a database (unspecified yet, mx.exe will make it nonymous), with a schema (derived from the module declaration) containing database objects (in our case only a table but user defined functions and such are possible too):

<?xml version="1.0" encoding="utf-8"?>
<mts:AnonymousDatabase Name="{x:Null}"
                       xmlns:mts="clr-namespace:Microsoft.TSQL10.SyntaxTree;assembly=Microsoft.M.Framework"
                       xmlns:scg="clr-namespace:System.Collections.Generic;assembly=mscorlib"
                       xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
                       xmlns:x2="http://schemas.microsoft.com/netfx/2008/xaml">
  <mts:AnonymousDatabase.Schemas>
    <mts:Schema Name="Geeks">
      <mts:Schema.DatabaseObjects>

Let’s take a look at the Table definition for our Languages type:

<mts:Table x:Name="__ Reference ID 0" Name="Languages">
  <mts:Table.AdditionalStatements>
    <mts:InsertStatement DeclareVariableName="{x:Null}" DerivedTable="{x:Null}" DmlTableSource="{x:Null}" ExecuteStatement="{x:Null}" Label="{x:Null}" OutputClause="{x:Null}" Relation="{x2:Reference Name=&quot;__ Reference ID 0&quot;}" TopExpression="{x:Null}" WithCommonTableExpression="{x:Null}" RelationDependencySatisfiedExternally="False" TableHints="None" TopClauseOptions="None" UseDefaultValues="False">
      <mts:InsertStatement.Columns>
        <mts:SimpleColumn DefaultValue="{x:Null}" x:Name="__ Reference ID 1" IdentityIncrement="1" IdentitySeed="1" IsFileStream="False" IsIdentity="False" IsNullable="False" IsRowGuidColumn="False" Name="Name">
          <mts:SimpleColumn.Type>
            <mts:TSqlType Length="20" Precision="0" Scale="0" Type="NVarChar" />
          </mts:SimpleColumn.Type>
        </mts:SimpleColumn>
        <mts:SimpleColumn DefaultValue="{x:Null}" x:Name="__ Reference ID 2" IdentityIncrement="1" IdentitySeed="1" IsFileStream="False" IsIdentity="False" IsNullable="False" IsRowGuidColumn="False" Name="Year">
          <mts:SimpleColumn.Type>
            <mts:TSqlType Length="0" Precision="0" Scale="0" Type="SmallInt" />
          </mts:SimpleColumn.Type>
        </mts:SimpleColumn>
        <mts:SimpleColumn DefaultValue="{x:Null}" x:Name="__ Reference ID 3" IdentityIncrement="1" IdentitySeed="1" IsFileStream="False" IsIdentity="False" IsNullable="False" IsRowGuidColumn="False" Name="Creator">
          <mts:SimpleColumn.Type>
            <mts:TSqlType Length="2147483647" Precision="0" Scale="0" Type="NVarChar" />
          </mts:SimpleColumn.Type>
        </mts:SimpleColumn>
      </mts:InsertStatement.Columns>
      <mts:InsertStatement.Values>
        <scg:List x:TypeArguments="mts:ISqlExpression" Capacity="4">
          <mts:Constant Literal="Lisp" Type="UnicodeString" />
          <mts:Constant Literal="1958" Type="Integer" />
          <mts:Constant Literal="McCarthy" Type="UnicodeString" />
        </scg:List>
      </mts:InsertStatement.Values>
    </mts:InsertStatement>
</mts:Table.AdditionalStatements> <mts:Table.Columns> <x2:Reference Name="__ Reference ID 1" /> <x2:Reference Name="__ Reference ID 3" /> <x2:Reference Name="__ Reference ID 2" /> </mts:Table.Columns> <mts:Table.Constraints> <mts:PrimaryKeyConstraint Name="PK_Languages"> <mts:PrimaryKeyConstraint.Columns> <mts:ColumnReference Column="{x2:Reference Name=&quot;__ Reference ID 1&quot;}" Parent="{x:Null}" Qualifier="{x:Null}" Name="Name" /> </mts:PrimaryKeyConstraint.Columns> </mts:PrimaryKeyConstraint> </mts:Table.Constraints> </mts:Table>

Quite a bit of code, but rather self-explanatory. The table is called Languages, and additional statements are required to populate it. “Simple” columns are defined based on TSqlTypes and insert statements are paired with corresponding values. Finally, the table has constraints, in our case just the definition of a primary key. Notice how cross-references are made within the document using string literals like “__ Reference ID 1”, for example to refer to a column in the primary key constraint declaration.

Next, what’s up with the compilationunit.xaml file? Again XAML, this time representing the syntax tree of our model definition:

<?xml version="1.0" encoding="utf-8"?>
<CompilationUnit FileName="" Span="(0:1,1)#(0:1,1)"
                 xmlns="http://schemas.microsoft.com/languages/2008/M"
                 xmlns:p="http://schemas.microsoft.com/netfx/2008/xaml/schema"
                 xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
                 xmlns:x2="http://schemas.microsoft.com/netfx/2008/xaml">
  <ModuleDeclaration FileName="C:\temp\oslo\geeks\geeks.m" Span="(0:1,1)#(583:18,2)">
    <ModuleDeclaration.Imports>
      <ModuleImportDirective FileName="" Span="(0:1,1)#(0:1,1)">
        <ImportedName Alias="{x:Null}" FileName="" Span="(0:1,1)#(0:1,1)">
          <ImportedName.Name>
            <DeclarationReference FileName="" Span="(0:1,1)#(0:1,1)">Language</DeclarationReference>
          </ImportedName.Name>
        </ImportedName>
      </ModuleImportDirective>
    </ModuleDeclaration.Imports>
    <ModuleDeclaration.Name>
      <Parsed x:TypeArguments="p:String" FileName="C:\temp\oslo\geeks\geeks.m" Span="(7:1,8)#(12:1,13)" Value="Geeks" />
    </ModuleDeclaration.Name>
    <TypeDeclaration FileName="C:\temp\oslo\geeks\geeks.m" Span="(21:3,5)#(167:8,27)">
      <TypeDeclaration.Name>
        <Parsed x:TypeArguments="p:String" FileName="C:\temp\oslo\geeks\geeks.m" Span="(26:3,10)#(34:3,18)" Value="Language" />
      </TypeDeclaration.Name>
      <ParameterizedExpression FileName="C:\temp\oslo\geeks\geeks.m" Span="(21:3,5)#(167:8,27)">
        <ParameterizedExpression.Operation>
          <DeclarationReference FileName="C:\temp\oslo\geeks\geeks.m" Span="(21:3,5)#(167:8,27)">$Operator$Where</DeclarationReference>
        </ParameterizedExpression.Operation>
        <EntityType FileName="C:\temp\oslo\geeks\geeks.m" Span="(40:4,5)#(146:8,6)">

Notice you can actually visualize this from within the IntelliPad tool through the “M Mode” menu, option “AST”:

image

Quite interesting too, but not as relevant for this discussion as the rest of the OPC package. I’ll talk more about AST representations in a subsequent post on MGrammar.

 

Deploy

Now that we have an image file, we’re ready to deploy our model to the repository using the mx.exe tool. In order to do this, you’ll need to have a SQL Server in the proximity. Assuming you have a default instance with Windows authentication available (and you’re logged in as a user that’s a dbo on the database), you can issue the following command:

mx –d:Demo –c –f –i:geeks.mx

This tells the M command-line utility to create (-c) a database called “Demo” (-d), forcing it (-f, causing existing databases to be recreated, warning!) and based on the image (-i) geeks.mx. Once we’ve done this, we can connect to the database (e.g. using sqlcmd.exe) and dump the data from our created model:

image

(For pretty-printing purposes, I’ve added a constraint to the creator field to limit its length to 20 characters too.) Notice where the module name went (database schema) and where the type name went (table name).

 

Recap

Wow, that was easy. Let’s recap:

  1. We used IntelliPad to write a model, consisting of a module containing a type (~ schema) and an extent (~ storage) with prepopulated data (~ DML).
  2. With m.exe we compiled the model definition to an image (~ object file).
  3. Using mx.exe the created model image was deployed to the SQL server.

And of course we detoured a little into the specifics of an M image (.mx file) in between steps 2 and 3. In a future post, I’ll be talking in more detail about M’s structural type system, core language features, and such. We’ll also explore other pieces of the puzzle, like MGrammar.

Stay tuned!

Del.icio.us | Digg It | Technorati | Blinklist | Furl | reddit | DotNetKicks

Filed under: ,

Comments

# Arjan`s World &raquo; LINKBLOG for February 13, 2009

Friday, February 13, 2009 12:58 PM by Arjan`s World » LINKBLOG for February 13, 2009

Pingback from  Arjan`s World    &raquo; LINKBLOG for February 13, 2009

# Getting Started with Oslo – Introducing “M”

Friday, February 13, 2009 5:14 PM by DotNetShoutout

Thank you for submitting this cool story - Trackback from DotNetShoutout

# Dew Drop - February 14, 2009 | Alvin Ashcraft's Morning Dew

Saturday, February 14, 2009 4:21 PM by Dew Drop - February 14, 2009 | Alvin Ashcraft's Morning Dew

Pingback from  Dew Drop - February 14, 2009 | Alvin Ashcraft's Morning Dew

# Newly Noted #18 | Patrick Verbruggen's Blog

Monday, February 16, 2009 11:04 AM by Newly Noted #18 | Patrick Verbruggen's Blog

Pingback from  Newly Noted #18 | Patrick Verbruggen's Blog

# Binzy Wu &raquo; Blog Archive &raquo; [Links] Bookmarks for 2009-02-17

Tuesday, February 17, 2009 4:18 PM by Binzy Wu » Blog Archive » [Links] Bookmarks for 2009-02-17

Pingback from  Binzy Wu  &raquo; Blog Archive   &raquo; [Links] Bookmarks for 2009-02-17

# The M Programming Language – Part 2 – Collections and Extents

Wednesday, February 18, 2009 2:11 AM by B# .NET Blog

Last time in this series, we looked at M’s structural type system , pointing out the differences compared

# pictures of dragons: The Tale of the Obscure Weblog Engine Bug &laquo; Law Blog

Pingback from  pictures of dragons: The Tale of the Obscure Weblog Engine Bug &laquo;  Law Blog

# Links 2009-02-28

Monday, March 02, 2009 1:03 AM by Gunnar Peipman's ASP.NET blog

SharePoint Customize a publishing page layout How-To Delete SharePoint Shared Services Provider (SSP

# Getting Started with Oslo Using 'M'

Monday, March 09, 2009 7:44 AM by US ISV Developer Evangelism Team

A community technology preview was released for Microsoft project code named "Oslo" in January and now

# The M Programming Language (Oslo)

Monday, March 09, 2009 2:08 PM by IHateSpaghetti {code}

” Oslo ” is the codename for Microsoft’s forthcoming modeling platform that helps you build your own

# Getting Started with Oslo – Introducing “M” - B# .NET Blog

Saturday, March 21, 2009 1:51 PM by DotNetShoutout

Thank you for submitting this cool story - Trackback from DotNetShoutout

# M: Obscure Programming Language of the Month

Thursday, October 08, 2009 8:10 AM by M: Obscure Programming Language of the Month

Pingback from  M: Obscure Programming Language of the Month