C# Related Posts by ThinqLinq

Reactive Extensions RX in Action the movie

I’ve been giving presentations for some time now. By far, my favorite talk to give is my Reactive Extensions in Action. This past summer, I had the talk recorded at the Code on the Beach conference. If you haven’t had a chance to see it live, now’s your chance. The focus of this talk is to focus on some of the practical uses of Rx rather than the underpinnings of how it works. I hope you enjoy and this sparks ideas from you on how you can use it in your applications.

 

Rx in Action recorded at Code on the Beach.
Posted on - Comment
Categories: C# - VB - Rx -

Left Outer Joins in LINQ with Entity Framework

As I spend more time reviewing code with clients and on public forums, I’m constantly seeing cases where people have issues with Outer joins in LINQ and the various flavors of LINQ. In this post, I’m going to look at a couple options from a syntax perspective that you can use to make working with outer joins easier with LINQ. Naturally, we have a much deeper discussion of outer joins in our book that you’re welcome to dig into as well.

Typically, if you do a join in LINQ, it will perform an inner join where it only returns the records that contain results in both sides of the evaluation (in both tables). If a child table doesn’t have a record for a parent row, that result will be excluded. Consider the case in Northwind where an Order record exists, but there are no order detail lines. The following LINQ query won’t return that record as part of its result set:

from o in Orders
join od in OrderDetails on o.OrderID equals od.OrderID
select new {o, od}

This translates into the following SQL statement (Note to DBA’s that are concerned by the * below: EF does list the individual columns in the actual SQL. I’m reducing them to * for the purposes of this post to focus on the join clauses):

SELECT 
    [Extent1].*, [Extent2.*
FROM  [dbo].[Orders] AS [Extent1]
    INNER JOIN [dbo].[OrderDetails] AS [Extent2] ON [Extent1].[OrderID] = [Extent2].[OrderID]

If you want to include the orders regardless of whether it has any detail lines, you would need to turn this inner join into an outer join using the DefaultIfEmpty extension method. LINQ only supports left outer joins. If you want a right outer join, you need to flip the logic of your query to turn it into a left outer join. In order to use the DefaultIfEmpty, you typically need to push the joined set into a temporary value first and then select from it using the DefaultIfEmpty method:

from o in Orders
join innerOD in OrderDetails on o.OrderID equals innerOD.OrderID into Inners
from od in Inners.DefaultIfEmpty()
select new {o, od}

This generates the expected LEFT outer join as shown below:

SELECT 
    [Extent1].*, [Extent2].*
FROM  [dbo].[Orders] AS [Extent1]
    LEFT OUTER JOIN [dbo].[OrderDetails] AS [Extent2] ON [Extent1].[OrderID] = [Extent2].[OrderID]
The problem that I have with this is that the syntax seems overly verbose to accomplish this change. As is often the case, Microsoft often gives multiple ways to accomplish the same goal. One method that I’ve started to find helpful is to revert to more of an ANSI 82 style syntax where the joins were accomplished in a where clause instead of a join clause. By mixing and matching the LINQ query comprehensions and lambda syntax, we can restate the above query as follows:
from o in Orders
from od in OrderDetails
    .Where(details => o.OrderID == details.OrderID)
    .DefaultIfEmpty()
select new {o, od}

If we check the SQL, we can see that this generates the same SQL as the more verbose version above. Often when I’m doing this in application code, I’ll put the where and DefaultIfEmpty on the same line to let me focus on what I’m fetching from, not how I’m joining them unless I need to focus on that.

There is an issue with using this syntax when joining nullable values or strings. Since the where statement doesn’t know about the cardinality relationship (1-Many, 0-1 – Many, etc), Entity Framework adds an additional check where the nullable value is not null to allow for join cases where both sides have a null value. Changing our query to use Northwind’s Customers and Orders which joins on the string CustomerID values, we can write the following LINQ query which is nearly identical to the one we created before for Orders and OrderDetails:

from c in Customers
from o in Orders
    .Where (o => o.CustomerID == c.CustomerID)
    .DefaultIfEmpty()
select new {c, o};

This results in the following SQL statement.

SELECT 
    1 AS [C1], 
    [Extent1].*, [Extent2}.*
    FROM  [dbo].[Customers] AS [Extent1]
    LEFT OUTER JOIN [dbo].[Orders] AS [Extent2] ON ([Extent2].[CustomerID] = [Extent1].[CustomerID])
       AND ([Extent2].[CustomerID] IS NOT NULL)

Note that in this case, we have an additional clause checking for Extent2.CustomerID IS NOT NULL. This may seem innocuous, but I have found in at least one case for the query execution to be significantly slower due to the use of an index scan rather than an index seek caused by this clause. Here you have to be careful when crafting your queries and monitor performance even more carefully to avoid performance bottlenecks unnecessarily.

While this version works better, I still prefer to use associations rather than joins to think about the problem from more of an object graph perspective rather than set based operations. As long as you have a natural association between the entities I much prefer using the associations to navigate through than to have to worry about building out the joins manually each time. We can restate our join even more simply as follows using associations:

from o in Orders
from od in o.Order_Details.DefaultIfEmpty()
select new {o, od}

Note that if you omit the DefaultIfEmpty clause, you would get an Inner join rather than left outer join.

If you have other ways of creating outer joins in LINQ that you prefer, let me know what you thinq.

Posted on - Comment
Categories: C# - Entity Framework - LINQ -

CodeLens for Visual Studio 2013 configuration

One of the new features of Visual Studio is the ability to automatically see statistics about your code while you are looking at it via a new feature called CodeLens. For example, you can see how many method calls are referencing your method, how many bugs and what tests are accessing the method.

image

While this information can be helpful when reviewing code, it sometimes gets in the way when writing it. For example, you can’t just click the 0 references line and start adding attributes and comments, you have to add them in a blank line above the CodeLens injected text (included as images here because you can’t copy and paste the CodeLens text).

image

If you don’t want the CodeLens information getting in the way, simply right click one of the tags and select CodeLens Options. Alternatively, in the Quick Launch window (Ctrl+Q), just type CodeLens to find the menu in config. From here, you can tweak which values are displayed inline with your code or disable CodeLens entirely to get it out of your way while writing code.

image

Posted on - Comment
Categories: C# - Visual Studio -

Rx for Windows Phone article now available

A couple of years ago I gave a presentation at the last MIX for doing asynchronous programming using Reactive Extensions (Rx) on the Windows Phone. The video of the presentation is still available for watching. Around the same time, I was approached by Code Magazine to write a similar article, which I was happy to oblige. It took a while but I was happy to see the article finally published in the Sep/Oct 2013 edition of Code Magazine.

In the article I demonstrate how to use Rx to build a dice rolling game (like Yhatzee) for the Windows Phone 7 or 8 and communicating with a service tier for rolling the dice (hiding the fact that you may be loading the results). While the particular solution was targeted to Windows Phone, the concepts are valid across the wide variety of platforms that Rx supports. Give the article a read and let me know what you thinq.

Posted on - Comment
Categories: C# - Rx - VB - WP7 -

Async in loops

In yesterday’s post, I mentioned that there is a performance penalty of using the new Async/Await feature inside of loops. Today, I want to point out another reason why you might not want to use Async/Await inside of a loop. Consider the following example that takes a web page as a string and finds all of the hyperlinks contained in that page (using a regular expression match) and downloads each of those child pages. Using Tasks from the TPL we can load the child pages like this:

      private void GetChildren(string parent)
        {
            var matches = Regex.Matches(parent, _HtmlRegex, RegexOptions.IgnoreCase);
            foreach (Match match in matches)
            {
                if (_CancelToken.IsCancellationRequested)
                    return;
                string childTarget = match.Groups[1].Value;
                WebClient client = new WebClient();
                Task<string> childTask = client.DownloadStringTaskAsync(new Uri(childTarget));
                childTask.ContinueWith(child =>
                {
                    _FoundUrls.Add(string.Format("{0} ({1})", childTarget, child.Result.Length));
                },
                    TaskContinuationOptions.ExecuteSynchronously);
            }
            TaskButton.IsEnabled = true;
        }

Here we use the DownloadStringTaskAsync passing each found child page url. We use the ContinueWith method to chain operations asynchronously to download the child pages. As a first pass on this using Async/Await, we can simply await the task returner and no longer need the ContinueWith clause. Our code just continues as if it were just a straight code flow:

        private async void GetChildrenAsync(string parent)
        {
            var matches = Regex.Matches(parent, _HtmlRegex, RegexOptions.IgnoreCase);
            foreach (Match match in matches)
            {
                if (_CancelToken.IsCancellationRequested)
                    return;
                string childTarget = match.Groups[1].Value;
                WebClient client = new WebClient();
                string child = await client.DownloadStringTaskAsync(new Uri(childTarget));
                _FoundUrls.Add(string.Format("{0} ({1})", childTarget, child.Length));
            }
            AsyncCtp2Button.IsEnabled = true;
        }

That sure looks cleaner. What’s the issue? The downside of await is that the code execution moves over to a background loop and stops executing until it receives a response. As a result, we loose the performance improvement of issuing multiple requests and processing them as they return. Although the UI thread can continue on, this background process is blocked until it completes. At which time we request the next page that was found.

When writing multiple asynchronous requests, it is often better to use the WaitAny, WaitAll (for blocking operations), or WhenAll for non-blocking operations. Unfortunately, Async/Await only allow for operating on a single task, not a group of tasks. So how can we modify the code to issue multiple requests and operate on the results as they come back? We need to translate each of the requests into tasks and then return a Task of the resulting string arrays. Here’s the resulting code:

        private Task<string[]> GetChildrenAsyncAll(string parent)
        {
            var matches = Regex.Matches(parent, _HtmlRegex, RegexOptions.IgnoreCase);
            var childTasks = matches.OfType<Match>()
                             .Select(match => 
                                 {
                                     string childTarget = match.Groups[1].Value;
                                     var client = new WebClient();
                                     var childTask = client.DownloadStringTaskAsync(new Uri(childTarget));
                                     childTask.ContinueWith(child => 
                                         Dispatcher.BeginInvoke(new Action(() => 
                                             _FoundUrls.Add(string.Format("{0} ({1})", childTarget, child.Result.Length)))));
                                     return childTask;
                                 });

            return Task.WhenAll(childTasks.ToArray());
       }

Ugh. That’s not quite as elegant. I never said async was easy. Async/Await do make the simple cases appear easier. However there are times where a bit more effort is needed to achieve the true results that you want.

As always, if you have recommendations on how to improve this code, I’m always eager to learn ways to improve. If you want to try this sample yourself, feel free to download the async project and kick the tires a bit. The download sample also includes a version using Rx if you want to compare that version as well.

Posted on - Comment
Categories: C# -

Rx samples with WinRT

Recently I decided to rebuild one of my presentation computers using the Windows 8 Consumer Preview and Visual Studio 11 Beta. While it is working quite well for my typical usage, I did run into a bit of an issue when prepping my Rx talks for the Rocky Mountain Trifecta this past weekend.

In the past, when I'm giving my Practical Rx talk, I show Silverlight, WPF, WP7 and RxJs samples. Unfortunately, the Windows 8 Consumer Preview does not support the Windows Phone 7 emulator and Visual Studio 11 does not support the WP7 targets. As an alternative, i decided to port the WP7 demos over to a Metro style Win-RT application, which you can download now.

The main changes were due to slightly different event names, but other all of the Rx methods that I needed were available in the Rx Beta 2 experimental build that Bart introduced on the Rx Blog. However, when preping the prototypical Dictionary Suggest sample, I ran into a TargetInvocationException when trying to dispose the event handler. Searching a bit further into the call-stack, I found that the exception was caused when disposing the observable subscriptions due to the TakeUntil clause. Even though the Throttle is what pushes this code off of the dispatcher thread. For reference, here's the broken code method:


    var svc = new SimpleService.SimpleServicesvcClient();
    var results = new ObservableCollection();
    Translations.ItemsSource = results;

    IObservable inputStrings =
        from keyup in Observable.FromEventPattern
            (InputText, "TextChanged")
        select InputText.Text;

    var svcResults = 
        from text in inputStrings
        .Throttle(TimeSpan.FromMilliseconds(250))
        from result in svc.DoSomethingCoolAsync(new DoSomethingCoolRequest { input = text })
        .ToObservable()
        .TakeUntil(inputStrings)
        select String.Format("{0} - {1}", text, result.DoSomethingCoolResult);

    svcResults
        .ObserveOnDispatcher()
        .Subscribe(result => results.Insert(0, result));
         

Thanks to a bit of additional snooping by the Rx and WinRT teams, we found that the real reason here is that we are attaching the TextChanged event handler on the core dispatcher thread. However, when we use the Throttle, the operation is shifted to another thread. When we add the TakeUntil, we instruct Rx to dispose of the current subscription when the second stream starts. Of course, the new keypress also starts another stream of observables.  While this works without errors in the .Net world (Silverlight/WP7/WPF), WinRt is a bit pickier and requires that the remove handler be called on the same thread that was used when the handle was added. To fix this situation, we need to explicitly force the subscription onto the Dispatcher thread using SubscribeOnDispatcher. Here's the fix:


    var svc = new SimpleService.SimpleServicesvcClient();
    var results = new ObservableCollection();
    Translations.ItemsSource = results;

    IObservable inputStrings =
        from keyup in Observable.FromEventPattern
            (InputText, "TextChanged")
        .SubscribeOnDispatcher()
        select InputText.Text;

    var svcResults = 
        from text in inputStrings
        .Throttle(TimeSpan.FromMilliseconds(250))
        from result in svc.DoSomethingCoolAsync(new DoSomethingCoolRequest { input = text })
        .ToObservable()
        .TakeUntil(inputStrings)
        select String.Format("{0} - {1}", text, result.DoSomethingCoolResult);

    svcResults
        .ObserveOnDispatcher()
        .Subscribe(result => results.Insert(0, result));
         
Posted on - Comment
Categories: Rx - WinRT - C# -

Select Many with Rx and RxJs

A while back, I showed how you could use the query syntax and Rx to code a drag-drop operation similar to the way you might describe  the process to your mother. As a review, let’s take another look at the code, this time in C# as a preparation for moving it over to javaScript.


var mouseDown = from evt in Observable.FromEventPattern<MouseButtonEventArgs>(image, "MouseLeftButtonDown")
                select evt.EventArgs.GetPosition(image);
var mouseUp = Observable.FromEventPattern&;lt;MouseButtonEventArgs>(this, "MouseLeftButtonUp");
var mouseMove = from evt in Observable.FromEventPattern<MouseEventArgs>(this, "MouseMove")
                select evt.EventArgs.GetPosition(this);

var q = from startLocation in mouseDown
        from endLocation in mouseMove.TakeUntil(mouseUp)
        select new
        {
            X = endLocation.X - startLocation.X,
            Y = endLocation.Y - startLocation.Y
        };

What might not be evident from this code is that the query syntax is actually using the SelectMany extension method when we use the from x in xs from y in ys above. If we wanted to, we could re-write this using the Lambda/method syntax as follows:


var q = mouseDown
        .SelectMany(startPos =>
              mouseMove
                    .TakeUntil(mouseUp)
                    .Select(movePos =>
                        new
                        {
                            X = movePos.X - startPos.X,
                            Y = movePos.Y - startPos.Y
                        })
                     );

Personally, I prefer the query syntax and find the lambda syntax a bit messy with SelectMany. Let’s try to tear this down a bit to see if we can understand what’s going on. Here we still have 3 different sets of Observables—mouseDown, mouseMove and mouseUp. The SelectMany extends mouseDown taking in the instance of each observed value as it is produced. SelectMany then takes a function (lambda) which will generate a new set of Observables of some other type which may or may not include values from the first observed value and values from the second variable.

In this case, we create the second observable set from the mouseMoves that we start listening to as the result of the SelectMany. We can then generate an new observable set by projecting (Select) the mouseDown’s startPos and mouseMove’s movePos offsets. For those who think pictures say more than words, here’s a marble diagram illustrating how SelectMany works:

image

In this case the Ox represents the MouseDown observables. fx represents the function supplied in the Lambda parameter and Oy represents the resulting Observable set of offsets. Wes and Jeffrey walk through SelectMany in their Channel 9 video as well.

Moving to RxJs

So what does all of this have to do with JavaScript? In the process of rewriting some of my Rx talks to RxJs, I wanted to include the MouseMove sample. Since JavaScript doesn’t support the syntactic sugar that enable query expressions, we are forced to use the method syntax. If you want to skip to the chase, you can download my RxJs samples and look at the DragDrop.htm to try this out. As we did with the Silverlight/WPF version of DragDrop, we start by setting up three sets of Observables to track the MouseDown, MouseMove and MouseUp events.


    <script type="text/javascript" src="Scripts/rx.js"></script>
    <script type="text/javascript" src="Scripts/jquery-1.4.1.min.js"></script>
    <script type="text/javascript" src="Scripts/rx.jQuery.js"></script>

    <script type="text/javascript">
        $(document).ready(function () {
            var dragTarget = $("img");
            var mouseUp = dragTarget.toObservable("mouseup");
            var mouseMove = dragTarget.toObservable("mousemove");
            var mouseDown = dragTarget.toObservable("mousedown")
                .Select(function (event) {
                    return {
                        left: event.clientX - dragTarget.offset().left,
                        top: event.clientY - dragTarget.offset().top
                    };
                });
    </script>
    <img alt="logo" src="Images/RxLogo.png" style="position: absolute; top:400; left:400; height: 100;
        width: 100" id="RxLogo" />

Now that we have our observables, we can merge the streams using SelectMany and subscribe to the end result to actually move the image on the screen by altering the css left and top positions accordingly.


            var moves = mouseDown.SelectMany(function (imageOffset) {
                return mouseMove
                        .Do(function (event) { event.preventDefault(); })
                        .Select(function (pos) {
                            return {
                               left: pos.clientX - imageOffset.left,
                                top: pos.clientY - imageOffset.top
                            };
                        })
                        .TakeUntil(mouseUp);
            });
            moves.Subscribe(function (pos) {
                dragTarget.css("left", pos.left);
                dragTarget.css("top", pos.top);
            });

With the exception of the JavaScript function syntax which I tend to think of as a hybrid of C# and VB’s lambda syntaxes, we essentially have the same code that we did with the C# lambda syntax for SelectMany.

It can take a bit of mind twisting to get your head around the lambda syntax for SelectMany, but once you’ve done that, you can start doing some powerful manipulations to extend and coordinate observable event streams in both .Net and RxJs.

Posted on - Comment
Categories: Rx - JQuery - C# -

devLINK RxJs and Async session materials available

I would like to thank everyone who came out to my DevLINQ sessions this week. The materials for both of the sessions are not available on the Files tab  of this site. In addition, here are the descriptions and direct links to each of these downloads:

  • Reactive Extensions for JavaScript (RxJs)
  • The Reactive Extensions allow developers to build composabile, asynchronous event driven methods over observable collections. In web applications, you can use this same model in client side processing using the RxJs framework. We'll show you how you can take advantage of this framework to simplify your complex asynchronous client side operations.

    Includes slides and samples demonstrating some of the uses of the Reactive Extensions for JavaScript (RxJs). The samples illustrate using RxJs for LINQ style queries, Timer based web page rotator, Mock observable sensor, Mouse drag drop, Dictionary Suggest, and Bing Translator over multiple languages.

  • Async Programming in .Net
  • While .Net 4 added parallel programming capabilities, these tools primarily help when your application is CPU bound. In many cases, parallel processing doesn't address IO Latency issues. In these cases, we need to provide the perception of responsive applications by using asynchronous programming tools. We will explore some of these options including Delegates, Callbacks, Iterators, Observers, and the new C# Async/Await keywords.

    Includes slides and samples demonstrating ways of performing Asynchronous operations from .Net 1.0 through .Net 5.0. In addition to the standard Visual Studio 2010 install, you will also need to download and in stall the following libraries to use these samples:

Posted on - Comment
Categories: Code Camp - C# - Rx - JQuery -

Does LINQ to SQL eliminate the possibility of SQL Injection

By default, LINQ to SQL uses parameterized queries rather than concatenated strings when executing your LINQ queries. As a result, if a user tries to perform SQL Injection by improperly escaping parts of the SQL, the escape is considered part of the parameter rather than part of the query and thus avoids the injection.

However, as we discussed in chapter 8 of LINQ in Action, LINQ to SQL greatly reduces the possibility of SQL Injection, but doesn't completely eliminate it. For example, if you are using Stored Procedures and call spExecuteSQL passing in a concatenated string inside of the SQL Stored Proc, you are still subject to SQL Injection. Of course, this applies regardless of the data access technology and shows that even stored procs can't eliminate the possibility of SQL Injection.

In addition, the LINQ to SQL DataContext does offer the ability to pass a SQL Pass-through query as a string which can be injectable. For example, the following returns all rows in the Authors table:

string searchName = "Good' OR ''='";
TypedDataContext context = this;

string sql = @"Select ID, LastName, FirstName, WebSite, TimeStamp " +
"From dbo.Author " +
"Where LastName = '" + searchName + "'";

IEnumerable<Author> authors = context.ExecuteQuery<Author>(sql);

LINQ to SQL does allow for a Injection safe version if you use string parameter place holders and pass the parameters into the overloaded version of ExecuteQuery accepting a paramarray of objects:

string sql = @"Select ID, LastName, FirstName, WebSite, TimeStamp    " +
"From dbo.Author " +
"Where LastName = {0}";

IEnumerable<Author> authors = context.ExecuteQuery<Author>(sql, searchName);

Thankfully, if you're standard LINQ methods, you're safe from SQL Injection with LINQ to SQL. The Entity Framework on the other hand actually does have some other potential injectionable areas if you're thinking of going down that route.

Posted on - Comment
Categories: LINQ - C# -

Dynamic Programming in a Statically Typed World

Tomorrow night, I’m giving my presentation discussing not only how, but when to consider using the dynamic features of C# 4 and VB (forever). If you attend the presentation, you can download the samples and slides from my download page. Part of the key in this presentation is  discussing the Why as compared to the simpler How of Dynamic. Here are some of my top reasons to use the dynamic features:

Testability

When trying to use unit testing, the compiler often limits the ability to follow-through on the test first methodology because you have to write at least a bare implementation before your tests and solution can compile. If you declare your objects as dynamic in your tests, you can write all of your tests without having to write the implementation. Naturally, the tests will fail at this point, but with test driven development, your tests should fail until you have written your implementation.

One of the main things to understand if you are using dynamic types is that you no longer have your compiler acting as a unit-test, checking your code to make sure that the methods and types that you are trying to consume actually exist. Instead, you need to take extra care to fully unit test your code to ensure that it truly works the way you intend. Frankly, I don’t care what testing framework you use or how much code coverage you have, just do it!

COM Iterop

One of the traditional advantages VB has had over the C based languages is the late binding ability to work naturally with COM interfaces like those exposed in Office. Using dynamic in C# 4, gives the curly-bracket world to work with office with less ceremony. VB still has an edge in that it can retain intellisense support because of the late binding rather than “object with dynamic semantics” typing that C# utilizes.

Flexible data structures

There are times when building frameworks where you might not know the structure of your data at compile time. I’ve written before about how to use the dynamic features for parsing CSV files. The same technique can be used to parse XML files as well. If you’ve used DataSets in the past, this is another such framework which currently uses strings as parameters to access tables and fields as objects. With Microsoft.Data in the WebMatrix libraries, Microsoft introduced a dynamic implementation over datasets as well, simplifying the programming model there as well. David Fowler wrote a series of blog posts introducing Microsoft.Data last year.

Scripting and DLR

Often, applications need the ability to allow for end user customization without the need to re-compile the application. With the DLR integration, you can add these script extension points in your application so that users can write code in Python, Ruby or any other DLR supported language. As long as you add the hooks in your application, you can take advantage of the user’s scripted changes.

In addition, the web contains a plethora of python and ruby code components that you can include in your .Net application by interacting with them via the DLR. Taking advantage of mature and well tested modules is often better than trying to re-invent the wheel yourself. Don’t fall into the trap of the “Not invented here” syndrome.

Increased separation of concerns

By using dynamic connections between your types, you can have components interact without needing to share version compatible interfaces and contracts. As long as your modules follow established conventions, it can work nicely without relying on the ceremony required otherwise. I demonstrated a while back how to connect MVC 1 views with anonymous types using VB’s late binding. More recent versions of MVC use dynamic features by default to bind to views with C# as well.

Along similar lines, Robert McCarter demonstrated using DynamicObject in the ViewModel of a MVVM pattern to eliminate the need to delegate all of the property Set/Get pairs between the View and the Model.

Summary

I’m sure this is just the tip of the iceberg of areas to use dynamic features. Can you thinq of others? The nice thing about VB and C# is that you can scope the use of dynamic features to just those parts of the application that benefit from them. This way you get the best of both worlds: the performance, tooling, and compiler support of static languages and simplicity, no compilation, implicitly typed objects and methods of dynamic languages.

Ultimately, when I have to choose between the two, I return to the mantra from Erik Meijer, et. al.

Use “Static typing where possible, dynamic typing when needed.

Posted on - Comment
Categories: VB - Dynamic - C# -

Rx Mock Accelerometer using Observable.GenerateWithTime

When people ask about resources to learn how to use Rx on the Windows Phone, I often point them to the How to guides, including the How to: Use Reactive Extensions to Emulate and Filter Accelerometer Data for Windows Phone. Unfortunately, I have a problem with the underlying implementation of the emulation method which returns values through an iterator (yield) and IEnumerable<T> and then cast the IEnumerable to IObservable. As I pointed out in a MSDN forum post, It would be better to recommend returning just an IObservable natively and replace the yield with OnNext(T).

In reviewing this for my MIX 2011 presentation, I decided to take another look at the implementation and found a much better solution. When mocking the accelerometer, we essentially want to issue new values at a regular interval. To start, let’s take a look at the mocking method from the original How to guide:


static IEnumerable<Vector3> EmulateAccelerometerReading()
{
    // Create a random number generator
    Random random = new Random();

    // Loop indefinitely
    for (double theta = 0; ; theta+=.1 )
    {
        // Generate a Vector3 in which the values of each axes slowly drift between -1 and 1 and
        // then normalize the vector
        Vector3 reading = new Vector3((float)Math.Sin(theta), (float)Math.Cos(theta * 1.1), (float)Math.Sin(theta * .7));
        reading.Normalize();
                
        // At random intervals, generate a random spike in the data
        if (random.NextDouble() > .95)
        {
            reading = new Vector3((float)(random.NextDouble() * 3.0 - 1.5),
             (float)(random.NextDouble() * 3.0 - 1.5),
             (float)(random.NextDouble() * 3.0 - 1.5));

        }

        // return the vector and then sleep
        yield return reading;
        Thread.Sleep(100);
    }
}

 

Essentially, this implementation sets up an infinite loop and issues new values using the yield return statement and then sleeps 100 milliseconds. As I wrote last year in the MSDN forum, a better option is to return an IObservable and use OnNext rather than yield return as follows:


public static IObservable<Vector3> GetEmulator()
{
     var obs = Observable.Create<Vector3>(subscriber =>
                {
                    Random rnd = new Random();
                    for (double theta = 0; ; theta += .1)
                    {
                        Vector3 reading = new Vector3(
                            (float)Math.Sin(theta),
                            (float)Math.Cos(theta * 1.1),
                            (float)Math.Sin(theta * .7));
                        reading.Normalize();
                        if (rnd.NextDouble() > .95)
                        {
                            reading = new Vector3(
                                (float)(rnd.NextDouble() * 3.0 - 1.5),
                                (float)(rnd.NextDouble() * 3.0 - 1.5),
                                (float)(rnd.NextDouble() * 3.0 - 1.5));
                        }
                        // return the vector and sleep before repeating
                        Thread.Sleep(100);
                        subscriber.OnNext(reading);
                    }
                });
      return obs;
}

In preparation for Mix, I decided to revisit this and replace the looping implementation with the native Observable.GenerateWithTime method as follows:


public static IObservable<Vector3> GetAccelerometer()
{
    var obs = Observable.GenerateWithTime<double, Vector3>(
                initialState: 0, 
                condition: _ => true,
                resultSelector: theta => new Vector3((float)Math.Sin(theta),
                                        (float)Math.Cos(theta * 1.1),
                                        (float)Math.Sin(theta * .7)),
                timeSelector: _ => TimeSpan.FromMilliseconds(100),
                iterate: theta => theta + .1)
                .ObserveOnDispatcher();

     return obs;
 }

Notice now we no longer have a loop in our code or a timer sleep. Those pieces are essentially handled for us inside the implementation of the GenerateWithTime. Let’s break this object instantiation down a bit to know what’s going on.

Observable.GenerateWithTime takes two generic type arguments, the double represents the type which we will increment as new values are generated. This allows us to set range values and incrementors similar to how we setup the “for” loop and potentially escape from the loop as necessary. The second generic type indicates the type of values that the observable will create. In this case, we return Vector3 types from the phone’s XNA libraries.

The first input parameter (initialState) sets the starting value of the for loop.

The condition parameter takes a lambda expression allowing us to evaluate if we should continue issuing values. In this case, we’ll keep issuing values as long as something is subscribed to us, thus we return “true”.

The resultSelector parameter takes a input value which is the current value of our iteration and returns some new value based on that input value. Here is where we generate our new vector value based on the current value.

The timeSelector parameter allows us to specify how often we want to issue new values. This would be the same as our sleep value from the original examples.

The iterate parameter allows us to increment our looping value. This is the same as the final portion of the original “for” loop declaration.

Before we return the observable, I add a method to ObserveOnDispatcher to make sure that we delegate back to the dispatcher thread. If we didn’t do this, the GenerateWithTime moves our execution context to the timer’s thread that it generates rather than leaving us on the calling thread.

Using the GenerateWithTime method allows us to abstract the looping implementation details and provides a more declarative/functional mechanism for generating the observable values. The nice thing is that we can consume this just as we would any other observable.

If you want to see this example in action, download my Rx for WP7 samples from MIX and try out the WP7 accelerometer sample. You will see that the mock value is used when you are inside the emulator and the actual accelerometer values are used when connected to a device. Regardless of how the values are generated, we process them with the same Rx subscription pipeline:


subscribed = accelReadings.Subscribe(args =>
             {
                 Single multiplier;
                 if (Single.TryParse(this.MoveMultiplier.Text, out multiplier))
                 {
                     ButtonTransform.X = args.X * multiplier;
                     ButtonTransform.Y = args.Y * multiplier;
                 }
             });
Posted on - Comment
Categories: WP7 - Rx - C# -

Using LINQPad with MongoDb and NoRM

I’ve recently been working on a project that might be a good fit for the rising wave of Document Databases espoused by the rising NoSQL movement. Unlike the traditional Relational Databases like SQL Server, Document Databases can store object hierarchies, or unshaped data for quick and efficient saving and retrieval. There are quite a number of these databases appearing, including MongoDb, CouchDb, RavenDb. Wikipedia also has a listing of Document Databases with links to other options.

Typically, these data stores expose their data through XML or JSON interfaces. Luckily,  many of these databases also have LINQ providers to make interfacing with the systems easier. For example, if you are using MongoDb, you can use a provider like the NoRM (No-ORM) provider. Today I had the pleasure of sitting in on a webcast that Michael Kennedy of DevelopMentor did showing MongoDb with NoRM. During the presentation the question came up regarding using LINQPad in the environment. I promised to tell them how to do this, so here you go:

Install and Startup Mongo

The first step you need to do is download and startup MongoDb. To download it, go to the MongoDb site and download the version appropriate to your OS. Once you’ve downloaded it, unzip the files to your favorite directory. From the command line, start Mongo by entering the following from the bin directory that you unzipped:

mongob -dbpath=c:\projects\data\mongo

Of course, this assumes you want to put the database in the c:\projects\data\mongo path. The path does need to be present before running this command. You will be prompted to open a firewall hole for Mongo. Once you’re done, Mongo should be up and running.

Preparing NoRM

Now that the database is running, we need to prepare the .Net provider. Download the NoRM provider through Github, or directly from their download site. Unzip these files and open the appropriate sln file in Visual Studio. NoRM works in VS 2008 and NoRM – VS2010 works in VS 2010. Once you open it, compile it. Now you’re ready to roll. If you’re not sure how to get started with the NoRM provider, take a look at the many unit tests that they’ve provided to get up and running quickly.

Using LINQPad

Assuming you’ve downloaded and installed LINQPad, you should now be ready to start consuming Mongo through NoRM in LINQPad. Start by opening LINQPad and creating a new query. In order to work with our database, we need to create some class structures. As a result, you need to select the C# Program or VB Program option under the Language drop-down. (I’m choosing to do this in C# this time because the NoRM provider doesn’t fully support VB expression trees at this point.)

image 

By selecting the Program option, we can now create not only LINQ queries, but entire classes and methods. Scroll to the bottom of the code window and add a class describing the shape that we want to save and retrieve.


class Post
{
	public ObjectId Id { get; set; }
	public string Title { get; set; }
	public string Content { get; set; }
	public DateTime PubDate { get; set; }
	public string[] Categories { get; set; }
}

Keeping with the typical samples I have on this site, I’m modeling a Blog post. This will create a hierarchical shape containing posts with a collection of categories. Now that the shape is defined, we need to figure out how to insert and retrieve it using LINQPad.

To start, press F4 (or select Query and Query Properties from the menu). This will bring up a dialog to add references. In this dialog, click the Add button and locate the Norm.dll that you generated when you compiled it above when Preparing NoRM.

image

Before you close this window, select the “Additional Namespace Imports” tab at the top. This will allow us to add the Norm imports/using clauses. Add an import for Norm and Norm.Linq. (Notice here, you don’t include the “using” or “imports” keywords, just the namespaces that you want to import.) Once you’ve added the reference, close the dialog so that we can continue consuming Mongo.

image

In the code window, enter the following inside the body of the void Main() method that was generated when we selected the Program option in LINQPad:


using (var cn = MongoQueryProvider.Create("mongodb://127.0.0.1/ThinqLinq?strict=false"))
{
	var posts = cn.DB.GetCollection<Post>();
	posts.Insert(new Post
	{
		Title="Testing Linqpad",
		Content="This is a test using LinqPad",
		PubDate=DateTime.Now,
		Categories = new string[] {"NoSql", "MongoDb", "LinqPad"}
	});

	var foundPosts = from p in new MongoQuery<Post>(cn)
			where p.Title == "Testing Linqpad"
			select p;
		
	foundPosts.Dump();
}

Let’s step through this code a bit. The first line sets up our connection to the Mongo Database. We pass in the URI of the server (mongodb://127.0.0.1/) along with the name of the “database” that we want to use, or create if it hasn’t already been created (“ThinqLinq”).

This connection behaves similarly to the LINQ to SQL DataContext or Entity Framework’s ObjectContext. As with those contexts, we next need to access the “Table” for the type that we are trying to create. We do that by accessing the GetCollection method of the DB object referencing the type that we want to get (Post).

With the reference to the posts collection, we can insert a new Post object into our database by calling the Insert method. Notice, unlike a traditional RMDBS, we have not actually created anything in the database yet. This is the great thing about a Document Database. We can save our objects directly in the database without having to create the structures ahead of time.

With the object inserted into the database, we can now fetch it back using a standard LINQ query. The only difference we see here to the pattern in other LINQ providers is that we access the data source by calling MongoQuery passing in the Type that we want to fetch and the connection object that we declared at the top of the method. We view the results by calling the LINQPad Dump extension method on the results. Once completed, here’s a screen shot including the final code and results. Notice that the results do include the three items we added in the Categories array along with the single post record.

image

Interestingly, the built-in performance timer and Lambda expression options in LINQPad still work here even though we aren’t accessing a traditional database. The SQL tab remains blank because we aren’t issuing a SQL statement to Mongo.

Document Databases and the whole NoSQL movement are quite intriguing. There are plenty of times where relational data stores are a better fit, particularly when you are needing to report on related objects rather than working with object hierarchies natively. However, tools like MongoDB and NoRM make working with non-traditional hierarchical document stores quite easy as well and point to some interesting options for data storage and retrieval for other needs.

Posted on - Comment
Categories: C# - LinqPad -

Visual Studio 2010 Keyboard Binding Poster

When I work with new teams, I often get asked how I did x using the keyboard instead of reaching for the mouse. I find if I can keep my hands on the keyboard, I can often be much more productive. So, how do you learn to take advantage of the keyboard shortcuts, I recommend downloading one of the Key binding posters like the ones that were just released for Visual Studio 2010. The posters are available for VB.Net, C#, F#, and C++.

Once you have then downloaded, pick a  a different key binding each day and dedicate yourself to using that one throughout the day. You’ll find some bindings work better for you than others. Each person has their favorite way of working, so try them all and see which are your favorites. So, what are you waiting for? Download them and get coding!

Posted on - Comment
Categories: Visual Studio - C# - VB -

Replace Foreach with LINQ

One of the best ways to start Thinqing in LINQ is to find places where you can replace iterative loops with LINQ queries. This won’t necessarily make your applications any faster at the moment, but by taking advantage of the declarative syntax, it will allow you to change your underlying implementation (ie. parallelize it with PLINQ) easier in the future. Recently, I came across some code that mimicked creating a control array as we used to have in VB6 by iterating over a list of fields and adding controls to a form adding the index to the end of the field name. As the fields are being added,we hook up event listeners along the way. Here’s the old code:


Dim Fields() As String = {"a", "one", "and", "a", "two"}

Dim index As Integer

For Each item As String In Fields
    Dim box As New TextBox
    With box
        .Name = item + index.ToString
        .Top = index * 30
        .Height = 24
        AddHandler .TextChanged, AddressOf OnValueChanged
    End With
    Controls.Add(box)
    index += 1
Next
To break this down, we're creating an index so that we won't have any repeats. We then iterate over our list of fields and create (Project) new textboxes for each of the items in our list. Once we create that value, we then add the handler. Finally we add this item to another list. If we think about this in a set based manner rather than iterative, we can start getting a grasp of what LINQ really has to offer. Let's rewrite this in LINQ starting with our source:

Dim Fields() As String = {"a", "one", "and", "a", "two"}
Dim b1 = From item In Fields
         Select New TextBox With
         {
             .Name = item,
             .Top = 30,
             .Height = 24
         }

Controls.AddRange(boxes)

In this example, we take our starting list. Project (Select) new objects from these values and then pass this list directly into the Controls collection using AddRange. No more For Each.

This is a start, but there's an issue. We need to be able to add the index to this set based operation. One of the little secrets in the LINQ operators is that there are overloads which expose the index. In VB, you can't access these using the LINQ query comprehensions. You have to use the extension methods and Lambda Functions directly as follows:


Dim Fields() As String = {"a", "one", "and", "a", "two"}
Dim boxes = Fields _
            .Select(Function(item, index) _
                New TextBox With {
                    .Name = item + index.ToString(),
                    .Top = index * 30,
                    .Height = 24})

Controls.AddRange(boxes.OfType(Of Control).ToArray)

We're almost there. We just need to add our handlers for each of our new text boxes. While we could call ForEach over an array, it would cause us to iterate over our field list twice (creating two sets of text boxes). We need a way to only iterate over it once. Here, we need to create a new method and using C# iterators. It will take an IEnumerable and return an IEnumerable. By using Yield, it will not cause the enumeration to happen multiple times, but rather to add a new step as each value is being pulled through the enumeration pipeline.


public static class Extensions
    {
       public static IEnumerable<T> WhileEnumerating<T>(this IEnumerable<T> source, Action<T> action)
       {
           foreach (T item in source)
           {
               action(item);
               yield return item;
           }
       }
    }

Now, we can inject methods into the pipeline as follows:


Dim boxes = Fields _
            .Select(Function(item, index) _
                New TextBox With {
                    .Name = item + index.ToString(),
                    .Top = index * 30,
                    .Height = 24}) _
            .WhileEnumerating(Sub(item) AddHandler item.TextChanged, AddressOf OnValueChanged)

Controls.AddRange(boxes.OfType(Of Control).ToArray)

If we wanted to inject more functions, we would just add more .WhileEnumerating methods. Make sure however that each of these methods do not have side effects on other methods of the set. There you have it. Go search for those For Each (foreach) loops in your code and see how you can clean them up with LINQ to Objects.

Posted on - Comment
Categories: C# - LINQ - VB - Visual Studio -

LINQ to CSV using DynamicObject

When we wrote LINQ in Action we included a sample of how to simply query against a CSV file using the following LINQ query:


From line In File.ReadAllLines(“books.csv”) 
Where Not Line.StartsWith(“#”) 
Let parts = line.Split(“,”c) 
Select Isbn = parts(0), Title = parts(1), Publisher = parts(3)

While this code does make dealing with CSV easier, it would be nicer if we could refer to our columns as if they were properties where the property name came from the header row in the CSV file, perhaps using syntax like the following:


From line In MyCsvFile
Select line.Isbn, line.Title, line.Publisher

With strongly typed (compile time) structures, it is challenging to do this when dealing with variable data structures like CSV files. One of the big enhancements that is coming with .Net 4.0 is the inclusion of Dynamic language features, including the new DynamicObject data type. In the past, working with dynamic runtime structures, we were limited to using reflection tricks to access properties that didn't actually exist. The addition of dynamic language constructs offers better ways of dispatching the call request over dynamic types. Let's see what we need to do to expose a CSV row using the new dynamic features in Visual Studio 2010.

First, let's create an object that will represent each row that we are reading. This class will inherit from the new System.Dynamic.DynamicObject base class. This will set up the base functionality to handle the dynamic dispatching for us. All we need to do is add implementation to tell the object how to fetch values based on a supplied field name. We'll implement this by taking a string representing the current row. We'll split that based on the separator (a comma). We also supply a dictionary containing the field names and their index. Given these two pieces of information, we can override the TryGetMember and TrySetMember to Get and Set the property based on the field name:


Imports System.Dynamic

Public Class DynamicCsv
    Inherits DynamicObject

    Private _fieldIndex As Dictionary(Of String, Integer)
    Private _RowValues() As String

    Friend Sub New(ByVal currentRow As String,
                   ByVal fieldIndex As Dictionary(Of String, Integer))
        _RowValues = currentRow.Split(","c)
        _fieldIndex = fieldIndex
    End Sub

    Public Overrides Function TryGetMember(ByVal binder As GetMemberBinder,
                                           ByRef result As Object) As Boolean
        If _fieldIndex.ContainsKey(binder.Name) Then
            result = _RowValues(_fieldIndex(binder.Name))
            Return True
        End If
        Return False
    End Function

    Public Overrides Function TrySetMember(ByVal binder As SetMemberBinder,
                                           ByVal value As Object) As Boolean
        If _fieldIndex.ContainsKey(binder.Name) Then
            _RowValues(_fieldIndex(binder.Name)) = value.ToString
            Return True
        End If
        Return False
    End Function
End Class

With this in place, now we just need to add a class to handle iterating over the individual rows in our CSV file. As we pointed out in our book, using File.ReadAllLines can be a significant performance bottleneck for large files. Instead we will implement a custom Enumerator. In our customer enumerable, we initialize the process with the GetEnumerator method. This method opens the stream based on the supplied filename. It also sets up our dictionary of field names based on the values in the first row. Because we keep the stream open through the lifetime of this class, we implement IDisposable to clean up the stream.

As we iterate over the results calling MoveNext, we will read each subsequent row and create a DynamicCsv instance object. We return this row as an Object (Dynamic in C#) so that we will be able to consume it as a dynamic type in .Net 4.0. Here's the implementation:


Imports System.Collections

Public Class DynamicCsvEnumerator
    Implements IEnumerator(Of Object)
    Implements IEnumerable(Of Object)

    Private _FileStream As IO.TextReader
    Private _FieldNames As Dictionary(Of String, Integer)
    Private _CurrentRow As DynamicCsv
    Private _filename As String

    Public Sub New(ByVal fileName As String)
        _filename = fileName
    End Sub

    Public Function GetEnumerator() As IEnumerator(Of Object) _
        Implements IEnumerable(Of Object).GetEnumerator

        _FileStream = New IO.StreamReader(_filename)
        Dim headerRow = _FileStream.ReadLine
        Dim fields = headerRow.Split(","c)
        _FieldNames = New Dictionary(Of String, Integer)
        For i = 0 To fields.Length - 1
            _FieldNames.Add(GetSafeFieldName(fields(i)), i)
        Next
       
        Return Me
    End Function

    Function GetSafeFieldName(ByVal input As String) As String
        Return input.Replace(" ", "_")
    End Function

    Public Function GetEnumerator1() As IEnumerator Implements IEnumerable.GetEnumerator
        Return GetEnumerator()
    End Function

    Public ReadOnly Property Current As Object Implements IEnumerator(Of Object).Current
        Get
            Return _CurrentRow
        End Get
    End Property

    Public ReadOnly Property Current1 As Object Implements IEnumerator.Current
        Get
            Return Current
        End Get
    End Property

    Public Function MoveNext() As Boolean Implements IEnumerator.MoveNext
        Dim line = _FileStream.ReadLine
        If line IsNot Nothing AndAlso line.Length > 0 Then
            _CurrentRow = New DynamicCsv(line, _FieldNames)
            Return True
        Else
            Return False
        End If
    End Function

    Public Sub Reset() Implements IEnumerator.Reset
        _FileStream.Close()
        GetEnumerator()
    End Sub

#Region "IDisposable Support"
    Private disposedValue As Boolean ' To detect redundant calls

    ' IDisposable
    Protected Overridable Sub Dispose(ByVal disposing As Boolean)
        If Not Me.disposedValue Then
            If disposing Then
                _FileStream.Dispose()
            End If
            _CurrentRow = Nothing
        End If
        Me.disposedValue = True
    End Sub

    ' This code added by Visual Basic to correctly implement the disposable pattern.
    Public Sub Dispose() Implements IDisposable.Dispose
        Dispose(True)
        GC.SuppressFinalize(Me)
    End Sub
#End Region

End Class

Now that we have our custom enumerable, we can consume it using standard dot notation by turning Option Strict Off in Visual Basic or referencing it as a Dynamic type in C#:

VB:



Public Sub OpenCsv()
    Dim data = New DynamicCsvEnumerator("C:\temp\Customers.csv")
    For Each item In data
        TestContext.WriteLine(item.CompanyName & ": " & item.Contact_Name)
    Next

End Sub

C#:


[TestMethod]
public void OpenCsvSharp()
{
    var data = new DynamicCsvEnumerator(@"C:\temp\customers.csv");
    foreach (dynamic item in data)
    {
        TestContext.WriteLine(item.CompanyName + ": " + item.Contact_Name);
    }
}

In addition, since we are exposing this as an IEnumerable, we can use all of the same LINQ operators over our custom class:

VB:


Dim query = From c In data
            Where c.City = "London"
            Order By c.CompanyName
            Select c.Contact_Name, c.CompanyName

For Each item In query
    TestContext.WriteLine(item.CompanyName & ": " & item.Contact_Name)
Next

C#:


[TestMethod]
public void LinqCsvSharp()
{
    var data = new DynamicCsvEnumerator(@"C:\temp\customers.csv");
    var query = from dynamic c in data 
                where c.City == "London"
                orderby c.CompanyName
                select new { c.Contact_Name, c.CompanyName };

    foreach (var item in query)
    {
        TestContext.WriteLine(item.CompanyName + ": " + item.Contact_Name);
    }
}

Note: This sample makes a couple assumptions about the underlying data and implementation. First, we take an extra step to translate header strings that contain spaces to replace the space with an underscore. While including spaces is legal in the csv header, it isn't legal in VB to say: " MyObject.Some Property With Spaces". Thus we'll manage this by requiring the code to access this property as follows: "MyObject.Some_Property_With_Spaces".

Second, this implementation doesn't handle strings that contain commas. Typically fields in CSV files that contain commas are wrapped by quotes (subsequently quotes are likewise escaped by double quotes). This implementation does not account for either situation. I purposely did not incorporate those details in order to focus on the use of DynamicObject in this sample. I welcome enhancement suggestions to make this more robust.

Posted on - Comment
Categories: LINQ - VB Dev Center - VB - C# - Dynamic -

Using Cast Or OfType Methods

When working with generic lists, often you want to work with a more specific type than the list natively exposes. Consider the following where Lions, Tigers and Bears derive from Circus Animal:


  Dim items As New List(Of Object)
  items.Add(New Lion)
  items.Add(New Tiger)
  items.Add(New Bear)
  
  Dim res1 = items.Cast(Of CircusAnimal)() 
  Dim res2 = items.OfType(Of CircusAnimal)()

In this case, both res1 and res2 will return an enumerable of CircusAnimal objects rather than just returning Object types. However what would happen if you added something that wasn’t a CircusAnimal (an operation perfectly legal for the items list since it will take any Object:


  Dim items As New List(Of Object)
  items.Add(New Lion)
  items.Add(New Tiger)
  items.Add(New Bear)
  items.Add(“Elephant”)

  Dim res1 = items.Cast(Of CircusAnimal)() 
  Dim res2 = items.OfType(Of CircusAnimal)()

In this case, evaluating res1 would give an InvalidCastException when evaluating the newly added Elephant (string). The OfType method would return only the Lion, Tiger, and Bear objects casting them to CircusAnimal and skips the “Elephant”. What’s going on here? Under the covers, Cast performs a DirectCast operation as we iterate over each result. OfType performs an extra operation to see if the source object is of the correct type before trying to cast it. If the type doesn’t match, we skip that value. If code helps to visualize the difference, here’s an approximation of what’s happening under the covers (don’t bother to use Reflector here as it doesn’t know how to simplify the yield operation):


public static IEnumerable<T> Cast<T>(this IEnumerable source) {
  foreach (object obj in source)
    yield return (T)obj;

public static IEnumerable<T> OfType<T>(this IEnumerable source) {
  foreach (object obj in source) {
    if (obj is T) 
      yield return (T)obj;
   }
}

Note that the main difference here is the added check to see if the source object can be converted to the target type before trying to cast it. So which should you use and when? If you know that the objects you are working with are all the correct target type, you can use Cast. For example if you wanted to work with the CheckedListItem values in a CheckedListBox’s SelectedItems collection (which returns Object), you can be sure that all values returned are CheckedListItem instances.


Dim Query = From item in Me.Options.Items.Cast(Of LIstItem) _
            Where item.Selected _
            Select item.Value

If you want to work with diverse object types and only return a subset, OfType is a better option. For example, to disable all buttons on a form (but not the other controls), we could do something like this:


For Each button In Me.Controls.OfType(Of Button)
   button.Enabled = False
Next

If you want to be safe, you can always use OfType. Realize that it will be slightly slower and may ignore errors that you may want to actually know about otherwise.

Posted on - Comment
Categories: VB - C# - LINQ -

Watch language differences when parsing Expression Trees

I’ve run into a number of cases where people have built LINQ providers and only tested them with C# clients. In one case, I was sitting next to the person who wrote the provider when we discovered that using a VB client didn’t work with their provider because they failed to test it.

I’ve even seen it in the sample Entity Framework provider  available on the MSDN Code Gallery when parsing the .Include() method. Hopefully, you will be able to access my VB translation of the EF sample provider once it finishes going through the legal hoops.

So, why are these providers failing when run against a different language? Much of LINQ is based around language syntactic sugar. The various compiler teams added different optimizations when translating your LINQ code into real framework code. (Update: the VB Team explained the difference in a blog post from 2 years ago.) Let’s take a look at a simple case which tripped up at least two providers I tested recently. The query in question is perhaps the most common query when looking at LINQ samples:


From c In Customers _
Where c.City = "London" _
Select c

This seems to be a very straight forward example. Let’s take a look at what the compiler translates is query into.

C#:


Customers.Where (c => (c.City == "London"))

VB:


Customers.Where (Function(c) => (Operators.CompareString (c.City, "London", False) == 0))
         .Select ((Function(c) => c)

Notice here that the C# code does a literal translation into an equality operator. However, the VB team uses what turns out to be a slightly faster CompareString operator implementation. This doesn’t seem like too much of an issue until you consider the differences that this causes in the Expression tree that you need to parse in your provider. Using the Expression Tree Visualizer, you can see how much of a difference these two expressions cause under the covers. In this case, the C# version is on the left and VB is on the right.

image

Notice that both versions start with the same constant expression and the right expression constant of the Binary Expression is the same in both cases. However, the remainder of the expression is different between the two source examples.

The moral of the story: If creating a framework or API which will be consumed by multiple languages, make sure to include tests in each of those languages.

Posted on - Comment
Categories: C# - VB - LINQ -

Iterators OR Excuse me waiter theres a goto in my C sharp

At Codestock '09, I gave my LINQ Internals talk and had a number of people express shock when I showed the underlying implementation of their beloved iterators when looking at the code through Reflector. Let's look first at the C# that we wrote. This is similar to the implementation of LINQ to Object's Where method as shown in the sequence.cs file that's part of the C# Samples.


public static IEnumerable Where(this QueryableString source, Func predicate)
{
   foreach (char curChar in source)
        if (predicate(curChar))
            yield return curChar;
}

C# Iterators aren't really first class citizens, but syntactic sugar around the actual implementation. The meat of the implementation occurs in a generated class that implements the actual MoveNext method as we foreach over the results. The results are much less pretty:


private bool MoveNext()
{
    bool CS$1$0000;
    try
    {
        switch (this.1__state)
        {
            case 0:
                break;

            case 2:
                goto Label_0087;

            default:
                goto Label_00A5;
        }
        this.1__state = -1;
        this.7__wrap2 = this.4__this.GetEnumerator();
        this.1__state = 1;
        while (this.7__wrap2.MoveNext())
        {
            this.<curString>5__1 = this.7__wrap2.Current;
            if (!this.predicate(this.<curString>5__1))
            {
                continue;
            }
            this.2__current = this.<curString>5__1;
            this.1__state = 2;
            return true;
        Label_0087:
            this.1__state = 1;
        }
        this.m__Finally4();
    Label_00A5:
        CS$1$0000 = false;
    }
    fault
    {
        this.System.IDisposable.Dispose();
    }
    return CS$1$0000;
}

As you can see, the iterator sets up a switch (Select Case) statement that checks to see where we are in the loop (using a state variable). Essentially this is a state machine. The first time through we set up the environment. As we iterate over the results, we call the predicate that was passed in. If the predicate evaluates as true, we exit out of the method returning true.

The next time we return to the MoveNext, we use goto Label_0087 to re-enter the loop and continue the iteration. It's at this point that the jaws dropped in my presentation. Yes, Virginia, there are "Goto's" in C#. Spaghetti code isn't limited to VB. It's this point in my presentation where I quipped that the reason why iterators aren't in VB yet is because we want to do them "Right". While this is partly a joke, there is a level of seriousness in the comment. If you want to dig deeper on iterators, I recommend the following for your reading pleasure (note, these are NOT for the faint of heart):

After reading these, I'm sure you will have a better understanding of why it is taking so long to get iterators in VB. In the mean time, you might also find Bill McCarthy's recent article on using Iterators in VB Now to be interesting.

Posted on - Comment
Categories: VB - C# - LINQ -

LINQ In Action Samples available in LINQPad

I've been asked for some time what I think about the LINQPad tool. For those of you unfamiliar with it, LINQPad is a small but powerful tool that allows you to test your LINQ queries along with other VB and C# code. With this tool, you insert your code snippets in the code window and run it directly. If you point it to a database connection, LINQPad will build the .DBML behind the scenes and let you access the generated classes just as you would inside visual studio. When you execute the code, you can see the results in a grid layout along with the corresponding Lambda expressions and generated SQL statements.

LINQPad is a nice tool from the author of C# 3.0 in a Nutshell, Joe Albahari. I've frequently told programmers interested in LINQ to try LINQPad to get up to speed and use it instead of SQL Management Studio for a week. After a week, see how difficult it is to go back to TSQL.

Joe recently added the ability to integrate with other sample sources and offered the opportunity to Fabrice, Steve, and myself to have the samples from LINQ in Action available in LINQPad. Although we have all of our book samples available already, adding them to LINQPad offered the advantage of making it easier for you to try the queries and change them to learn better. You can then save your queries and re-run them later.

So, how do you use these samples? First Download the LINQPad executable and run it. When you open it, you will see a Samples tab that has a link to "download more samples."

LinqPad1

Clicking on this link will bring you to a dialog listing the additional samples available. Currently, there is only one additional sample, so it should be easy to find the samples from LINQ in Action.

image

Once you download the samples, you will see the LINQ in Action samples appear in the samples list broken up by chapter including both the VB and C# versions of each sample. You can then run each sample independently as shown below:

image

We have currently included chapters 1-8 which covers LINQ to Objects and LINQ to SQL. We plan to integrate the LINQ to XML and other chapters as we have the time. I hope you take the opportunity to try out this free product and our new samples.  Let us know if you find them helpful.

Posted on - Comment
Categories: VB - LINQ - C# -

LINQ to SQL support for POCO

One of the strengths that LINQ to SQL has over the upcoming Entity Framework is its support for POCO, or Plain Old Class Objects. With LINQ to SQL, the framework doesn't require any particular base classes, interfaces or even reliance on the 3.5 framework for the resulting objects. I demonstrated this in the talk I did at the Teched Tweener weekend. Download the demo project to see this in action.

In this sample, I created two separate projects. The first class library project, I created only targeting the 2.0 framework. As a result the project can not use any LINQ specific techniques. This will also allow us to consume the resulting objects in projects that do not have access to the newer framework, or to all of the namespaces. This is particularly important in cases like Silverlight. To call attention to the differences in the projects, I declared the 2.0 project in C# and the LINQ enabled project in VB.

The 2.0 class library project consists of a single class file. This represents the Subject entity from the Linq In Action database.

namespace UnmappedClasses
{
    public class Subject
    {
        public Guid ID { get; set; }
        public string Name { get; set; }
        public string Description { get; set; }
    }

Notice here, there are no interfaces, base classes or custom attributes. Excluding the attributes is critical here because the standard <Table> and <Column> attributes reside in the System.Data.Linq.Mapping namespace which would not be supported in the 2.0 framework.

Admittedly, it consists of three auto-implemented properties. Auto-implemented properties are used for brevity here and are consumable by the .Net 2.0 Framework because it relies on compiler features rather than runtime features.

Because we can't allow the class structure to include the attributes, we can't use the LINQ to SQL designer classes or SQL Metal to generate our classes. We do need to have a way to indicate the mapping to our data store. Here is where the XML Mapping file comes in handy.

When instantiating the DataContext, we can either rely on the inline attributes, or an external mapping file. Luckily, the XML mapping file's structure is concise and very similar to the attributes that would have been applied to the class otherwise. The main difference we need to do is indicate the Type that is used for a given table since we are not directly annotating the class itself. The other difference you may notice is that I don't include the Storage attribute. While there is nothing to stop me from using that in a Mapping source, we can't identify the backing field when using auto-implemented properties.

<?xml version="1.0" encoding="utf-8"?>
<Database Name="lia" xmlns="http://schemas.microsoft.com/linqtosql/mapping/2007">
  <Table Name="dbo.Subject" Member="Subject">
    <Type Name="UnmappedClasses.Subject">
      <Column Name="ID" Member="ID"  DbType="UniqueIdentifier NOT NULL" IsPrimaryKey="true" />
      <Column Name="Name" Member="Name" DbType="VarChar(100) NOT NULL" CanBeNull="false" />
      <Column Name="Description" Member="Description" DbType="VarChar(200)" />
    </Type>
  </Table>
</Database> 

Now, with that out of the way, we can get to the LINQ portion of the work. Actually, that is quite easy. In our 3.5 enabled project, we will create a XmlMappingSource, pass it into the constructor of the DataContext and then fetch the object from this context as we would any other LINQ enabled class.

Dim map = XmlMappingSource.FromXml(XDocument.Load("C:\Projects\LINQ\AdvancedLinqToSql\WinformDemo\lia.map").ToString)
 Using dc As New DataContext(My.Settings.liaConnectionString, map)
    Me.SubjectBindingSource.DataSource = dc.GetTable(Of UnmappedClasses.Subject)()
 End Using  
 
This example happens to bind the results to a Winform object binding source, but you could expose it to ASP directly, through an encapsulation layer, like a repository pattern, or a service interface.
Posted on - Comment
Categories: C# - LINQ - VB -

LINQ enabled Personal Web Starter Kit in C Sharp

I love it when projects take a life of their own. A while back, I posted my LINQ enabled  Personal Web Starter Kit in VB and received several requests to provide a C# port. Thankfully, one brave soul stepped up and did the port for me. Thanks go to Stephen Murray for undertaking the challenge. As is often the case, one of the best ways to learn a technology is to use it.

If you're interested in this sample, you can check out the project at the MSDN code center. Specifically, you can access the original VB version or Stephen's C# Port. As always, let us know what you Thinq.

Posted on - Comment
Categories: C# - LINQ -

Consuming a DataReader with LINQ

Last night at the Atlanta MS Pros meeting, I gave the first my new talks on LINQ Migration Strategies. If you missed it, you can catch me at the Raleigh, NC and Huntsville, AL code camps. Baring that, check out the presentation slides. This talk focuses on how you can extend existing structures using the power of LINQ. Among other things, this includes LINQ to DataSets and the XML Bridge classes.

During the presentation, fellow MVP, Keith Rome asked a question that I couldn't let sit. Is it possible to us LINQ with a data reader? I answered that it should be possible if you combine the power of C# (sorry, VB doesn't have this yet) iterators with the concept of using the fields collection in Datasets. Essentially an untyped dataset is an array of arrays. The first array is consumed by LINQ through the iterator.

The challenge here is that LINQ works on anything that implements IEnumerable, but the DataReader doesn't implement that; at least not natively. Here's where the fun of Extension Methods comes to play. With a C# extension method, we can expose an IEnumerable pattern as we iterate over the rows that we read.

To create an extension method in C#, we create a static method in a static class. We then decorate the first parameter of the method with the "this" keyword to indicate that we are extending that type. In this sample, I wanted to expose the results as an IEnumerable<IDataRecord>, but I couldn't figure out how to get a handle on the current record to yield it as we are iterating. I did find that you can push a row's data into a object collection, so that's what I did in this example. I welcome other recommendations to keep things more strongly typed. Here's the extension method implementation.

public static class DataReaderExtension
{
    public static IEnumerable<Object[]> DataRecord(this System.Data.IDataReader source)
    {
        if (source == null)
            throw new ArgumentNullException("source");
 
        while (source.Read())
        {
            Object[] row = new Object[source.FieldCount];
            source.GetValues(row);
            yield return row;
        }
    }
}

With this extension method, we can now create a data reader and query it using LINQ to Objects as follows:

Using cn As New System.Data.SqlClient.SqlConnection(MyConnectionString)
    cn.Open()
    Using cm = cn.CreateCommand
        cm.CommandType = Data.CommandType.Text
        cm.CommandText = "Select IsApproved, EnteredDate, Creator from Comments"
       
Using dr = cm.ExecuteReader
            Me.Listbox1.DataSource = _
                From row In dr.DataRecord _
                Where CBool(row(0)) _
                Order By CDate(row(1)) _
                Select CStr(row(2)) Distinct
           
Listbox1.DataBind()
        End Using
   
End Using
End Using

I am not happy about the excessive casting to and from object in this implementation. As a result of the extra casting, I suspect that it doesn't perform as well as more native implementations even though we are consuming a data reader, but I haven't had the chance to actually run performance comparisons on the alternatives. Alternative solutions are welcome.

Posted on - Comment
Categories: C# - LINQ - VB -

LINQ to SQL Compiled Queries

As LINQ nears release, people are starting to consider the performance implications that the extra overhead brings. Currently there are two threads on this: thread1, thread2. For those that are intested in the performance implications, I highly recommend checking out the outstanding series of posts by Rico Mariani.

Ultimately if you want to get the best performance, you need to use the Compile function of the CompiledQuery class. Let's consider the following query (Note, turn this into C# by adding a semi-colon at the end if necessary):

From p In context.Products() _
            Where (p.Price >= minimumPrice) _
            Select p)

In this query, we are searching for the products that are "Expensive" in that their price exceeds a price value that we set. If we regularly consume this, we can eliminate the overhead of building the query pipeline. See the Matt Warren talk about this pipeline on the deep dive video at on Charlie Calvert's blog for more information regarding the overhead that is necessary to evaluate a query.

To compile the query, we can leverage the CompiledQuery.Compile method. This method takes an argument list as input and a result type. It returns a delegate as a variable that we will be able to consume. Thus in this case, we can pass the DataContext instance and the minimumPrice variable in. We will return a IQueryable(of Product) object. thus we can start the definition of our compiled query as follows:

VB: 
     CompiledQuery.Compile( _
         Function(context As MyDataContext, minimumPrice As Decimal) _
            From p In context.Products() _
            Where (p.Price >= minimumPrice) _
                    Select p)

C#:
     CompiledQuery.Compile(
            (MyDataContext context, decimal minimumPrice) =>
                    from p in context.Products
                    where p.Price >= minimumPrice
                    select p);

In this case we are defining our query as a Lambda function which the Compile can convert to the corresponding expression tree representation. If you are unfamiliar with Lambdas in VB, check out Timothy Ng's recent MSDN Magazine article.

With the delegate functions declared, all that is left is to actually assign them to a variable that we can consume later. To do this, we define a static/shared function which returns the Func anonymous delegate type. By defining it as a static/shared function, the compilation will only occur once per AppDomain and will remain cached through the rest of the application's lifetime. Notice, we are defining the signature of the query, not the results. We are free to change the parameter without needing to re-compile the query's structure. Here's the completed function calls in VB and C#:

Public Shared ExpensiveProducts As Func(Of MyDataContext, Decimal, IQueryable(Of Product)) = _
       CompiledQuery.Compile(Function(context As MyDataContext, minimumPrice As Decimal) _
            From p In context.Products() _
            Where (p.Price >= minimumPrice) _
            Select p)

   public static Func<MyDataContext, decimal, IQueryable<Product>>
         ExpensiveProducts =
           
 CompiledQuery.Compile((MyDataContext context, decimal minimumPrice) =>
                    from p in context.Products
                    where p.Price >= minimumPrice
                    select p);

The syntax for using the compiled query takes a bit of thinking to get your head around. We are essentially creating a function which returns a function rather than returning a value. If you're familiar with functional programming such as OCaml and F#, this concept should be easy to grok. For the rest of us, piece out the individual method components as laid out above and you should be able to understand it after a while ;-)

To consume our new function, we simply instantiate a DataContext instance and pass it along with the minimumPrice we want to set as our minimum value.

  Dim context As New MyContext(My.Settings.MyConnectionString)
  Dim Items = ExpensiveProducts(context, minimumPrice)

Posted on - Comment
Categories: LINQ - VB - C# -

Auto Implemented Properties

I was looking over the help documentation included as part of the March Orcas CTP which I recently downloaded. I noticed that most of the documentation that came with the previous previews appears to have not made it into this release yet. If you want documentation and samples telling you what LINQ is all about, I still recommend grabbing the MAY CTP which you can find from my LINQ Links post a while back. If you prefer, soon you will be able to catch a book on LINQ I'm helping out on. Check out LinqInAction.com for updates.

I did notice one new feature I don't recall seeing elsewhere before in the C# LINQ overview offered in the integrated Orcas Help file called Auto-Implemented Properties. With Auto-Implemented Properties, you can create a private field and public property accessor with a single line of C# code like this:

public string Test { get; set; }

Typically when working in OOP environments, we have to choose between structures and classes to represent our data in memory. Structures have an advantage in that they require a bit less memory overhead and are more consise to type. However they suffer in their implementation due to issues with instantiation (which typically offsets the memory overhead) and marshalling. Structures are value types which are passed byRef where-as with objects, the pointer to the object is passed rather than the contents. Many people are caught offguard when passing a structure expecting the values to be updated and finding that they aren't. Objects typically take a bit more effort to type (private fields, public accessors, etc) and are slightly slower to instantiate. The benifits of classes easily outweigh those of structures in most cases.

C# 3.0 offers a new way to declare simple properties in classes to make them nearly as easy to create as properites in structures. Consider the following existing C# code snippet:

private string _Test1;
public string Test1
{
get { return _Test1; }
set { _Test1 = value; }
}

Here, we are creating a simple private field and exposing a public accessor for it. We aren't adding any business logic to the property, rather implementing a simple object persistance mecanism. In C# 3.5, we will now be able to replace the above code with a one-line statement:

public string Test { get; set }

Thanks to Lutz Roder's recent release of Reflector, we can now look under the covers of .Net 3.5 code. Here is what the compiler translates the one line of code above into:

// Fields
[CompilerGenerated]
private string <>k__AutomaticallyGeneratedPropertyField0;

// Properties
public string Test
{
[CompilerGenerated]
get
{
return this.<>k__AutomaticallyGeneratedPropertyField0;
}
[CompilerGenerated]
set
{
this.<>k__AutomaticallyGeneratedPropertyField0 = value;
}
}

In essense, the compiler is doing all of the hard work for use. It creates the expected accessors and adds an anonymous type for the private field, in this case called <>k__AutomaticallyGeneratedPropertyField0. Unfortunately, if we want to include any business logic, including change managment (implementing IChangeTracking or other such interfaces), we will need to do that ourself as the property setter does not include the code necessary to handle those notifications.

Posted on - Comment
Categories: C# -

Extension Methods are powerful and dangerous

Much of the querying functionality present in LINQ and the associated techologies leverage a new language feature known as Extension methods. With an extension method you can "extend" any type simply by adding the "this" keyword before the first parameter of the method definition. The LINQ samples primiarly extend IEnumerable<T>. However Extension methods can be used to extend any type.

Perhaps the following sample class can help explain the Extension method. In this example, I will simply extend the string data type with a new method that capitolizes the first letter of each word. I begin with a normal static method on a static class.

public static class MyExtension
{
public static string TitleCase(string source)
{
return new System.Globalization.CultureInfo("en-US").TextInfo.ToTitleCase(source);
}
}

The standard implementation would be as follows:

string test = "this is a test"
Console.WriteLine(MyExtension.TitleCase(test));

So far there is nothing new. To change the TitleCase method into an extension method, all we need to do is add "this" prior to the first parameter as follows:

public static string TitleCase(this string source)

As long as the MyExtension class is in scope, usage of the new extension method is as simple as the following:

string test = "this is a test"
Console.WriteLine(test.TitleCase());

In fact, you can shorten it to the following:

string test = "this is a test".TitleCase();

Here I have extended the String type with a method that it would not otherwise have. A word of warning is important here. Use Extension methods only after all other options have been exhausted. There are a number of reasons why an extension method is dangerous, including potential namespace collisions and increased difficulty maintaining code when a method is not directly present on a class. Extension methods can be powerful, but dangerous if mis-used.

Posted on - Comment
Categories: C# -