LINQ and Related Topics

I love the fact that Microsoft makes it’s conference materials available for those unfortunate enough not to be able to attend. I also love watching the videos on my Zune. Even better is when I can use the Zune podcasting ability to download these videos. So far, I wasn’t able to find such a feed. Thankfully, fellow MVP, Bill McCarthy posted some quick LINQ to XML code to generate HTML tables based on the Xessions XML that was used for PDC. You can read his post here: http://msmvps.com/blogs/bill/archive/2010/11/03/pdc-2010-sessions.aspx.

To take this a step further, I modified his code to generate a quick RSS feed that I can use in the Zune software to download them as if they were a podcast. Here’s the revised code:

Dim doc = XDocument.Load("http://videoak.microsoftpdc.com/pdc_schedule/Schedule.xml")
        Response.Write(<?xml version="1.0" encoding="UTF-8"?>
                       <rss version="2.0">
                           <channel>
                               <title>PDC Videos</title>
                               <link>http://www.Microsoftpdc.com</link>
                               <description>Download content files for PDC 2010.</description>
                               <generator>LINQ</generator>
                               <%= From session In doc...<Session>
                                   From content In session...<Content>
                                   Where content.@Url.EndsWith("Low.wmv")
                                   Select <item>
                                              <title><%= session.<ShortTitle>.Value & " - " & content.@Title %></title>
                                              <link><%= session.@ShortUrl %></link>
                                              <enclosure url=<%= content.@Url %>/>
                                          </item> %>
                           </channel>
                       </rss>)

Note: This code does require VB10. If you want to do it with VB9, just add the line continuators (_).

One of the great things about the Reactive Extensions is that they allow you to express rather complex interactions simply. For this example, we’ll consider the mouse drag drop operation in Silverlight. Note: The identical code works in both the web based Silverlight and the Windows 7 phone. If you want to download the phone version of this code, it is available in C# or VB.

One of the indicators of simplicity is if you can explain a concept to your mother. How would you explain drag-drop to your mother?  It may go something like this:

• Record the start location when you press the mouse button down.
• While you are moving the mouse, record the end location until you let the mouse button up.
• Calculate the difference between the start and end locations and move the image you dragged accordingly.

Now, let’s see how we can do this in code. The beauty of the Reactive programming model is that we can compose multiple expressions together in a concise manner. Here’s how we accomplish the above process:

Dim q = From startLocation In mouseDown
        From endLocation In mouseMove.TakeUntil(mouseUp)
        Select New With {
            .X = endLocation.X - startLocation.X,
            .Y = endLocation.Y - startLocation.Y
        }

As you can see, this code reads nearly the same as the description you gave to your mother. It is actually a bit more concise than the slightly more verbose description.

At this point we have a couple details to wrap up. First, we need to declare the variables for mouseDown, mouseMove and mouseUp that we used above. We do this by using RX to subscribe to the events as discussed in this post.

Dim mouseDown = From evt In Observable.FromEvent(Of MouseButtonEventArgs)(image, "MouseLeftButtonDown")
                Select evt.EventArgs.GetPosition(image)
Dim mouseUp = Observable.FromEvent(Of MouseButtonEventArgs)(Me, "MouseLeftButtonUp")
Dim mouseMove = From evt In Observable.FromEvent(Of MouseEventArgs)(Me, "MouseMove")
                Select evt.EventArgs.GetPosition(Me)

While we’re here, notice that we are grabbing the MouseLeftButtonDown event of the image. However, we track the mouseMove and mouseUp on the form itself. We could use the MouseMove and MouseLeftButtonUp events of the image, but if we try to move too fast, the time Silverlight takes to calculate that the mouse is moving on the image rather than the canvas can mean that your movement is detected off of the image before you’ve been able to move the image. Tracking on the form itself drastically increases performance and reduces the possibility that you will move off of the image too soon.

The last thing we need to do is to move the image to the new location. In this sample, we placed the image on a Canvas. We just need to use the distance we recorded in our query and subscribe to the observable with an action that moves the image:

q.Subscribe(Sub(value)
                Canvas.SetLeft(image, value.X)
                Canvas.SetTop(image, value.Y)
            End Sub)

If you want to see this code in action, the VB version is available in the WPF samples and WP7 samples on the download page. The C# sample is in the Silverlight RX samples and WP7 samples also on the download page.

For a while, those of us who love Visual Basic have been struggling to make sure that newly released platforms include support for VB. When platforms that cater to the hobbyist, such as the new Windows Phone 7 tools are introduced without support for VB, we are particularly saddened. Happily, the team worked hard to overcome this shortcoming and announced today availability of the Windows Phone 7 tools in Visual Basic using Silverlight. You can download the tools now.

In celebration of this opportunity, I have converted my RX samples over to VB and made them available on my downloads page. I’ll post explanations of each of the samples over the next few days. For now, feel free to download the samples and try them out for yourself. Here’s the list of samples that are included:

• Implementing Drag and Drop with Reactive Extensions
• Emulating the Accelerometer
• Bing Translation client (Note, you will need to supply your own AppID to use this sample)

As I present more about the Reactive Framework, I often get people asking where they can learn more about it. Here are some resources that I’ve found useful:

• Rx center on Dev Labs (Start here for the bits)
• Hands on Labs:
  • Rx .NET HOL C#
  • Rx .NET HOL VB
  • Rx JS HOL
• Rx Forum
• RxWiki (Samples and more)
• RX Videos on Channel 9
• Matt Podwysocki’s blog (particularly for Rx JS)

Of course there’s nothing that beats learning by doing, so go out and try the bits yourself. Don’t be surprised when you hit a wall, but that’s when the real learning starts, trying to figure out how to overcome the challenges.

Is there a resource that I’m missing here that’s helped you? I’m happy to add it.

When I started looking at the Reactive Framework, one of the first things I did was to try creating some of the same standard LINQ queries that I’ve used against LINQ to Objects:

        Dim unsorted = From s In Sensor
                       Where s.SensorType = "2"
                       Order By s.SensorValue
                       Select s

If you try this where Sensor is an IObservable rather than IEnumerable, you will find that the Order By clause generates the following compiler error in VB: Definition of Method OrderBy is not accessible in this context. C# generates a similar but different error: “Could not find an implementation of the query pattern for source type IObservable<T>. OrderBy not found.” Essentially, the compiler is telling you that there isn’t an extension method called OrderBy that extends IObservable. Did the reactive team make a mistake and forget to implement sorting? Far from it.

Let’s consider the uses of the standard query operators  over a data source where you don’t necessarily know when the source ends. “From” doesn’t really exist, it’s just a query holder for identifying the local variable name (s) used later in the query and the source of the data (Sensor).

With “Where”, we are filtering the results. We can filter results over an ongoing stream without needing to know when the stream will end. As a result, filtering isn’t much of an issue.

Similarly, “Select” simply takes the input type and transforms it into another type. This is commonly referred to as a Projection. Since projections work equally well over data streams, we are fine implementing that in Reactive.

Sorting on the other hand is a bit more problematic. Consider the case where we process the following values: 1, 2, 4, 3, 5. It’s not difficult to sort these values and return them. However, what would happen to our sort if the next value that was sent was 0? We would need to reevaluate the entire result set and inject our new value before the first value that came in. In dealing with continuous event streams, we have no way of knowing whether the next value we are going to receive will need to be inserted prior to other results.

As a result, we need to partition the sets of data we receive if we need to sort these values so that we can be assured of knowing when the last value is received from this set. The Reactive Framework supports a number of partitioning methods, including BufferWithTime, BufferWithCount, and BufferWithTimeOrCount. With these methods, we can partition our streams into pre-determined chunks based on a timespan, and/or item count. The result is a new stream of IObserverable objects that contain an IList of the original data type. In the case of our Sensors, we can partition our result sets as follows:

Dim segmented = Sensor.BufferWithTime(TimeSpan.FromSeconds(3))

This creates a variable of type IObservable(Of IList(Of SensorInfo)). If we wanted, we could then display the sorted values in the partitioned lists as follows:

  segmented.Subscribe(Sub(val) FilteredList.ItemsSource =
                                         From v In val
                                         Order By v.SensorValue)

As you can see, you CAN sort values using the Reactive Framework using partitioning schemes, but it doesn’t make as much sense over data streams as it does with IEnumerable data sources typically encountered with LINQ.