Dimitre 的个人资料XSLT: Riding the challen...日志列表网络 工具 帮助

日志


7月28日

Part 2: The Swiss Army Knife of Data Structures ... in C#

Update: In a personal communication with Eric Lippert it was revealed that, in his words:
 
"I worked up a full implementation as well but I decided that it was too complicated to post in the blog. What I was really trying to get across was that immutable data structures were possible and not that hard; a full-on finger tree implementation was a bit too much for that purpose."
 
I am happy to correct my post and acknowledge that a previous C# implementation did exist. Hopefully, Eric will publish his implementation.
So, the work discussed here is most probably the first published Finger Tree  implementation in C#.
 
 
In Part 1 I described my experience in implementing the Finger Tree data structure in C#.
 
Here is a comparison of the performance of some common operations on sequences of length 100,000 as implemented by the C# Finger Tree and the .NET 3.5 Enumerable extension methods (applied on a List<UInt32> object).
 
Disclaimer: This comparison was carried out on my 4-year old PC, having CPU speed of 3GHz and RAM of 2GB and running Windows XP.  Results will largely vary on other configurations, depending on CPU and memory speed, available memory, cache size, ..., etc. Another factor to consider is the time necessary for jitting the .NET IL code on initial method execution. However the observed ratios (speedup) should remain closely the same. Below is a table, summarizing the results:
 

Performance Comparison:
FSeq   vs.
 .NET Collection with 100,000 elements

IEnumerable<uint>         vs.                 FSeq

 

Method

.NET Av. Time

FSeq Av. time

FSeq SpeedUp

Concat()

5268 µs

56 µs

94.07

ElementAt(),

[ ]

1 µs

46 µs

---

Skip()

2022 µs

49 µs

41.27

Take()

1513 µs

50 µs

30.26

Reverse()

4859 µs

93 µs

52.25

 

 

 
As seen from this comparison, using a Finger Tree as a general sequence data structure in .NET may speed up sequence operations (roughly) 30 to 94 times (extreme cases to more than 100 times). The only exception is element access, and only in the case of an array (note that the List<T> class in .NET is actually an ArrayList). If the instance on which the Enumerable methods are called is of some other type, say LinkedList<T>, then the speedup for all operations would be even bigger. Because the tested methods of Enumerable are using deferred execution, it was necessary to add an additional operation that references an item located the end of the result-sequence. 
 
Another problem I had to deal with was how to test for "typical" or "average" operations. In other words, how to make the tests representative. I ended up with the decision to include in the Concat(), Insert(), Remove() and Substr() tests strings with a maximum variety of lengths. What in contrast I call extreme cases is when the sequences/strings to be inserted/concatenated/removed/extracted are all with the same maximum length (close to the length of the first collection/string, on which the operation is performed -- in this case close to 100,000)
 
  

 

Another interesting comparison can be made if a string is represented as a sequence of characters. As the table below shows, for long strings the potential speedup of using a Finger-tree-based string implementation is 3 to 5 times (extreme cases 5 to 25 times):
 

Performance Comparison:
FString   vs.
 .NET string with 100,000 elements

string               vs.            FString

 

Method

.NET Av. Time

FString Av. time

FSeq SpeedUp

Concat(),  +

655 µs

129µs

5.08

Insert()

653 µs

220µs

2.97

Remove()

 

176 µs

55 µs

3.20

Substring()

167 µs

54 µs

3.09

 

 
 
The source code of the performance tests is here. Take notice that in its present state this code doesn't produce any output at all. To see the timing results, run under the VS2008 Debugger and put breakpoints at suitable locations.
 
In the third part of my Finger-Tree series I will present a brief performance comparison between a Finger-Tree-based sequence and XPath sequences, as implemented in existing XSLT 2.0 processors.
7月20日

The Swiss Army Knife of Data Structures ... in C#

Actually, it is not a knife and is known under the name of Finger Tree.  Smile

 
Update: In a personal communication with Eric Lippert it was revealed that, in his words:
 
"I worked up a full implementation as well but I decided that it was too complicated to post in the blog. What I was really trying to get across was that immutable data structures were possible and not that hard; a full-on finger tree implementation was a bit too much for that purpose."
 
I am happy to correct my post and acknowledge that a previous C# implementation did exist. Hopefully, Eric will publish his implementation.
So, the work discussed here is most probably the first published Finger Tree  implementation in C#.

 

Background:
Created by Ralf Hinze and Ross Paterson in 2004, and based to a large extent on the work of Chris Okasaki on Implicit Recursive Slowdown and Catenable Double-Ended Queus, this data structure, to quote the abstract of the paper introducing Finger Trees, is:

"a functional representation of persistent sequences supporting access to the ends in amortized constant time, and concatenation and splitting in time logarithmic in the size of the smaller piece. Representations achieving these bounds have appeared previously, but 2-3 finger trees are much simpler, as are the operations on them. Further, by defining the split operation in a general form, we obtain a general purpose data structure that can serve as a sequence, priority queue, search tree, priority search queue and more."

Why the finger tree deserves to be called the Swiss knife of data structures can best be explained by again quoting the introduction of the paper:

"The operations one might expect from a sequence abstraction include adding and removing elements at both ends (the deque operations), concatenation, insertion and deletion at arbitrary points, finding an element satisfying some criterion, and splitting the sequence into subsequences based on some property. Many efficient functional implementations of subsets of these operations are known, but supporting more operations efficiently is difficult. The best known general implementations are very complex, and little used.
This paper introduces functional 2-3 finger trees, a general implementation that performs well, but is much simpler than previous implementations with similar bounds. The data structure and its many variations are simple enough that we are able to give a concise yet complete executable description using the functional programming language Haskell (Peyton Jones, 2003). The paper should be accessible to anyone with a basic knowledge of Haskell and its widely used extension to multiple-parameter type classes (Peyton Jones et al., 1997). Although the structure makes essential use of laziness, it is also suitable for strict languages that provide a lazy evaluation primitive.
"

Efficiency and universality are the two most attractive features of finger trees. Not less important is simplicity, as it allows easy understanding, straightforward implementation and uneventful maintenance.

Stacks support efficient access to the first item of a sequence only, queues and deques support efficient access to both ends, but not to an randomly-accessed item. Arrays allow extremely efficient O(1) access to any of their items, but are poor at inserting, removal, splitting and concatenation. Lists are poor (O(N)) at locating a randomly indexed item.

Remarkably, the finger tree is efficient with all these operations. One can use this single data structure for all these types of operations as opposed to having to use several types of data structures, each most efficient with only some operations.

Note also the words functional and persistent, which mean that the finger tree is an immutable data structure.

In .NET the IList<T> interface specifies a number of void methods, which change the list in-place (so the instance object is mutable). To implement an immutable operation one needs first to make a copy of the original structure (List<T>, LinkedList<T>, ..., etc). An achievement of .NET 3.5 and LINQ is that the set of new extension methods (of the Enumerable class) implement immutable operations.

In the year 2008, Finger Tree implementations have been known only in a few programming languages: in Haskell, in OCaml, and in Scala. At least this is what the popular search engines say.

What about a C# implementation? In February Eric Lippert had a post in his blog about finger trees. The C# code he provided does not implement all operations of a Finger Tree and probably this is the reason why this post is referred to by the Wikipedia only as "Example of 2-3 trees in C#", but not as an implementation of the Finger Tree data structure. Actually, he did have a complete implementation at that time (see the Update at the start of this post), but desided not to publish it.

My modest contribution is what I believe to be the first published complete C# implementation of the Finger Tree data structure as originally defined in the paper by Hinze and Paterson (only a few exercises have not been implemented).

Programming a Finger Tree in C# was as much fun as challenge. The finger tree structure is defined in an extremely generic way. At first I even was concerned that C# might not be sufficiently expressive to implement such rich genericity. It turned out that C# lived up to the challenge perfectly. Here is a small example of how the code uses multiple types and nested type constraints:

// Types:

//     U -- the type of Containers that can be split

//     T -- the type of elements in a container of type U

//     V -- the type of the Measure-value when an element is measured

public class Split<U, T, V>
   
where U : ISplittable<T, V>
       
where T : IMeasured<V>

{
// ..........................................................
}

Another challenge was to implement lazy evaluation (the .NET term for this is "deferred execution") for some of the methods. Again, C# was up to the challenge with its IEnumerable interface and the ease and finesse of using the "yield return" statement.

The net result: it was possible to write code like this:

public override IEnumerable<T> ToSequence()
{
     ViewL<T, M> lView = LeftView();
     yield return lView.head;

     foreach (T t in lView.ftTail.ToSequence())
          yield return t;
}

Another challenge, of course, was that one definitely needs to understand Hinze's and Ross' article before even trying to start the design of an implementation. While the text should be straightforward to anyone with some Haskell and functional programming experience, it requires a bit of concentration and some very basic understanding of fundamental algebraic concepts.  In the text of the article one will find a precise and simple definition of a Monoid. My first thought was that such academic knowledge would not really be necessary for a real-world programming task. Little did I know... It turned out that the Monoid plays a central role in the generic specification of objects that have a Measure.

I was thrilled to code my own version of a monoid in C#:

public class Monoid<T>

{

  T theZero;

  
  public delegate T monOp(T t1, T t2);

 

  public monOp theOp;

 

  public Monoid(T tZero, monOp aMonOp)

  {

    theZero = tZero;

 

    theOp = aMonOp;

  }

 

  public T zero

  {

    get

    {

      return theZero;

    }

  }

}

 

Without going into too-much details, here is how the correct Monoids are defined in suitable auxiliary classes to be used in defining a Random-Access Sequence, Priority Queue and Ordered Sequence:

 

public static class Size

{

    public static Monoid<uint> theMonoid =

         new Monoid<uint>(0, new Monoid<uint>.monOp(anAddOp));

       

    public static uint anAddOp(uint s1, uint s2)

    {

        return s1 + s2;

    }

}

 

 

public static class Prio

{

    public static Monoid<double> theMonoid =

         new Monoid<double>

            (double.NegativeInfinity,

             new Monoid<double>.monOp(aMaxOp)

             );

 

    public static double aMaxOp(double d1, double d2)

    {

        return (d1 > d2) ? d1 : d2;

    }

}

 

 

 

public class Key<T, V> where V : IComparable

{

    public delegate V getKey(T t);

       

    // maybe we shouldn't care for NoKey, as this is too theoretic

    public V NoKey;

 

    public getKey KeyAssign;

 

    public Key(V noKey, getKey KeyAssign)

    {

        this.KeyAssign = KeyAssign;

    }

}

 

public class KeyMonoid<T, V> where V : IComparable

{

    public Key<T, V> KeyObj;

 

    public Monoid<V> theMonoid;

 

    public V aNextKeyOp(V v1, V v2)

    {

        return (v2.CompareTo(KeyObj.NoKey) == 0) ? v1 : v2;

    }

 

    //constructor

    public KeyMonoid(Key<T, V> KeyObj)

    {

        this.KeyObj = KeyObj;

 

        this.theMonoid =

         new Monoid<V>(KeyObj.NoKey,

                       new Monoid<V>.monOp(aNextKeyOp)

                      );

    }

}

 

 

Yet another challenge was to be able to create methods dynamically, as currying was essentially used in the specification of finger trees with measures. Once again it was great to make use of the existing .NET 3.5 infrastructure. Below is my simple FP static class, which essentially uses the .NET 3.5 Func object and a lambda expression in order to implement currying:

 

public static class FP

{

  public static Func<Y, Z> Curry<X, Y, Z>
            (
this Func<X, Y, Z> func, X x)

  {

    return (y) => func(x, y);

  }

}

 

And here is a typical usage of the currying implemented above:

 

public T ElemAt(uint ind)

{
  return treeRep.Split

(new MPredicate<uint>

   (

      FP.Curry<uint, uint, bool>(theLessThanIMethod2, ind)

   ),

     0

      ).splitItem.Element;

}

 

Now, for everyone who have reached this point of my post, here is the link to the complete implementation.

Be reminded once again that .NET 3.5 is needed for a successful build.

In my next posts I will analyze the performance of this Finger Tree implementation and how it fares compared to existing implementations of sequential data structures as provided by different programming languages and environments.