leonardo

Pythagorean triples

Thu, 01 May 2008 18:18:18 GMT

A little puzzle:
Given an array of n integers, find all Pythagorean triples in the array, that is, three elements such that a^2 + b^2 = c^2. Do this in O(n^2) time.

A simple solution is to test the equation with a hash of the squares on every unordered (a^2,b^2) pair (see the first Python version below).

First I have written a Python solution that uses a set, then I have translated it to D using my set module, then using D associative arrays, then I have translated the code to C adapting a simple hash code to integer keys. Then I have simplified the C code, removing the values from the hash.

That's the faster general solution, but the code can be improved if the problem is made less general. So I have tried to count the number of distinct Pythagorean triples among the integers [1,10_000].
So I have tried to simplify the hash some more. I have tried a really simple hash, and while counting the collision frequency I have found a very simple hash configuration where every number is present at its place in the array, or it's absent. Then as usual I have backported that code to Psyco and D.

Click to see the source code

# Short general Python version
def xpairs(seq):
    len_seq = len(seq)
    for i, e1 in enumerate(seq):
        for j in xrange(i+1, len_seq):
            yield e1, seq[j]

squares = [x*x for x in xrange(1, 10000)]
set_squares = set(squares)
print sum(1 for a,b in xpairs(squares) if a+b in set_squares)


// General D version
import std.conv: toInt;
import d.func: map, fromKeys, xrange, putr;

void main(string[] args) {
    int n = args.length ? toInt(args[1]) : 1_000;
    auto squares = map((int x){return x * x;}, xrange(1, n));
    auto squares_set = fromKeys(squares, 0);
    squares_set.rehash;

    int ntriples;
    foreach (i, aa; squares[0 .. $-1])
        foreach (bb; squares[i+1 .. $])
            if (aa + bb in squares_set)
                ntriples++;

    putr("Triples in [1, ", n, "): ", ntriples);
}


// C version
#include "stdlib.h"
#include "stdio.h"

#define N 10000
#define M 24593

int main() {
    int i, j;
    int* squares = malloc((N-1) * sizeof(int));
    int* squares_set = malloc(M * sizeof(int));

    for (i = 1; i < N; i++) {
        squares[i-1] = i * i;
        squares_set[(i * i) % M] = i * i;
    }

    int ntriples = 0;
    for (i = 0; i < N - 2; i++) {
        int aa = squares[i];
        for (j = i+1; j < N - 1; j++) {
            int aabb = aa + squares[j];
            if (aabb == squares_set[aabb % M])
                ntriples++;
        }
    }

    printf("%d\n", ntriples);
    return 0;
}


// D2 version
import d.func: map, xrange, putr;

void main() {
    auto xnumbers = xrange(1, 10_000);
    auto squares = map((int x){return x * x;}, xnumbers);

    auto squares_set = new int[24_593];
    foreach (sq; squares)
        squares_set[sq % squares_set.length] = sq;

    int ntriples;
    foreach (i, aa; squares[0 .. $-1])
        foreach (bb; squares[i+1 .. $])
            if ((aa + bb) == squares_set[(aa + bb) % squares_set.length])
                ntriples++;

    putr(ntriples);
}


# Psyco (Python) version:
from array import array

def main():
    N = 10000
    M = 24593
    squares = array("l", [x * x for x in xrange(1, N)])

    squares_set = array("l", [0]) * M
    for i in xrange(1, N):
        ii = i * i
        squares_set[ii % M] = ii

    ntriples = 0
    for i in xrange(N - 2):
        aa = squares[i]
        for j in xrange(i+1, N - 1):
            aabb = aa + squares[j]
            if squares_set[aabb % M] == aabb:
                ntriples += 1

    print ntriples

import psyco; psyco.full()
main()

Using MinGW 4.2.1 I have compiled that C code with:
gcc -O3 -s -march=pentiumpro -fprofile-generate triples.c -o triples
Followed by a run, followed by:
gcc -O3 -s -march=pentiumpro -fprofile-use triples.c -o triples

The general D version works with any group of input numbers (not tested below).

The D1 version is really close to the C version. The D2 version is faster and higher-level, it uses my d libs:
www.fantascienza.net/leonardo/so/libs_d.zip

The timing results (Pentium3 500 MHz):

Timings triples, N = 10_000:
  D general: 13.38 s
  Psyco:      6.89 s   (4.12 X)   (1.87 X)
  D1:         4.02 s   (2.40 X)   (1.09 X)
  D2:         3.68 s   (2.20 X)   (1    X)
  C:          1.67 s   (1    X)   (0.45 X)

Triples in [1, 10000): 12467
Triples in [1, 20000): 27171
Triples in [1, 40000): 58728
Triples in [1, 80000): 394138

The timings are quite good for Psyco but quite bad for D: the C-like D version is 2.4 times slower than the C version, and the Psyco version is just 1.7 times slower than the C-like D version (and the Psyco version is 1.9 times slower than the faster D version). This means that with this tiny program there's not that much purpose in using D instead of Psyco.

Jolly Jumpers

Tue, 29 Apr 2008 15:00:26 GMT

A little programming challenge, "Jolly Jumpers" in D:
http://mikael.jansson.be/journal/2008/03/jolly-jumpers-challenge

The version jolly() below comes from a C version by the Reddit user manannan (indents reduced to 2 spaces for LiveJournal).
The version jolly2() is for maniacs :-) It uses 1/8 of memory of jolly() using a bitmap with the bts compiler intrinsic, and alloca() to speed up memory allocation.

import std.math: abs, ceil;
import std.intrinsic: bts;
import std.c.stdlib: alloca;
import std.stdio: writefln;

bool jolly(int[] arr) {
  auto flags = new ubyte[arr.length - 1];
  foreach (i, el; arr[0 .. $-1]) {
    auto diff = abs(arr[i+1] - el) - 1;
    if (diff < 0 || diff > arr.length || flags[diff]++)
      return false;
  }
  return true;
}

bool jolly2(int[] arr) {
  int nuints = cast(int)ceil(arr.length / cast(float)(uint.sizeof * 8));
  uint[] flags = (cast(uint*)alloca(nuints * uint.sizeof))[0 .. nuints];
  if (flags.ptr is null)
  	throw new Exception("Stack overflow");
  flags[] = 0;
  foreach (i, el; arr[0 .. $-1]) {
    int diff = abs(arr[i+1] - el) - 1;
    if (diff < 0 || diff > arr.length || bts(flags.ptr, diff))
      return false;
  }
  return true;
}

void main() {
  foreach (a; [[1, 4, 2, 3], [1, 4, 2, -1, 6]])
    writefln(jolly(a), " ", jolly2(a));
}

Updates and few links

Fri, 18 Apr 2008 13:44:02 GMT

Slow updates because of limited net access.

The C language is designed for the hardware of the computer.
Python is designed for the programmer.
The D language is designed for both the designer of its compiler and for its compiler.

-----------------------

Few links:

Many free video courses, for science and engineering, there are an couple of them for Python too, with interesting PDF files, that show how to use Python in various scientific situations:
http://www.datawrangling.com/hidden-video-courses-in-math-science-and-engineering.html

Very nice, how to teach some Computer Science without computer to young children, some of the things shown are nice, like the sorting algorithms:
http://csunplugged.com/
But I don't know if it's an advantage for young children to learn sorting nets and other things. I think they have more important things to learn first, like reading complex texts, writing well, some mathematics, statistics, geometry, combinatorics, etc.

A really interesting parser+interepreter, PyMeta, it's a version of OMeta+Python:
http://washort.twistedmatrix.com/
OMeta is a part of a dream to create now foundations of computer programming, this is the third part of an article that discusses related matters:
http://www.moserware.com/2008/04/towards-moores-law-software-part-3-of-3.html

-----------------------

As soon as possible I'll update my D libs, there are many news.
New things:
- editDistance, editDistanceFast: they find the Levenshtein distance between two arrays (strings too).
- ALL: template that's true if the template predicate is true for all the items of the given tuple.
- MAP: template that maps a template on all the items of the given tuple and return the resulting tuple.
- BKtree: fast implementation of BK-trees to find nearby objects, given a distance function.
- polyCentroid: to find area and centroid coordinate of a given polygon.
- delVoidC, delVoidGC, NewVoidCArray, NewVoidCMatrix, NewVoidGCArray, NewVoidGCMatrix: high-performance functions to allocate/deallocate 1D/2D void arrays.
- xchain, Chainable: to chain any number of any kind of iterable, plus now all x-something lazy iterable classes support the ~ operator to chain them.
Changes:
- Removed boxarray and strarray functions because they are of little use.
- ExprType and xsplit simplified and improved.
- 'select' module renamed to 'ranking' to avoid any name clashing with func.select.
- Removed dependencies from 'meta' package, it's not included anymore.

-----------------------

I have developed the editDistanceFast/BKtree code in Python and D, and I have found that there are some advantages in developing in two (quite different) languages at a time.
- If during the development I put a bug in one of the two implementations, they may give different outputs, so I am able to see there's a bug.
- Optimizing the code for D allows me to focus to low level things. While if I program just in Python I seem not much able to "see" some of such details; and in this code I have found that every time I have invent a way to speed up the D code, the same trick was able to speed up the Psyco (the Python JIT) version too.
- Python allows me to quickly write a program that works, that is nearly bug-free. It's great for prototyping, while developing in D is slower. Later I can usually translate the code to D. Python allows me to write very short code too, and short programs are very useful, because they often hide less bugs and you can understand them better and faster. Later I may use my D libs to translate the Python code to a very short D code (using the high-order functions, etc). While if I program in D from the start I may miss spots where I can shorten the code with little performance loss.
- The short original Python code, plus the wonderful doctests allow me to set a starting point, to define what's the correct output of the code. Later I can translate the Python doctests to D unittests, and even later I can use the short and easy Python version to test if the D version is correct.
- So jumping between the two languages allows me to find many things that I may miss when I develop in just one language. The result is code that is readable, debugged, short, and fast :-)

As an example of the results of such language jumping, this is the D version of the editDistance, that's back-translated from Python, showing a quite short and readable code (I have created a pair of editDistance and editDistanceFast in both languages, the fast version has limits in the length of the possible input iterables/arrays, and avoid dynamic allocations):

int editDistance(T)(T[] s1, T[] s2) {
    if (len(s1) > len(s2)) { auto sa = s1; s1 = s2; s2 = sa; }
    auto r1 = range(len(s2) + 1);
    auto r2 = new int[len(r1)];
    foreach (i, c1; s1) {
        r2[0] = i + 1;
        foreach (j, c2; s2)
            r2[j+1] = c1 == c2 ? r1[j] : min(r2[j], r1[j], r1[j+1]) + 1;
        auto ra = r1; r1 = r2; r2 = ra;
    }
    return r1[$ - 1];
}

While the following Psyco code is essentially back-translated from the fast D version:

def editDistanceFast(s1, s2, r1=[0]*35, r2=[0]*35):
    if s1 == s2: return 0
    if len(s1) > len(s2):
        s1, s2 = s2, s1
    len_s2 = len(s2)
    assert len(s2) <= 34, "Error: one input sequence"\
           "is too much long (> 34), use editDistance()."
    for i in xrange(len_s2 + 1):
        r1[i] = i
        r2[i] = 0
    i = 0
    for c1 in s1:
        r2[0] = i + 1
        j = 0
        for c2 in s2:
            if c1 == c2:
                r2[j+1] = r1[j]
            else:
                a1 = r2[j]
                a2 = r1[j]
                a3 = r1[j+1]
                if a1 > a2:
                    if a2 > a3:
                        r2[j+1] = 1 + a3
                    else:
                        r2[j+1] = 1 + a2
                else:
                    if a1 > a3:
                        r2[j+1] = 1 + a3
                    else:
                        r2[j+1] = 1 + a1
            j += 1
        aux = r1; r1 = r2; r2 = aux
        i += 1
    return r1[len_s2]

There you can even see those r1=[0]*35 and r2=[0]*35 that are a way to implement a kind of the static arrays I have used in the fast D version. The end result is that the fast D version is just 4.5 times faster than the fast Psyco version.

You can find the Python version here:
http://www.fantascienza.net/leonardo/so/bktree.py

Links

Tue, 08 Apr 2008 18:17:25 GMT

How the JVM compiles Java code, it shows some quite interesting optimizazions:
http://weblogs.java.net/blog/kohsuke/archive/2008/03/deep_dive_into.html

Faster object message sending in Objective-C:
http://www.ridiculousfish.com/blog/archives/2005/08/01/objc_msgsend/

Nest functions, and ways to manage them (one of such ways, the delegate-based one is used by D, but there are two other ones):
http://ridiculousfish.com/blog/archives/2006/02/05/nest/

Slow D

Tue, 08 Apr 2008 01:25:42 GMT

Here I have collected few benchmarks where the D language, or the DMD compiler, or the Phobos std lib don't shine in their performance. Two of the following performance problems (see 'gc1' and 'sort' benchmarks) have already a solution.

http://www.fantascienza.net/leonardo/js/slow_d.zip

--------------------

Compilers/interpreters used:
Digital Mars D Compiler v1.028

gcc version 4.2.1-dw2 (mingw32-2)

Python 2.5.2 (r252:60911, Feb 21 2008, 13:11:45) [MSC v.1310 32 bit (Intel)] on win32

Psyco V.1.5.2 (JIT for Python)

javac 1.6.0_03, Java version "1.6.0_03"

All timings are "warm", best of 3, in seconds.
CPU used is a Pentium3 500 MHz, 256 MB RAM, with Win.

Compilation flags used (when not specified otherwise):
gcc: -O3 -s
dmd: -O -release -inline
Python: no flags.

-------------------

BENCHMARKS:

recursive, n = 38:
  C: 3.89 s (with __builtin_expect)
  C: 3.99 s
  D: 4.95 s


hash, n = 500_000:
  D:     7.07 s (7.15 s with GC patched)
  D:     5.89 s (GC disabled)
  Psyco: 4.23 s

(Note that Python+Psyco has much bigger overhead compared to D in any instruction, simple loops too).

wordcount, on a txt file of 13_312_768 bytes:
  C: 0.73 s
  D: 5.77 s


gc1, with 6.3 MB of text:
        loading   splitting   total
         time        time     time
  D:       1.87 s     3.58 s   13.76 s
  Python:  0.28 s     1.98 s    3.52 s
  Python:  0.1 s      2.0 s     3.38 s (load with 'rb', same result)
With a Patch to the D GC, plus disabling the GC:
  D:       0.08 s     0.72 s    1.36 s (GC patched + GC disabled after load)


gc2, n=1_000, m=10_000:
               seconds  MB
  DMD class:   18.95    1.7
  GDC class:   17.91    1.8
  DMD struct:  11.77    1.7
  GDC struct:  12.31    1.8
  Python:      37.10    3.1
  Psyco:       15.68    3.5
  Java:         2.19    7.3


gc3 (binarytrees), n = 15:
  Java:   9.12 s
  D:     35.01 s


sort, const n = 1_000_000:
  Random distribution:
    sort: 2.934
    fastSort: 0.881
  Already sorted arrays:
    sort: 1.723
    fastSort: 0.41
 Inverted order arrays:
    sort: 1.853
    fastSort: 0.651

(For a better and more complete version of fastSort see my D libs:
www.fantascienza.net/leonardo/so/libs_d.zip ).

Returning '(()) isn't difficult

Mon, 07 Apr 2008 14:19:42 GMT

A little amusing post about language syntax:
http://cadrlife.blogspot.com/2008/03/returning-considered-difficult.html

"Suppose you want a function to find all subsequences of a list with a certain length. For instance, the subsequences of (1...5) with length 2 are as follows."

(1 2) (1 3) (1 4) (1 5)
(2 3) (2 4) (2 5)
(3 4) (3 5)
(4 5)

In a lisp dialect:

(def subn (n li)
 (if (is 0 n) '(())
     (no li) '()
       (join (map [cons (car li) _] (subn (- n 1) (cdr li)))
             (subn n (cdr li)))))

Haskell:

subn 0 _      = [[]]
subn _ []     = []
subn n (x:xs) = map (x:) (subn (n-1) xs) ++ subn n xs

Python:

def subn(n, a):
    if not n: return [[]]
    if not a: return []
    return [[a[0]] + sub for sub in subn(n-1, a[1:])] + subn(n, a[1:])

The Scala version too uses pattern matching, showing how much it can be useful (it may be useful for D language too):

def subn[T](n: int, li : List[T]) : List[List[T]] = (n,li) match {
   case (0,_)     => List(Nil)
   case (_,Nil)   => Nil
   case (_,x::xs) => (subn(n-1, xs) map (x :: _)) ::: subn(n, xs)
}

D version using my libs:

import d.func: putr, map, not, range, xrange, select, BaseType, array;

T[][] subn(T)(int n, T[] a) {
    if (!n) return new T[][](1, 0);
    if (not(a)) return new T[][0];
    return map((T[] s){return a[0..1] ~ s;}, subn(n-1, a[1..$])) ~ subn(n, a[1..$]);
}

If you want shorter lines you can use select (reverted from the 'list' name):

T[][] subn2(T)(int n, T[] a) {
    if (!n) return new T[][](1, 0);
    if (not(a)) return new T[][0];
    T[] s;
    return select(a[0..1] ~ s, s, subn2(n-1, a[1..$])) ~ subn2(n, a[1..$]);
}

This version is able to digest any iterable:

BaseType!(TyIter)[][] subn3(TyIter)(int n, TyIter iter) {
    auto a = array(iter);
    alias BaseType!(TyIter) T;
    if (!n) return new T[][](1, 0);
    if (not(a)) return new T[][0];
    return map((T[] s){return a[0..1] ~ s;}, subn3(n-1, a[1..$])) ~ subn3(n, a[1..$]);
}

void main() {
    putr( subn(2, range(5)) );
    // Output: [[0, 1], [0, 2], [0, 3], [0, 4], [1, 2], [1, 3], [1, 4],
    //          [2, 3], [2, 4], [3, 4]]  

    putr( subn2(3, range(5)) );
    // [[0, 1, 2], [0, 1, 3], [0, 1, 4], [0, 2, 3], [0, 2, 4], [0, 3, 4],
    //  [1, 2, 3], [1, 2, 4], [1, 3, 4], [2, 3, 4]]

    putr( subn3(2, xrange(5)) );
    // Output: [[0, 1], [0, 2], [0, 3], [0, 4], [1, 2], [1, 3], [1, 4],
    //          [2, 3], [2, 4], [3, 4]]    
}

Self-recursive generators in D

Sat, 05 Apr 2008 17:44:54 GMT

I have (slightly) adapted the Python-like generators for D written by Witold Baryluk to my libs. They allow a simpler syntax that can be used for single iterations (not parallel scan allowed, you have to use some threads or light threads). So I have used them to translate to D some self-recursive generators from Python code by David Eppstein:

http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/221457

As an example this yields Fibbinary numbers (no consecutive ones in binary representation):
http://www.research.att.com/projects/OEIS?Anum=A003714

struct A003714 {
  void generator() {
    yield(1);
    foreach(x; A003714()) {
      yield(2 * x);
      if (!(x & 1))
        yield(2 * x + 1);
    }
  }
  mixin Generator!(int);
}

The original Python version:

def A003714():
  yield 1
  for x in A003714():
    yield 2 * x
    if not (x & 1):
      yield 2 * x + 1

The syntax isn't as clean as Python one, but it's simple enough anyway. They are 2-3 times slower than handwritten opApply struct methods, so you may want to use them only where code size/readability matters more.

import std.string: format;
import d.func: Tuple, putr, array, xslice, map;
import d.generators: Generator, IterBreak, IterLast;

/** Sequence of moves to solve Towers of Hanoi; has many other interpretations.
For more info see http://www.research.att.com/projects/OEIS?Anum=A001511 */
struct A001511 {
    void generator() {
        yield(1);
        foreach(x; A001511()) {
            yield(x + 1);
            yield(1);
        }
    }
    mixin Generator!(int);
}

/** Fibbinary numbers (no consecutive ones in binary representation).
For more info see http://www.research.att.com/projects/OEIS?Anum=A003714 */
struct A003714 {
    void generator() {
        yield(1);
        foreach(x; A003714()) {
            yield(2 * x);
            if (!(x & 1))
                yield(2 * x + 1);
        }
    }
    mixin Generator!(int);
}

/** Inverse Gray code; each n occurs at position n^(n>>1) of the sequence.
For more info see http://www.research.att.com/projects/OEIS?Anum=A006068 */
struct A006068 {
    void generator() {
        yield(0);
        foreach(x; A006068()) {
            if (x & 1) {
                yield(2 * x + 1);
                yield(2 * x);
            } else {
                if (x)
                    yield(2 * x);
                yield(2 * x + 1);
            }
        }
    }
    mixin Generator!(int);
}

/// Using usual D syntax. It's 2-3 times faster but the code is longer
struct A006068b {
    int opApply(int delegate(ref int) dg) {
        int result, aux;
        aux = 0; result = dg(aux); if (result) return result;
        foreach(x; A006068b()) {
            if (x & 1) {
                aux = 2 * x + 1; result = dg(aux); if (result) break;
                aux = 2 * x; result = dg(aux); if (result) break;
            } else {
                if (x)
                    { aux = 2 * x; result = dg(aux); if (result) break; }
                aux = 2 * x + 1; result = dg(aux); if (result) break;
            }
        }
        return result;
    }
}

/** Thue-Morse sequence; binary sequence with no triply-repeated block.
For more info see http://www.research.att.com/projects/OEIS?Anum=A010060 */
struct A010060 {
    void generator() {
        yield(0);
        int omit = 1;
        foreach(x; A010060()) {
            if (x) {
                yield(1);
                yield(0);
            } else {
                if (!omit)
                    yield(0);
                yield(1);
            }
            omit = 0;
        }
    }
    mixin Generator!(int);
}

/** Odious numbers: numbers with an odd number of ones in their binary representation.
For more info, see http://www.research.att.com/projects/OEIS?Anum=A000069 */
struct A000069 {
    void generator() {
        yield(1);
        foreach (x; A000069()) {
            if (x & 1 && x > 1)
                yield(2 * x - 1);
            yield(2 * x);
            if (!(x & 1))
                yield(2 * x + 3);
        }
    }
    mixin Generator!(int);
}

void main() {
    foreach(seq; Tuple!(A001511, A003714, A006068, A006068b, A010060, A000069))
        putr(array(xslice(seq(), 20)));
    putr();

    putr( map((int n){ return format("%b", n); }, xslice(A003714(), 30)) );
}

The whole output:

[1, 2, 1, 3, 1, 2, 1, 4, 1, 2, 1, 3, 1, 2, 1, 5, 1, 2, 1, 3]
[1, 2, 4, 5, 8, 9, 10, 16, 17, 18, 20, 21, 32, 33, 34, 36, 37, 40, 41, 42]
[0, 1, 3, 2, 7, 6, 4, 5, 15, 14, 12, 13, 8, 9, 11, 10, 31, 30, 28, 29]
[0, 1, 3, 2, 7, 6, 4, 5, 15, 14, 12, 13, 8, 9, 11, 10, 31, 30, 28, 29]
[0, 1, 1, 0, 1, 0, 0, 1, 1, 0, 0, 1, 0, 1, 1, 0, 1, 0, 0, 1]
[1, 2, 4, 7, 8, 11, 13, 14, 16, 19, 21, 22, 25, 26, 28, 31, 32, 35, 37, 38]

["1", "10", "100", "101", "1000", "1001", "1010", "10000", "10001", "10010", "10100", "10101", "100000", "100001", "100010", "100100", "100101", "101000", "101001", "101010", "1000000", "1000001", "1000010", "1000100", "1000101", "1001000", "1001001", "1001010", "1010000", "1010001"]

Euler problem 14

Sat, 05 Apr 2008 15:23:16 GMT

The Euler problem n.14:

The following iterative sequence is defined for the set of positive integers:
n -> n/2 (n is even)
n -> 3n + 1 (n is odd)
Using the rule above and starting with 13, we generate the following sequence of 10 terms:
13 -> 40 -> 20 -> 10 -> 5 -> 16 -> 8 -> 4 -> 2 -> 1
Although it has not been proved yet (Collatz Problem), it is thought that all starting numbers finish at 1. Which starting number, under one million, produces the longest chain of Collatz numbers?

My Python solution, that uses memoization of results to speed up computation:

import array

def main(m):
  lens = array.array('H', [0]) * m
  max_steps = max_steps_pos = 1
  for i in xrange(1, m):
    n = i
    n_steps = 1
    while n != 1:
      if n < i:
        n_steps += lens[n] - 1
        break
      n = 3 * n + 1 if n & 1 else n // 2
      n_steps += 1
    if n_steps > max_steps:
      max_steps = n_steps
      max_steps_pos = i
    lens[i] = n_steps
  return max_steps, max_steps_pos

import psyco; psyco.bind(main)
print "max_steps, max_steps_pos:", main(1000000)

Psyco is often able to manage array.array very efficiently.
'H' represents unsigned shorts [0 .. 2^16-1], because the chains are never too much long. This allows to reduce memory used and improve CPU cache efficiency (mostly in the D version). In Python you can use 'l' that is safer and doesn't slow down the code much, but uses twice the memory (so you can manage smaller problems).

The D version is essentially the same, the only significant difference is that the array isn't pre-initialized:

import std.stdio, std.c.stdlib;

void main() {
  const uint m = 1_000_000;
  auto lens = (cast(ushort*)malloc(m * ushort.sizeof))[0 .. m];
  uint max_steps = 1;
  uint max_steps_pos = 1;

  for (uint i = 1; i < m; i++) {
    uint n = i;
    uint n_steps = 1;

    while (n != 1) {
      if (n < i) {
        n_steps += lens[n] - 1;
        break;
      }
      n = n & 1 ? 3 * n + 1 : n / 2;
      n_steps++;
    }
    if (n_steps > max_steps) {
      max_steps = n_steps;
      max_steps_pos = i;
    }
    lens[i] = n_steps;
  }
  writefln("max_steps, max_steps_pos: ", max_steps, " ", max_steps_pos);
}


Timings:
m = 1_000_000:
D/C:    0.29 s  ( 1   X)
Psyco:  1.60 s  ( 5.5 X) with array 'H'
Python: 19.20 s (66   X) with list

A C version is very similar to that D version, and the running time (with GCC 4.2.1) is the same, but the executable is almost 20 times smaller.

The D code scans m = 100_000_000 in 25 s finding a chain of length 758 on n= 83_706_505.

An Haskell version:
http://www.haskell.org/haskellwiki/Euler_problems/11_to_20#Problem_14

To improve cache efficiency I have tried to store half of the values, but the code is slower (it allows to scan twice the values with the same max memory).

Links

Tue, 01 Apr 2008 21:35:56 GMT

Counting polyominos in Haskell, slow but very elegant code:
http://alaska-kamtchatka.blogspot.com/2007/11/counting-cattle.html

This wonderful article by Lockharts (I'd like it to be more widely read) teaches you what mathematics is, and why people like to develop/enjoy it. I agree with about 99% of the things he/she says:
http://www.maa.org/devlin/LockhartsLament.pdf

Easy intro to the pdb ("Python DeBugger"):
http://www.ferg.org/papers/debugging_in_python.html

This shows you some of the very nice things (useful too) you can do with Python generators (part of those things are possible with a similar number of lines of code in D with my D libs):
http://www.dabeaz.com/generators/Generators.pdf

Very nice regular expression generator, starting from text, I'd like to have this stand-alone instead of online:
http://www.txt2re.com/index-python.php3

Babbage Difference Engine in LEGO!
http://acarol.woz.org/

A nice "Quickhull" demonstration in JavaScript, sometimes nowdays Java applet are a less necessary:
http://infoecho.net/blogs/echo/archives/2007/03/13/230/

Look and say Sequence - 4

Fri, 28 Mar 2008 21:03:30 GMT

GCC supports gotos to variables addresses too, they can be useful in some special situations, like when you want to optimize an interpreter (and sometimes a finite state machine too, see below), you can find a short post about gcc-specific C extensions:
http://majek4.blogspot.com/2008/03/useful-c-extensions-gcc-specific.html

This is example code from that blog post:

#include "stdio.h"
#include "assert.h"

int main(int argc, char** argv) {
    int value = 2;
    if (argc == 2)
        value = atoi(argv[1]);
    assert(value >= 0 && value <= 2);

    const void *labels[] = {&&VAL0, &&VAL1, &&VAL2};

    goto *labels[value];
    assert(0);

    VAL0:
        puts("The value is 0");
        goto END;
    VAL1:
        puts("The value is 1");
        goto END;
    VAL2:
        puts("The value is 2");
        goto END;
    END:
        return 0;
}

Compiling that C code with MinGW (GCC) V.4.2.1 it produces the following assembly (two parts in the middle of the code):

...
LC1:
	.ascii "value >= 0 && value <= 2\0"
LC2:
	.ascii "The value is 0\0"
LC3:
	.ascii "The value is 1\0"
LC4:
	.ascii "The value is 2\0"
...

...
	call	__assert   ; corrisponde alla riga assert(0);
L6:
	movl	$L7, -20(%ebp)
	movl	$L8, -16(%ebp)
	movl	$L9, -12(%ebp)
	movl	-8(%ebp), %eax
	movl	-20(%ebp,%eax,4), %eax
	jmp	*%eax
L7:
	movl	$LC2, (%esp)
	call	_puts
	jmp	L10
L8:
	movl	$LC3, (%esp)
	call	_puts
	jmp	L10
L9:
	movl	$LC4, (%esp)
	call	_puts
L10:
	movl	$0, %eax
	addl	$36, %esp
	popl	%ecx
	popl	%ebp
	leal	-4(%ecx), %esp
	ret
...

Where the jmp *%eax is the computed goto.
Note that in ANSI C you can write the middle part of that code as:

switch (value) {
    case 0:
        puts("The value is 0");
        break;
    case 1:
        puts("The value is 1");
        break;
    case 2:
        puts("The value is 2");
        break;
}
return 0;

But in more complex situations it may help.

I have tried to use that kind of goto in the C version of the Audioactive series generator, and it speeds up the code about 3%. Inside the S0 state the following code:

switch (*p1++) {
    case '1': goto S1;
    case '2': goto S2;
    case '3': goto S3;
}

Is replaced by:

goto *labelsS123[*p1++ - '1'];

Where labelsS123 is:

const void *labelsS123[] = {&&S1, &&S2, &&S3};

Convex Hull in D

Fri, 28 Mar 2008 02:56:35 GMT

From Wikipedia I have found a link to simple C++ code to find the Convex Hull of points in 2D (about 795 lines with comments, and it crashes with n=10000 random integer 2D points):
http://www.microplop.com/assets_c/2007/11/convexhull.h
http://www.microplop.com/assets_c/2007/11/main.cpp

This is (nice) Python code written by David Eppstein (Mar 2002):
http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/117225

And the following templated function is its translation to D (about 14 lines without comments):

/** Returns the array of the points on the convex hull of a list of 2D
points in O(n log n) time. A point is a static array of length 2. */
T[2][] convexHull2D(T)(T[2][] points) {
  // Andrew's monotone chain algorithm
  T[2][] U, L;
  // Return positive number if p-q-r are clockwise, neg if ccw, zero if colinear
  T orientation(T[2] p, T[2] q, T[2] r) {
    return (q[1]-p[1])*(r[0]-p[0]) - (q[0]-p[0])*(r[1]-p[1]);
  }
  foreach(p; points.sort) {
    while (U.length > 1 && orientation(U[$-2],U[$-1],p) <= 0)
      U.length = U.length - 1;
    while (L.length > 1 && orientation(L[$-2],L[$-1],p) >= 0)
      L.length = L.length - 1;
    U ~= p; L ~= p;
  }
  return U.length ? U[0 .. $-1] ~ L.reverse : U;
}

import std.stdio: writefln;
void main() {
  int[2][] points = [[2, 284], [12, 465], [339, 495], [641, 815], [171, 37],
                     [99, 694], [207, 633], [648, 855], [713, 300], [331, 563]];
  writefln(convexHull2D(points));
}

If you want to make it faster you can use my D libs and replace this line:

foreach(p; points.sort) {

With:

foreach(p; points.fastSort()) {

This convexHull2D() code isn't optimized for speed, but for small practical situations it's enough, it's able to find the convex hull of 1_000_000 float 2D points uniform in [0,1]^2 in 6.5 seconds (convex hull of 38 points) on a Pentium3 500 MHz.
On 200_000 2D points (double) it needs 1.5 seconds, while the same code in Python (a different dataset) needs 12.6 s and Psyco 5.2 s. This shows that in some situations D allows to write (templated) programs not much more complex than Python ones, despite running quite faster.

This page on Wikipedia shows another simpler way to speed up the code:
http://en.wikipedia.org/wiki/Convex_hull_algorithms

/>The idea is to quickly exclude many points that would not be part of the convex hull anyway. This method is based on the following idea. Find the two points with the lowest and highest x-coordinates, and the two points with the lowest and highest y-coordinates. (Each of these operations takes O(n).) These four points form a convex quadrilateral, and all points that lie in this quadrilateral (except for the four initially chosen vertices) are not part of the convex hull. Finding all of these points that lie in this quadrilateral is also O(n), and thus, the entire operation is O(n). Optionally, the points with smallest and largest sums of x- and y-coordinates as well as those with smallest and largest differences of x- and y-coordinates can also be added to the quadrilateral, thus forming an irregular convex octagon, whose insides can be safely discarded.<

Loading tables in D

Thu, 20 Mar 2008 00:56:05 GMT

In the Census 2000 U.S. Gazetteer Files page:
http://www.census.gov/geo/www/gazetteer/places2k.html

You can find zip codes and the coordinates:
http://www.census.gov/tiger/tms/gazetteer/zcta5.zip

The file looks like this, where ... are other fields, the last two ones are latitude and longitude:

AL35004 5-Digit ZCTA ... 33.606379 -86.502492
AL35005 5-Digit ZCTA ... 33.592585 -86.959690
AL35006 5-Digit ZCTA ... 33.451714 -87.239578
...

With Python you can read such data into a dict with a line of code:

zips = dict((l[2:7], (float(l[136:146]), float(l[147:157]))
            for l in file("zcta5.txt"))

In D language with my libs you can do something similar with little code:

import std.string: stripl;
import std.conv: toFloat;
import d.xio: xfile;
import d.func: Struct, toStruct, putr;

void main() {
  float[][string] zips;
  foreach(l; xfile("zcta5.txt"))
    zips[l[2..7]] = [toFloat(stripl(l[136..146])),
                     toFloat(stripl(l[147..157]))];
}

You can reduce memory used putting the two coordinates into a struct defined on the fly, instead of a dynamic array. Disabling the GC speeds up this code more:

... // imports
import std.gc: disable;

void main() {
  disable;
  Struct!(float, float)[string] zips;
  foreach(l; xfile("zcta5.txt"))
    zips[l[2..7]] = toStruct(toFloat(stripl(l[136..146])),
                             toFloat(stripl(l[147..157])) );
}

In theory you can reduce the memory necessary for the keys too (and speed up their usage), putting the zip code into a fixed len 5-char array (the first two chars are digits, they can be packed into a single byte, so you need only 4 bytes, that are managed very quickly by the CPU, but that's overkill):

... // imports

void main() {
  disable;
  struct Char5 { char[5] k; }
  Struct!(float, float)[Char5] zips;

 foreach(l; xfile("zcta5.txt")) {
   Char5 c5;
   c5.k[] = l[2..7];
   zips[c5] = toStruct(toFloat(stripl(l[136..146])),
                       toFloat(stripl(l[147..157])) );
  }
  char[5] czip;
  czip[0..5] = "76252"[0..5];
  putr(zips[czip]);
}

In practice this may be a little slower.

High-level Spelling Corrector in D

Mon, 17 Mar 2008 19:16:10 GMT

In the last blog posts I have shown low-level examples of coding in D, to simulate a finite state machine used to generate terms of the audioactive sequence.

And now an example of high-level style coding in D. The following is related to the article "How to Write a Spelling Corrector" by the famous Peter Norvig:
http://norvig.com/spell-correct.html

Python version (21 lines, not counting the caller last one):

import re, collections
from string import ascii_lowercase as alphabet

words = lambda text: re.findall(r'[a-z]+', text.lower())

def train(features):
    model = collections.defaultdict(lambda: 1)
    for f in features:
        model[f] += 1
    return model

nwords = train(words(file('big3.txt').read()))

def edits1(word):
    n = len(word)
    return set([word[0:i]+word[i+1:] for i in xrange(n)] +                     # deletion
               [word[0:i]+word[i+1]+word[i]+word[i+2:] for i in xrange(n-1)] + # transposition
               [word[0:i]+c+word[i+1:] for i in xrange(n) for c in alphabet] + # alteration
               [word[0:i]+c+word[i:] for i in xrange(n+1) for c in alphabet])  # insertion

def known_edits2(word):
    return set(e2 for e1 in edits1(word) for e2 in edits1(e1) if e2 in nwords)

known = lambda words: set(w for w in words if w in nwords)

def correct(word):
    candidates = known([word]) or known(edits1(word)) or known_edits2(word) or [word]
    return max(candidates, key=lambda w: nwords[w] if w in nwords else -1)

print correct('speling'), "\n", correct('korrecter')

D version using my libs (23 lines, not counting the caller last one):
www.fantascienza.net/leonardo/so/libs_d.zip

import std.string, std.regexp, std.file, d.all;

int[string] nwords;
static this() {
    foreach (w; RegExp("[a-z]+", "g").match(inToLower(cast(string)read("big3.txt"))))
        nwords[w]++;
}

Set!(string) edits1(string w) {
    int i, n = len(w);
    char c;
    return set(list(w[0..i] ~ w[i+1..$], i, xrange(n)) ~                     // deletion
               list(w[0..i] ~ w[i+1] ~ w[i] ~ w[i+2..$], i, xrange(n-1)) ~   // transposition
               list(w[0..i] ~ c ~ w[i+1..$], i, xrange(n), c, lowercase[]) ~ // alteration
               list(w[0..i] ~ c ~ w[i..$], i, xrange(n+1), c, lowercase[])); // insertion
}

Set!(string) knownEdits2(string word) {
    string s1, s2;
    return set(list(s2, s1, edits1(word), s2, edits1(s1), s2 in nwords));
}

Set!(string) known(T)(T words) { return set(words) & nwords; }

string correct(string w) {
    auto found = lazyOr(known([w]), known(edits1(w)), knownEdits2(w), set([w]));
    return max(found, (string wo){ return wo in nwords ? nwords[wo] : -1; });
}

void main() { putr( correct("speling"), \n, correct("korrecter")); }

This D program has found one performance bug (problem already fixed, fix not available yet) in the D garbage collector(s), one possible bug using AAs with the ?: operator, a bug in my libs, has made me add that lazyOr function to my libs, and it was fun to write :-)
I have created lazyOr() to write this program (but it will be useful elsewhere, it's similar to the Python 'or' operator), if you don't want to use lazyOr() you can define just this line:

T or(T)(T x, lazy T y) { return isTrue(x) ? x : y(); }

That you can use like this:

auto found = or(or(or(known([w]), known(edits1(w))), knownEdits2(w)), set([w]));

That "g" in the Phobos REgex API is ugly, that API needs some cleaning and improviments (maybe PCRE can just be plug into Phobos (with an improved API) intead of the current regex engine, this will allow to remove most regex bugs, improve its speed, work with a very standard re syntax, etc.).

Note that Python program has corner-case bugs, is fragile, it's quite slow, it's not a good example of programming, etc. Much better Python programs can be written for this purpose. Probably its only good side is to be short, so it can show in a short space the idea behind a spelling corrector. The D code shares most of the same problems. Such high-level style coding in D can be useful to weite prototypes too, that later if necessary can be translated to faster lower-level style D (or even assembly) code.

With the patch to the GC the D version runs a little faster than the Python version (about 7.5 seconds on my very old PC, probably a little more than 1.5 seconds on a more modern PC).

Note the "big3.txt" is just a large block of texts in English, it comes from this version:
http://norvig.com/big.txt
But I have removed non-ASCII chars (> 127).

Links

Sun, 16 Mar 2008 10:46:28 GMT

To edit source in collaboration:
http://collabedit.com/

Ten Emerging Technologies:
http://www.technologyreview.com/specialreports/specialreport.aspx?id=25
"Probabilistic Chips" will probably became really important in the future, as electronics becomes smaller and smaller.

Removing braces from C/C++ (they have some problems, but a better version can be created, probably using a proper C/D/C++ parser), I'd like to create something like this for the D language:
http://www.imitationpickles.org/pyplus/
http://blog.micropledge.com/2007/09/nobraces
http://micropledge.com/projects/nobraces
http://blogstatic.micropledge.com/2007/09/nobraces.ci.txt

Look and say Sequence - 3

Sun, 16 Mar 2008 10:46:01 GMT

I have removed a bug from the D/C code, and I have added a faster Python version. Psyco is able to speed it up *25* times, so it's just 3 times slower than the faster C version (that uses a different algorithm), and it uses essentially the same memory:
http://www.fantascienza.net/leonardo/js/audioactive.zip

Beside that Python version, I have tried to translate to Python the finite state automata version too, producing something like this:

p1 = p2 = 0
pend = len1
S0, S1, S2, S3, S11, S22, S33, END = xrange(8)
state = S0

while True:
    if state == S0:
        if p1 == pend: state = END
        elif s1[p1] == "1": p1 += 1; state = S1
        elif s1[p1] == "2": p1 += 1; state = S2
        elif s1[p1] == "3": p1 += 1; state = S3

    elif state == S1:
        if p1 == pend: s2[p2] = '1'; p2 += 1; s2[p2] = '1'; p2 += 1; state = END
        elif s1[p1] == "1": p1 += 1; state = S11
        elif s1[p1] == "2": p1 += 1; s2[p2] = '1'; p2 += 1; s2[p2] = '1'; p2 += 1; state = S2
        elif s1[p1] == "3": p1 += 1; s2[p2] = '1'; p2 += 1; s2[p2] = '1'; p2 += 1; state = S3
...

But it results a bit slower than the version (derived from the switc-based C/D version that I have later removed) that you can find in the audioactive zip.

"The Good Guest"

Wed, 12 Mar 2008 12:34:34 GMT

I have written another little short story, "The Good Guest":
http://www.fantascienza.net/leonardo/le/the_good_guest.htm

This is only the 4th story I write in English (while I have written many more in my language, but I have stopped writing few years ago, so in the meantime I have lost some of my writing skills).
My English is far from perfect, so I appreciate any suggestion or problem you may spot.
Goldfur is the inspiration of many of my ideas and literary world still, even after 10 years. And I owe hir a lot.

Tue, 11 Mar 2008 22:29:29 GMT

Similar to the usual paste sites, but this allows to run the code too (inside a sandbox, but with 40 MB RAM or more, and for some seconds, so it can run interesting programs too). It supports C++, C, D, Python, etc, in D it supports Tango too:
http://codepad.org/

Photo-realistic rendering keeps improving, this is an open source renderer that shows some stunning images:
http://www.luxrender.net/

"What makes Mathematics hard to learn" article on the OLPC site by the good Marvin Minsky, he talks mostly about children:
http://wiki.laptop.org/go/What_makes_Mathematics_hard_to_learn%3F

An interesting (new?) way to test programs, Test-Driven Completion (TDC) (beside using ShedSkin to find types of a dynamically typed program):
http://www.eigenclass.org/repos/rcodetools/head/README

Look and say Sequence - 2

Tue, 11 Mar 2008 02:44:41 GMT

I have updated the Look and say Sequence generators:
http://www.fantascienza.net/leonardo/js/audioactive.zip

See for information:
http://en.wikipedia.org/wiki/Look-and-say_sequence
http://www.research.att.com/~njas/sequences/A005150

This is a very simple sequence, but it can be generated in many different ways, and I have explored many different interesting things. First of all I have found my fastest Python/Psyco versions, looking among more than ten versions.

The faster Python versions use a kind of pattern matching, splitting the precedent term (seen as a string of digits) with groupby, in groups of equal digits, then replacing those patterns using a dictionary of patterns.

Once I have created the fastest Python/Psyco versions, I have tried to use D to generate the terms of the Look and say sequence. First of all I have used my D libs to produce programs as short as Python ones. Then I have tried to write longer but much faster programs.

A first quite fast D version of mine uses two nested switches, and appends the corresponding pair of digits:

foreach (c; last) {
    if (count == 0) {
        cchar = c;
        count++;
    } else {
        if (c == cchar)
            count++;
        else {
            switch (cchar) {
                case '1':
                    switch (count) {
                        case 1: r ~= "11"; break;
                        case 2: r ~= "21"; break;
                        case 3: r ~= "31"; break;
                    }
                    break;
                case '2':
                    switch (count) {
                        case 1: r ~= "12"; break;
                        case 2: r ~= "22"; break;
                        case 3: r ~= "32"; break;
                    }
                    break;
                case '3':
                    switch (count) {
                        case 1: r ~= "13"; break;
                        case 2: r ~= "23"; break;
                        case 3: r ~= "33"; break;
                    }
                    break;
            }
            cchar = c;
            count = 1;
        }
    }
}

But later I have kept improving that. I have found an expression that gives an upper bound of the number of digits of the successive term of the series, so I have used that to pre-allocate the whole term, and avoid array appends. Later I have used char pointer to speed up the code, and I have allocated memory from the non-GC heap. I have stated to use C too, because the GCC 4.2.1 compiler is a better optimizer than the DMD v.1.027 D compiler.

One strategy that leads to some of the faster programs is to turn the program into a finite state machine. With a bit of thinking I have found the automata and I have simplified it to just seven states:

S0:
    if (p1 == pend) goto END;
    switch(*p1++) {
        case '1': goto S1;
        case '2': goto S2;
        case '3': goto S3;
    }

S1:
    if (p1 == pend) { *p2++ = '1'; *p2++ = '1'; goto END; }
    switch (*p1++) {
        case '1': goto S11;
        case '2': *p2++ = '1'; *p2++ = '1'; goto S2;
        case '3': *p2++ = '1'; *p2++ = '1'; goto S3;
    }

S2:
    if (p1 == pend) { *p2++ = '1'; *p2++ = '2'; goto END; }
    switch (*p1++) {
        case '1': *p2++ = '1'; *p2++ = '2'; goto S1;
        case '2': goto S22;
        case '3': *p2++ = '1'; *p2++ = '2'; goto S3;
    }

S3:
    if (p1 == pend) { *p2++ = '1'; *p2++ = '3'; goto END; }
    switch (*p1++) {
        case '1': *p2++ = '1'; *p2++ = '3'; goto S1;
        case '2': *p2++ = '1'; *p2++ = '3'; goto S2;
        case '3': goto S33;
    }

S11:
    if (p1 == pend) { *p2++ = '2'; *p2++ = '1'; goto END; }
    switch (*p1++) {
        case '1': *p2++ = '3'; *p2++ = '1'; goto S0;
        case '2': *p2++ = '2'; *p2++ = '1'; goto S2;
        case '3': *p2++ = '2'; *p2++ = '1'; goto S3;
    }

S22:
    if (p1 == pend) { *p2++ = '2'; *p2++ = '2'; goto END; }
    switch (*p1++) {
        case '1': *p2++ = '2'; *p2++ = '2'; goto S1;
        case '2': *p2++ = '3'; *p2++ = '2'; goto S0;
        case '3': *p2++ = '2'; *p2++ = '2'; goto S3;
    }

S33:
    if (p1 == pend) { *p2++ = '2'; *p2++ = '3'; goto END; }
    switch (*p1++) {
        case '1': *p2++ = '2'; *p2++ = '3'; goto S1;
        case '2': *p2++ = '2'; *p2++ = '3'; goto S2;
        case '3': *p2++ = '3'; *p2++ = '3'; goto S0;
    }

END:
    s2.length = p2 - s2.ptr;
    s1 = s2;

Converting that to C the code is really fast. One of those blocks is few (3-6) assembly instructions (the assembly code is full of jumps). Later I have performed the allocation just once up front, for both strings.
In the C versions I have used the __builtin_expect() that allows me to tell the compiler that usually p1 != pend. The resulting versions are audioactive_seq2.d and audioactive_seq3.c, they are almost the same, but the C versions is faster: it's able to finds the 66th term in 7.1s (62_226_614 digits) on a Pentium3 @ 500MHz with 256MB RAM.
With that I have found the 72th term, that has 305_351_794 digits in few minutes. Now it's clear that this program is fast enough, but pure speed isn't enough, few minutes of running time show that a different solution must be found. Someone has found the 91th term, that has 47_032_657_188 digits, this is incredible.

So I have to reduce memory used. There are just 3 digits, so 2 bits/digits suffice. So 4 digits can be packed in a byte, but this makes the finite state machine really complex, probably too much complex for the time I can use to this problem. A simpler compromise is to use 1 byte to store 2 digits, and seeing that all term lengths are even, that simplifies code.

Later I have read that digits aren't equally represented, "1" is more common (frequencies: 50%, 32%, 18%). A simple way to encode the digits as bits is to use 0 for "1", 10 for "2", and 11 for "3". This leads to an efficient (Huffman) encoding of the symbols. It turns out that it's easy enough to modify the state machine to manage such bits, and D language offers compiler intrinsic to manage arrays of bits in a very simple way. Here are the first two states of the 7 state machine:

S0:
    if (p1 == pend) goto END;
    if (testbit(s1, p1++)) {
        if (testbit(s1, p1++)) {
            goto S3;
        } else {
            goto S2;
        }
    } else {
        goto S1;
    }

S1:
    if (p1 == pend) { writezero(s2, p2++); writezero(s2, p2++); goto END; }
    if (testbit(s1, p1++)) {
        if (testbit(s1, p1++)) {
            writezero(s2, p2++); writezero(s2, p2++); goto S3;
        } else {
            writezero(s2, p2++); writezero(s2, p2++); goto S2;
        }
    } else {
        goto S11;
    }
...

This allows to use about 1.5 bits/digit, it's almost the optimal encoding (for single digits. But the digits of the look and say sequence contain less than 100 different blocks of digits, using such blocks as macro-symbols, plus a conversion table, the next term of the sequence can be computed in very short time, using a tiny amount of memory. This may be the trick to allow to compute the len of the 91th term). The resulting code is in audioactive_seq4.d, it's about two times slower than audioactive_seq2.d, so it shows that despite all the speed of this computation is CPU-bound, it's not bound to the transfer rate from cache and CPU (because audioactive_seq4.d needs such transfer rate to be much lower, just about 18% of the precedent one).

This has allowed me to find the 77th term, with 1_149_440_192 digits, in few minutes. So the limiting factor is memory still, it's not computing time yet.

Probably there is a way to pack the two sequences (two adjacent terms of the series) more in memory, because they are scanned monotonically and almost in parallel. So most of the same memory can be shared for both sequences. This allows to use almost half the (RAM) memory. Both sequences must be scanned in the same forward direction because backwards memory scanning is slower (but this isn't a problem in Python. So in Python you can just perform an alternating scanning of a single array.array("c"). In Python too an automata can be used, and with the help of Psyco this may lead to a fast program, faster than the "fastest ones" of audioactive_seq.py. I have seen that often a good way to speed-optimize a Python program is to translate it to D/C, optimize that again and again, and then back-translating it to Psyco).

Another simple way to "solve" the memory problem is to write the term on disk (encoded with 1.5 bit/digit), then read it to compute the next one, etc. The reads and writes are sequential, so they are fast enough (a memory mapped file can be an easy solution).

Look and say Sequence

Fri, 07 Mar 2008 21:45:38 GMT

I have written the faster Python and D generators for the "look and say" integer sequence:
http://www.fantascienza.net/leonardo/js/audioactive.zip

See:
http://en.wikipedia.org/wiki/Look-and-say_sequence
http://www.research.att.com/~njas/sequences/A005150

The D version uses some low-level tricks to be able to find the 66th term (62_226_614 digits!) in 12.5 seconds on a Pentium3 at 500MHz with 256MB RAM (in D simulating a state machine with 40 states it may be possible to go a little faster).
The Python version is more than 100 times slower, but it's rather fast anyway.

The D version is much longer than the Python versions, but using my D libs you can create a D version with essentially the same size of the Python versions (updated Mar 8 2008), fast enough using an associative array:

import d.func;

string[] audioactive1(int n) {
  string[string] conv = ["11"[]: "21"[], "22": "22", "222": "32",
                         "333": "33", "1": "11", "111": "31", "33":
                         "23", "3": "13", "2": "12"];
  string[] r = ["1"];
  foreach (_; xrange(n - 1))
    r ~= map((string g){return conv[g];}, xgroupBy(r[$-1])).joinArr();
  return r;
}

// Or just even:

string[] audioactive2(int n) {
  string[] r = ["1"];
  foreach (_; xrange(n - 1))
    r ~= map((string g){return str(len(g))~g[0];}, xgroupBy(r[$-1])).joinArr();
  return r;
}

Beside being useful to write less code and reduce bugs in not-speed critical parts of the code, I haveseen such high level functions also allow to prototype code, that later can be optimized.

Another look at Python Vs my D libs

Fri, 07 Mar 2008 00:58:32 GMT

The last pages of this PDF document show how not program in Python (it looks like Java):
http://www.keithbraithwaite.demon.co.uk/professional/presentations/2003/ot/why_learn_python.pdf

This blog post too talks about it:
http://wordaligned.org/articles/why-python-programmers-should-learn-python

This is a Python version of mine:

def inverter(n):
    n2txt = "zero one two three four five six seven eight nine".split()
    txt2n = dict((digit, str(i)) for i, digit in enumerate(n2txt))
    if isinstance(n, basestring):
        return int("".join(txt2n[p] for p in n.split()))
    else:
        return " ".join(n2txt[int(d)] for d in str(n))

print inverter(752)
print inverter("seven five two")

This is it translated to D using my D libs (isn't it neat?):

import std.string, d.func, std.conv, d.string;

If!(IsString!(T), long, string) inverter(T)(T n) {
    auto n2txt = "zero one two three four five six seven eight nine".split();
    auto txt2n = dict(zip(n2txt, range(10)));
    static if (IsString!(n))
        return toInt(map((string p){return str(txt2n[p]);}, n.split()).join(""));
    else
        return map((char d){return n2txt[toInt(d~"")];}, str(n)).join(" ");
}

void main() {
    putr(inverter(752));
    putr(inverter("seven five two"));
}

As you can see the D version uses map() and various anonymous functions that clutter code making it not much readable, despite those many helpers. So in the end I am not going to use such D code, while that Python despite being a bit slow is almost acceptable.

Just to be complete, the following is a Python version that uses map instead of list comps/iterators (like people used to write Python before the introduction of list comps and iterators):

def inverter(n):
    n2txt = "zero one two three four five six seven eight nine".split()
    txt2n = dict(zip(n2txt, xrange(10)))
    if isinstance(n, basestring):
        return int("".join(map(lambda p: str(txt2n[p]), n.split())))
    else:
        return " ".join(map(lambda d: n2txt[int(d)], str(n)))

print inverter(752)
print inverter("seven five two")

Programming languages for bioinformatics

Thu, 06 Mar 2008 15:48:59 GMT

I have found this interesting suite of benchmarks:
http://www.bioinformatics.org/benchmark/

Related to this "A comparison of common programming languages used in bioinformatics" article:
http://www.biomedcentral.com/1471-2105/9/82

See this too:
http://hackmap.blogspot.com/2008/02/fast-python-with-shedskin.html

I have tried to repeat some of your benchmarks, but I can't repeat the 9+ GB file used for one of them.
So far I have tested only the "alignment.py" program, because it's the only with data available.

Few comments on that work:

1) For Python, for such 3 benchmarks the Psyco JustInTime compiler helps a lot. Just install it:
http://psyco.sourceforge.net/
and add the following line to the code:
import psyco; psyco.full()

2) I haven't run the program "parse.py", but it shows various inefficiencies:
- No Psyco
- heavy-loop processing code outside funtions. Just put that big while 1: inside a main() function will help.
- Binary file loading is faster than normal one, just read the file with "rb".
- Python files are iterables, so to scan a file you just need:
for line in file("somefile", "rb"): print line
- To take the next line you just need:
f = file("somefile", "rb"):
print f.next()
- You don't need to use string functions, Python strings have methods, so you can do "something".replace("some", "any")
- Sometimes in Python you can replace REs with string methods.
- This is a bad way to strip a line:
line = string.replace(line,'\n','')
This is better and probably faster:
line = line.rstrip()

3) I like the D language, so I have tried it:
http://www.digitalmars.com/d/1.0/index.html

My timings of the "alignment" program:
  D3:        0.96 s    41 MB    66 lines
  C:         1.05 s    41 MB   146 lines
  C++:       1.22 s    41 MB    87 lines
  D2:        1.95 s    54 MB    60 lines
  D1:        2.73 s    57 MB    58 lines
  Java:      3.72 s    53 MB    79 lines
  Psyco:    11.22 s   168 MB    64 lines
  Python2: 115.3  s  ~160 MB    63 lines

Notes:
3a) C is the C code compiled with:
-pipe -O3 -s -ffast-math -fomit-frame-pointer -funroll-loops -march=pentiumpro -fprofile-generate
-pipe -O3 -s -ffast-math -fomit-frame-pointer -funroll-loops -march=pentiumpro -fprofile-use
3b) D code (compiler V. 1.026) compiled with:
-O -release -inline
D1 is a direct translation of the Java version
D2: inverts the building of the strings, and reverses them at the end (appending at the start of an array is slow thing in any language)
D3: is like D4 but it doesn't pre-clear the allocated memory.

Finally, my D code can be found here:
http://www.fantascienza.net/leonardo/js/bio_bench.zip

Update Mar 7 2008: later I have found one version of the L Gene of the Hantaan virus:
http://beta.uniprot.org/uniprot/P23456.fasta

toUpper

Tue, 04 Mar 2008 18:51:57 GMT

Updated my D libs again, among the improvements I have added a fast toUpper/inToUpper that converts a UTF8 string to uppercase (in-place too, even), it uses the integer SIMD uppercase(string) on UTF-8 data v2 by Paul Hsieh:

uint x = *p;
p++;
uint b = 0x80808080U | x;
uint c =   b - 0x61616161U;
uint d = ~(b - 0x7b7b7b7bU);
uint e = (c & d) & (~x & 0x80808080U);
*raux = x - (e >> 2);

It's 2-3 times faster than the std.string.toupper of Phobos, expecially the inplace version.

Links

Sat, 01 Mar 2008 16:24:40 GMT

I have updated my D libs nearly daily. I have added xprimes() too, that yields prime numbers quickly (probably a little more than 1 second to find the first 1_000_000_000 primes), original C code by Achim Flammenkamp.

The best praise I have read for languages with mandatory indentation:
http://okasaki.blogspot.com/2008/02/in-praise-of-mandatory-indentation-for.html
It tells me that other languages too (like D) can enjoy sharing that with Python.

A very nice free game you will love, it's an example of the "games of the future", where the PC uses physical simulation to simulate a model of reality in an almost transparent way:
http://www.kloonigames.com/blog/games/crayon
http://www.kloonigames.com/download.php?file=crayon.zip
>You play with crayons and physics. The goal of the game is to move the red ball so that it collects the stars. You can cause the red ball to move by drawing physical objects. With left mouse button you can draw and with right you can remove objects.<

Lisp in a single Python expression:
http://thinkpython.blogspot.com/2008/02/lisp-in-single-expression.html

Fri, 22 Feb 2008 22:59:20 GMT

"Graphics Programming Black Book" by Michael Abrash, free pdf files:
http://www.byte.com/abrash/

2D simulation, very nice game (but the actual game crashes on my PC):
http://www.youtube.com/watch?v=0H5g9VS0ENM

All the chapters of a nice book about software and programming:
http://www.codinghorror.com/blog/archives/000346.html
Some articles are very nice, like this one:
http://www.shirky.com/writings/group_enemy.html

Links

Tue, 19 Feb 2008 20:30:15 GMT

Many examples of mathematical art, with many links to interesting sites:
http://www.bridgesmathart.org/art-exhibits/jmm08/

Some basic but well written ways to write slow programs (mostly from a C/C++ perspective):
http://www.artima.com/cppsource/how_to_go_slow.html

The article talks a lot about RAM-speed-bound problems, this little table can show where those problems come from:

Store:     Size:     Access cost (clocks):
Registers  64B       1
L1 cache   32KB      3
L2 cache   4MB       14
DRAM       4GB       200
Disk       500GB     15M

That incredible table comes from this, a rather good set of slides:
http://www.nwcpp.org/Downloads/2007/Machine_Architecture_-_NWCPP.pdf