BraneCloud Evolution

New Post: Interested in the design decision process for porting to .NET vs using IKVM

bstabile — Sun, 14 Apr 2013 16:12:50 GMT

Hi jperraud,

The conversion supplied here is meant to be an INITIAL port to native .NET.

It only makes sense for those (such as me) who will wish to CHANGE it in some significant way.

The primary goal for the baseline conversion was to make it functionally EQUIVALENT to the original ECJ. That is, it should be able to parse parameter files used by the original java version and produce similar (if not exact) results.

The main reason I didn't take the conversion any further than what has been posted here, is because the changes I am making in a private fork won't make sense for those targeting different domains. In the future, it is possible I may post more radical changes if the end result might be useful to a broad audience.

Some of the changes I am working on privately include...

Major refactoring to separate interfaces from plug-in algorithmic implementations and concrete types.
The widespread use of generics.
The use of WCF for distributed communications and interaction.
The use of WF for long-running process logic and state persistence.
The use of TPL for parallel and asynchronous processing (including Async-Await semantics).
Heavy usage of things like Enterprise Library for Logging, Instrumentation, Configuration, and so on.

On top of all of these fundamental changes, I have need for numerous domain-specific optimizations and capabilities that not everyone would particularly desire. Leaving the conversion in its most basic form, I felt, was the best way to provide a starting point for others who might have completely different objectives.

While IKVM is a great way to take advantage of java libraries that might be useful "as-is", it doesn't really address my own particular (and perhaps peculiar) requirements.

Regards,
Ben

New Post: Interested in the design decision process for porting to .NET vs using IKVM

jperraud — Thu, 28 Mar 2013 22:38:21 GMT

Hi,

Very interesting work. I wish Codeplex was better at flagging users with new projects of interest...

I am considering ways to reuse existing libraries (ECJ, JMetal, and others) from a C# metaheuristics framework of mine, so your project caught my attention. My framework is largely meant as a consistent set of pattern in C# and I have no intention nor time to reimplement the algorithms.

You mention in the project description that you do need a C# port, but some users may not. I'd be very interested in the design criteria you have and in particular whether you considered using IKVM and found out difficulties or missed on some C#/.NET idioms, hence the port to C#. I'd just like not to rediscover something you already found out.

Regards

Commented Issue: tmp_s.ToUpper() throws a NullReferenceException [15309]

bstabile — Tue, 09 Oct 2012 00:12:14 GMT

Evolve.cs Line 441 "else if (V_SEED_TIME.ToUpper().Equals(tmp_s.ToUpper()) || (tmp_s == null && auto))"
 
You have to flip the conditions of the "or" because when "tmp_s" is null it crashes with a NullReferenceException when trying to call ToUpper() of null,

Comments: ** Comment from web user: bstabile **

Aronmek,

Thanks for reporting this bug.

In "Alpha3" (under construction), many changes will be included, both to the original source ECJ repository code, and to this converted C# fork.

Now there is a "Null Guard" that throws an exception right away if the seed isn't set in the default parameters (or overridden in later params specifications, or set automatically). In other words, if the framework hasn't initialized this somewhere, it causes an "intentional" exception.

Alpha 3 is in the works, but I don't want to release anything to fix small conversion anomalies such as this. Rest assured, there are much bigger things to deal with as the fork takes on extensive .NET reconstruction, to exploit platform specific features.

Again, let me thank you for spotting the bug, but you'll be happy, I hope, to know that much bigger fish are being fried behind the scenes.

Regards,

bstabile

Source code checked in, #96133

Project Collection Service Accounts — Mon, 01 Oct 2012 21:40:31 GMT

Upgrade: New Version of LabDefaultTemplate.xaml. To upgrade your build definitions, please visit the following link: http://go.microsoft.com/fwlink/?LinkId=254563

Source code checked in, #96132

Project Collection Service Accounts — Mon, 01 Oct 2012 21:33:03 GMT

Checked in by server upgrade

Updated Wiki: Home

bstabile — Thu, 26 Jul 2012 03:05:07 GMT

Project Description

Industrial-Strength Evolutionary Computation for .NET 4.0

This is an independent port of Sean Luke's venerable ECJ project (GMU EC Lab) Version 20

NOTE: This conversion to C# cannot and will not in any way be supported by Sean or any other member of his team at GMU. Please do not even ask! It is incredible that software of this caliber is made available in open-source form, and that is more than enough support!

It is hard to imagine how many hours Sean and his team have put into this over the course of 13 years. But suffice it to say:
ECJ is an 800 lb Gorilla in the realm of Evolutionary Computation and Stochastic Optimization.

MILD WARNING: ECJ is used for some very serious business (and this C# version has, or will shortly have, all of the same capabilities). But you don't need to get scared off by that. A comprehensive collection of samples is provided with ECJ, and the documentation that you can browse on Sean's site covers all of the basics and well beyond. I highly recommend that newbies work with ECJ in JAVA before you decide if this converted fork makes sense for you. The need for tight native .NET platform integration is potentially uncalled for, and you must ask yourself, "Do I really Need This?". Chances are, you don't. But I personally really DO need this, and I’m happy to spare others with similar requirements all of the heavy lifting required to get it this far. Trust me, it is a DAUNTING challenge to preserve delicate algorithms when there are so many moving parts.

I have made all of the samples bundled in the original Java version equally accessible to those of you who need orientation in C#. Each sample is set up as a bare-bones console app that you can "fiddle" with to your heart's content. But also, and this is something that is NOT provided in the original software, there are over 500 unit tests that I wrote during the conversion process. This turns out to be quite a useful side-effect of going through the whole torturous exercise in the first place! These tests, although perhaps still a little incoherent in places (because they focus on specific conversion issues), make it even easier to dig deep without stumbling around in total bewilderment (well, you may still stumble around, but they do provide a bit of a handrail to lean on when you start feeling sick to your stomach). \'-)

Side Note: There are likely to wind up being several thousands of tests associated with this software. I only covered what was causing difficulty during the conversion process, and that is but a small fraction of the full coverage this software deserves. ("sheesh", as Sean is wont to say, I must be freaking crazy!)

If you are not familiar with the purpose of this software, you can purchase Sean's excellent book entitled The Essentials of Metaheuristics (or download the electronic version for free). You can also find numerous articles and academic papers on the website listed above, covering a variety of related topics. If you plan on doing really serious work in this field, you might also consider starting out with the early works of John Koza (Genetic Programming I, II, et al) and then move on through the ever-expanding body of research that grew from there. Or if you really get serious about this stuff, you will probably want to start all the way back with John Holland, who as many of you may already know, is generally considered the "father" of research into the field of Genetic Algorithms.

ECJ targets Java, because large scale evolutionary processes often need to interoperate seamlessly between workstations, which may or may not be hosting Windows, and back-end "heavy metal" hosting one of the UNIX/LINUX variants. Such environments are typical in many large enterprise and academic research settings, as you all well know (or know all too well!). If that kind of heterogeneous environment describes where YOU develop complex systems to solve difficult problems, then you will definitely want to trot on over to Sean's lab and familiarize yourself with the Java "source" code.

On the other hand..., if you develop your software for homogenous environments with Windows7 and Server 2003/2008/2008R2, then you may want to consider downloading the converted Open Source C# code hosted here to take full advantage of tight integration with the latest features provided by .NET 4.0.

Obviously I won't be able to deliver anywhere near the same level of support that a dedicated team can provide (almost none at all in fact). But if you are willing to work hard at it, or if you just want to experiment with the basics, everything you need to go the full distance is (or will shortly be) available in the download. I didn't leave anything out. But I should repeat the warning given above: some things have NOT YET been covered in the unit tests. I haven't had the time, for example, to test whether the Master / Slave communications are working (it uses simple sockets, so I'm not too worried about that, because I'll be migrating it to WCF4). Same goes for Island Exchange for speciation, coevolution, and maybe another thing or two. The ECJ team was actively revamping some of the code for a robotics exercise while I was still in the middle of the conversion. Those gaps in coverage will be closed in due course.

I'll also revisit the ECJ SVN repository in coming weeks to pick up any new bits that show up and integrate any bug fixes and significant changes.

The goal for this project is to try and deliver a conversion of the latest bits as they accrue in ECJ, while maintaining a robust and flexible platform on which to develop in C#, F#, IronPython, IronRuby, VB.NET and so on. I will also be keeping my eye out for other juicy plums that will integrate nicely with this software. In fact, ECJ has a sibling project for "Multi-Agent Simulation of Neighborhoods, or Networks (or something)" (aka MASON) that fits the bill perfectly ("sheesh", I must be out of my freaking mind!). And integration with Microsoft's Project Trident collaborative scientific research platform seems like a yes-braner. But there are also many other interesting possibilities. Stay tuned!

NOTE: This software is a central feature in a much larger development picture that is partially laid out in the Vision statement. Additional information will be made available in the Documentation.

BTW: BraneCloud is linked to a registered domain (net|com) for future things like name/schema resolution in the cloud. So the only thing I ask if you want to derive your own fork from this one is that you please use a different root namespace to avoid collisions. Other that that, you are free to do with this what you will, so long as it abides by the stated licensing terms:

ECJ License (AFL 3.0)
BraneCloud.Evolution.EC (Apache 2.0)

(Sean informs me that he may eventually be switching to the Apache license when he gets around to it.)

Updated Wiki: Home

bstabile — Thu, 26 Jul 2012 03:02:11 GMT

Project Description

Industrial-Strength Evolutionary Computation for .NET 4.0

This is an independent port of Sean Luke's venerable ECJ project (GMU EC Lab) Version 20

NOTE: This conversion to C# cannot and will not in any way be supported by Sean or any other member of his team at GMU. Please do not even ask! It is incredible that software of this caliber is made available in open-source form, and that is more than enough support!

It is hard to imagine how many hours Sean and his team have put into this over the course of 13 years. But suffice it to say:
ECJ is an 800 lb Gorilla in the realm of Evolutionary Computation and Stochastic Optimization.

MILD WARNING: ECJ is used for some very serious business (and this C# version has, or will shortly have, all of the same capabilities). But you don't need to get scared off by that. A comprehensive collection of samples is provided with ECJ, and the documentation that you can browse on Sean's site covers all of the basics and well beyond. I highly recommend that newbies work with ECJ in JAVA before you decide if this converted fork makes sense for you. The need for tight native .NET platform integration is potentially uncalled for, and you must ask yourself, "Do I really Need This?". Chances are, you don't. But I personally really DO need this, and I’m happy to spare others with similar requirements all of the heavy lifting required to get it this far. Trust me, it is a DAUNTING challenge to preserve delicate algorithms when there are so many moving parts.

I have made all of the samples bundled in the original Java version equally accessible to those of you who need orientation in C#. Each sample is set up as a bare-bones console app that you can "fiddle" with to your heart's content. But also, and this is something that is NOT provided in the original software, there are over 500 unit tests that I wrote during the conversion process. This turns out to be quite a useful side-effect of going through the whole torturous exercise in the first place! These tests, although perhaps still a little incoherent in places (because they focus on specific conversion issues), make it even easier to dig deep without stumbling around in total bewilderment (well, you may still stumble around, but they do provide a bit of a handrail to lean on when you start feeling sick to your stomach). \'-)

Side Note: There are likely to wind up being several thousands of tests associated with this software. I only covered what was causing difficulty during the conversion process, and that is but a small fraction of the full coverage this software deserves. ("sheesh", as Sean is wont to say, I must be freaking crazy!)

If you are not familiar with the purpose of this software, you can purchase Sean's excellent book entitled The Essentials of Metaheuristics (or download the electronic version for free). You can also find numerous articles and academic papers on the website listed above, covering a variety of related topics. If you plan on doing really serious work in this field, you might also consider starting out with the early works of John Koza (Genetic Programming I, II, et al) and then move on through the ever-expanding body of research that grew from there. Or if you really get serious about this stuff, you will probably want to start all the way back with John Holland, who as many of you may already know, is generally considered the "father" of research into the field of Genetic Algorithms.

ECJ targets Java, because large scale evolutionary processes often need to interoperate seamlessly between workstations, which may or may not be hosting Windows, and back-end "heavy metal" hosting one of the UNIX/LINUX variants. Such environments are typical in many large enterprise and academic research settings, as you all well know (or know all too well!). If that kind of heterogeneous environment describes where YOU develop complex systems to solve difficult problems, then you will definitely want to trot on over to Sean's lab and familiarize yourself with the Java "source" code.

On the other hand..., if you develop your software for homogenous environments with Windows7 and Server 2003/2008/2008R2, then you may want to consider downloading the converted Open Source C# code hosted here to take full advantage of tight integration with the latest features provided by .NET 4.0.

Obviously I won't be able to deliver anywhere near the same level of support that a dedicated team can provide (almost none at all in fact). But if you are willing to work hard at it, or if you just want to experiment with the basics, everything you need to go the full distance is (or will shortly be) available in the download. I didn't leave anything out. But I should repeat the warning given above: some things have NOT YET been covered in the unit tests. I haven't had the time, for example, to test whether the Master / Slave communications are working (it uses simple sockets, so I'm not too worried about that, because I'll be migrating it to WCF4). Same goes for Island Exchange for speciation, coevolution, and maybe another thing or two. The ECJ team was actively revamping some of the code for a robotics exercise while I was still in the middle of the conversion. Those gaps in coverage will be closed in due course.

I'll also revisit the ECJ SVN repository in coming weeks to pick up any new bits that show up and integrate any bug fixes and significant changes.

The goal for this project is to try and deliver a conversion of the latest bits as they accrue in ECJ, while maintaining a robust and flexible platform on which to develop in C#, F#, IronPython, IronRuby, VB.NET and so on. I will also be keeping my eye out for other juicy plums that will integrate nicely with this software. In fact, ECJ has a sibling project for "Multi-Agent Simulation of Neighborhoods, or Networks (or something)" (aka MASON) that fits the bill perfectly ("sheesh", I must be out of my freaking mind!). And integration with Microsoft's Project Trident collaborative scientific research platform seems like a yes-braner. But there are also many other interesting possibilities. Stay tuned!

ECJ License (AFL 3.0)
BraneCloud.Evolution.EC (Apache 2.0)

(Sean informs me that he may eventually be switching to the Apache license when he gets around to it.)

Created Issue: tmp_s.ToUpper() throws a NullReferenceException [15309]

aronmek — Mon, 18 Jun 2012 22:11:42 GMT

Evolve.cs Line 441 "else if (V_SEED_TIME.ToUpper().Equals(tmp_s.ToUpper()) || (tmp_s == null && auto))"

You have to flip the conditions of the "or" because when "tmp_s" is null it crashes with a NullReferenceException when trying to call ToUpper() of null,

Updated Wiki: Home

bstabile — Thu, 24 May 2012 18:55:41 GMT

Project Description

Industrial-Strength Evolutionary Computation for .NET 4.0

This is an independent port of Sean Luke's venerable ECJ project (GMU EC Lab) Version 20

NOTE: This conversion to C# cannot and will not in any way be supported by Sean or any other member of his team at GMU. Please do not even ask! It is incredible that software of this caliber is made available in open-source form, and that is more than enough support!

It is hard to imagine how many hours Sean and his team have put into this over the course of 13 years. But suffice it to say:
ECJ is an 800 lb Gorilla in the realm of Evolutionary Computation and Stochastic Optimization.

MILD WARNING: ECJ is used for some very serious business (and this C# version has, or will shortly have, all of the same capabilities). But you don't need to get scared off by that. A comprehensive collection of samples is provided with ECJ, and the documentation that you can browse on Sean's site covers all of the basics and well beyond. I highly recommend that newbies work with ECJ in JAVA before you decide if this converted fork makes sense for you. The need for tight native .NET platform integration is potentially uncalled for, and you must ask yourself, "Do I really Need This?". Chances are, you don't. But I personally really DO need this, and I’m happy to spare others with similar requirements all of the heavy lifting required to get it this far. Trust me, it is a DAUNTING challenge to preserve delicate algorithms when there are so many moving parts.

I have made all of the samples bundled in the original Java version equally accessible to those of you who need orientation in C#. Each sample is set up as a bare-bones console app that you can "fiddle" with to your heart's content. But also, and this is something that is NOT provided in the original software, there are over 500 unit tests that I wrote during the conversion process. This turns out to be quite a useful side-effect of going through the whole torturous exercise in the first place! These tests, although perhaps still a little incoherent in places (because they focus on specific conversion issues), make it even easier to dig deep without stumbling around in total bewilderment (well, you may still stumble around, but they do provide a bit of a handrail to lean on when you start feeling sick to your stomach). \'-)

Side Note: There are likely to wind up being several thousands of tests associated with this software. I only covered what was causing difficulty during the conversion process, and that is but a small fraction of the full coverage this software deserves. ("sheesh", as Sean is wont to say, I must be freaking crazy!)

If you are not familiar with the purpose of this software, you can purchase Sean's excellent book entitled The Essentials of Metaheuristics (or download the electronic version for free). You can also find numerous articles and academic papers on the website listed above, covering a variety of related topics. If you plan on doing really serious work in this field, you might also consider starting out with the early works of John Koza (Genetic Programming I, II, et al) and then move on through the ever-expanding body of research that grew from there. Or if you really get serious about this stuff, you will probably want to start all the way back with John Holland, who as many of you may already know, is generally considered the "father" of research into the field of Genetic Algorithms.

ECJ targets Java, because large scale evolutionary processes often need to interoperate seamlessly between workstations, which may or may not be hosting Windows, and back-end "heavy metal" hosting one of the UNIX/LINUX variants. Such environments are typical in many large enterprise and academic research settings, as you all well know (or know all too well!). If that kind of heterogeneous environment describes where YOU develop complex systems to solve difficult problems, then you will definitely want to trot on over to Sean's lab and familiarize yourself with the Java "source" code.

On the other hand..., if you develop your software for homogenous environments with Windows7 and Server 2003/2008/2008R2, then you may want to consider downloading the converted Open Source C# code hosted here to take full advantage of tight integration with the latest features provided by .NET 4.0.

Obviously I won't be able to deliver anywhere near the same level of support that a dedicated team can provide (almost none at all in fact). But if you are willing to work hard at it, or if you just want to experiment with the basics, everything you need to go the full distance is (or will shortly be) available in the download. I didn't leave anything out. But I should repeat the warning given above: some things have NOT YET been covered in the unit tests. I haven't had the time, for example, to test whether the Master / Slave communications are working (it uses simple sockets, so I'm not too worried about that, because I'll be migrating it to WCF4). Same goes for Island Exchange for speciation, coevolution, and maybe another thing or two. The ECJ team was actively revamping some of the code for a robotics exercise while I was still in the middle of the conversion. Those gaps in coverage will be closed in due course.

I'll also revisit the ECJ SVN repository in coming weeks to pick up any new bits that show up and integrate any bug fixes and significant changes.

The goal for this project is to try and deliver a conversion of the latest bits as they accrue in ECJ, while maintaining a robust and flexible platform on which to develop in C#, F#, IronPython, IronRuby, VB.NET and so on. I will also be keeping my eye out for other juicy plums that will integrate nicely with this software. In fact, ECJ has a sibling project for "Multi-Agent Simulation of Neighborhoods, or Networks (or something)" (aka MASON) that fits the bill perfectly ("sheesh", I must be out of my freaking mind!). And integration with Microsoft's Project Trident collaborative scientific research platform seems like a no-brainer. But there are also many other interesting possibilities. Stay tuned!

ECJ License (AFL 3.0)
BraneCloud.Evolution.EC (Apache 2.0)

(Sean informs me that he may eventually be switching to the Apache license when he gets around to it.)

Updated Release: BraneCloud.Evolution.EC.2.0.Alpha2 (Apr 10, 2012)

bstabile — Thu, 24 May 2012 18:50:12 GMT

Fix for GeneVectorIndividual - Setter for GenomeLength - Filling "genome" should have been filling "newGenome" when the new length is greater than the old length.

Bug Report filed with ECJ team.

NOTE: If you encounter initial build errors, simply build again. The errors are probably thrown because Visual Studio Multi-Core optimization is enabled.

BRS

Released: BraneCloud.Evolution.EC.2.0.Alpha2 (Apr 10, 2012)

Thu, 24 May 2012 18:50:12 GMT

Updated Release: BraneCloud.Evolution.EC.2.0.Alpha2 (Apr 10, 2012)

bstabile — Wed, 11 Apr 2012 01:24:01 GMT

Released: BraneCloud.Evolution.EC.2.0.Alpha2 (Apr 10, 2012)

Wed, 11 Apr 2012 01:24:01 GMT

Created Release: BraneCloud.Evolution.EC.2.0.Alpha2 (Apr 10, 2012)

bstabile — Wed, 11 Apr 2012 01:22:55 GMT

New Post: Bug Report

bstabile — Tue, 10 Apr 2012 17:05:06 GMT

Roey,

You are right! I've reported the bug to the ECJ team, and I'll try and post a new version here as soon as I have a smidgen of free time.

Thanks!

Ben

New Post: Bug Report

roey_nissim — Tue, 27 Mar 2012 06:22:03 GMT

I think I found a bug in GeneVectorIndividual.cs in GenomeLength setter the original code is:

set
 {
 var s = (GeneVectorSpecies)Species;
 var newGenome = new VectorGene[value];
 Array.Copy(genome, 0, newGenome, 0, genome.Length < newGenome.Length ? genome.Length : newGenome.Length);
 for (var x = genome.Length; x < newGenome.Length; x++)
 {
 if (genome[x] == null)
 {
 genome[x] = (VectorGene)s.GenePrototype.Clone(); // not reset
 }
 }
 genome = newGenome;
 }

The red lines are wrong since the new gene prototype clones should be added to newGenome and not genome the fix is:

set
 {
 var s = (GeneVectorSpecies)Species;
 var newGenome = new VectorGene[value];
 Array.Copy(genome, 0, newGenome, 0, genome.Length < newGenome.Length ? genome.Length : newGenome.Length);
 for (var x = genome.Length; x < newGenome.Length; x++)
 {
 if (newGenome[x] == null)
 {
 newGenome[x] = (VectorGene)s.GenePrototype.Clone(); // not reset
 }
 }
 genome = newGenome;
 }

This issue is raised only when trying to work with lists, in vectors there is not problem since the length is set and equal

Roey

New Post: Problem with NSGA

bstabile — Sun, 12 Feb 2012 06:37:06 GMT

Sorry Rogala,

I still do not understand the specific nature of your problem.

It is standard practice to let the problem in standard GA architectures consider "diversity". But this can be very specific to particular "domains". I cannot really help you without additional information.

There are downsides to enforcement related to "convergence" under some circumstances. You might want to look at the C++ implementation of "pure" GA algorithms in something like Mathew Wall's GALib (http://lancet.mit.edu/ga/). This is a very "traditional' treatment that is used in many commercial and research projects that may help you decide which way to proceed. (It includes, by the way, tree-based representation, as well as standard array-based representation, so you are not limited to a particular representation when you analyze the algorithms).

ECJ, and my C# variant of that, is capable of doing anything you want (provided you fully understand what you are trying to accomplish!).

I hope this general pointer helps.

As I originally stated: For a Masters Thesis, I don't feel as though I should be explicitly wrinting any custom code for you. This is something that you should strive to solve on your own, given the flexibility of the framework provided. \;-)

In all likelihood, you are smarter than I am! I am just a hack with some "Idiot Savant" talents that suit my proprietary needs quite nicely!

Best Regards,

Ben

New Post: Problem with NSGA

rogala — Mon, 06 Feb 2012 14:59:49 GMT

Hi,

Thank you for your reply, but my problem is different. I would like the genome, which is represented in an array of integer, contains diffrent value.

Przemek

New Post: Problem with NSGA

bstabile — Sun, 29 Jan 2012 21:44:57 GMT

rogala,

ALSO note that memory usage will probably NOT be a problem unless the scale of your problem is quite large. There could also be situations where one might be doing, say, "Island Exhange" for speciation. If you needed to ensure GLOBAL uniqueness, then that could put some pressure on you to use very compact representations. You would probably NOT want to serialize the actual trees across the network as .NET object graphs, but would instead rely on each Island to report on uniqueness for genomes that you pass over in string form. The built-in ability to represent trees as compact LISP-like "S" expressions, is very useful for such purposes.

In the extreme case, if you wanted GLOBAL uniqueness across all generations, and islands, and even runs... then you would have to think carefully about efficiency. This is also heavily dependent on what types of evolution you wish to support. If you are bringing the top 10 percent of your genomes into a new population, and you want them all to be cloned uniquely (with respect to all of the other genomes that have ever been evaluated anywhere), then you have some analysis and work to do.

In some cases, such as when I am trying to analyze the evolutionary process itself, I use a relational database to store all kinds of statistics, along with the representation of all genomes that were produced by the system in many cases. This is a lot of overhead for a non-trivial Problem. Using compression on a single S-Expression is probably not going to be very helpful. But compressing thousands of them in a single stream provides significant benefit.

Ben

New Post: Problem with NSGA

bstabile — Sun, 29 Jan 2012 20:24:13 GMT

rogala,

Also note that it is NOT GOOD ENOUGH to compare string hash codes. Different strings can return the same hash code. Therefore, you will probably run into memory difficulties for a very large number of genomes using the string comparison technique. It is possible to use certain compression techniques to alleviate this problem. You might want to examine the notion of "Suffix Sorting", or more specifically something like the "Burrows-Wheeler Transform" compression strategy to compactly represent uniqueness.

I have to warn you though, this gets into the realm of Information or "Knowledge" Theory. There may be other tree comparison techniques with which I am not (off the top of my head) familiar.

Ben