Friday, July 22, 2011

Reconstructing Other Models Using Galacticus

One of the original goals of the Galacticus project was to be able to reproduce the behavior of other semi-analytic models - and to do so not just approximately, but precisely and algorithmically (i.e. using precisely the same algorithms as those models). Why would you want to do this? Read on......

Semi-analytic models are complex - Galacticus is around 100KLOC (kilo-lines of code), not a huge software package by commercial standards, but more than big enough to mean it takes a lot of work to maintain and to understand all of its behavior. Given this complexity it's impossible to fully define such a model by describing the algorithms in journal articles - the traditional way of communicating how a calculation was performed. If done well, a description of this form can probably get close to fully defining a model, but there will always be details that are missed.

The best, and perhaps only, way to fully define the model then is through the source code - which is why Galacticus is Free and Open Source Software. Want to know how a specific calculation is implemented? Download the source and go look.

This problem is crucial because the scientific method rests on the idea of reproducibility of results - if you can't reproduce my results, then they're probably wrong. You could argue that any model which so complex that you can't reproduce its results from the published description in a journal isn't really a robust physical model. But that's not the point! Whether you want to run my code to run your own calculations, or instead want to see precisely what I did so you can check whether your own implementation of the same calculation gives the same result, the ultimate, precise, complete definition of the algorithm is the source code itself (and, perhaps, the relevant programming language specificiation).

While Galacticus is Free and Open Source Software, other semi-analytic codes are not. But Galacticus is highly modular, so in principle we can implement the algorithms of those other models in Galacticus and it should give the same results that they do. This would be very useful for understanding differences between models, and allowing us to test the reproducibility of those results.

The problem of course, is that typically all we have to go on is the published journal articles describing those other models....... So, reconstructing them with precision might be impossible, and will certainly be very difficult. But, let's try anyway!

Specifically, I've been working on recreating the Baugh et al. (2005) model - this has some features which differed significantly from what was available in Galacticus, making it sufficiently challenging. I'd love to have a video montage to insert here, showing me coding while inspiring music plays..... But I don't, so you'll have to imagine it instead.

The bottom line is that it took about three weeks to get this to work - the biggest challenges were introducing the concept of "halo formation events" into Galacticus, and in precisely matching the adiabatic contraction algorithm (crucial since it determines the rotation speeds of galaxies, and star formation rates are proportional to the third power of this in the Baugh et al. model).  But, once those were done correctly, the results are very good. Here's a simple example showing, as a function of redshift, the mass density in galaxies and in stars in the original Baugh et al. model and in the reconstructed version using Galacticus.
You can see that the match is very good - although not perfect. There remain some differences - which can be traced to the fundamentally different way in which the two models handle timestepping - but overall, it works! Other quantities (e.g. sizes, luminosities etc.) of galaxies also look good.

So, while it's not easy, it is possible to reconstruct other models using Galacticus. Hopefully I'll repeat this process for some other published models - so that we can compare them on an equal footing. For now, if you want to run the Baugh et al. model using Galacticus, a suitable input parameter file is available here - works with revision 510 or later of Galacticus v0.9.0.

No comments:

Post a Comment