Run Time Issues

Okay, I should say I mean this not quite in the normal programmer sort of way. I am worried running a reasonable number of simulations is going to take weeks of computing time on my poor three year old 2.2 GHz AMD Turion 64 laptop.

Part of me (the part which knows the software is single thread mathematics) wonders if I should just go find some 3.x Ghz Pentium 4 box with just enough hard drive and RAM to install Xubuntu and R and then simply run it on there expecting (yes, with massive power consumption) the kind of speed increase which would justify it.

On the other hand, it would be even better to run it on about ten machines in parrallel and then move all the individual datasets to one file. Too bad I’m not in college anymore and ergo don’t have free access to ten desktops for a weekend at a time.

Not Happy with ABCIMA

ABCIMA with the working (and very nice BCIMA software) has failed to achieve. It was a 250 point total sample chain with five 50 point subchains.  The last three each stuck at a single value, all three of which weren’t anywhere near where they should have been. Rather odd, really, that a single BCIMA run should work well in tests and ABCIMA should work out poorly.

Right now the adaptation only takes the most recent subchain to do the adaptation to the proposal. I might try changing that to it using the entire existing chain.

Still can’t get images uploaded, so I’ll post it when I can.

Chaining Works Now

And I got my wish and the backchaining worked. Once again, once I get a chance to upload the images I’ll post them. Right now it looks really good for a backchain and ran pretty fast too.

Coming up next: a test of the ABCIMA code with the news BCIMA core.

Bugs in Reverse

As I noted, there were problems with backward coupling. Oh, it ran, but it wouldn’t couple. Going back over the .backstep code revealed a small bug. The stop condition I had originally coded was this

checkCondition <- (cauchy/(proposal*truncEff)) - (comp/inf$infimumK);
STOP <- if(checkCondition >= 0) {TRUE} else {FALSE};

but the cauchy variable is on a log scale, so it needed to be written this way:

checkCondition <- (exp(cauchy)/(proposal*truncEff)) - (comp/inf$infimumK);
STOP <- if(checkCondition >= 0) {TRUE} else {FALSE};

Maybe this will fix the issue with backward coupling. Of course, it could and I still never see it, so I’m hoping for a quick, successful coupling so I can at least know it works in theory.

Handling A Failure to Couple

I may have found a good way to handle the failure to couple. It works like this:

1. Check whether or not coupling ocoured.
2. If so, proceed as normal
3. If not, skip making the forward chain and return a value of NULL

This should work for one and only one reason (at least with respect to the ABCIMA code) and that is because in R, we can append a NULL to a matrix and not change the matrix. Therefore, when the ABCIMA code gets a NULL back for a particular backward coupling instance, it will simply ignore it and try again as if nothing has happened.

As for how to handle this in the large scale numerical experiments, I’ll write a quick if-else which will map NULL values for the BCIMA result to a pair of NA’s for the point estimate and then this will make the distance NA and I can then both (1) count how many times BCIMA failed and (2) ignore the failures in the three-way comparison.

Failing Gracefully

I added the code to test whether or not the coupling succeeded or failed. At this point, I also realize while yes, the code does work (just had one fail out on me) simply having it stop isn’t such a graceful failure. The .backstep function passes a success or failure variable and that’s a good thing, but the BCIMA (in order to work with the ABCIMA software and my eventual hopes of automated numerical experiments) needs to do something other than simply fail loudly (no matter how ideal this may ordinarily be).

What I need it to do, in some sense, is to go ahead and tell ABCIMA “Hey, I failed” and for ABCIMA to go “Okay, I’ll just try again with a different sample setup.” I’m sure there’s a way to work that out and still keep a simple control structure, so I’m investigating the break() function to see what I can see.

ABCIMA Results and Run Time Issues

Yesterday I did a trial run of the ABCIMA software which ran to completion. It also took forever to run. While the image indicates an okay estimate (the most recent BCIMA run was actually better . . . not sure why) If the results of the eventual set of numerical experiments don’t bear out significantly better estimates ABCIMA may end up a bit of a bust.

Thinking about that extra time, though, it makes sense. The algorithm works like this:

1. Take a starting point in the feasible region
2. chain backwards until coupling (right now there’s a 750 point hard stop, at which point that back chain has failed to converge)
3. No go forward as many steps as we went backwards
4. Now generate a sample of specified size using this point as the start point.
5. Do this over and over again until the desired total sample size is reached.

In my trial run I did five fifty point samples for a total of 250 points. Each fifty point sample reqires that same up to 1500 points of back chain and burn in totallying up to 7500 non-sample points vice the mere max of 1500 for the BCIMA chain with the same parameters. Small wonder then that ABCIMA is so slow to run: it has to do five times as much work for the same result.

And once I get the image uploaded, I’ll post it here. Note, I did get the BCIMA image up.

Waiting For ABCIMA Test Results

I’m sitting here waiting for the last fifty datapoints of what could be my first successful ABCIMA analysis run ever to compute.

How do I know I’m down to the last fifty? There’s a little piece of legacy code that tells me when the backchain portion of the BCIMA process completed. So I know when I see that come up for the fifth time I’m close to realizing a dream I’ve had for over a year now.

Testing the ABCIMA Code

That’s exactly what’s happening right now.

Time

Turns out using the command Sys.time() it is possible to get a nice, intuitive computation of how long it takes for various pieces of code to run. This is getting added to the eventual list of things to do for the IMA, BCIMA, and ABCIMA code in order to facilitate some sort of breakdown of how long different things take to run.