Setting up

Templating logic

Suppose we decide that only one gigabyte of memory is pushing it for the simulations where N is set to 300. We could simply change the memory parameter to “2g”, but as discussed in the introduction, this might not be ideal. It would be nice if we could set all the settings where N is less than 300 to run with 1 gigabyte of RAM, and those where N is 300 (and eventually greater) to run with 2 gigabytes of RAM. We can accomplish this using templating logic.

Some background

glurmo uses Golang’s templating engine, which is extremely powerful. While it is worth reading through the previously linked page to get a sense of what it is capable of, its most important capability for our purposes is setting the value of a certain templating parameter based on a condition.

Example

To see what this looks like in practice, check the slurm template in conditional-lm/.glurmo/slurm_template. You’ll notice that line 6 looks quite a bit different than it did previously:

#SBATCH --mem={{if lt (atoi .N) 300}}{{.low_memory}}{{else}}{{.high_memory}}{{end}}

There’s quite a bit going on here, so we’ll go over it in parts.

First, the broad structure on the right side of the equal sign is that of an if statement:

{{if condition}} {{.param_1}} {{else}} {{.param_2}} {{end}}

If the condition is true, this expression takes the value of param_1; otherwise, it takes the value of param_2.

The condition itself looks like this:

lt (atoi .N) 300

This highlights two strange things about Golang’s templating logic. The first is that rather than using the typical symbols (<, >, etc.), it uses functions: lt for less than, gt for greater than, and so on. You can find the full details here. The second is that functions are called as follows:

func-name arg1 arg2 ...

We can therefore see that this line is checking that the result of the expression (atoi .N) is less than 300. Finally, note that atoi is itself a function that converts a string to an integer.

Taken together, this line becomes much clearer: it is checking if the value of N is less than 300. If it is, it uses the low_memory parameter, which you can see in settings.json is “1g”. Otherwise, it sets the value to high_memory, which is “2g”.

Running setup

Despite all of this, setting up and running the simulation are exactly the same as before:

$ glurmo  -s .

Let’s take a moment to make sure it worked as we expected. Run the following command:

$ cat N_100/beta_1_1/slurm/slurm_0

While the exact output will vary depending on the path to your simulation study, the top part should look like this:

#!/bin/sh

#SBATCH --job-name=conditional_lm_100_1___0
#SBATCH --time=00:01:00
#SBATCH --mail-user=your.email@university.edu
#SBATCH --mail-type=END,FAIL
#SBATCH --mem=1g
...

Because N is 100 in this setting, we only request 1 gigabyte of data. Now let’s check a simulation with N = 300:

$ cat N_300/beta_1_1/slurm/slurm_0

The top of your output should look a bit different:

#!/bin/sh

#SBATCH --job-name=conditional_lm_300_1___0
#SBATCH --time=00:01:00
#SBATCH --mail-user=your.email@university.edu
#SBATCH --mail-type=END,FAIL
#SBATCH --mem=2g

As expected, for this simulation, we request 2 gigabytes of memory. It worked!