Block Runner

This is a notebook to show how to use the BMTOOL.SLURM module¶

By Gregory Glickert¶

First we will import the packages we need¶

In [2]:

from bmtool.SLURM import SimulationBlock, BlockRunner
from pathlib import Path
import shutil

from bmtool.SLURM import SimulationBlock, BlockRunner from pathlib import Path import shutil

Next we will set up the pathing the runner will use¶

In [4]:

output_name = "example_output_name" # can be any string
basePath = Path.cwd()  # Equivalent to os.getcwd() gets our current working directory

# Path to 'Run-Storage' directory this is just setting up where i want things stored so this would go ../Run-Storage/example_output_name dir
run_storage_dir = basePath.parent / 'Run-Storage'  
target_dir = run_storage_dir / output_name  # Path to 'Run-Storage/CurrentResults'
display(target_dir)

output_name = "example_output_name" # can be any string basePath = Path.cwd() # Equivalent to os.getcwd() gets our current working directory # Path to 'Run-Storage' directory this is just setting up where i want things stored so this would go ../Run-Storage/example_output_name dir run_storage_dir = basePath.parent / 'Run-Storage' target_dir = run_storage_dir / output_name # Path to 'Run-Storage/CurrentResults' display(target_dir)

PosixPath('/home/gjgpb9/cortex_modeling/bmtool/examples/Run-Storage/example')

You can also manually define the target path like this¶

In [5]:

from pathlib import PosixPath
manual_path = "gjgpb9/bmtool/examples/Run-Storage/example"

# Create a PosixPath object
target_dir = PosixPath(manual_path)
display(target_dir)

from pathlib import PosixPath manual_path = "gjgpb9/bmtool/examples/Run-Storage/example" # Create a PosixPath object target_dir = PosixPath(manual_path) display(target_dir)

PosixPath('gjgpb9/bmtool/examples/Run-Storage/example')

It is then good practice to check if that path already exists and if so delete it¶

In [ ]:

if target_dir.exists() and target_dir.is_dir():
    shutil.rmtree(target_dir)

if target_dir.exists() and target_dir.is_dir(): shutil.rmtree(target_dir)

Now we can define out simulation_cases. This is a dictionary with the key being the name of the case and the value is the command used to run that simulation. Below is an example of what that could look like.¶

In [ ]:

simulation_cases = {
    "baseline": "mpirun nrniv -mpi -python run_network.py simulation_config_baseline.json False",
    "short": "mpirun nrniv -mpi -python run_network.py simulation_config_short.json False",
    "long": "mpirun nrniv -mpi -python run_network.py simulation_config_long.json False"
}

simulation_cases = { "baseline": "mpirun nrniv -mpi -python run_network.py simulation_config_baseline.json False", "short": "mpirun nrniv -mpi -python run_network.py simulation_config_short.json False", "long": "mpirun nrniv -mpi -python run_network.py simulation_config_long.json False" }

Next we define our block parameters. These are parameters that will be used to allocate the SLURM resources. The below example will allocate resources for 4 hours on the partition named batch. The partition will be different depending on your system setup. The job will use 40 cores and 80G of memory. The output_base_dir will be the directory where the output of the simulation will be stored¶

In [ ]:

block_params = {
'time': '04:00:00',
'partition': 'batch',
'nodes': 1,
'ntasks': 40,
'mem': '80G',
'output_base_dir': target_dir
}

block_params = { 'time': '04:00:00', 'partition': 'batch', 'nodes': 1, 'ntasks': 40, 'mem': '80G', 'output_base_dir': target_dir }

There may also be some additional commands you to run in the script before running your model. An example could be loading modules. You can add those like this. If your model doesn't need any additional commands you can just not include this.¶

In [ ]:

additional_commands = [
    "module purge",
    "module load slurm",
    "module load cpu/0.17.3b",
    "module load gcc/10.2.0/npcyll4",
    "module load openmpi/4.1.1",
    "export HDF5_USE_FILE_LOCKING=FALSE"
]

additional_commands = [ "module purge", "module load slurm", "module load cpu/0.17.3b", "module load gcc/10.2.0/npcyll4", "module load openmpi/4.1.1", "export HDF5_USE_FILE_LOCKING=FALSE" ]

Next we set up the paths for changing our json file. The module supports changing any key in the json file. The component_path should be the relative path to the BMTK components directory. The json_file_path should be the path FROM the components folder to the json file you wish to edit.¶

In [ ]:

#full_path = 'components/synaptic_models/synapses_STP/FSI2LTS.json'
component_path = 'components'
json_file_path = 'synaptic_models/synapses_STP/FSI2LTS.json'
param_name = 'initW' # key in json
param_values = [3.1,5] # values in json

#full_path = 'components/synaptic_models/synapses_STP/FSI2LTS.json' component_path = 'components' json_file_path = 'synaptic_models/synapses_STP/FSI2LTS.json' param_name = 'initW' # key in json param_values = [3.1,5] # values in json

Now we set up the blocks.¶

In [ ]:

num_blocks = len(param_values)
# Create a list to hold the blocks
blocks = []
for i in range(1, num_blocks + 1):
    block_name = f'block{i}'
    block = SimulationBlock(block_name, **block_params, simulation_cases=simulation_cases,
                            additional_commands=additional_commands,component_path=component_path)
    blocks.append(block)

num_blocks = len(param_values) # Create a list to hold the blocks blocks = [] for i in range(1, num_blocks + 1): block_name = f'block{i}' block = SimulationBlock(block_name, **block_params, simulation_cases=simulation_cases, additional_commands=additional_commands,component_path=component_path) blocks.append(block)

Then we can get the Runner ready to submit. You can also have the option to put a webhook link to send messages on progress of the jobs. Currently, only Microsoft Teams has been tested. If you want to only do one run of the network then you can just set param_values to be None instead.¶

In [ ]:

flow_url = None
runner = BlockRunner(blocks=blocks,json_file_path=json_file_path,
                     param_name=param_name,param_values=param_values,webhook=flow_url)

flow_url = None runner = BlockRunner(blocks=blocks,json_file_path=json_file_path, param_name=param_name,param_values=param_values,webhook=flow_url)

Then finally we submit the run¶

In [ ]:

runner.submit_blocks_parallel()

runner.submit_blocks_parallel()

Something to note is if you are using a webhook and run this last line the code run until the last job is finished. This can take some time, so it may be easier to run the file on SLURM so you don't have to keep your code running on your end. You could also just close the code after all the jobs are submitted and just not worry about knowing when the jobs are done.¶