parallel

Multi-threaded functions for parameter sweeps

Classes

Config(callsig, num_threads, queue_size)	Configuration variables for ParallelFor
DefaultStatus(params, sequence)	Default status reporter
Factory(variables, fixtures, produce, config)	Produces Deferred jobs for ParallelFor
Hooks()	Holds functions which hook into ParallelFor
ParallelFor(factory[, make_result])	Keep track of progress while running _parallel_for

Functions

_parallel_for(sequence, produce, retire[, ...])	Multi-threaded for loop
_sequential_for(sequence, produce, retire)	Simple single-threaded for loop
ndsweep(factory[, plot, labels, tags, silent])	Do a multi-threaded n-dimensional parameter sweep
parallel_for(factory[, make_result])	Multi-threaded loop feed by the factory function
parallelize([num_threads, queue_size])	A decorator which creates factory functions for parallel_for()
sweep(factory[, plot, labels, tags, silent])	Do a multi-threaded parameter sweep

ndsweep(factory, plot=None, labels=None, tags=None, silent=False)

Do a multi-threaded n-dimensional parameter sweep

Parameters:

factoryFactory

Factory function created with the parallelize() decorator.

plotcallable

Plotting functions which takes a NDSweep result as its only argument.

labels, tagsdict

Forwarded to NDSweep object.

silentbool

Don’t print status messages.

Returns:

NDSweep

parallel_for(factory, make_result=None)

Multi-threaded loop feed by the factory function

Parameters:

factoryFactory

Factory function created with the parallelize() decorator.

make_resultcallable, optional

Creates the final result from raw data. This result is also the final return value of parallel_for().

Returns:

array_like

A result for each loop iteration.

Examples

@parallelize(x=np.linspace(0, 1, 10))
def factory(x):
    pb.Model(...)  # depends on `x`
    kpm = pb.kpm(model)
    return kpm.deferred_ldos(...)  # may also depend on `x`

results = parallel_for(factory)

parallelize(num_threads=num_cores, queue_size=num_cores, **kwargs)

A decorator which creates factory functions for parallel_for()

The decorated function must return a Deferred compute kernel.

Parameters:

num_threadsint

Number of threads that will run in parallel. Defaults to the number of cores in the current machine.

queue_sizeint

Number of Deferred jobs to be queued up for consumption by the worker threads. The maximum number of jobs that will be kept in memory at any one time will be queue_size + num_threads.

**kwargs

Variables which will be iterated over in parallel_for() and passed to the decorated function. See example.

Examples

@parallelize(a=np.linspace(0, 1, 10), b=np.linspace(-2, 2, 10))
def factory(a, b):
    pb.Model(...)  # depends on `a` and `b`
    kpm = pb.kpm(model)
    return kpm.deferred_ldos(...)  # may also depend on `a` and `b`

results = parallel_for(factory)

sweep(factory, plot=<function <lambda>>, labels=None, tags=None, silent=False)

Do a multi-threaded parameter sweep

Parameters:

factoryFactory

Factory function created with the parallelize() decorator.

plotcallable

Plotting functions which takes a Sweep result as its only argument.

labels, tagsdict

Forwarded to Sweep object.

silentbool

Don’t print status messages.

Returns:

Sweep