pytask API

class pytask.pytask.PyTask(redis_instance, update_task_interval=5, **kwargs)

Bases: pytask.helpers._PyTaskRedisConf

A daemon that starts/stops tasks & replicates that to a Redis instance tasks can be control via Redis pubsub.

Redis Instance:
The first argument can be either a Redis client or a list of host/port details. When using a list, pytask will use redis-py with one host and redis-py-cluster where multiple hosts are present.
Parameters:
  • redis_instance (client or list) – Redis client or list of (host, port) tuples
  • task_set (str) – name of task set
  • task_prefix (str) – prefix for task names
  • new_queue (str) – queue to read new task IDs from
  • end_queue (str) – where to push complete task IDs
  • update_task_interval (int) – interval in s to update task times
exception PyTaskException

Bases: exceptions.Exception

exception PyTask.StopTask

Bases: pytask.pytask.PyTaskException

PyTask.add_exception_handler(handler)

Add an exception handler.

PyTask.add_task(task_class)

Add a task class.

PyTask.add_tasks(*task_classes)

Add multiple task classes.

PyTask.run(task_map=None)

Run pytask, basically a wrapper to handle KeyboardInterrupt.

PyTask.start_local_task(task_name, **task_data)

Used to start local tasks on this worker, which will start when .run is called.

Helpers

class pytask.helpers.PyTaskHelpers(redis_instance, task_set='tasks', task_prefix='task-', new_queue='new-task', end_queue='end-task')

Bases: pytask.helpers._PyTaskRedisConf

Helper functions for managing task data within Redis. All PyTask instances have an instance attached on their helpers attribute.

Parameters:
  • redis_instance (client or list) – Redis client or list of (host, port) tuples
  • task_set (str) – name of task set
  • task_prefix (str) – prefix for task names
  • new_queue (str) – queue to read new task IDs from
  • end_queue (str) – where to push complete task IDs
get_active_task_ids()

Get a list of active task_ids.

get_end_task_ids()

Get task IDs in the end queue.

get_new_task_ids()

Get task IDs in the new queue.

get_task(task_id, keys=None)

Get task hash data.

reload_task(task_id)

Reload a task.

remove_task(task_id)
restart_if_state(task_id, states)
restart_task(task_id)
set_task(task_id, data, value=None)

Set task hash data.

start_task(task_name, task_id=None, cleanup=True, **task_data)

Start a new task.

stop_task(task_id)

Stop a task.

pytask.helpers.run_loop(function, interval)

Like JavaScripts setInterval, slight time drift as usual, useful in long running tasks.

Included Tasks

class pytask.tasks.Cleanup(task_handler=None)

Bases: pytask.task.Task

A pytask which cleans up other pytasks! Drains the end queue and optionally triggers a handler/callback function.

Parameters:task_handler (str) – module:attribute type string
NAME = 'pytask/cleanup'
start()
class pytask.tasks.Monitor(loop_interval=10, task_timeout=60)

Bases: pytask.task.Task

A pytask which monitors pytasks! Checks all task state in Redis at a configured interval, will re-queue tasks which timeout.

NAME = 'pytask/monitor'
check_tasks()
start()
stop()