oio.xcute package

Subpackages

• oio.xcute.common package
  • Submodules
  • oio.xcute.common.backend module
  • oio.xcute.common.job module
  • oio.xcute.common.worker module
  • Module contents
• oio.xcute.jobs package
  • Submodules
  • oio.xcute.jobs.blob_mover module
  • oio.xcute.jobs.blob_rebuilder module
  • oio.xcute.jobs.common module
  • oio.xcute.jobs.meta2_decommissioner module
  • oio.xcute.jobs.meta2_rebuilder module
  • oio.xcute.jobs.tester module
  • Module contents

Submodules

oio.xcute.client module

class oio.xcute.client.XcuteClient(conf, endpoint=None, proxy_endpoint=None, refresh_delay=3600.0, logger=None, **kwargs)

Bases: oio.api.base.HttpApi

Simple client API for the xcute service.

job_create(job_type, job_config=None)

job_delete(job_id)

job_list(limit=None, marker=None)

job_pause(job_id)

job_resume(job_id)

job_show(job_id)

lock_list()

lock_show(lock)

xcute_request(method, action, params=None, **kwargs)

Make a request to the xcute service.

oio.xcute.orchestrator module

class oio.xcute.orchestrator.XcuteOrchestrator(conf, logger=None)

Bases: object

DEFAULT_DISPATCHER_TIMEOUT = 2

DEFAULT_MAX_JOBS_PER_BEANSTALKD = 1024

DEFAULT_REFRESH_TIME_BEANSTALKD_WORKERS = 30

compute_total_tasks(job_id, job_type, job_info, job)

dispatch_tasks(job_id, job_type, job_info, job)

dispatch_tasks_batch(beanstalkd_workers, job_id, job_type, job_config, tasks)

Try sending a task until it’s ok

exit_gracefully(*args, **kwargs)

get_beanstalkd_workers()

Yield beanstalkd workers following a loadbalancing strategy

handle_backend_errors(func, *args, **kwargs)

handle_running_job(job_id, job_type, job_info)

First launch the computation of total number of tasks, then launch the dispatchnig of all tasks across the platform.

listen_beanstalkd_reply_forever()

Process this orchestrator’s job replies

listen_loop(listener)

One iteration of the listening loop

make_beanstalkd_payload(job_id, job_type, job_config, tasks)

process_reply(beanstalkd_job_id, encoded_reply)

refresh_beanstalkd_workers_forever()

Refresh beanstalkd workers by looking at the score, existing tubes and tube statistics.

run_forever()

Take jobs from the queue and spawn threads to dispatch them

safe_compute_total_tasks(job_id, job_type, job_info, job)

Compute the total number of tasks and update the backend.

safe_dispatch_tasks(job_id, job_type, job_info, job)

Dispatch all tasks across the platform and update the backend.

safe_handle_running_job(job_info)

safe_run_forever()

oio.xcute.server module

class oio.xcute.server.XcuteServer(conf, logger=None)

Bases: oio.common.wsgi.WerkzeugApp

on_job_create(req, *args, **kwargs)

on_job_delete(req, *args, **kwargs)

on_job_list(req, *args, **kwargs)

on_job_pause(req, *args, **kwargs)

on_job_resume(req, *args, **kwargs)

on_job_show(req, *args, **kwargs)

on_lock_list(req, *args, **kwargs)

on_lock_show(req, *args, **kwargs)

on_status(*args, **kwargs)

oio.xcute.server.access_log(func)

oio.xcute.server.create_app(conf)

oio.xcute.server.handle_exceptions(func)

Module contents