oio.directory package

Submodules

oio.directory.admin module

class oio.directory.admin.AdminClient(conf, **kwargs)

Bases: oio.common.client.ProxyClient

Low level database administration client.

copy_base_from(service_type=None, account=None, reference=None, cid=None, **kwargs)

Copy a base to another service, using DB_PIPEFROM.

Parameters:

• svc_from – id of the source service.
• svc_to – id of the destination service.

copy_base_to(service_type=None, account=None, reference=None, cid=None, **kwargs)

Copy a base to another service, using DB_PIPETO. Source service is looked after in service directory.

Parameters:svc_to – id of the destination service.

election_debug(service_type=None, account=None, reference=None, cid=None, **kwargs)

Get debugging information about an election.

election_leave(service_type=None, account=None, reference=None, cid=None, **kwargs)

Force all peers to leave the election.

election_ping(service_type=None, account=None, reference=None, cid=None, **kwargs)

Trigger or refresh an election.

election_status(service_type=None, account=None, reference=None, cid=None, **kwargs)

Get the status of an election (trigger it if necessary).

Returns:a dict with ‘master’ (str), ‘slaves’ (list), ‘peers’ (dict) and ‘type’ (str)

example

{

‘peers’: {

‘127.0.0.3:6014’: {

‘status’:

{‘status’: 303,

‘message’: ‘127.0.0.1:6015’},

‘body’: u’’},

‘127.0.0.1:6015’: {

‘status’:

{‘status’: 200,

‘message’: ‘OK’},

‘body’: u’’},

‘127.0.0.2:6016’: {

‘status’:

{‘status’: 303,

‘message’: ‘127.0.0.1:6015’},

‘body’: u’’}

}, ‘master’: ‘127.0.0.1:6015’, ‘slaves’: [‘127.0.0.3:6014’, ‘127.0.0.2:6016’], ‘type’: ‘meta1’

}

election_sync(service_type=None, account=None, reference=None, cid=None, **kwargs)

Try to synchronize a dubious election.

get_properties(service_type=None, account=None, reference=None, cid=None, **kwargs)

Get user and system properties from the admin table of an sqliterepo base.

has_base(service_type=None, account=None, reference=None, cid=None, **kwargs)

Ask each peer if base exists.

remove_base(service_type=None, account=None, reference=None, cid=None, **kwargs)

Remove specific base.

service_flush_cache(svc_id, **kwargs)

Flush the resolver cache of an sqlx-bases service.

set_peers(service_type=None, account=None, reference=None, cid=None, **kwargs)

Force the new peer set in the replicas of the old peer set.

set_properties(service_type=None, account=None, reference=None, cid=None, **kwargs)

Set user or system properties in the admin table of an sqliterepo base.

oio.directory.admin.loc_params(func)

Wrap database localization parameters in request parameters

oio.directory.client module

class oio.directory.client.DirectoryClient(conf, **kwargs)

Bases: oio.common.client.ProxyClient

Mid-level client for OpenIO SDS service directory (meta0, meta1).

create(account=None, reference=None, properties=None, **kwargs)

Create a reference (a service container).

Parameters:

• account – name of the account where the reference must be created
• reference – name of the reference to create
• properties (dict) – dictionary of properties to set on the reference

Returns:

True if the reference has been created, False if it already existed

del_properties(account=None, reference=None, properties=None, cid=None, **kwargs)

Delete properties for a reference.

delete(account=None, reference=None, cid=None, **kwargs)

Delete a reference.

destroy(*args, **kwargs)

Deprecated:use delete

force(account=None, reference=None, service_type=None, services=None, cid=None, autocreate=False, replace=False, **kwargs)

Associate the specified services to the reference.

get_properties(account=None, reference=None, properties=None, cid=None, **kwargs)

Get properties for a reference.

has(account=None, reference=None, cid=None, **kwargs)

link(account=None, reference=None, service_type=None, cid=None, autocreate=False, **kwargs)

Poll and associate a new service to the reference.

list(account=None, reference=None, cid=None, service_type=None, **kwargs)

List the services linked to the reference.

renew(account=None, reference=None, service_type=None, cid=None, **kwargs)

Re-poll and re-associate a set of services to the reference. Will increment the sequence number.

set_properties(account=None, reference=None, properties=None, cid=None, **kwargs)

Set properties for a reference.

show(*args, **kwargs)

Deprecated:use list

unlink(account=None, reference=None, service_type=None, cid=None, **kwargs)

Remove an associated service from the reference

oio.directory.meta module

class oio.directory.meta.MetaMapping(conf, service_types, admin_client=None, conscience_client=None, logger=None, **kwargs)

Bases: object

Represents the content of the meta_n0 database

admin

apply(moved=None, **kwargs)

Upload the current mapping to the meta_n0 services, and set peers accordingly in meta_n1 databases.

Parameters:moved – list of bases that have moved.

conscience

static get_cid_and_seq(base)

reset()

Reset the base allocations and reload the services from Conscience.

oio.directory.meta0 module

Meta0 client and meta1 balancing operations

class oio.directory.meta0.Meta0Client(conf, **kwargs)

Bases: oio.common.client.ProxyClient

Meta0 administration client

force(mapping, **kwargs)

Force the meta0 prefix mapping. The mapping may be partial to force only a subset of the prefixes.

list(**kwargs)

Get the meta0 prefix mapping

class oio.directory.meta0.Meta0PrefixMapping(meta0_client, replicas=3, digits=None, min_dist=1, **kwargs)

Bases: oio.directory.meta.MetaMapping

Represents the content of the meta0 database

assign_services(base, services, fail_if_already_set=False)

Assign services to manage base.

Parameters:fail_if_already_set – raise ValueError if base is already managed by some services

bootstrap(strategy=None)

Build self.num_bases() assignations from scratch, using strategy to find new services.

check_replicas()

Check that all bases have the right number of replicas

count_pfx_by_svc()

Build a dict with service addresses as keys and the number of managed bases as values.

decommission(svc, bases_to_remove=None, strategy=None)

Unassign all bases of bases_to_remove from svc, and assign them to other services using strategy.

static dist_between(loc1, loc2)

find_services_less_bases(known=None, min_score=1, **_kwargs)

Find replicas services, including the ones of known

find_services_m1_pool(known=None, **_kwargs)

Find replicas services, including the ones of known, by calling the proxy’s load balancer.

find_services_random(known=None, **_kwargs)

Find replicas services, including the ones of known

get_loc(svc, default=None)

Get the location of a service. If location is not defined, return: - service IP address if default is None or “addr” - default for any other value.

get_managed_bases(svc)

Get the list of bases managed by the service

get_score(svc)

Get the score of a service, or 0 if it is unknown

load(json_mapping=None, swap_bytes=True, **kwargs)

Load the mapping from the cluster, from a JSON string or from a dictionary.

num_bases()

Get total the number of bases according to self.digits

prefix_siblings(pfx)

Get the list of prefixes that share the same base as pfx.

prefix_to_base(pfx)

Get the name of the base the prefix will be saved in. When self.digits is 4, the name of the base is pfx.

rebalance(max_loops=65536)

Reassign bases from the services which manage the most

services

to_json(bases=None)

Serialize the mapping to a JSON string suitable as input for ‘meta0_force’ request.

oio.directory.meta0.count_prefixes(digits)

Returns the number of real prefixes in meta0/meta1. Raises an exception if the prefix number is not acceptable.

oio.directory.meta0.generate_prefixes(digits)

oio.directory.meta0.generate_short_prefixes(digits)

oio.directory.meta1 module

Meta1 client and meta2 balancing operations

class oio.directory.meta1.Meta1RefMapping(namespace, directory_client=None, **kwargs)

Bases: oio.directory.meta.MetaMapping

Represents the content of the meta1 database

move(src_service, dest_service, base_name, service_type, **kwargs)

Move a base of src_service to dest_service

reference

Module contents