oio.blob package

Submodules

oio.blob.auditor module

class oio.blob.auditor.BlobAuditor(conf, **kwargs)[source]

Bases: oio.common.daemon.Daemon

Walk through the chunks of a volume, and check for incoherencies: missing extended attributes, invalid hash, position or size, or orphaned chunk.

run(*args, **kwargs)[source]
class oio.blob.auditor.BlobAuditorWorker(conf, logger, volume)[source]

Bases: object

audit_pass()[source]
chunk_audit(path, chunk_id)[source]
chunk_file_audit(chunk_file, chunk_id)[source]
safe_chunk_audit(path)[source]
class oio.blob.auditor.ChunkReader(fp, size, md5_checksum)[source]

Bases: object

close()[source]

oio.blob.client module

class oio.blob.client.BlobClient(conf=None, perfdata=None, logger=None, connection_pool=None, **kwargs)[source]

Bases: object

A low-level client to rawx services.

chunk_copy(*args, **kwargs)[source]
chunk_delete(*args, **kwargs)[source]
chunk_delete_many(**kwargs)[source]
Return type:list of either urllib3.response.HTTPResponse or urllib3.exceptions.HTTPError, with an extra “chunk” attribute.
chunk_get(*args, **kwargs)[source]
chunk_head(*args, **kwargs)[source]
chunk_put(*args, **kwargs)[source]
resolve_url(url)[source]
oio.blob.client.extract_headers_meta(headers)[source]
oio.blob.client.update_rawx_perfdata(func)[source]

oio.blob.converter module

class oio.blob.converter.BlobConverter(conf, logger=None, **kwargs)[source]

Bases: object

cid_from_name(account, container)[source]
content_id_from_name(cid, path, version, search=False)[source]
convert_chunk(fd, chunk_id)[source]
converter_pass(input_file=None)[source]
decode_fullpath(fullpath)[source]
decode_old_fullpath(old_fullpath)[source]
encode_fullpath(fd, chunk_id, account, container, path, version, content_id)[source]
get_chunk_id_and_fullpath(fd, chunk_pos, container_id, path, version, chunk_id=None, account=None, container=None, content_id=None)[source]
is_fullpath_error(err)[source]
name_from_cid(cid)[source]
paths_gen(input_file=None)[source]
rdir

Get an instance of RdirClient.

recover_chunk_fullpath(path, chunk_id=None)[source]
safe_convert_chunk(path, chunk_id=None)[source]
save_xattr(fd, chunk_id, xattr)[source]
class oio.blob.converter.CacheDict(size=262144)[source]

Bases: collections.OrderedDict

oio.blob.indexer module

class oio.blob.indexer.BlobIndexer(conf, **kwargs)[source]

Bases: oio.common.daemon.Daemon

index_pass()[source]
report(tag, start_time)[source]
run(*args, **kwargs)[source]
safe_recover_fullpath(path)[source]
safe_update_index(path)[source]
update_index(path, chunk_id)[source]
oio.blob.indexer.random() → x in the interval [0, 1).

oio.blob.mover module

class oio.blob.mover.BlobMover(conf, **kwargs)[source]

Bases: oio.common.daemon.Daemon

run(*args, **kwargs)[source]
class oio.blob.mover.BlobMoverWorker(conf, logger, volume)[source]

Bases: object

chunk_move(path, chunk_id)[source]
load_chunk_metadata(path, chunk_id)[source]
mover_pass(**kwargs)[source]
safe_chunk_move(path)[source]

oio.blob.registrator module

class oio.blob.registrator.BlobRegistrator(conf, logger, volume, container_ids)[source]

Bases: object

BEAN_TYPES = ('alias', 'header', 'chunk')
DEFAULT_CHUNKS_PER_SECOND = 30
DEFAULT_REPORT_INTERVAL = 3600
log_report(status, force=False)[source]
pass_bean(meta, bean)[source]
pass_chunk_file(path)[source]
pass_volume()[source]

oio.blob.utils module

oio.blob.utils.check_volume(volume_path)[source]

Check if volume_path points to a rawx directory.

Returns:the namespace name and the service ID
Raises:oio.common.exceptions.OioException – when the specified path does not belong to a rawx service or misses some attributes.
oio.blob.utils.check_volume_for_service_type(volume_path, required_type)[source]

Check if volume_path points to a directory for the specified service type.

Returns:the namespace name and the service ID
Raises:oio.common.exceptions.OioException – when the specified path does not belong to a service from the specified type or is missing some attributes.
oio.blob.utils.read_chunk_metadata(fd, chunk_id, for_conversion=False)[source]

Module contents