oio package
Subpackages
- oio.account package
- oio.api package
- oio.blob package
- oio.check_service package
- oio.cli package
- Subpackages
- Submodules
- oio.cli.clientmanager module
- Module contents
- oio.common package
- Submodules
- oio.common.autocontainer module
- oio.common.client module
- oio.common.configuration module
- oio.common.constants module
- oio.common.daemon module
- oio.common.decorators module
- oio.common.easy_value module
- oio.common.exceptions module
- oio.common.fullpath module
- oio.common.green module
- oio.common.http module
- oio.common.http_eventlet module
- oio.common.http_urllib3 module
- oio.common.json module
- oio.common.logger module
- oio.common.redis_conn module
- oio.common.storage_functions module
- oio.common.storage_method module
- oio.common.timestamp module
- oio.common.utils module
- oio.common.wsgi module
- oio.common.xattr module
- Module contents
- oio.conscience package
- oio.container package
- oio.content package
- oio.crawler package
- oio.directory package
- oio.ecd package
- oio.event package
- oio.rdir package
- oio.rebuilder package
- oio.zk package
Module contents
OpenIO SDS Python API.
Basic object storage example:
>>> from oio import ObjectStorageApi
>>> api = ObjectStorageApi(namespace="OPENIO")
>>> api.object_create("myaccount", "mycontainer", "/etc/magic")
([{u'url': u'http://127.0.0.1:6008/DEADBEEFCAFEBABE1EE7',
u'score': 65,
u'hash': '8de4989188593b0419d387099c9e9872',
u'pos': '0',
u'size': 113}],
113,
'8de4989188593b0419d387099c9e9872')
-
class
oio.
ObjectStorageApi
(namespace, logger=None, **kwargs)[source] Bases:
object
The Object Storage API.
High level API that wraps AccountClient, ContainerClient and DirectoryClient classes.
Every method that takes a kwargs argument accepts the at least the following keywords:
- headers: dict of extra headers to pass to the proxy
- connection_timeout: float
- read_timeout: float
- write_timeout: float
-
EXTRA_KEYWORDS
= ('chunk_checksum_algo', 'autocreate')
-
TIMEOUT_KEYS
= ('connection_timeout', 'read_timeout', 'write_timeout')
-
account_create
(**kwargs)[source] Create an account.
Parameters: account (str) – name of the account to create Returns: True if the account has been created
-
account_del_properties
(account=None, *args, **kwargs)[source]
-
account_delete
(account=None, *args, **kwargs)[source] Delete an account.
Parameters: account (str) – name of the account to delete
-
account_flush
(account=None, *args, **kwargs)[source] Flush all containers of an account
Parameters: account (str) – name of the account to flush
-
account_list
(**kwargs)[source] List known accounts.
Notice that account creation is asynchronous, and an autocreated account may appear in the listing only after several seconds.
-
account_refresh
(account=None, *args, **kwargs)[source] Refresh counters of an account.
Parameters: account (str) – name of the account to refresh, or None to refresh all accounts (slow)
-
account_set_properties
(account=None, *args, **kwargs)[source]
-
account_show
(account=None, *args, **kwargs)[source] Get information about an account.
-
account_update
(account=None, *args, **kwargs)[source]
-
all_accounts_refresh
(**kwargs)[source] Deprecated: call account_refresh(None) instead
-
blob_client
A low-level client to rawx services.
Return type: oio.blob.client.BlobClient
-
container_create
(**kwargs)[source] Create a container.
Parameters: - account (str) – account in which to create the container
- container (str) – name of the container
- properties (dict) – properties to set on the container
Returns: True if the container has been created, False if it already exists
-
container_create_many
(**kwargs)[source] Create Many containers
Parameters: - account (str) – account in which to create the containers
- containers (list) – names of the containers
- properties (dict) – properties to set on the containers
-
container_del_properties
(account, container, *args, **kwargs)[source] Delete properties of a container.
Parameters: - account (str) – name of the account
- container (str) – name of the container to deal with
- properties (list) – a list of property keys
-
container_delete
(account, container, *args, **kwargs)[source] Delete a container.
Parameters: - account (str) – account from which to delete the container
- container (str) – name of the container
-
container_flush
(account, container, *args, **kwargs)[source] Flush a container
Parameters: - account (str) – account from which to delete the container
- container (str) – name of the container
- fast (bool) – flush container quickly, may put high pressure on the event system
-
container_get_properties
(account, container, *args, **kwargs)[source] Get information about a container (user and system properties).
Parameters: - account (str) – account in which the container is
- container (str) – name of the container
- properties – ignored
Returns: a dict with “properties” and “system” entries, containing respectively a dict of user properties and a dict of system properties.
-
container_list
(account=None, *args, **kwargs)[source] Get the list of containers of an account.
Parameters: - account (str) – account from which to get the container list
- limit (int) – maximum number of results to return
- marker (str) – name of the container from where to start the listing
- end_marker –
- prefix –
- delimiter –
Returns: the list of containers of an account
Return type: list of items (list) with 5 fields: name, number of objects, number of bytes, 1 if the item is a prefix or 0 if the item is actually a container, and modification time.
-
container_purge
(account, container, *args, **kwargs)[source]
-
container_refresh
(account, container, *args, **kwargs)[source]
-
container_set_properties
(account, container, *args, **kwargs)[source] Set properties on a container.
Parameters: - account (str) – name of the account
- container (str) – name of the container where to set properties
- properties (dict) – a dictionary of properties
- clear (bool) –
- system – dictionary of system properties to set
-
container_show
(account, container, *args, **kwargs)[source] Get information about a container (user properties).
Parameters: - account (str) – account in which the container is
- container (str) – name of the container
Returns: a dict with “properties” containing a dict of user properties.
-
container_snapshot
(account, container, *args, **kwargs)[source] Create a copy of the container (only the content of the database)
Parameters: - account (str) – account in which the target is
- container (str) – name of the target
- dst_account (str) – account in which the snapshot will be.
- dst_container (str) – name of the snapshot
-
container_touch
(account, container, *args, **kwargs)[source] Trigger a notification about the container state.
Parameters: - account (str) – account from which to delete the container
- container (str) – name of the container
-
container_update
(account, container, metadata, clear=False, **kwargs)[source] Deprecated: use container_set_properties
-
object_analyze
(*args, **kwargs)[source] Deprecated: use object_locate
-
object_create
(account, container, *args, **kwargs)[source] Create an object or append data to object in container of account with data taken from either data (str or generator) or file_or_path (path to a file or file-like object). The object will be named after obj_name if specified, or after the base name of file_or_path.
Parameters: - account (str) – name of the account where to create the object
- container (str) – name of the container where to create the object
- file_or_path (str or file-like object) – file-like object or path to a file from which to read object data
- data (str or generator) – object data (if file_or_path is not set)
- etag (str) – entity tag of the object
- obj_name – name of the object to create. If not set, will use the base name of file_or_path.
- mime_type (str) – MIME type of the object
- properties (dict) – a dictionary of properties
- policy (str) – name of the storage policy
- key_file –
- append (bool) – if set, data will be append to existing object (or object will be created if unset)
- autocreate – if set to false, autocreation of container will be
disabled :type autocreate: bool
Parameters: - perfdata – optional dict that will be filled with metrics of time spent to resolve the meta2 address, to do the meta2 requests, and to upload chunks to rawx services.
- deadline (float seconds) – deadline for the request, in monotonic time (oio.common.utils.monotonic_time). This supersedes timeout or read_timeout keyword arguments.
Returns: list of chunks, size and hash of the what has been uploaded
-
object_del_properties
(account, container, obj, *args, **kwargs)[source] Delete some properties from an object.
Parameters: properties (list) – list of property keys to delete Returns: True if the property has been deleted (or was missing)
-
object_delete
(account, container, obj, *args, **kwargs)[source] Delete an object from a container. If versioning is enabled and no version is specified, the object will be marked as deleted but not actually deleted.
Parameters: - account (str) – name of the account the object belongs to
- container (str) – name of the container the object belongs to
- obj – name of the object to delete
- version – version of the object to delete
Returns: True on success
-
object_delete_many
(**kwargs)[source] Delete several objects.
Parameters: objs – an iterable of object names (should not be a generator) Returns: a list of tuples with the name of the object and a boolean telling if the object has been successfully deleted Return type: list of tuple
-
object_drain
(**kwargs)[source] Remove all the chunks of a content, but keep all the metadata.
Parameters: - account (str) – name of the account where the object is present
- container (str) – name of the container where the object is present
- obj – name of the object to drain
-
object_fastcopy
(*args, **kwargs)[source] Deprecated: use object_link.
-
object_fetch
(*args, **kwargs)[source] Download an object.
Parameters: - account – name of the account in which the object is stored
- container – name of the container in which the object is stored
- obj – name of the object to fetch
- version (str) – version of the object to fetch
- ranges (list of tuple) – a list of object ranges to download
- key_file – path to the file containing credentials
- properties (bool) – should the request return object properties along with content description (True by default)
- perfdata – optional dict that will be filled with metrics of time spent to resolve the meta2 address, to do the meta2 request, and the time-to-first-byte, as seen by this API.
Returns: a dictionary of object metadata and a stream of object data
Return type: tuple
-
object_get_properties
(account, container, obj, *args, **kwargs)[source] Get the description of an object along with its user properties.
Parameters: - account – name of the account in which the object is stored
- container – name of the container in which the object is stored
- obj – name of the object to query
Returns: a dict describing the object
-
example
- {‘hash’: ‘6BF60C17CC15EEA108024903B481738F’,
‘ctime’: ‘1481031763’, ‘deleted’: ‘False’, ‘properties’: {
u’projet’: u’OpenIO-SDS’},‘length’: ‘43518’, ‘hash_method’: ‘md5’, ‘chunk_method’: ‘ec/algo=liberasurecode_rs_vand,k=6,m=3’, ‘version’: ‘1481031762951972’, ‘policy’: ‘EC’, ‘id’: ‘20BF2194FD420500CD4729AE0B5CBC07’, ‘mime_type’: ‘application/octet-stream’, ‘name’: ‘Makefile’}
-
object_head
(account, container, obj, trust_level=0, **kwargs)[source] Check for the presence of an object in a container.
Parameters: trust_level (int) – 0: do not check chunks; 1: check if there are enough chunks to read the object; 2: check if all chunks are present.
-
object_link
(*args, **kwargs)[source] Make a shallow copy of an object. Works across accounts and across containers.
-
object_list
(account, container, *args, **kwargs)[source] Lists objects inside a container.
Parameters: - properties – if True, list object properties along with objects
- versions – if True, list all versions of objects
- deleted – if True, list also the deleted objects
Returns: a dict which contains * ‘objects’: the list of object descriptions * ‘prefixes’: common prefixes (only if delimiter and prefix are set) * ‘properties’: a dict of container properties * ‘system’: a dict of system metadata * ‘truncated’: a bool telling if the listing was truncated * ‘next_marker’: a str to be used as marker to get the next
page of results (in case the listing was truncated)
-
object_locate
(account, container, obj, *args, **kwargs)[source] Get a description of the object along with the list of its chunks.
Parameters: - account – name of the account in which the object is stored
- container – name of the container in which the object is stored
- obj – name of the object to query
- version – version of the object to query
- chunk_info – if True, fetch additional information about chunks from rawx services (slow). The second element of the returned tuple becomes a generator (instead of a list).
- properties (bool) – should the request return object properties along with content description
Returns: a tuple with object metadata dict as first element and chunk list as second element
-
object_set_properties
(account, container, obj, *args, **kwargs)[source] Set properties on an object.
Parameters: - account – name of the account in which the object is stored
- container – name of the container in which the object is stored
- obj – name of the object to query
- properties – dictionary of properties
-
object_show
(account, container, obj, *args, **kwargs)[source] Get the description of an object along with the dictionary of user-set properties.
Deprecated: prefer using object_get_properties, for consistency with container_get_properties.
-
object_touch
(**kwargs)[source] Trigger a notification about an object (as if it just had been created).
Parameters: - account (str) – name of the account where to create the object
- container (str) – name of the container where to create the object
- obj – name of the object to touch
-
object_truncate
(account, container, obj, *args, **kwargs)[source] Truncate object at specified size. Only shrink is supported. A download may occur if size is not on chunk boundaries.
Parameters: - account – name of the account in which the object is stored
- container – name of the container in which the object is stored
- obj – name of the object to query
- version – version of the object to query
- size – new size of object
-
object_update
(account, container, obj, metadata, version=None, clear=False, **kwargs)[source] Deprecated: use object_set_properties
-
proxy_client