RAWX service
The RAWX service is a share-nothing service responsible to store the chunks. The interface uses a subset of the WebDAV commands augmented with custom headers.
GET /{CHUNKID}
Download a chunk from the target RAWX service. The chunk ID is given as a token of the URL. No particular header is expected in the request. The attributes of the chunk will be returned as headers of the reply.
| Token | Description |
|---|---|
| CHUNKID | a string of 64 hexadecimal characters |
| Header | Description |
|---|---|
| Range | Both chunked and inline are managed |
| Header | Description |
|---|---|
| X-oio-chunk-meta-full-path | A string representing a complete OpenIO URL |
| X-oio-chunk-meta-container-id | String of 64 hexadecimal characters |
| X-oio-chunk-meta-content-path | A string |
| X-oio-chunk-meta-content-version | A strictly positive integer, less than (2^63). That integer SHOULD match the number of bytes of the targeted chunk. |
| X-oio-chunk-meta-content-id | String of hexadecimal characters, usually of 32 characters. The only constraint is that the length must be even, to be convertible to a binary form. |
| X-oio-chunk-meta-content-storage-policy | A string, no check will be performed, at the RAWX level the string might even represent a non-existing storage policy |
| X-oio-chunk-meta-content-chunk-method | A string. At the RAWX level no check will be performed and that string might even be an invalid chunk-method description. |
| X-oio-chunk-meta-metachunk-size | A null or positive number |
| X-oio-chunk-meta-metachunk-hash | A string of 32 hexadecimal characters |
| X-oio-chunk-meta-chunk-id | A string of 64 hexadecimal characters that must math the CHUNKID present in the URL |
| X-oio-chunk-meta-chunk-pos | Either a positive integer (including 0) or a compound of 2 positive integers gathered with a dot. |
| X-oio-chunk-meta-chunk-size | A null or positive integer |
| X-oio-chunk-meta-chunk-hash | A string of 32 hexadecimal characters |
| X-oio-chunk-meta-oio-version | A string describing the versin of the headers. |
Example
GET /CE456217C7DBAC618A7F0EBFBCDB6C8F184ED8ADCBC6F0B6F493A51EE095D86A HTTP/1.1
Host: 127.0.0.1:6009
User-Agent: curl/7.55.1
HTTP/1.1 200 OK
Date: Tue, 21 Nov 2017 16:19:57 GMT
Server: Apache
Last-Modified: Tue, 21 Nov 2017 16:18:29 GMT
ETag: "55e808daf7710"
Accept-Ranges: bytes
Content-Length: 100
X-oio-chunk-meta-full-path: ACCT/JFS/magic/1511281109448048/5835AF8D805E0500AAB7F6808F50900A
X-oio-chunk-meta-container-id: 9006CE70B59E5777D6BB410C57944812EB05FCDB5BA85D520A14B3051D1D094F
X-oio-chunk-meta-content-path: magic
X-oio-chunk-meta-content-version: 1511281109448048
X-oio-chunk-meta-content-id: 5835AF8D805E0500AAB7F6808F50900A
X-oio-chunk-meta-content-storage-policy: EC
X-oio-chunk-meta-content-chunk-method: ec/algo=liberasurecode_rs_vand,k=6,m=3
X-oio-chunk-meta-metachunk-size: 111
X-oio-chunk-meta-metachunk-hash: 272913026300E7AE9B5E2D51F138E674
X-oio-chunk-meta-chunk-id: CE456217C7DBAC618A7F0EBFBCDB6C8F184ED8ADCBC6F0B6F493A51EE095D86A
X-oio-chunk-meta-chunk-pos: 0.3
X-oio-chunk-meta-chunk-size: 100
X-oio-chunk-meta-chunk-hash: 527EC56D67EF8DA68E3FB93158552272
X-oio-chunk-meta-oio-version: 4.2
...
PUT /{CHUNKID}
Upload a chunk on the target RAWX service.
| Token | Description |
|---|---|
| CHUNKID | a string of 64 hexadecimal characters |
| Header | Description |
|---|---|
| Transfer-Encoding | Both chunked and inline are managed |
| Header | Description | Mandatory |
|---|---|---|
| X-oio-chunk-meta-full-path | A string representing a complete OpenIO URL | True |
| X-oio-chunk-meta-container-id | String of 64 hexadecimal characters | False |
| X-oio-chunk-meta-content-path | A string | False |
| X-oio-chunk-meta-content-version | A strictly positive integer, less than (2^63). That integer SHOULD match the number of bytes of the targeted chunk. | False |
| X-oio-chunk-meta-content-id | String of hexadecimal characters, usually of 32 characters. The only constraint is that the length must be even, to be convertible to a binary form. | False |
| X-oio-chunk-meta-content-storage-policy | A string, no check will be performed, at the RAWX level the string might even represent a non-existing storage policy | True |
| X-oio-chunk-meta-content-chunk-method | A string. At the RAWX level no check will be performed and that string might even be an invalid chunk-method description. | True |
| X-oio-chunk-meta-metachunk-size | A null or positive number | True if the storage policy is EC, False otherwise |
| X-oio-chunk-meta-metachunk-hash | A string of 32 hexadecimal characters | True if the storage policy is EC, False otherwise |
| X-oio-chunk-meta-chunk-id | A string of 64 hexadecimal characters that must math the CHUNKID present in the URL | False |
| X-oio-chunk-meta-chunk-pos | Either a positive integer (including 0) or a compound of 2 positive integers gathered with a dot. | True |
| X-oio-chunk-meta-chunk-size | A null or positive integer | False |
| X-oio-chunk-meta-chunk-hash | A string of 32 hexadecimal characters | False |
| X-oio-chunk-meta-oio-version | A string describing the versin of the headers. | False |
Example
PUT /0123456789ABCDEF0123456789ABCDEF0123456789ABCDEF0123456789ABCDEF HTTP/1.1
Content-Length: 12345
X-oio-chunk-meta-full-path: ACCT/JFS/magic/1511281109448048/5835AF8D805E0500AAB7F6808F50900A
X-oio-chunk-meta-container-id: 9006CE70B59E5777D6BB410C57944812EB05FCDB5BA85D520A14B3051D1D094F
X-oio-chunk-meta-content-path: magic
X-oio-chunk-meta-content-version: 1511281109448048
X-oio-chunk-meta-content-id: 5835AF8D805E0500AAB7F6808F50900A
X-oio-chunk-meta-content-storage-policy: EC
X-oio-chunk-meta-content-chunk-method: ec/algo=liberasurecode_rs_vand,k=6,m=3
X-oio-chunk-meta-metachunk-size: 111
X-oio-chunk-meta-metachunk-hash: 272913026300E7AE9B5E2D51F138E674
X-oio-chunk-meta-chunk-id: CE456217C7DBAC618A7F0EBFBCDB6C8F184ED8ADCBC6F0B6F493A51EE095D86A
X-oio-chunk-meta-chunk-pos: 0.3
X-oio-chunk-meta-chunk-size: 100
X-oio-chunk-meta-chunk-hash: 527EC56D67EF8DA68E3FB93158552272
X-oio-chunk-meta-oio-version: 4.2
-snip-
HTTP/1.1 200 OK
Content-Length: 0
DELETE /{CHUNKID}
Delete a chunk stored on the target RAWX service.
| Token | Description |
|---|---|
| CHUNKID | a string of 64 hexadecimal characters |
Example
DELETE /0123456789ABCDEF0123456789ABCDEF0123456789ABCDEF0123456789ABCDEF HTTP/1.1
Content-Length: 0
HTTP/1.1 204 No content
Content-Length: 0
GET /update/{CHUNKID}
Despite the GET verb, this route alters the metadata of the chunk.
| Token | Description |
|---|---|
| CHUNKID | a string of 64 hexadecimal characters |
| Header | Description |
|---|---|
| X-oio-chunk-meta-container-id | String of 64 hexadecimal characters |
| X-oio-chunk-meta-content-path | A string |
| X-oio-chunk-meta-content-id | String of hexadecimal characters, usually of 32 characters. The only constraint is that the number must be even, to be convertible to a binary form. |
| X-oio-chunk-meta-content-version | A strictly positive integer, less than (2^63). That integer SHOULD match the number of bytes of the targeted chunk. |
| X-oio-chunk-meta-content-storage-policy | A string, no check will be performed, at the RAWX level the string might even represent a non-existing storage policy |
| X-oio-chunk-meta-content-chunk-method | A string. At the RAWX level no check will be performed and that string might even be an invalid chunk-method description. |
| X-oio-chunk-meta-metachunk-size | A null or positive number |
| X-oio-chunk-meta-metachunk-hash | A string of 32 hexadecimal characters |
| X-oio-chunk-meta-chunk-id | A string of 64 hexadecimal characters that must math the CHUNKID present in the URL |
| X-oio-chunk-meta-chunk-size | A null or positive integer |
| X-oio-chunk-meta-chunk-hash | A string of 32 hexadecimal characters |
| X-oio-chunk-meta-chunk-pos | Either a positive integer (including 0) or a compound of 2 positive integers gathered with a dot. |
| X-oio-chunk-meta-oio-version | A string describing the versin of the headers. |
| X-oio-chunk-meta-full-path | A string representing a complete OpenIO URL |
Example
GET /update/0123456789ABCDEF0123456789ABCDEF0123456789ABCDEF0123456789ABCDEF HTTP/1.1
Content-Length: 0
X-oio-chunk-meta-container-id: 9006CE70B59E5777D6BB410C57944812EB05FCDB5BA85D520A14B3051D1D094F
X-oio-chunk-meta-content-path: magic
X-oio-chunk-meta-content-id: 5835AF8D805E0500AAB7F6808F50900A
X-oio-chunk-meta-content-version: 1511281109448048
X-oio-chunk-meta-content-storage-policy: EC
X-oio-chunk-meta-content-chunk-method: ec/algo=liberasurecode_rs_vand,k=6,m=3
X-oio-chunk-meta-metachunk-size: 111
X-oio-chunk-meta-metachunk-hash: 272913026300E7AE9B5E2D51F138E674
X-oio-chunk-meta-chunk-id: CE456217C7DBAC618A7F0EBFBCDB6C8F184ED8ADCBC6F0B6F493A51EE095D86A
X-oio-chunk-meta-chunk-size: 100
X-oio-chunk-meta-chunk-hash: 527EC56D67EF8DA68E3FB93158552272
X-oio-chunk-meta-chunk-pos: 0.3
X-oio-chunk-meta-oio-version: 4.2
X-oio-chunk-meta-full-path: ACCT/JFS/magic/1511281109448048/5835AF8D805E0500AAB7F6808F50900A
HTTP/1.1 200 OK
Content-Length: 0
GET /info
Returns some static information about the targeted RAWX service. No particular header is expected, neither in the request nor in the reply.
GET /info HTTP/1.1
Host: 127.0.0.1
Content-Length: 0
HTTP/1.1 200 OK
Content-Length: 60
namespace OPENIO
path /home/jfs/.oio/sds/data/OPENIO-rawx-4
GET /stat
Returns some volatile counters and gauges about what’s happening in the targeted RAWX service.
Sample exchange:
GET /stat HTTP/1.1
Host: 127.0.0.1
Content-Length: 0
HTTP/1.1 200 OK
Content-Length: 625
counter req.time 436106
counter req.time.put 0
counter req.time.get 0
counter req.time.del 0
counter req.time.stat 3612
counter req.time.info 3613
counter req.time.raw 0
counter req.time.other 0
counter req.hits 7225
counter req.hits.put 0
counter req.hits.get 0
counter req.hits.del 0
counter req.hits.stat 3612
counter req.hits.info 3613
counter req.hits.raw 0
counter req.hits.other 0
counter rep.hits.2xx 7225
counter rep.hits.4xx 2
counter rep.hits.5xx 0
counter rep.hits.other 0
counter rep.hits.403 0
counter rep.hits.404 0
counter rep.bread 0
counter rep.bwritten 0