Try OpenIO SDS

Build from Source

Set the $SDS variable
Configure Repository
Build Dependencies
Download Source Code
Build
Install
Sandbox Setup

This section describes how to compile OpenIO SDS from the official source downloaded from GitHub.

The build procedure is available for Ubuntu 18.04 (Bionic Beaver). Howerver, other distributions are currently supported as explained at Supported Linux Distributions.

Set the $SDS variable

Throughout this guide, the environment variable SDS will be used several times. You should ensure this variable is always defined. It should be set to the name of directory where you are building OpenIO SDS.

export SDS=$HOME/local

Configure Repository

We provide easy access to build and runtime dependency versions not available in common distributions.

# Ubuntu Bionic
echo "deb http://mirror.openio.io/pub/repo/openio/sds/20.04/Ubuntu/ bionic/" | sudo tee /etc/apt/sources.list.d/openio-sds.list
curl http://mirror.openio.io/pub/repo/openio/APT-GPG-KEY-OPENIO-0 | sudo apt-key add -
sudo apt -y update

Build Dependencies

Build tools

sudo apt -y install git cmake

Build dependencies

sudo apt -y install \
    flex bison \
    libcurl4-gnutls-dev \
    libglib2.0-dev \
    libapreq2-dev \
    libsqlite3-dev \
    libjson-c-dev \
    liblzo2-dev \
    libzmq3-dev \
    libattr1-dev \
    libzookeeper-mt-dev \
    asn1c \
    openio-gridinit \
    liberasurecode-dev \
    python-dev \
    python-pbr \
    python-setuptools \
    libleveldb-dev
sudo apt -y install golang-go
go get gopkg.in/ini.v1 gopkg.in/tylerb/graceful.v1 golang.org/x/sys/unix

Download Source Code

The official OpenIO SDS source code is available from Github.

git clone https://github.com/open-io/oio-sds.git -b 7.0.0

Build

Perform the build in a separate folder from sources.

mkdir build && cd build
cmake \
    -DCMAKE_INSTALL_PREFIX=${SDS} \
    -DLD_LIBDIR=lib \
    -DCMAKE_BUILD_TYPE=Debug \
    ../oio-sds
make

Install

make install
( cd ../oio-sds && python setup.py install --user --install-scripts=${SDS}/bin)

Binaries and scripts are installed in $SDS/bin. Libraries are installed in $SDS/lib. Note that for Python, output is in local user installation $HOME/.local/.

Sandbox Setup

Environment

Set a few environment variables so everything we built previously is correctly found and used.

echo "export PATH=${PATH}:$SDS/bin" >> $HOME/.bashrc
echo "export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$SDS/lib" >> $HOME/.bashrc
source $HOME/.bashrc

Runtime Dependencies

To run the sandbox, you need additional runtime dependencies:

External services:

Redis: advanced key-value store used by account services.
Beanstalkd: simple and fast work queue used by OpenIO to run background jobs.

Libraries:

Python dependencies: several services and tools in OpenIO are built with Python

sudo apt install -y \
    apache2 \
    python-cliff \
    python-eventlet \
    python-gunicorn \
    python-redis \
    python-requests \
    python-werkzeug \
    python-xattr \
    python-yaml \
    python-zookeeper \
    redis-server \
    beanstalkd

We used to start redis from systemctl, but an instance will be started along with OpenIO SDS services.

Create Sandbox

oio-reset.sh -f etc/bootstrap-preset-MINIMAL.yml