Add a Meta2 Service

Description

This documentation explains how to add a new Meta2 service on your cluster.

It will rely on the following 3 nodes cluster:

+--------------------+  +--------------------+  +--------------------+
|        node1       |  |        node2       |  |        node3       |
|                    |  |                    |  |                    |
|      10.0.0.36     |  |      10.0.0.37     |  |      10.0.0.38     |
+--------------------+  +--------------------+  +--------------------+
|                    |  |                    |  |                    |
|  account           |  |  account           |  |  account           |
|  meta0             |  |  meta0             |  |  meta0             |
|  meta1             |  |  meta1             |  |  meta1             |
|  meta2             |  |  meta2             |  |  meta2             |
|  rawx              |  |  rawx              |  |  rawx              |
|  rdir (rawx node2) |  |  rdir (rawx node3) |  |  rdir (rawx node1) |
|                    |  |                    |  |                    |
+--------------------+  +--------------------+  +--------------------+

Add a new meta2 using ansible

This part of the documentation explains how to add a new meta2 service using ansible.

In this example, we will add a new mounted volume on the node1, which will automatically install a new meta2 service.

First, add a new volume in the host_vars/node1.yml configution file:

host_vars/node1.yml
---
openio_data_mounts:
  - { partition: '/dev/sdb', mountpoint: "/var/lib/oio/sds" }
openio_metadata_mounts:
  - { partition: '/dev/sdb', mountpoint: "/var/lib/oio/sds" }
  - { partition: '/dev/sdb1', mountpoint: "/var/lib/oio/sds1", meta2_count: 1, meta1_count: 0  }

Run the following command to configure your new mounted volume:

oiosds/products/sds/deploy.sh
# ./deploy.sh

Check that the service has been correctly added:

 # gridinit_cmd status
 KEY                         STATUS      PID GROUP
 OPENIO-account-0            UP        17232 OPENIO,account,0
 OPENIO-beanstalkd-0         UP        16654 OPENIO,beanstalkd,0
 OPENIO-conscienceagent-0    UP         8420 OPENIO,conscienceagent,0
 OPENIO-memcached-0          UP        21677 OPENIO,memcached,0
 OPENIO-meta0-0              UP        21113 OPENIO,meta0,0
 OPENIO-meta1-0              UP        21196 OPENIO,meta1,0
 OPENIO-meta2-0              UP        18261 OPENIO,meta2,0
 OPENIO-meta2-1              UP        22503 OPENIO,meta2,1
 OPENIO-oio-blob-indexer-0   UP        19725 OPENIO,oio-blob-indexer,0
 OPENIO-oio-blob-rebuilder-0 UP        20371 OPENIO,oio-blob-rebuilder,0
 OPENIO-oio-event-agent-0    UP        20629 OPENIO,oio-event-agent,0
 OPENIO-oioproxy-0           UP        16979 OPENIO,oioproxy,0
 OPENIO-oioswift-0           UP        22457 OPENIO,oioswift,0
 OPENIO-rawx-0               UP        19401 OPENIO,rawx,0
 OPENIO-rdir-0               UP        20127 OPENIO,rdir,0
 OPENIO-redis-0              UP        15877 OPENIO,redis,0
 OPENIO-redissentinel-0      UP         3636 OPENIO,redissentinel,0
 OPENIO-zookeeper-0          UP         2944 OPENIO,zookeeper,0

Then, on node1, restart the conscience agent:

# gridinit_cmd restart @conscienceagent

Finally, you must unlock your new service:

 # openio cluster unlock meta2 10.0.0.36:6122

Add a new meta2 manually

Prerequisites

You must have the IP address and port of your new Meta2 service. In this example, the new Meta2 will listen on 10.0.0.36:6122.

Configuration

Create an new directory meta2-1 in /var/lib/oio/sds/OPENIO/.

# mkdir /var/lib/oio/sds/OPENIO/meta2-1

Give the rights for this directory to the openio user:

# chown openio:openio /var/lib/oio/sds/OPENIO/meta2-1/

Create a new configuration file (OPENIO-meta2-1.conf) in the /etc/gridinit.d/ directory:

/etc/gridinit.d/OPENIO-meta2-1
# OpenIO managed
[Service.OPENIO-meta2-1]
command=/usr/bin/oio-meta2-server -p /run/oio/sds/OPENIO-meta2-1.pid -s OIO,OPENIO,meta2,1 -O Endpoint=10.0.0.36:6122  -O Config=/etc/oio/sds/OPENIO/meta2-1/meta2-1.conf OPENIO /var/lib/oio/sds/OPENIO/meta2-1
enabled=true
start_at_boot=true
on_die=respawn
group=OPENIO,meta2,1
uid=openio
gid=openio
env.PATH=/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin

Give the rights for this file to the openio user:

# chown openio:openio /etc/gridinit.d/OPENIO-meta2-1.conf

Create a new configuration file (meta2-1.yml) in the /etc/oio/sds/OPENIO/watch directory:

/etc/oio/sds/OPENIO/watch/meta2-1.yml
# OpenIO managed
---
host: 10.0.0.36
port: 6122
type: meta2
location: node-1.0
checks:
 - {type: tcp}
stats:
 - {type: volume, path: /var/lib/oio/sds/OPENIO/meta2-0}
 - {type: meta}
 - {type: system}
...

Give the rights for this file to the openio user:

# chown openio:openio /etc/oio/sds/OPENIO/watch/meta2-1.yml

To make your new meta2 service available, you must reload the configuration and start the service:

# gridinit_cmd reload
# gridinit_cmd start OPENIO-meta2-1

And restart the conscience agent:

# gridinit_cmd restart @conscienceagent

Finally, you must unlock your new service:

# openio cluster unlock meta2 10.0.0.36:6122

You can check that your new service is available using the openio cluster list command:

 # openio cluster list meta2

 +---------+----------------+------------+---------------------------------+------------+-------+------+-------+
 | Type    | Addr           | Service Id | Volume                          | Location   | Slots | Up   | Score |
 +---------+----------------+------------+---------------------------------+------------+-------+------+-------+
 | meta2   | 10.0.0.36:6122 | n/a        | /var/lib/oio/sds/OPENIO/meta2-1 | node-1     | n/a   | True |    90 |
 | meta2   | 10.0.0.38:6121 | n/a        | /var/lib/oio/sds/OPENIO/meta2-0 | node-3     | n/a   | True |    91 |
 | meta2   | 10.0.0.36:6121 | n/a        | /var/lib/oio/sds/OPENIO/meta2-0 | node-1     | n/a   | True |    90 |
 | meta2   | 10.0.0.37:6121 | n/a        | /var/lib/oio/sds/OPENIO/meta2-0 | node-2     | n/a   | True |    92 |
 +---------+----------------+------------+---------------------------------+------------+-------+------+-------+