Ceph RBD - how to create and use a Rados Block Device

Last updated: 15-May-2016

I assume you have a working Ceph cluster as descibed in the previous section. Now we can create and maintain Block Devices.
In this example I will create a Rados Block Device (RBD), and use in in another VM. This must not be one of the VM's of the Ceph cluster.
Step 1 is to create a new Centos 7 VM. From a terminal like GitBash type the following:


        
$ vagrant init centos/7; vagrant up --provider virtualbox

Everything in Ceph is stored as an object in the Ceph Object Store (RADOS). Logically an object is stored in a Placement Group, and Placement Groups are clustered in Pools. See picture below.

Now we SSH into the Ceph admin node, and we cd to the my-cluster folder, as explained in the installation section.
We will first create our own pool, called testpool with the recommended 128 Placement Groups. And we show a list of all our pools to see that it actually succeeded.


        
$ ceph osd pool create testpool 128
$ ceph osd lspools

Next we create a new Rados Block Device in the testpool. The name of this device will be newrbd, and the size will be 100Mb.
Show a list of all RBD's in the pool. And try some other commands to get more info about the Block Device.


        
$ rbd create testpool/newrbd --size 100
$ rbd ls -p testpool
$ rbd --image newrbd info -p testpool
$ rados df

Before you go to the client VM, you need the secret. It can be found in the my-cluster folder. Copy the key.


        
$ cat ceph.client.admin.keyring

Ok time to go to the new VM you just created. Or go to some other VM you would like to use for this. Just make sure the kernel supports RBD. A simple modprobe rbd will tell you if this will work. If not you can manually install RBD support. Read here how that must be done.

The following command maps our RBD to the client VM. The 3 IP addresses are our Ceph Monitors. Paste the secret you just copied from the admin keyring. Note that you also have to specify the pool and the name of the RBD.


      $ modprobe rbd
$ echo "192.168.33.81,192.168.33.82,192.168.33.83 \
       name=admin,secret=AQAiIzdXKMhwCBAAafWtIRuJmtuvgBEegt9CDg== \
	   testpool newrbd" > /sys/bus/rbd/add

Follow the steps below to put a filesystem on the new device, and to mount it.


      $ ll /dev/rbd*
$ mkfs.xfs -L newrbd /dev/rbd0
$ mkdir /mnt/newrbd
$ mount /dev/rbd0 /mnt/newrbd/
$ df -h

Let's put some stuff in the new device to see if it's working.


      $ cd /mnt/newrbd
$ mkdir somefolder
$ cd somefolder
$ wget http://something.from.somewhere
$ df -h