Notes on Ceph librados Client

Cluster Connection

• A client is an application that uses librados to connect to a Ceph cluster.

• It needs a cluster object populatd with cluster info (cluster name, info from ceph.conf)

• Then the client do a rados_connect and cluster handle is populated.

• A cluster handle can bind with different pools.

​

Cluster IO context

• The I/O happens on a pool so the connection needs to bind to a pool.

• The connection to a pool gives the client an I/O context.

• The client only species an object name/xattr and librados maps it to a PG & OSD in the cluster.

• An obhect write to rados require key, value, and value size.

• librados::bufferlist is primarily used for storing object value.

​

References

• http://docs.ceph.com/docs/luminous/rados/api/librados-intro/

​

​

Written with StackEdit.

How to secure pools in a Ceph cluster

We have three flags to set per pool: nopgchange, nodelete, and nosizechange

$ for pool in $(rados lspools); do ceph osd pool set $pool nopgchange true; done

Set application on all pools
for pool in $(rados lspools); do ceph osd pool application enable $pool rgw; done

Ceph radosgw: ERROR: endpoints not configured for upstream zone, meta sync: ERROR: failed to fetch mdlog info

The s3cmd client was getting error code 500 for bucket creation.

rgw logs were showing following errors:

2018-06-06 18:57:01.808147 7f0114b8d700  0 ERROR: endpoints not configured for upstream zone
2018-06-06 18:57:01.808164 7f0114b8d700  0 meta sync: ERROR: failed to fetch mdlog info
2018-06-06 18:57:31.808298 7f0114b8d700  0 ERROR: endpoints not configured for upstream zone
2018-06-06 18:57:31.808326 7f0114b8d700  0 meta sync: ERROR: failed to fetch mdlog info

This problem can occur with an improperly configured zone.

Solution

In my case I resolved it by running the following commands:
 
$ radosgw-admin zone modify --rgw_realm=my_realm --rgw-zonegroup=in --rgw-zone=north --master --default

$ radosgw-admin period update

$ radosgw-admin period commit

Notes from http://lists.ceph.com/pipermail/ceph-users-ceph.com/2016-July/011950.html

#!/bin/sh

set -x

RADOSGW_ADMIN=radosgw-admin

echo "Exercise initialization code"
$RADOSGW_ADMIN user info --uid=foo # exercise init code (???)

echo "Get default zonegroup"
$RADOSGW_ADMIN zonegroup get --rgw-zonegroup=default | sed 's/"id":.*/"id": "default",/g' | sed 's/"master_zone.*/"master_zone": "default",/g' > default-zg.json

echo "Get default zone"
$RADOSGW_ADMIN zone get --zone-id=default > default-zone.json

echo "Creating realm"
$RADOSGW_ADMIN realm create --rgw-realm=myrealm

echo "Creating default zonegroup"
$RADOSGW_ADMIN zonegroup set --rgw-zonegroup=default < default-zg.json

echo "Creating default zone"
$RADOSGW_ADMIN zone set --rgw-zone=default < default-zone.json

echo "Setting default zonegroup to 'default'"
$RADOSGW_ADMIN zonegroup default --rgw-zonegroup=default

echo "Setting default zone to 'default'"
$RADOSGW_ADMIN zone default --rgw-zone=default

How to delete a commit in Git

$ git rebase -p --onto SHA^ SHA

The SHA is the commit ID.

Reference