Openstack with Ceph Storage Backend Provisioning

Halo, kali ini kita akan membahas bagaimana deploy openstack dengan ceph sebagai backend storage nya. Pada artikel ini openstack yang digunakan versi Ussuri dan Ceph Octopus dengan kolla-ansible. letsgoo

Prerequisite

Konfigurasi Network:

public api: 10.20.10.0/24 (ens3)
internal api: 10.20.11.0/24 (ens4)
self-service: 10.20.12.0/24 (ens5)
provider: 10.20.13.0/24 (ens6)
ceph public: 10.20.14.0/24 (ens7)
ceph cluster: 10.20.15.0/24 (ens8)

Service yang akan diinstall:

Core service (keystone, nova, neutron, glance)
Ceph backend (Cinder)
Horizon (dashboard)

Preparation

Sebelum masuk ke tahap instalasi lakukan langkah - langkah persiapan terlebih dahulu

Note: Jalankan perintah dibawah ini pada semua node

Update & Upgrades paket pada semua node

apt update -y && sudo apt upgrade -y

Tambahkan host untuk semua node

nano /etc/hosts
....
10.20.11.211 do-os-controller1 do-os-controller1.internal.donny.lab
10.20.10.211 do-os-controller1.public.donny.lab

10.20.11.212 do-os-controller2 do-os-controller2.internal.donny.lab
10.20.10.212 do-os-controller2.public.donny.lab

10.20.11.213 do-os-controller3 do-os-controller3.internal.donny.lab
10.20.10.213 do-os-controller3.public.donny.lab

10.20.11.214 do-os-compute1 do-os-compute1.internal.donny.lab
10.20.10.214 do-os-compute1.public.donny.lab

10.20.11.215 do-os-compute2 do-os-compute2.internal.donny.lab
10.20.10.215 do-os-compute2.public.donny.lab

10.20.11.216 do-os-compute3 do-os-compute3.internal.donny.lab
10.20.10.216 do-os-compute3.public.donny.lab

10.20.11.200 internal.donny.lab
10.20.10.200 public.donny.lab
....

Membuat SSH keypair dan copy ke semua node agar dapat ssh tanpa menggunakan password

ssh-keygen -t rsa
tee -a /root/.ssh/authorized_keys << EOF
<ssh pub key do-os-controller1>
<ssh pub key do-os-controller2>
<ssh pub key do-os-controller3>
<ssh pub key do-os-compute1>
<ssh pub key do-os-compute2>
<ssh pub key do-os-compute3>
EOF

Instalasi Ceph

Note: Jalankan perintah dibawah ini di deployer node

Install paket pip3

apt-get install python3-pip -y

Clone file dari repo ceph-ansible

git clone https://github.com/ceph/ceph-ansible.git
cd ceph-ansible
git checkout stable-5.0

Install dependensi yang dibutuhkan

pip3 install -r requirements.txt

Buat file konfigurasi ceph

cp site.yml.sample site.yml
cd group_vars/
cp all.yml.sample all.yml
cp mons.yml.sample mons.yml
cp osds.yml.sample osds.yml
cp mgrs.yml.sample mgrs.yml

nano all.yml
....
ceph_origin: repository 
ceph_repository: community 
ceph_stable_release: octopus 
monitor_interface: ens7
osd_objectstore: bluestore 
public_network: 10.20.14.0/24 
cluster_network: 10.20.15.0/24 
dashboard_enabled: True 
dashboard_admin_user: admin 
dashboard_admin_password: BHEJmFDWEDmFzdWOMoMm1a/j6+HeVoHUROKY378J7pE=
grafana_admin_user: admin
grafana_admin_password: yb2DWtryOY1B6HXJTJd44mjqVd43aE4vcyG6NWepz94=
ntp_daemon_type: timesyncd
....

nano osds.yml
....
devices:
  - /dev/vdb
  - /dev/vdc
  
osd_auto_discovery: false
....

Buat ansible file konfigurasi

cd ../
nano ansible.cfg
....
[defaults] 
inventory=hosts
....

Buat ansible inventory file

nano hosts
....
[mons] 
do-os-controller1 
do-os-controller2 
do-os-controller3 
 
[mgrs] 
do-os-controller1 
do-os-controller2 
do-os-controller3 
 
[osds] 
do-os-compute1 
do-os-compute2 
do-os-compute3 
 
[grafana-server] 
do-os-controller1 
....

Cek koneksi ansible host

ansible -m ping all

deploy ceph

ansible-playbook site.yml

Buat pool pada ceph

ceph osd pool create volumes
ceph osd pool create images
ceph osd pool create backups
ceph osd pool create vms

Set pool ke rbd ceph

rbd pool init volumes
rbd pool init images
rbd pool init backups
rbd pool init vms

Buat keyring ceph

ceph auth get-or-create client.glance mon 'allow r' osd 'allow class-read object_prefix rbd_children, allow rwx pool=images' -o /etc/ceph/ceph.client.glance.keyring
ceph auth get-or-create client.cinder mon 'allow r' osd 'allow class-read object_prefix rbd_children, allow rwx pool=volumes, allow rwx pool=images' -o /etc/ceph/ceph.client.cinder.keyring
ceph auth get-or-create client.nova mon 'allow r' osd 'allow class-read object_prefix rbd_children, allow rwx pool=vms, allow rx pool=images' -o /etc/ceph/ceph.client.nova.keyring
ceph auth get-or-create client.cinder-backup mon 'allow r' osd 'allow class-read object_prefix rbd_children, allow rwx pool=backups' -o /etc/ceph/ceph.client.cinder-backup.keyring

Instalasi Openstack

Note: Jalankan perintah dibawah ini di deployer node

Instal dependensi yang dibutuhkan

apt-get install python3-dev libffi-dev gcc libssl-dev python3-selinux python3-setuptools -y

Instal kolla-ansible menggunakan pip3

pip3 install kolla-ansible==10.2.0

Melakukan konfigurasi kolla-ansible

mkdir ~/openstack
cd ~/openstack
mkdir -p /etc/kolla
chown -R $USER:$USER /etc/kolla
cp -r /usr/local/share/kolla-ansible/etc_examples/kolla/* /etc/kolla
cp /usr/local/share/kolla-ansible/ansible/inventory/* .

Membuat ansible.cfg file

mkdir /etc/ansible
nano /etc/ansible/ansible.cfg
....
[defaults]
host_key_checking=False
pipelining=True
forks=100
interpreter_python=/usr/bin/python3
....

Membuat ansible inventory

nano multinode
....
[control] 
do-os-controller1 
do-os-controller2 
do-os-controller3 
 
[network] 
do-os-controller1 
do-os-controller2 
do-os-controller3 
 
[compute] 
do-os-compute1 
do-os-compute2 
do-os-compute3 
 
[monitoring] 
do-os-controller1 
do-os-controller2 
do-os-controller3 
 
[storage] 
do-os-controller1 
do-os-controller2 
do-os-controller3 
 
[deployment] 
localhost       ansible_connection=local
....

Verifikasi koneksi ansible invetory

ansible -i multinode all -m ping

Generate password untuk semua service openstack

kolla-genpwd

Membuat global konfigurasi kolla-ansible

nano /etc/kolla/globals.yml
....
# global
kolla_base_distro: "ubuntu"
kolla_install_type: "source"
openstack_release: "ussuri"
nova_compute_virt_type: "kvm"
## network
kolla_internal_vip_address: "10.20.11.200"
kolla_internal_fqdn: "internal.donny.lab"
kolla_external_vip_address: "10.20.10.200"
kolla_external_fqdn: "public.donny.lab"
kolla_external_vip_interface: "ens3"
api_interface: "ens4"
tunnel_interface: "ens5"
neutron_external_interface: "ens6"
neutron_plugin_agent: "ovn"
neutron_ovn_distributed_fip: "yes"
enable_neutron_provider_networks: "yes"
## tls
kolla_enable_tls_internal: "yes"
kolla_enable_tls_external: "yes"
kolla_copy_ca_into_containers: "yes"
kolla_enable_tls_backend: "yes"
openstack_cacert: "/etc/ssl/certs/ca-certificates.crt"
## openstack service
enable_openstack_core: "yes"
enable_cinder: "yes"
enable_fluentd: “no”
## ceph
ceph_cinder_keyring: "ceph.client.cinder.keyring"
ceph_glance_keyring: "ceph.client.glance.keyring"
ceph_nova_keyring: "ceph.client.nova.keyring"
ceph_cinder_backup_keyring: "ceph.client.cinder-backup.keyring"
glance_backend_ceph: "yes"
cinder_backend_ceph: "yes"
nova_backend_ceph: "yes"
## docker
enable_docker_repo: false
docker_apt_package: docker.io

Generate TLS certificate

kolla-ansible -i multinode certificates

Membuat konfigurasi direktori kolla-ansible

mkdir /etc/kolla/config
mkdir /etc/kolla/config/nova
mkdir /etc/kolla/config/glance
mkdir -p /etc/kolla/config/cinder/cinder-volume
mkdir /etc/kolla/config/cinder/cinder-backup

Copy keyring ceph.conf ke direktori kolla-ansible

cp /etc/ceph/ceph.conf /etc/kolla/config/cinder/
cp /etc/ceph/ceph.conf /etc/kolla/config/nova/
cp /etc/ceph/ceph.conf /etc/kolla/config/glance/
cp /etc/ceph/ceph.client.glance.keyring /etc/kolla/config/glance/
cp /etc/ceph/ceph.client.nova.keyring /etc/kolla/config/nova/
cp /etc/ceph/ceph.client.cinder.keyring /etc/kolla/config/nova/
cp /etc/ceph/ceph.client.cinder.keyring /etc/kolla/config/cinder/cinder-volume/
cp /etc/ceph/ceph.client.cinder.keyring /etc/kolla/config/cinder/cinder-backup/
cp /etc/ceph/ceph.client.cinder-backup.keyring /etc/kolla/config/cinder/cinder-backup/

Deploy Openstack

kolla-ansible -i ./multinode bootstrap-servers
kolla-ansible -i ./multinode prechecks
kolla-ansible -i ./multinode deploy
kolla-ansible -i ./multinode post-deploy

Menambahkan path CA cert pada rc file

echo "export OS_CACERT=/etc/ssl/certs/ca-certificates.crt" | tee -a /etc/kolla/admin-openrc.sh

Menambahkan root CA cert pada file ca-certicates

echo "export OS_CACERT=/etc/ssl/certs/ca-certificates.crt" | tee -a /etc/kolla/admin-openrc.sh

Install openstack client untuk menjalankan openstack dari CLI

cd ~
apt install python3-venv
python3 -m venv osclient
source osclient/bin/activate
pip3 install python-openstackclient

Test service openstack

source /etc/kolla/admin-openrc.sh
openstack service list
openstack compute service list
openstack volume service list
openstack network agent list

Openstack with Ceph Storage Backend Provisioning

Prerequisite

Preparation

Note: Jalankan perintah dibawah ini pada semua node

Instalasi Ceph

Note: Jalankan perintah dibawah ini di deployer node

Instalasi Openstack

Note: Jalankan perintah dibawah ini di deployer node

More articles from DO IT

Adding Octavia Openstack Service

Openstack with Ceph Storage Backend Provisioning