Force a mongodb in a replica set to be a primary

There was one situation, where our production mongo server suddenly becomes secondary, causing any write and read to the server to fail. Searching in the mongo documentation, we found an easy solution. Below are the steps.

First step:
Check that your replica is running fine
mongo> rs.status()

Second step:
Freeze all mongo node in the replica that you do not want to be primary for lets say, 120 seconds. Access the mongo shell, and run below command, do this for all nodes that you do not want to be primary
mongo> rs.freeze(120) 

Third step:
Demote the current primary, so that other node that has not been frozen, will take over as primary. Run this in mongo shell, to demote the node from being a primary, for 120 seconds
mongo> rs.stepdown(120)

That's it, run rs.status() again to make sure that your desired server is now a primary.

Reference: https://docs.mongodb.com/manual/tutorial/force-member-to-be-primary/

Use centos repository in RHEL

Get the repository url in your country, by visiting http://mirror-status.centos.org/. In my case, since I am in Malaysia, I chose http://centos.ipserverone.com/centos/

Create a repo file in /etc/yum.repos.d
# cat > /etc/yum.repos.d/centos.repo
[centos]
name=CentOS-7
baseurl=http://centos.ipserverone.com/centos/7/os/x86_64/
enabled=1
gpgcheck=1
gpgkey=http://centos.ipserverone.com/centos/7/os/x86_64/RPM-GPG-KEY-CentOS-7

Press ctrl-d to exit and save

Check whether your new centos repo is listed and ready to be used
# yum repolist

Done

Backup RHEV/RHV/ovirt disk image to external drive

Login to RHV/RHEV/ovirt administrator interface

Search for your VM, lets say I want to search for windows VM

Right click on the VM name, and choose 'Create Snapshot'

Give a proper name, and tick "Save Memory" if you want to save current state of memory as well. Please be warned, that by choosing "Save Memory", the VM will be paused while the snapshooting is in progress. 

If you check the Snapshots tab, the status of the snapshot will be Locked, and the VM will be paused

Once completed, the status will be changed to Ok

Now, open a terminal inside the rhev machine, and search for the disk image snapshot by running below command:

# virsh -r dumpxml windows | grep "source file"

Your newly created snapshot is on the last line of the above command. Copy the file location, and put it into your external drive

# cp /rhev/data-center/00000002-0002-0002-0002-00000000035d/747b7d84-68d5-4436-98f5-baeec68381e1/images/8af6c761-94ac-4adb-bf33-f009e3dd3dc7/../8af6c761-94ac-4adb-bf33-f009e3dd3dc7/../8af6c761-94ac-4adb-bf33-f009e3dd3dc7/../8af6c761-94ac-4adb-bf33-f009e3dd3dc7/../8af6c761-94ac-4adb-bf33-f009e3dd3dc7/e42c8e85-9566-4953-b881-70c7ce97de0b /mnt/my_external_drive/windows-snapshot-20181219.raw

If you want a different format, use qemu-img command to change it. For example, you require this image to be vmware compatible

# qemu-img -f raw -O vmdk /mnt/my_external_drive/windows-snapshot-20181219.raw /mnt/my_external_drive/windows-snapshot-20181219.vmdk

Send postgresql log to syslog

Enable syslog to listen to udp port 514 (the port number can be changed to suit your need)

# vi /etc/rsyslog.conf

$ModLoad imudp

$UDPServerRun 514

Restart rsyslog

# systemctl restart rsyslog

Check if rsyslog is now listening to 514 udp port

# ss -tulpn | grep 514

udp    UNCONN     0      0                      *:514                   *:*      users:(("rsyslogd",15281,3))

udp    UNCONN     0      0                     :::514                  :::*      users:(("rsyslogd",15281,4))

Set postgres to log to syslog

# vi /var/lib/pgsql/9.5/data/postgresql.conf

log_destination = 'syslog'

Restart postgres

# systemctl restart postgresql-9.5

Check if your postgres log is now being logged by syslog

# tail -f /var/log/messages 

...

Dec 11 08:39:21 mypostgres postgres[21518]: [3-1] < 2018-12-11 08:39:21.932 UTC >LOG:  autovacuum launcher started

Install nvidia driver for Tesla card on ubuntu 16.04

Check what driver your card is currently using, in this case, the card is using the opensource nvidia drivers called nouveau
$ lspci -v | grep -A8 NVIDIA
3d:00.0 3D controller: NVIDIA Corporation Device 1bb3 (rev a1)
        Subsystem: NVIDIA Corporation Device 11d8
        Flags: bus master, fast devsel, latency 0, IRQ 40
        Memory at b7000000 (32-bit, non-prefetchable) [size=16M]
        Memory at 4bfe0000000 (64-bit, prefetchable) [size=256M]
        Memory at 4bff0000000 (64-bit, prefetchable) [size=32M]
        Capabilities:
        Kernel driver in use: nouveau
        Kernel modules: nvidiafb, nouveau

Download the driver for your card from nvidia download website

Choose the correct card model, and operating system, and click SEARCH

Click DOWNLOAD

Click AGREE & DOWNLOAD to download

If you want to download using wget, right click on the AGREE & DOWNLOAD button, and choose "Copy Link Address". Run wget against the link

$ wget http://us.download.nvidia.com/tesla/410.79/nvidia-diag-driver-local-repo-ubuntu1604-410.79_1.0-1_amd64.deb

Install the driver

$ sudo apt install ./nvidia-diag-driver-local-repo-ubuntu1604-410.79_1.0-1_amd64.deb

Make sure you have now nvidiafb in kernel modules 

$ lspci -v | grep -A15 NVIDIA

3d:00.0 3D controller: NVIDIA Corporation Device 1bb3 (rev a1)

        Subsystem: NVIDIA Corporation Device 11d8

        Flags: bus master, fast devsel, latency 0, IRQ 40

        Memory at b7000000 (32-bit, non-prefetchable) [size=16M]

        Memory at 4bfe0000000 (64-bit, prefetchable) [size=256M]

        Memory at 4bff0000000 (64-bit, prefetchable) [size=32M]

        Capabilities: [60] Power Management version 3

        Capabilities: [68] MSI: Enable+ Count=1/1 Maskable- 64bit+

        Capabilities: [78] Express Endpoint, MSI 00

        Capabilities: [100] Virtual Channel

        Capabilities: [250] Latency Tolerance Reporting

        Capabilities: [128] Power Budgeting

        Capabilities: [420] Advanced Error Reporting

        Capabilities: [600] Vendor Specific Information: ID=0001 Rev=1 Len=024

        Capabilities: [900] #19

        Kernel driver in use: nouveau

        Kernel modules: nvidiafb, nouveau

Update

$ sudo apt update

Install cuda-drivers

$ sudo apt install cuda-drivers

Reboot your machine

$ sudo reboot

Check whether your card is using nvidia driver, and not nouveau

$ lspci -v | grep -A8 NVIDIA

3d:00.0 3D controller: NVIDIA Corporation Device 1bb3 (rev a1)

        Subsystem: NVIDIA Corporation Device 11d8

        Flags: bus master, fast devsel, latency 0, IRQ 88

        Memory at b7000000 (32-bit, non-prefetchable) [size=16M]

        Memory at 4bfe0000000 (64-bit, prefetchable) [size=256M]

        Memory at 4bff0000000 (64-bit, prefetchable) [size=32M]

        Capabilities:

        Kernel driver in use: nvidia

        Kernel modules: nvidiafb, nouveau, nvidia_410_drm, nvidia_410

Using tsocks with apt to bypass proxy

Recently, I encountered an error while trying to update ubuntu server 16.04. The error is as below:

E: Failed to fetch http://my.archive.ubuntu.com/ubuntu/dists/xenial/InRelease  Clearsigned file isn't valid, got 'NOSPLIT' (does the network require authentication?)

It seems there is a proxy somewhere in the network, that I do not know. Asking around, even the owner of the server does not even know any proxy server inside their network. So, what I did was, create a socks proxy on my own, to a server outside of the network, and tunnel the apt connection using an application called tsocks.

First, you need to install tsocks. This is a bit tricky since we cannot use the apt command to download and install it from the internet. So I downloaded the deb file, using wget.

$ wget -c http://archive.ubuntu.com/ubuntu/pool/universe/t/tsocks/tsocks_1.8beta5-9.3_amd64.deb

Once downloaded, install using apt on the local file.

$ sudo apt install ./tsocks*.deb

Once installed, create a socks proxy to a server outside of the network, on localhost port 8888 (this is just my favorite number for socks proxy, you can use any number above 1000). You need to be able to ssh to the outside server for this to be possible.

$ ssh foo@server.outside -D 8888

Change the tsocks config file, to suite your new socks proxy

$ sudo cat /etc/tsocks.conf

# You just need to change these 3 lines

server = 127.0.0.1

server_type = 5

server_port = 8888

$ sudo -i 

# tsocks apt update

# tsocks apt upgrade

P/S: You can also set an environment variable called http_proxy, if you do not want to install tsocks, but this setting will be only active on your current bash session. This can be accomplished by:

# export http_proxy='socks5://localhost:8888'

# apt update && apt upgrade

Create a persistent reverse tunnel for a server behind firewall

To do this, you need to have a middleman server (middleman) to act as intermediaries between your workstation and the server behind firewall (target). The best is to have your middleman server running ssh server on the usual port that firewall allows, for example 80 and 443.

Step 1: In the target server, create a passwordless ssh access to your middleman server. Please refer here on how to accomplish that

Step 2: Create a simple bash script in target server that will check for the reverse tunnel connection, and restart the tunnel if the tunnel is broken. Lets say in this case, my middleman ssh server is running on port 443, you want to create a reverse tunnel on port 2222 on middleman server, and you want to use a user called foo in the middleman server. Don't forget to make the script executable by the owner.

$ cat /home/foo/bin/tunnelcheck.sh

#!/bin/bash

SERVER=middleman

SPORT=443

PORT=2222

USER=foo

ssh $USER@$SERVER -p $SPORT -t nc -vz localhost $PORT > /dev/null 2>&1

if [ $? -ne 0 ];

  then ssh -R $PORT:localhost:22 -l $USER -fN $SERVER -p $SPORT

fi

$ chmod u+x /home/foo/bin/tunnelcheck.sh


Step 3: Set a crontab to run the above script every 10 minutes (or whatever interval you think is appropriate)
$ crontab -e
*/10 * * * * /home/foo/bin/tunnelcheck.sh


Step 4: Test the persistency by killing the ssh tunnel, and wait for crontab to run the tunnelcheck.sh script, and restart the tunnel

Step 5: You are now able to access the target server, simply by ssh'ng into port 2222 on middleman server
$ ssh foo@middleman -p 2222

How to test that your usb live cd is working

To do this, you need qemu-kvm. Install it:
$ sudo apt install qemu-kvm


Check the address of your usb, by using dmesg
$ dmesg | grep sd


Once you have the address, you can start it using below command (make sure you allocate at least 1024M memory, else it will return kernel panic)
$ sudo qemu-system-x86_64 -m 1024 -hda /dev/sdb


To use a cdrom, you can use below command
$ sudo qemu-system-x86_64 -m 1024 -cdrom /dev/cdrom


You can also test an iso file, using below command
$ sudo qemu-system-x86_64 -m 1024 -cdrom centos.iso

Using ssh-agent to do ssh agent forwarding

Normally, in order to use a key to login to a server, you need to have the ssh private key inside the machine you are initiating your ssh connection from, and the server needs to have a matching public key already recorded in .ssh/known_hosts. The private key is supposed to be private, and cannot be put everywhere and anywhere.

To overcome this, we can use ssh-agent command, with ssh agent forwarding (-A) feature of ssh.

Start ssh-agent

$ eval `ssh-agent`

OR

$ ssh-agent bash 

Use the agent forwarding feature of ssh to jump through many servers using the same key

$ ssh -A -i mine.key mine@server1

When you want to jump to server2, just run:

$ ssh -A mine@server2

and you will be connected to server2 using the same key (provided you already register the matching public key inside server2)

Once you have finished, do not forget to exit your ssh-agent, by typing exit, or killing the ssh-agent pid

$ kill `pidof ssh-agent`

Deploying cloud based openvpn server easily

Openvpn is an opensource software application that implements virtual private network techniques to create secure point-to-point or site-to-site connections in routed or bridged configurations and remote access facilities. One of the usage of openvpn, is so that we can browse the web without exposing our public ip address. With cloud server nowadays, it is truly cheap and easy to own an openvpn server. In this tutorial, I will use vultr cloud provider to setup my openvpn server.

Create your vultr account here

Once created, login to your account. You will be presented with below dashboard

Click on the + button to create your new cloud server

Choose your desired server location, in my case I choose Singapore

For server type, click on Application tab, and choose openVPN

For server size, choose $5/month for time being. You can always upgrade it to bigger size, if there is a need in the future.

Leave options 4, 5 and 6 as default

Put in your desired server hostname

Click on Deploy Now to deploy the server

Wait while the server is being installed

Once the status turned to "Running", your openvpn server is ready.

Click on the server name, to see the details of your openvpn. Follow the url shown in the details of your server, to get to the openvpn login page

From there, you can download the client for your device, and start using your own, brand new openvpn server, with just $5 a month.

Enjoy :)

Adding new virtual hard disk to existing RHV virtual machine

Login to your RHV/RHEV/ovirt console

Enter your administrator username and password



Once inside, search for your VM name

Scroll down, and choose Disk tab

Add details of your new virtual hard disk and click OK once done.



Make sure that your newly created disk is listed in the Disk tab

Login to your vm, and run lsblk to check your newly created disk
# lsblk 
NAME              MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT
...
vdc               252:32   0   200G  0 disk 

Create partition on the new disk

# cfdisk /dev/vdc

Choose New --> Primary --> Set size --> Write --> answer 'yes' --> Quit

Run lsblk again to check if your partition is successfully created 

# lsblk 
NAME              MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT
...
vdc               252:32   0   200G  0 disk 
└─vdc1            252:33   0   200G  0 part 



Make filesystem for your partition, in this case I want an ext4 partition
# mkfs.ext4 /dev/vdc1 
mke2fs 1.42.9 (28-Dec-2013)
Filesystem label=
OS type: Linux
...
Writing superblocks and filesystem accounting information: done   

Once done, you can mount your partition wherever you want
# mount /dev/vdc1 /mnt
# df -Th /mnt/
Filesystem     Type  Size  Used Avail Use% Mounted on
/dev/vdc1      ext4  197G   61M  187G   1% /mnt

Score!!

Easily checking your domain name availability online

You can easily check this using an online NS checker by ping.eu

Fill in the domain that you would like to check in the white box as shown by the red arrow, like so:

You should be getting response like above, if the domain name is accessible.

If you are getting like below, the dns for that domain is down.

Download rpm with dependencies without installing

The tool we are going to use is yum-utils. First, you need to install yum-utils

# yum install yum-utils

Then, use yumdownloader command, to download the package, adding --destdir for destination the files are going to be downloaded, and --resolve to make yumdownloader download the dependencies as well. For example, I want to download rpm for elinks, and its dependencies, to /tmp:

# yumdownloader --destdir=/tmp --resolve elinks

That's all folks, happy rpm downloading :)

How to install wordpress on ubuntu 18.04

There are 3 main components to be installed, before wordpress can be deployed, which are apache2, php and mariadb-server.

Install apache2

$ sudo apt install apache2 libapache2-mod-php 

Install php and its components
$ sudo apt install php php-curl php-gd php-mbstring php-xml php-xmlrpc php-mysql

Install mariadb-server
$ sudo apt install mariadb-server

Create new database
$ mysql -u root
MariaDB [none]> create database mywordpressdb;
MariaDB [none]> grant all on mywordpressdb.* to mywordpressuser@localhost identified by 'password';
MariaDB [none]> flush privileges;

Test the new database using the new user created, you should be able to access the database using the credential created above
$ mysql -u mywordpressuser -p mywordpressdb 
MariaDB [wordpress_db]>

Download wordpress
$ wget -c https://wordpress.org/latest.tar.gz

Extract to /var/www/html
$ tar -xvf latest.tar.gz
$ sudo mv wordpress /var/www/html/mywordpress
$ sudo chown -R www-data /var/www/html/mywordpress

Access your wordpress site using browser, at http://localhost/mywordpress, and follow the onscreen instruction

How to install joomla 3.8.10 on ubuntu 18.04

3 main components to make joomla work are nginx, mariadb and php.

Install nginx
$ sudo apt install nginx

Install mariadb
$ sudo apt install mariadb-server

Install php and required components
$ sudo apt install php php-zlib php-xml php-json php-mcrypt 

Download joomla
$ wget -c https://downloads.joomla.org/cms/joomla3/3-8-10/Joomla_3-8-10-Stable-Full_Package.tar.bz2

Extract joomla
$ mkdir joomla
$ tar -xvf Joomla_3-8-10-Stable-Full_Package.tar.bz2 -C joomla

Deploy to DocumentRoot
$ sudo mv joomla /var/www/html/
$ sudo chown www-data -R /var/www/html/joomla

Create database
$ sudo mysql
MariaDB [(none)]> create database joomla;
MariaDB [(none)]> grant all on joomla.* to joomla@localhost identified by 'password';
MariaDB [(none)]> flush privileges;

Edit nginx config as per below:
server {
    listen 80;
    listen [::]:80;
    root /var/www/html/joomla;
    index  index.php index.html index.htm;
    server_name  192.168.10.100;

    location / {
        try_files $uri $uri/ /index.php?$args;
    }

    location ~ \.php$ {
         include snippets/fastcgi-php.conf;
         fastcgi_pass unix:/var/run/php/php7.2-fpm.sock;
         fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
         include fastcgi_params;
    }

}

Test for syntax error
$ sudo nginx -t

Restart nginx
$ sudo systemctl restart nginx

Browse 192.168.10.100 and follow the joomla installation wizard until finish.

Creating openstack new security group, and allowing inbound port

To see existing security group
$ openstack security group list

To create a new security group named ssh-allow
$ openstack security group create  ssh-allow

To allow port 22 inbound (ingress)
$ openstack security group rule create --ingress --proto tcp --dst-port 22 ssh-allow

To see if our port allowance is successfully implemented (use --fit-width to format the output properly)
$ openstack security group show ssh-allow --fit-width

Setting up mongodb replication

Mongodb needs at least 2 servers, preferably 3, to setup a proper mongodb replication. In this article, we will use below hostname as our mongodb nodes:

192.168.0.10 mongo-1 (primary)
192.168.0.11 mongo-2
192.168.0.12 mongo-3


Make sure mongodb is installed in all servers.

Set mongodb repo:

mongo-1: $ cat > mongodb.repo << EOF >[mongodb]
>name=MongoDB Repository
>baseurl=http://downloads-distro.mongodb.org/repo/redhat/os/x86_64/
>gpgcheck=0
>enabled=1
>EOF
mongo-1: $ sudo mv mongodb.repo /etc/yum.repos.d/

Install mongodb:
mongo-1: $ sudo yum install -y mongodb-org

Set /etc/hosts for each server as below:
mongo-1: $ cat >> hosts << EOF 

192.168.0.10 mongo-1

192.168.0.11 mongo-2

192.168.0.12 mongo-3

EOF 

mongo-1: $ sudo mv /etc/hosts /etc/hosts.original

mongo-1: $ sudo mv hosts /etc/


To ease up this installation, turn off firewall and set selinux to permissive mode, temporarily, in all servers.
mongo-1: $ sudo systemctl stop firewalld mongo-1: $ sudo setenforce 0

Edit /etc/mongod.conf in every server, to be similar as below (assuming we are using myreplica as our replSet)

mongo-1: $ sudo cat /etc/mongod.conf 

logpath=/var/log/mongodb/mongod.log 

logappend=true 

fork=true 

dbpath=/var/lib/mongo 

pidfilepath=/var/run/mongodb/mongod.pid 

replSet=myreplica

Once editing is done, restart mongodb in each server

mongo-1: $ sudo systemctl restart mongod

On the first server initiate mongo replica:

mongo-1: $ sudo mongo

MongoDB shell version: x.x.x

connecting to: test

Server has startup warnings: 

2018-05-28T04:39:22.580+0000 [initandlisten] 

2018-05-28T04:39:22.580+0000 [initandlisten] ** WARNING: Readahead for /var/lib/mongo is set to 4096KB

2018-05-28T04:39:22.580+0000 [initandlisten] **          We suggest setting it to 256KB (512 sectors) or less

2018-05-28T04:39:22.580+0000 [initandlisten] **          http://dochub.mongodb.org/core/readahead

myreplica:PRIMARY> rs.initiate() 

Add the other server, namely mongo-2 and mongo-3 to the replicaset

myreplica:PRIMARY> rs.add("mongo-2")

myreplica:PRIMARY> rs.add("mongo-3")

Run rs.status() to see the status of our replica
myreplica:PRIMARY> rs.status() 

{

        "set" : "myreplica",

        "date" : ISODate("2018-05-28T05:32:10Z"),

        "myState" : 1,

        "members" : [

                {

                        "_id" : 0,

                        "name" : "mongodb01.novalocal.local:27017",

                        "health" : 1,

                        "state" : 1,

                        "stateStr" : "PRIMARY",

                        "uptime" : 11,

                        "optime" : Timestamp(1604978545, 5),

                        "optimeDate" : ISODate("2018-05-28T05:32:10Z"),

                        "electionTime" : Timestamp(1604626688, 1),

                        "electionDate" : ISODate("2018-05-28T05:32:10Z"),

                        "self" : true

                },

        ],

        "ok" : 1

}

In order to rectify the "stateStr: UNKNOWN" and "lastHeartbeatMessage: still initializing", simply add the name of the primary server, as given by mongodb in /etc/hosts of all secondary servers

mongo-2: $ cat /etc/hosts

192.168.0.10 mongo-1 mongodb-1.novalocal
192.168.0.11 mongo-2

192.168.0.12 mongo-3 

mongo-3: $ cat /etc/hosts

192.168.0.10 mongo-1 mongodb-1.novalocal
192.168.0.11 mongo-2

192.168.0.12 mongo-3 

You should be getting "syncingTo : mongodb-1.novalocal:27017", and "stateStr: SECONDARY" when you run rs.status() in primary server

myreplica:PRIMARY> rs.status()

...

{

                        "_id" : 2,

                        "name" : "mongo-3:27017",

                        "health" : 1,

                        "state" : 2,

                        "stateStr" : "SECONDARY",

                        "uptime" : 368,

                        "optime" : Timestamp(1527485519, 1),

                        "optimeDate" : ISODate("2018-05-28T05:31:59Z"),

                        "lastHeartbeat" : ISODate("2018-05-28T05:38:06Z"),

                        "lastHeartbeatRecv" : ISODate("2018-05-28T05:38:06Z"),

                        "pingMs" : 1,

                        "syncingTo" : "mongodb-1.novalocal:27017"

                }

...

Your replica is now complete. To test it out:

Create new database in primary server, and fill up with data

myreplica:PRIMARY> use mynewdb

myreplica:PRIMARY> db.stack.save(

... {

...     "name": "myreplica",

...     "description":  "this is my new mongodb replica",

...     "hosts" : [ "mongo-1", "mongo-2", "mongo-3" ],

... })

WriteResult({ "nInserted" : 1 })

myreplica:PRIMARY> show dbs

admin      (empty)

local      2.077GB

mynewdb    0.078GB

myreplica:PRIMARY> show collections;

stack

system.indexes

myreplica:PRIMARY> db.stack.find()

{ "_id" : ObjectId("5b0b97f9aca2dd0afb9d86a5"), "name" : "myreplica", "description" : "this is my new mongodb replica", "hosts" : [ "mongo-1", "mongo-2", "mongo-3" ] }

Login to secondary servers, sync (by running "rs.slaveOk()" ) and check whether the data gets replicated

myreplica:SECONDARY> use mynewdb

switched to db mynewdb

myreplica:SECONDARY> show collections

2018-05-28T05:51:42.601+0000 error: { "$err" : "not master and slaveOk=false", "code" : 13435 } at src/mongo/shell/query.js:131

myreplica:SECONDARY> rs.slaveOk()

myreplica:SECONDARY> show collections

stack

system.indexes

myreplica:SECONDARY> db.stack.find()

{ "_id" : ObjectId("5b0b97f9aca2dd0afb9d86a5"), "name" : "myreplica", "description" : "this is my new mongodb replica", "hosts" : [ "mongo-1", "mongo-2", "mongo-3" ] }

Done :)

Creating image from a running instance

Sometimes we want to spawn off a few new instances, with the same spec and operating systems, but we do not want to go through the hassle of setting up each OS manually, and then update it one by one. In order to do that efficiently, openstack provides a very good way, which is to create an image from a running instance, and this image can be used to spawn off new instances afterwards.

Before we turn any instance to an image, we need to know its instance ID

$ openstack server list

We can then create an image from the above instance ID

$ openstack server image create --name centos7-updated-20180525 21e78f23-8b67-423a-9622-d46c8487f829

To make sure our image is created correctly, check using:

$ openstack image list

To create a new instance from the image, please refer here

Creating a new instance on openstack

In order to create new instance (it is called server in openstack command), you need to know beforehand a few information to feed to the create instance command. Refer below for those information:

check available flavor

$ openstack flavor list

check available images

$ openstack image list

check available network

$ openstack network list

check available security group

$ openstack security group list

check available keypair

$ openstack keypair list

Once you get all the above information, to create the new instance, just use below command, providing the above information as option to openstack server create command

$ openstack server create \

--image centos-7-20180520 \

--key-name my-keypair \

--flavor m1.medium \

--security-group defaults \

--network private-140 \

thenewinstancename

To check whether your new instance has been created and active:

$ openstack server list

Simple http server using nodejs http-server

This method is particularly useful if you want to make some files on your server, downloadable on the internet, via any port you choose, directly using your ip (or public ip)

Install epel repository
# yum install epel-release

Install npm
# yum install npm

Check whether npm (nodejs package manager) is installed
# npm -v

Install http-server globally
# npm install http-server -g

Start http-server on port 80, serving content in /tmp/test
# http-server /tmp/test -p 80

You should now able to browse to the ip address of the machine (in this example, 139.x.x.x) using a browser, and should be able to download the content of /tmp/test

Sending email using netcat (nc) command

Sometimes you just have to send an email from a server you are currently working on, or you just need to test whether your email server is working, either way, you can always count on netcat to to the job for you. Below command will connect to the mail server you want, on port 25, and send email from there. Please take note, that this command is used in CentOS 7 server, using nmap-ncat-6.40-7.el7 package.

Connect to the mail.foo.bar on port 25

$ nc mail.foo.bar 25
220 mail.foo.bar ESMTP Postfix

Once connected, say hello to the email server

$ nc my.email.server 25 220 mail.foo.bar ESMTP Postfix

HELO foo.bar

250 mail.foo.bar

Specify a return address, just in case the email bounced. This is an SMTP protocol requirement.

$ nc my.email.server 25 220 mail.foo.bar ESMTP Postfix
HELO foo.bar 

250 mail.foo.bar 

MAIL FROM: mrbean@foo.bar 

250 2.1.0 Ok

Specify recipient of the email.

$ nc my.email.server 25 220 mail.foo.bar ESMTP Postfix
HELO foo.bar
250 mail.foo.bar 

MAIL FROM: mrbean@foo.bar 

250 2.1.0 Ok 

RCPT TO: teddy@foo.bar 

250 2.1.5 Ok

Put in you message (any message will do, but below is just how normal email usually formatted). Once you are done typing in your message, put dot (.) as a signal you are done with the DATA, and server will reply with your queue number.

$ nc my.email.server 25 220 mail.foo.bar ESMTP Postfix
HELO foo.bar
250 mail.foo.bar 

MAIL FROM: mrbean@foo.bar
250 2.1.0 Ok 

RCPT TO: teddy@foo.bar
250 2.1.5 Ok 

DATA
354 End data with .
From: mrbean@foo.bar
To: teddy@foo.bar
Subject: A gift
Date: Thu, 22 Mar 2018 12:00:00 +0000
This is merely a gift for you. Enjoy.
.
250 2.0.0 Ok: queued as 7F571241267C

To exit, just type quit, and you are done.

$ nc my.email.server 25 220 mail.foo.bar ESMTP Postfix
HELO foo.bar
250 mail.foo.bar 

MAIL FROM: mrbean@foo.bar
250 2.1.0 Ok 

RCPT TO: teddy@foo.bar
250 2.1.5 Ok 

DATA
354 End data with .
From: mrbean@foo.bar
To: teddy@foo.bar
Subject: A gift
Date: Thu, 22 Mar 2018 12:00:00 +0000
This is merely a gift for you. Enjoy.
.
250 2.0.0 Ok: queued as 7F571241267C 

QUIT
221 2.0.0 Bye

That's all,  the recipient should already received the email, or you should get a return email if it is bounced.

Creating large gpt partition using gdisk

Check the current partition

# parted /dev/sda print
Model: Cisco (scsi)
Disk /dev/sda: 78124994560s
Sector size (logical/physical): 512B/512B
Partition Table: msdos
Disk Flags:
Number  Start    End                Size                Type       File system  Name                             Flags
 1            2048s  4095s             2048s             primary                       Linux Filesystem
 2            4096s  4294967295s 4294963200s primary   xfs               Linux Filesystem           boot

Run gdisk on the disk

# gdisk /dev/sda
GPT fdisk (gdisk) version 0.8.6
Partition table scan:
  MBR: MBR only
  BSD: not present
  APM: not present
  GPT: not present
***************************************************************
Found invalid GPT and valid MBR; converting MBR to GPT format.
THIS OPERATION IS POTENTIALLY DESTRUCTIVE! Exit by typing 'q' if
you don't want to convert your MBR partitions to GPT format!
***************************************************************
Command (? for help):

Press 'p' to print current partition, and to see how much free space is available

Command (? for help): p
Disk /dev/sda: 78124994560 sectors, 36.4 TiB
Logical sector size: 512 bytes
Disk identifier (GUID): F0A6B5A3-3C91-47CE-BD70-FAD8A88DD132
Partition table holds up to 128 entries
First usable sector is 34, last usable sector is 78124994526
Partitions will be aligned on 2048-sector boundaries
Total free space is 73830029245 sectors (34.4 TiB)
Number  Start (sector)    End (sector)  Size              Code  Name
   1            2048               4095              1024.0 KiB  8300  Linux filesystem
   2            4096               4294967295  2.0 TiB         8300  Linux filesystem

Press 'n' to create new partition, and choose the default partition type, which is Linux filesystem

Command (? for help): n
Partition number (3-128, default 3):
First sector (34-78124994526, default = 4294967296) or {+-}size{KMGTP}:
Last sector (4294967296-78124994526, default = 78124994526) or {+-}size{KMGTP}:
Current type is 'Linux filesystem'
Hex code or GUID (L to show codes, Enter = 8300):
Changed type of partition to 'Linux filesystem'

Press 't', and type 'ef02' to change the first partition type, to bios boot partition. This to make sure that that this partition will be used for grub reinstallation later

Command (? for help): t
Partition number (1-3): 1
Current type is 'Linux filesystem'
Hex code or GUID (L to show codes, Enter = 8300): L
0700 Microsoft basic data  0c01 Microsoft reserved    2700 Windows RE
4200 Windows LDM data      4201 Windows LDM metadata  7501 IBM GPFS
7f00 ChromeOS kernel       7f01 ChromeOS root         7f02 ChromeOS reserved
8200 Linux swap            8300 Linux filesystem      8301 Linux reserved
8e00 Linux LVM             a500 FreeBSD disklabel     a501 FreeBSD boot
a502 FreeBSD swap          a503 FreeBSD UFS           a504 FreeBSD ZFS
a505 FreeBSD Vinum/RAID    a580 Midnight BSD data     a581 Midnight BSD boot
a582 Midnight BSD swap     a583 Midnight BSD UFS      a584 Midnight BSD ZFS          a585 Midnight BSD Vinum    a800 Apple UFS             a901 NetBSD swap
a902 NetBSD FFS            a903 NetBSD LFS            a904 NetBSD concatenated
a905 NetBSD encrypted      a906 NetBSD RAID           ab00 Apple boot
af00 Apple HFS/HFS+        af01 Apple RAID            af02 Apple RAID offline
af03 Apple label           af04 AppleTV recovery      af05 Apple Core Storage
be00 Solaris boot          bf00 Solaris root          bf01 Solaris /usr & Mac Z
bf02 Solaris swap          bf03 Solaris backup        bf04 Solaris /var
bf05 Solaris /home         bf06 Solaris alternate se  bf07 Solaris Reserved 1
bf08 Solaris Reserved 2    bf09 Solaris Reserved 3    bf0a Solaris Reserved 4
bf0b Solaris Reserved 5    c001 HP-UX data            c002 HP-UX service
ed00 Sony system partitio  ef00 EFI System            ef01 MBR partition scheme
ef02 BIOS boot partition   fb00 VMWare VMFS           fb01 VMWare reserved
fc00 VMWare kcore crash p  fd00 Linux RAID
Hex code or GUID (L to show codes, Enter = 8300): ef02
Changed type of partition to 'BIOS boot partition'

Press 'p' to check the changes that you have made

Command (? for help): p
Disk /dev/sda: 78124994560 sectors, 36.4 TiB
Logical sector size: 512 bytes
Disk identifier (GUID): F0A6B5A3-3C91-47CE-BD70-FAD8A88DD132
Partition table holds up to 128 entries
First usable sector is 34, last usable sector is 78124994526
Partitions will be aligned on 2048-sector boundaries
Total free space is 2014 sectors (1007.0 KiB)
Number  Start (sector)    End (sector)  Size       Code  Name
   1            2048            4095   1024.0 KiB  EF02  BIOS boot partition
   2            4096      4294967295   2.0 TiB     8300  Linux filesystem
   3      4294967296     78124994526   34.4 TiB    8300  Linux filesystem

Press 'w' to write the changes to disk, and answer 'Y' for confirmation

Command (? for help): w
Final checks complete. About to write GPT data. THIS WILL OVERWRITE EXISTING
PARTITIONS!!
Do you want to proceed? (Y/N): Y
OK; writing new GUID partition table (GPT) to /dev/sda.
Warning: The kernel is still using the old partition table.
The new table will be used at the next reboot.
The operation has completed successfully.

Reinstall grub, so that it will be located in the first partition of the hardisk

# grub2-install /dev/sdaInstalling for i386-pc platform.
Installation finished. No error reported.

Reboot the machine, for the new partition table to be activated

# reboot

Once booted, check whether the new partition has been successfully created, with GPT partition table 

# parted /dev/sda printModel: Cisco (scsi)
Disk /dev/sda: 40.0TB
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Disk Flags:
Number  Start       End        Size         File system  Name                        Flags
 1            1049kB  2097kB  1049kB                       BIOS boot partition  bios_grub
 2            2097kB  2199GB 2199GB  xfs               Linux filesystem
 3            2199GB 40.0TB   37.8TB                       Linux filesystem

Extracting and creating war files in centos

To manipulate war files, you need a command called jar, and this command is available from openjdk-devel package.

Install openjdk-devel -y

# yum install java-1.8.0-openjdk-devel

You should be able to use jar command

# which jar

/usr/bin/jar

View the content of a war file

# jar tf sample.jar

Extract the content of a war file

# jar xf sample.jar

Create war from files

# jar cf sample.jar /path/to/files

Rsync to a different ssh port

In order to use rsync with different ssh post, you can use -e flag, and supply ssh command with port option. For example, I want to transfer a file called /root/book.txt from a server called project.local with sshd running on port 55522 to local partition /mnt, I can use this command:

$ rsync -avz --progress -e "ssh -p 55522" root@project.local:/root/book.txt /mnt

where:
-a for archive mode
-v for verbose
-z to compress data during transfer
--progress for showing progress of the copy
-e to specify the remote shell to use, in this case to use ssh to port 55522

Resizing partition in centos after vmware hardisk extension

This is usually done, when the hardisk space for the CentOS VM is running low.

Check your current disk size

# fdisk -lu /dev/sda

To increase the size of the hardisk image, shutdown the centos VM

# poweroff

Increase the hardisk image in VMware

Power on the centos VM

Once logged in, check back whether you have increased your disk size

# fdisk -lu /dev/sda

Create new partition on the new disk space

# fdisk /dev/sda

Press:

n   {new partition}

p   {primary partition}

3   {partition number}

t    {change partition id}

8e {Linux LVM partition}

w  {write partition change to disk}

Reboot the vm

# reboot

Check the new partition

# fdisk -lu /dev/sda

Create physical volume (PV) on the new partition

# pvcreate /dev/sda3

Check list of PV

# pvs

Extend the current Volume Group (VG) to include the new PV, and check the new VG size

# vgextend centos /dev/sda3

# vgs

Extend the current Logical Volume (LV), and check the new size of logical volume 

# lvextend /dev/mapper/centos-root /dev/sda3

# lvs

Resize / partition

# resize2fs /dev/mapper/centos-root

Check the new size

# df -h /

before resize:

after resize: