linuxwave: 2018

Saturday, December 29, 2018

Force a mongodb in a replica set to be a primary

There was one situation, where our production mongo server suddenly becomes secondary, causing any write and read to the server to fail. Searching in the mongo documentation, we found an easy solution. Below are the steps.

First step:
Check that your replica is running fine
mongo> rs.status()

Second step:
Freeze all mongo node in the replica that you do not want to be primary for lets say, 120 seconds. Access the mongo shell, and run below command, do this for all nodes that you do not want to be primary
mongo> rs.freeze(120)

Third step:
Demote the current primary, so that other node that has not been frozen, will take over as primary. Run this in mongo shell, to demote the node from being a primary, for 120 seconds
mongo> rs.stepdown(120)

That's it, run rs.status() again to make sure that your desired server is now a primary.

Reference: https://docs.mongodb.com/manual/tutorial/force-member-to-be-primary/

Tuesday, December 18, 2018

Use centos repository in RHEL

Get the repository url in your country, by visiting http://mirror-status.centos.org/. In my case, since I am in Malaysia, I chose http://centos.ipserverone.com/centos/

Create a repo file in /etc/yum.repos.d
# cat > /etc/yum.repos.d/centos.repo
[centos]
name=CentOS-7
baseurl=http://centos.ipserverone.com/centos/7/os/x86_64/
enabled=1
gpgcheck=1
gpgkey=http://centos.ipserverone.com/centos/7/os/x86_64/RPM-GPG-KEY-CentOS-7

Press ctrl-d to exit and save

Check whether your new centos repo is listed and ready to be used
# yum repolist

Done

Backup RHEV/RHV/ovirt disk image to external drive

Search for your VM, lets say I want to search for windows VM

Right click on the VM name, and choose 'Create Snapshot'

Give a proper name, and tick "Save Memory" if you want to save current state of memory as well. Please be warned, that by choosing "Save Memory", the VM will be paused while the snapshooting is in progress.

If you check the Snapshots tab, the status of the snapshot will be Locked, and the VM will be paused

Once completed, the status will be changed to Ok

Now, open a terminal inside the rhev machine, and search for the disk image snapshot by running below command:

# virsh -r dumpxml windows | grep "source file"

Your newly created snapshot is on the last line of the above command. Copy the file location, and put it into your external drive

# cp /rhev/data-center/00000002-0002-0002-0002-00000000035d/747b7d84-68d5-4436-98f5-baeec68381e1/images/8af6c761-94ac-4adb-bf33-f009e3dd3dc7/../8af6c761-94ac-4adb-bf33-f009e3dd3dc7/../8af6c761-94ac-4adb-bf33-f009e3dd3dc7/../8af6c761-94ac-4adb-bf33-f009e3dd3dc7/../8af6c761-94ac-4adb-bf33-f009e3dd3dc7/e42c8e85-9566-4953-b881-70c7ce97de0b /mnt/my_external_drive/windows-snapshot-20181219.raw

If you want a different format, use qemu-img command to change it. For example, you require this image to be vmware compatible

# qemu-img -f raw -O vmdk /mnt/my_external_drive/windows-snapshot-20181219.raw /mnt/my_external_drive/windows-snapshot-20181219.vmdk

Tuesday, December 11, 2018

Send postgresql log to syslog

Enable syslog to listen to udp port 514 (the port number can be changed to suit your need)

# vi /etc/rsyslog.conf

$ModLoad imudp

$UDPServerRun 514

Restart rsyslog

# systemctl restart rsyslog

Check if rsyslog is now listening to 514 udp port

# ss -tulpn | grep 514

udp UNCONN 0 0 *:514 *:* users:(("rsyslogd",15281,3))

udp UNCONN 0 0 :::514 :::* users:(("rsyslogd",15281,4))

Set postgres to log to syslog

# vi /var/lib/pgsql/9.5/data/postgresql.conf

log_destination = 'syslog'

Restart postgres

# systemctl restart postgresql-9.5

Check if your postgres log is now being logged by syslog

# tail -f /var/log/messages

...

Dec 11 08:39:21 mypostgres postgres[21518]: [3-1] < 2018-12-11 08:39:21.932 UTC >LOG: autovacuum launcher started

Tuesday, December 4, 2018

Install nvidia driver for Tesla card on ubuntu 16.04

Check what driver your card is currently using, in this case, the card is using the opensource nvidia drivers called nouveau
$ lspci -v | grep -A8 NVIDIA
3d:00.0 3D controller: NVIDIA Corporation Device 1bb3 (rev a1)
Subsystem: NVIDIA Corporation Device 11d8
Flags: bus master, fast devsel, latency 0, IRQ 40
Memory at b7000000 (32-bit, non-prefetchable) [size=16M]
Memory at 4bfe0000000 (64-bit, prefetchable) [size=256M]
Memory at 4bff0000000 (64-bit, prefetchable) [size=32M]
Capabilities:
Kernel driver in use: nouveau
Kernel modules: nvidiafb, nouveau

Download the driver for your card from nvidia download website

Choose the correct card model, and operating system, and click SEARCH

Click DOWNLOAD

Click AGREE & DOWNLOAD to download

If you want to download using wget, right click on the AGREE & DOWNLOAD button, and choose "Copy Link Address". Run wget against the link

$ wget http://us.download.nvidia.com/tesla/410.79/nvidia-diag-driver-local-repo-ubuntu1604-410.79_1.0-1_amd64.deb

Install the driver

$ sudo apt install ./nvidia-diag-driver-local-repo-ubuntu1604-410.79_1.0-1_amd64.deb

Make sure you have now nvidiafb in kernel modules

$ lspci -v | grep -A15 NVIDIA

3d:00.0 3D controller: NVIDIA Corporation Device 1bb3 (rev a1)

Subsystem: NVIDIA Corporation Device 11d8

Flags: bus master, fast devsel, latency 0, IRQ 40

Memory at b7000000 (32-bit, non-prefetchable) [size=16M]

Memory at 4bfe0000000 (64-bit, prefetchable) [size=256M]

Memory at 4bff0000000 (64-bit, prefetchable) [size=32M]

Capabilities: [60] Power Management version 3

Capabilities: [68] MSI: Enable+ Count=1/1 Maskable- 64bit+

Capabilities: [78] Express Endpoint, MSI 00

Capabilities: [100] Virtual Channel

Capabilities: [250] Latency Tolerance Reporting

Capabilities: [128] Power Budgeting

Capabilities: [420] Advanced Error Reporting

Capabilities: [600] Vendor Specific Information: ID=0001 Rev=1 Len=024

Capabilities: [900] #19

Kernel driver in use: nouveau

Kernel modules: nvidiafb, nouveau

Update

$ sudo apt update

Install cuda-drivers

$ sudo apt install cuda-drivers

Reboot your machine

$ sudo reboot

Check whether your card is using nvidia driver, and not nouveau

$ lspci -v | grep -A8 NVIDIA

3d:00.0 3D controller: NVIDIA Corporation Device 1bb3 (rev a1)

Subsystem: NVIDIA Corporation Device 11d8

Flags: bus master, fast devsel, latency 0, IRQ 88

Memory at b7000000 (32-bit, non-prefetchable) [size=16M]

Memory at 4bfe0000000 (64-bit, prefetchable) [size=256M]

Memory at 4bff0000000 (64-bit, prefetchable) [size=32M]

Capabilities:

Kernel driver in use: nvidia

Kernel modules: nvidiafb, nouveau, nvidia_410_drm, nvidia_410

Monday, December 3, 2018

Using tsocks with apt to bypass proxy

Recently, I encountered an error while trying to update ubuntu server 16.04. The error is as below:

E: Failed to fetch http://my.archive.ubuntu.com/ubuntu/dists/xenial/InRelease Clearsigned file isn't valid, got 'NOSPLIT' (does the network require authentication?)

It seems there is a proxy somewhere in the network, that I do not know. Asking around, even the owner of the server does not even know any proxy server inside their network. So, what I did was, create a socks proxy on my own, to a server outside of the network, and tunnel the apt connection using an application called tsocks.

First, you need to install tsocks. This is a bit tricky since we cannot use the apt command to download and install it from the internet. So I downloaded the deb file, using wget.

$ wget -c http://archive.ubuntu.com/ubuntu/pool/universe/t/tsocks/tsocks_1.8beta5-9.3_amd64.deb

Once downloaded, install using apt on the local file.

$ sudo apt install ./tsocks*.deb

Once installed, create a socks proxy to a server outside of the network, on localhost port 8888 (this is just my favorite number for socks proxy, you can use any number above 1000). You need to be able to ssh to the outside server for this to be possible.

$ ssh foo@server.outside -D 8888

Change the tsocks config file, to suite your new socks proxy

$ sudo cat /etc/tsocks.conf

# You just need to change these 3 lines

server = 127.0.0.1

server_type = 5

server_port = 8888

$ sudo -i

# tsocks apt update

# tsocks apt upgrade

P/S: You can also set an environment variable called http_proxy, if you do not want to install tsocks, but this setting will be only active on your current bash session. This can be accomplished by:

# export http_proxy='socks5://localhost:8888'

# apt update && apt upgrade

Monday, November 26, 2018

Create a persistent reverse tunnel for a server behind firewall

To do this, you need to have a middleman server (middleman) to act as intermediaries between your workstation and the server behind firewall (target). The best is to have your middleman server running ssh server on the usual port that firewall allows, for example 80 and 443.

Step 1: In the target server, create a passwordless ssh access to your middleman server. Please refer here on how to accomplish that

Step 2: Create a simple bash script in target server that will check for the reverse tunnel connection, and restart the tunnel if the tunnel is broken. Lets say in this case, my middleman ssh server is running on port 443, you want to create a reverse tunnel on port 2222 on middleman server, and you want to use a user called foo in the middleman server. Don't forget to make the script executable by the owner.

$ cat /home/foo/bin/tunnelcheck.sh

#!/bin/bash

SERVER=middleman

SPORT=443

PORT=2222

USER=foo

ssh $USER@$SERVER -p $SPORT -t nc -vz localhost $PORT > /dev/null 2>&1

if [ $? -ne 0 ];

then ssh -R $PORT:localhost:22 -l $USER -fN $SERVER -p $SPORT

$ chmod u+x /home/foo/bin/tunnelcheck.sh

Step 3: Set a crontab to run the above script every 10 minutes (or whatever interval you think is appropriate)
$ crontab -e
*/10 * * * * /home/foo/bin/tunnelcheck.sh

Step 4: Test the persistency by killing the ssh tunnel, and wait for crontab to run the tunnelcheck.sh script, and restart the tunnel

Step 5: You are now able to access the target server, simply by ssh'ng into port 2222 on middleman server
$ ssh foo@middleman -p 2222

Tuesday, November 6, 2018

How to test that your usb live cd is working

To do this, you need qemu-kvm. Install it:
$ sudo apt install qemu-kvm

Check the address of your usb, by using dmesg
$ dmesg | grep sd

Once you have the address, you can start it using below command (make sure you allocate at least 1024M memory, else it will return kernel panic)
$ sudo qemu-system-x86_64 -m 1024 -hda /dev/sdb

To use a cdrom, you can use below command
$ sudo qemu-system-x86_64 -m 1024 -cdrom /dev/cdrom

You can also test an iso file, using below command
$ sudo qemu-system-x86_64 -m 1024 -cdrom centos.iso

Thursday, October 25, 2018

Using ssh-agent to do ssh agent forwarding

Normally, in order to use a key to login to a server, you need to have the ssh private key inside the machine you are initiating your ssh connection from, and the server needs to have a matching public key already recorded in .ssh/known_hosts. The private key is supposed to be private, and cannot be put everywhere and anywhere.

To overcome this, we can use ssh-agent command, with ssh agent forwarding (-A) feature of ssh.

Start ssh-agent

$ eval `ssh-agent`

$ ssh-agent bash

Use the agent forwarding feature of ssh to jump through many servers using the same key

$ ssh -A -i mine.key mine@server1

When you want to jump to server2, just run:

$ ssh -A mine@server2

and you will be connected to server2 using the same key (provided you already register the matching public key inside server2)

Once you have finished, do not forget to exit your ssh-agent, by typing exit, or killing the ssh-agent pid

$ kill `pidof ssh-agent`

Tuesday, August 28, 2018

Deploying cloud based openvpn server easily

Openvpn is an opensource software application that implements virtual private network techniques to create secure point-to-point or site-to-site connections in routed or bridged configurations and remote access facilities. One of the usage of openvpn, is so that we can browse the web without exposing our public ip address. With cloud server nowadays, it is truly cheap and easy to own an openvpn server. In this tutorial, I will use vultr cloud provider to setup my openvpn server.

Create your vultr account here

Once created, login to your account. You will be presented with below dashboard

Click on the + button to create your new cloud server

Choose your desired server location, in my case I choose Singapore

For server type, click on Application tab, and choose openVPN

For server size, choose $5/month for time being. You can always upgrade it to bigger size, if there is a need in the future.

Leave options 4, 5 and 6 as default

Put in your desired server hostname

Click on Deploy Now to deploy the server

Wait while the server is being installed

Once the status turned to "Running", your openvpn server is ready.

Click on the server name, to see the details of your openvpn. Follow the url shown in the details of your server, to get to the openvpn login page

From there, you can download the client for your device, and start using your own, brand new openvpn server, with just $5 a month.

Enjoy :)

Tuesday, August 14, 2018

Adding new virtual hard disk to existing RHV virtual machine

Enter your administrator username and password

Once inside, search for your VM name

Scroll down, and choose Disk tab

Add details of your new virtual hard disk and click OK once done.

Make sure that your newly created disk is listed in the Disk tab

Login to your vm, and run lsblk to check your newly created disk
# lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
...
vdc 252:32 0 200G 0 disk

Create partition on the new disk

# cfdisk /dev/vdc

Choose New --> Primary --> Set size --> Write --> answer 'yes' --> Quit

Run lsblk again to check if your partition is successfully created

# lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
...
vdc 252:32 0 200G 0 disk
└─vdc1 252:33 0 200G 0 part

Make filesystem for your partition, in this case I want an ext4 partition
# mkfs.ext4 /dev/vdc1
mke2fs 1.42.9 (28-Dec-2013)
Filesystem label=
OS type: Linux
...
Writing superblocks and filesystem accounting information: done

Once done, you can mount your partition wherever you want
# mount /dev/vdc1 /mnt
# df -Th /mnt/
Filesystem Type Size Used Avail Use% Mounted on
/dev/vdc1 ext4 197G 61M 187G 1% /mnt

Score!!

Monday, August 13, 2018

Easily checking your domain name availability online

You can easily check this using an online NS checker by ping.eu

Fill in the domain that you would like to check in the white box as shown by the red arrow, like so:

You should be getting response like above, if the domain name is accessible.

If you are getting like below, the dns for that domain is down.

Saturday, August 4, 2018

Download rpm with dependencies without installing

The tool we are going to use is yum-utils. First, you need to install yum-utils

# yum install yum-utils

Then, use yumdownloader command, to download the package, adding --destdir for destination the files are going to be downloaded, and --resolve to make yumdownloader download the dependencies as well. For example, I want to download rpm for elinks, and its dependencies, to /tmp:

# yumdownloader --destdir=/tmp --resolve elinks

That's all folks, happy rpm downloading :)

Thursday, July 5, 2018

How to install wordpress on ubuntu 18.04

There are 3 main components to be installed, before wordpress can be deployed, which are apache2, php and mariadb-server.

Install apache2

$ sudo apt install apache2 libapache2-mod-php

Install php and its components
$ sudo apt install php php-curl php-gd php-mbstring php-xml php-xmlrpc php-mysql

Install mariadb-server
$ sudo apt install mariadb-server

Create new database

$ mysql -u root

MariaDB [none]> create database mywordpressdb;

MariaDB [none]> grant all on mywordpressdb.* to mywordpressuser@localhost identified by 'password';

MariaDB [none]> flush privileges;

Test the new database using the new user created, you should be able to access the database using the credential created above

$ mysql -u mywordpressuser -p mywordpressdb 

MariaDB [wordpress_db]>

Download wordpress
$ wget -c https://wordpress.org/latest.tar.gz

Extract to /var/www/html

$ tar -xvf latest.tar.gz 

$ sudo mv wordpress /var/www/html/mywordpress

$ sudo chown -R www-data /var/www/html/mywordpress

Access your wordpress site using browser, at http://localhost/mywordpress, and follow the onscreen instruction

Tuesday, July 3, 2018

How to install joomla 3.8.10 on ubuntu 18.04

3 main components to make joomla work are nginx, mariadb and php.

Install nginx
$ sudo apt install nginx

Install mariadb
$ sudo apt install mariadb-server

Install php and required components
$ sudo apt install php php-zlib php-xml php-json php-mcrypt

Download joomla
$ wget -c https://downloads.joomla.org/cms/joomla3/3-8-10/Joomla_3-8-10-Stable-Full_Package.tar.bz2

Extract joomla

$ mkdir joomla

$ tar -xvf Joomla_3-8-10-Stable-Full_Package.tar.bz2 -C joomla

Deploy to DocumentRoot

$ sudo mv joomla /var/www/html/

$ sudo chown www-data -R /var/www/html/joomla

Create database

$ sudo mysql

MariaDB [(none)]> create database joomla;

MariaDB [(none)]> grant all on joomla.* to joomla@localhost identified by 'password';

MariaDB [(none)]> flush privileges;

Edit nginx config as per below:

server {
    listen 80;
    listen [::]:80;
    root /var/www/html/joomla;
    index  index.php index.html index.htm;
    server_name  192.168.10.100;

    location / {
        try_files $uri $uri/ /index.php?$args;
    }

    location ~ \.php$ {
         include snippets/fastcgi-php.conf;
         fastcgi_pass unix:/var/run/php/php7.2-fpm.sock;
         fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
         include fastcgi_params;
    }

}

Test for syntax error
$ sudo nginx -t

Restart nginx
$ sudo systemctl restart nginx

Browse 192.168.10.100 and follow the joomla installation wizard until finish.

Thursday, May 31, 2018

Creating openstack new security group, and allowing inbound port

To see existing security group
$ openstack security group list

To create a new security group named ssh-allow
$ openstack security group create ssh-allow

To allow port 22 inbound (ingress)
$ openstack security group rule create --ingress --proto tcp --dst-port 22 ssh-allow

To see if our port allowance is successfully implemented (use --fit-width to format the output properly)
$ openstack security group show ssh-allow --fit-width

Monday, May 28, 2018

Setting up mongodb replication

Mongodb needs at least 2 servers, preferably 3, to setup a proper mongodb replication. In this article, we will use below hostname as our mongodb nodes:

192.168.0.10 mongo-1 (primary)

192.168.0.11 mongo-2

192.168.0.12 mongo-3

Make sure mongodb is installed in all servers.

Set mongodb repo:

mongo-1: $ cat > mongodb.repo << EOF >[mongodb]
>name=MongoDB Repository
>baseurl=http://downloads-distro.mongodb.org/repo/redhat/os/x86_64/
>gpgcheck=0
>enabled=1
>EOF
mongo-1: $ sudo mv mongodb.repo /etc/yum.repos.d/

Install mongodb:
mongo-1: $ sudo yum install -y mongodb-org

Set /etc/hosts for each server as below:
mongo-1: $ cat >> hosts << EOF

192.168.0.10 mongo-1

192.168.0.11 mongo-2

192.168.0.12 mongo-3

EOF

mongo-1: $ sudo mv /etc/hosts /etc/hosts.original

mongo-1: $ sudo mv hosts /etc/

To ease up this installation, turn off firewall and set selinux to permissive mode, temporarily, in all servers.
mongo-1: $ sudo systemctl stop firewalld mongo-1: $ sudo setenforce 0

Edit /etc/mongod.conf in every server, to be similar as below (assuming we are using myreplica as our replSet)

mongo-1: $ sudo cat /etc/mongod.conf

logpath=/var/log/mongodb/mongod.log

logappend=true

fork=true

dbpath=/var/lib/mongo

pidfilepath=/var/run/mongodb/mongod.pid

replSet=myreplica

Once editing is done, restart mongodb in each server

mongo-1: $ sudo systemctl restart mongod

On the first server initiate mongo replica:

mongo-1: $ sudo mongo

MongoDB shell version: x.x.x

connecting to: test

Server has startup warnings:

2018-05-28T04:39:22.580+0000 [initandlisten]

2018-05-28T04:39:22.580+0000 [initandlisten] ** WARNING: Readahead for /var/lib/mongo is set to 4096KB

2018-05-28T04:39:22.580+0000 [initandlisten] ** We suggest setting it to 256KB (512 sectors) or less

2018-05-28T04:39:22.580+0000 [initandlisten] ** http://dochub.mongodb.org/core/readahead

myreplica:PRIMARY> rs.initiate()

Add the other server, namely mongo-2 and mongo-3 to the replicaset

myreplica:PRIMARY> rs.add("mongo-2")

myreplica:PRIMARY> rs.add("mongo-3")

Run rs.status() to see the status of our replica
myreplica:PRIMARY> rs.status()

{

"set" : "myreplica",

"date" : ISODate("2018-05-28T05:32:10Z"),

"myState" : 1,

"members" : [

{

"_id" : 0,

"name" : "mongodb01.novalocal.local:27017",

"health" : 1,

"state" : 1,

"stateStr" : "PRIMARY",

"uptime" : 11,

"optime" : Timestamp(1604978545, 5),

"optimeDate" : ISODate("2018-05-28T05:32:10Z"),

"electionTime" : Timestamp(1604626688, 1),

"electionDate" : ISODate("2018-05-28T05:32:10Z"),

"self" : true

"ok" : 1

}

In order to rectify the "stateStr: UNKNOWN" and "lastHeartbeatMessage: still initializing", simply add the name of the primary server, as given by mongodb in /etc/hosts of all secondary servers

mongo-2: $ cat /etc/hosts

192.168.0.10 mongo-1 mongodb-1.novalocal
192.168.0.11 mongo-2

192.168.0.12 mongo-3

mongo-3: $ cat /etc/hosts

192.168.0.10 mongo-1 mongodb-1.novalocal
192.168.0.11 mongo-2

192.168.0.12 mongo-3

You should be getting "syncingTo : mongodb-1.novalocal:27017", and "stateStr: SECONDARY" when you run rs.status() in primary server

myreplica:PRIMARY> rs.status()

...

{

"_id" : 2,

"name" : "mongo-3:27017",

"health" : 1,

"state" : 2,

"stateStr" : "SECONDARY",

"uptime" : 368,

"optime" : Timestamp(1527485519, 1),

"optimeDate" : ISODate("2018-05-28T05:31:59Z"),

"lastHeartbeat" : ISODate("2018-05-28T05:38:06Z"),

"lastHeartbeatRecv" : ISODate("2018-05-28T05:38:06Z"),

"pingMs" : 1,

"syncingTo" : "mongodb-1.novalocal:27017"

}

...

Your replica is now complete. To test it out:

Create new database in primary server, and fill up with data

myreplica:PRIMARY> use mynewdb

myreplica:PRIMARY> db.stack.save(

... {

... "name": "myreplica",

... "description": "this is my new mongodb replica",

... "hosts" : [ "mongo-1", "mongo-2", "mongo-3" ],

... })

WriteResult({ "nInserted" : 1 })

myreplica:PRIMARY> show dbs

admin (empty)

local 2.077GB

mynewdb 0.078GB

myreplica:PRIMARY> show collections;

stack

system.indexes

myreplica:PRIMARY> db.stack.find()

{ "_id" : ObjectId("5b0b97f9aca2dd0afb9d86a5"), "name" : "myreplica", "description" : "this is my new mongodb replica", "hosts" : [ "mongo-1", "mongo-2", "mongo-3" ] }

myreplica:SECONDARY> use mynewdb

switched to db mynewdb

myreplica:SECONDARY> show collections

2018-05-28T05:51:42.601+0000 error: { "$err" : "not master and slaveOk=false", "code" : 13435 } at src/mongo/shell/query.js:131

myreplica:SECONDARY> rs.slaveOk()

myreplica:SECONDARY> show collections

stack

system.indexes

myreplica:SECONDARY> db.stack.find()

{ "_id" : ObjectId("5b0b97f9aca2dd0afb9d86a5"), "name" : "myreplica", "description" : "this is my new mongodb replica", "hosts" : [ "mongo-1", "mongo-2", "mongo-3" ] }

Done :)

Friday, May 25, 2018

Creating image from a running instance

Sometimes we want to spawn off a few new instances, with the same spec and operating systems, but we do not want to go through the hassle of setting up each OS manually, and then update it one by one. In order to do that efficiently, openstack provides a very good way, which is to create an image from a running instance, and this image can be used to spawn off new instances afterwards.

Before we turn any instance to an image, we need to know its instance ID

$ openstack server list

We can then create an image from the above instance ID

$ openstack server image create --name centos7-updated-20180525 21e78f23-8b67-423a-9622-d46c8487f829

To make sure our image is created correctly, check using:

$ openstack image list

To create a new instance from the image, please refer here

Creating a new instance on openstack

In order to create new instance (it is called server in openstack command), you need to know beforehand a few information to feed to the create instance command. Refer below for those information:

check available flavor

$ openstack flavor list

check available images

$ openstack image list

check available network

$ openstack network list

check available security group

$ openstack security group list

check available keypair

$ openstack keypair list

Once you get all the above information, to create the new instance, just use below command, providing the above information as option to openstack server create command

$ openstack server create \

--image centos-7-20180520 \

--key-name my-keypair \

--flavor m1.medium \

--security-group defaults \

--network private-140 \

thenewinstancename

To check whether your new instance has been created and active:

$ openstack server list

Monday, April 16, 2018

Simple http server using nodejs http-server

This method is particularly useful if you want to make some files on your server, downloadable on the internet, via any port you choose, directly using your ip (or public ip)

Install epel repository
# yum install epel-release

Install npm
# yum install npm

Check whether npm (nodejs package manager) is installed
# npm -v

Install http-server globally
# npm install http-server -g

Start http-server on port 80, serving content in /tmp/test
# http-server /tmp/test -p 80

You should now able to browse to the ip address of the machine (in this example, 139.x.x.x) using a browser, and should be able to download the content of /tmp/test

Thursday, March 22, 2018

Sending email using netcat (nc) command

Sometimes you just have to send an email from a server you are currently working on, or you just need to test whether your email server is working, either way, you can always count on netcat to to the job for you. Below command will connect to the mail server you want, on port 25, and send email from there. Please take note, that this command is used in CentOS 7 server, using nmap-ncat-6.40-7.el7 package.

Connect to the mail.foo.bar on port 25

$ nc mail.foo.bar 25
220 mail.foo.bar ESMTP Postfix

Once connected, say hello to the email server

$ nc my.email.server 25 220 mail.foo.bar ESMTP Postfix

HELO foo.bar

250 mail.foo.bar

Specify a return address, just in case the email bounced. This is an SMTP protocol requirement.

$ nc my.email.server 25 220 mail.foo.bar ESMTP Postfix
HELO foo.bar

250 mail.foo.bar

MAIL FROM: mrbean@foo.bar

250 2.1.0 Ok

Specify recipient of the email.

$ nc my.email.server 25 220 mail.foo.bar ESMTP Postfix
HELO foo.bar
250 mail.foo.bar

MAIL FROM: mrbean@foo.bar

250 2.1.0 Ok

RCPT TO: teddy@foo.bar

250 2.1.5 Ok

Put in you message (any message will do, but below is just how normal email usually formatted). Once you are done typing in your message, put dot (.) as a signal you are done with the DATA, and server will reply with your queue number.

$ nc my.email.server 25 220 mail.foo.bar ESMTP Postfix
HELO foo.bar
250 mail.foo.bar

MAIL FROM: mrbean@foo.bar
250 2.1.0 Ok

RCPT TO: teddy@foo.bar
250 2.1.5 Ok

DATA
354 End data with .
From: mrbean@foo.bar
To: teddy@foo.bar
Subject: A gift
Date: Thu, 22 Mar 2018 12:00:00 +0000
This is merely a gift for you. Enjoy.
.
250 2.0.0 Ok: queued as 7F571241267C

To exit, just type quit, and you are done.

$ nc my.email.server 25 220 mail.foo.bar ESMTP Postfix
HELO foo.bar
250 mail.foo.bar

MAIL FROM: mrbean@foo.bar
250 2.1.0 Ok

RCPT TO: teddy@foo.bar
250 2.1.5 Ok

DATA
354 End data with .
From: mrbean@foo.bar
To: teddy@foo.bar
Subject: A gift
Date: Thu, 22 Mar 2018 12:00:00 +0000
This is merely a gift for you. Enjoy.
.
250 2.0.0 Ok: queued as 7F571241267C

QUIT
221 2.0.0 Bye

That's all, the recipient should already received the email, or you should get a return email if it is bounced.

Wednesday, March 21, 2018

Creating large gpt partition using gdisk

Check the current partition

# parted /dev/sda print
Model: Cisco (scsi)
Disk /dev/sda: 78124994560s
Sector size (logical/physical): 512B/512B
Partition Table: msdos
Disk Flags:
Number Start End Size Type File system Name Flags
1 2048s 4095s 2048s primary Linux Filesystem
2 4096s 4294967295s 4294963200s primary xfs Linux Filesystem boot

Run gdisk on the disk

# gdisk /dev/sda
GPT fdisk (gdisk) version 0.8.6
Partition table scan:
MBR: MBR only
BSD: not present
APM: not present
GPT: not present
***************************************************************
Found invalid GPT and valid MBR; converting MBR to GPT format.
THIS OPERATION IS POTENTIALLY DESTRUCTIVE! Exit by typing 'q' if
you don't want to convert your MBR partitions to GPT format!
***************************************************************
Command (? for help):

Press 'p' to print current partition, and to see how much free space is available

Command (? for help): p
Disk /dev/sda: 78124994560 sectors, 36.4 TiB
Logical sector size: 512 bytes
Disk identifier (GUID): F0A6B5A3-3C91-47CE-BD70-FAD8A88DD132
Partition table holds up to 128 entries
First usable sector is 34, last usable sector is 78124994526
Partitions will be aligned on 2048-sector boundaries
Total free space is 73830029245 sectors (34.4 TiB)
Number Start (sector) End (sector) Size Code Name
1 2048 4095 1024.0 KiB 8300 Linux filesystem
2 4096 4294967295 2.0 TiB 8300 Linux filesystem

Press 'n' to create new partition, and choose the default partition type, which is Linux filesystem

Command (? for help): n
Partition number (3-128, default 3):
First sector (34-78124994526, default = 4294967296) or {+-}size{KMGTP}:
Last sector (4294967296-78124994526, default = 78124994526) or {+-}size{KMGTP}:
Current type is 'Linux filesystem'
Hex code or GUID (L to show codes, Enter = 8300):
Changed type of partition to 'Linux filesystem'

Press 't', and type 'ef02' to change the first partition type, to bios boot partition. This to make sure that that this partition will be used for grub reinstallation later

Command (? for help): t
Partition number (1-3): 1
Current type is 'Linux filesystem'
Hex code or GUID (L to show codes, Enter = 8300): L
0700 Microsoft basic data 0c01 Microsoft reserved 2700 Windows RE
4200 Windows LDM data 4201 Windows LDM metadata 7501 IBM GPFS
7f00 ChromeOS kernel 7f01 ChromeOS root 7f02 ChromeOS reserved
8200 Linux swap 8300 Linux filesystem 8301 Linux reserved
8e00 Linux LVM a500 FreeBSD disklabel a501 FreeBSD boot
a502 FreeBSD swap a503 FreeBSD UFS a504 FreeBSD ZFS
a505 FreeBSD Vinum/RAID a580 Midnight BSD data a581 Midnight BSD boot
a582 Midnight BSD swap a583 Midnight BSD UFS a584 Midnight BSD ZFS a585 Midnight BSD Vinum a800 Apple UFS a901 NetBSD swap
a902 NetBSD FFS a903 NetBSD LFS a904 NetBSD concatenated
a905 NetBSD encrypted a906 NetBSD RAID ab00 Apple boot
af00 Apple HFS/HFS+ af01 Apple RAID af02 Apple RAID offline
af03 Apple label af04 AppleTV recovery af05 Apple Core Storage
be00 Solaris boot bf00 Solaris root bf01 Solaris /usr & Mac Z
bf02 Solaris swap bf03 Solaris backup bf04 Solaris /var
bf05 Solaris /home bf06 Solaris alternate se bf07 Solaris Reserved 1
bf08 Solaris Reserved 2 bf09 Solaris Reserved 3 bf0a Solaris Reserved 4
bf0b Solaris Reserved 5 c001 HP-UX data c002 HP-UX service
ed00 Sony system partitio ef00 EFI System ef01 MBR partition scheme
ef02 BIOS boot partition fb00 VMWare VMFS fb01 VMWare reserved
fc00 VMWare kcore crash p fd00 Linux RAID
Hex code or GUID (L to show codes, Enter = 8300): ef02
Changed type of partition to 'BIOS boot partition'

Press 'p' to check the changes that you have made

Command (? for help): p
Disk /dev/sda: 78124994560 sectors, 36.4 TiB
Logical sector size: 512 bytes
Disk identifier (GUID): F0A6B5A3-3C91-47CE-BD70-FAD8A88DD132
Partition table holds up to 128 entries
First usable sector is 34, last usable sector is 78124994526
Partitions will be aligned on 2048-sector boundaries
Total free space is 2014 sectors (1007.0 KiB)
Number Start (sector) End (sector) Size Code Name
1 2048 4095 1024.0 KiB EF02 BIOS boot partition
2 4096 4294967295 2.0 TiB 8300 Linux filesystem
3 4294967296 78124994526 34.4 TiB 8300 Linux filesystem

Press 'w' to write the changes to disk, and answer 'Y' for confirmation

Command (? for help): w
Final checks complete. About to write GPT data. THIS WILL OVERWRITE EXISTING
PARTITIONS!!
Do you want to proceed? (Y/N): Y
OK; writing new GUID partition table (GPT) to /dev/sda.
Warning: The kernel is still using the old partition table.
The new table will be used at the next reboot.
The operation has completed successfully.

Reinstall grub, so that it will be located in the first partition of the hardisk

# grub2-install /dev/sdaInstalling for i386-pc platform.
Installation finished. No error reported.

Reboot the machine, for the new partition table to be activated

# reboot

Once booted, check whether the new partition has been successfully created, with GPT partition table

# parted /dev/sda printModel: Cisco (scsi)
Disk /dev/sda: 40.0TB
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Disk Flags:
Number Start End Size File system Name Flags
1 1049kB 2097kB 1049kB BIOS boot partition bios_grub
2 2097kB 2199GB 2199GB xfs Linux filesystem
3 2199GB 40.0TB 37.8TB Linux filesystem

Thursday, February 8, 2018

Extracting and creating war files in centos

To manipulate war files, you need a command called jar, and this command is available from openjdk-devel package.

Install openjdk-devel -y

# yum install java-1.8.0-openjdk-devel

You should be able to use jar command

# which jar

/usr/bin/jar

View the content of a war file

# jar tf sample.jar

Extract the content of a war file

# jar xf sample.jar

Create war from files

# jar cf sample.jar /path/to/files

Tuesday, January 9, 2018

Rsync to a different ssh port

In order to use rsync with different ssh post, you can use -e flag, and supply ssh command with port option. For example, I want to transfer a file called /root/book.txt from a server called project.local with sshd running on port 55522 to local partition /mnt, I can use this command:

$ rsync -avz --progress -e "ssh -p 55522" root@project.local:/root/book.txt /mnt

where:
-a for archive mode
-v for verbose
-z to compress data during transfer
--progress for showing progress of the copy
-e to specify the remote shell to use, in this case to use ssh to port 55522