Posts

How push notification works for FCM/Apple push (APNs) in network level

Question : How push notification works for FCM/Apple push (APNs) in network level? There are so many articles describes the push notification work flow in application level. But how APN servers sends the notifications to clients in network level are not explained in many articles. The answer is long polling connections from client devices(Mobiles, Laptops) to APN servers. why long polling connection? if its short connections then it leads to notification delays based on polling interval. APN servers are configured with long keepalive timeout(usually this timeout is very short around 60 secs to 300 secs in normal application servers) so push notifications will deliver to client devices immediately through same long lived TCP connection once APN receives the notification from app server side why polling comes here? TCP connection always disconnects if there is no internet or network change or device restarts, etc,… In this case, client device always keep trying/polling to the AP...

Remove Large files in Git history

You pushed some large unnecessary files mistakenly in git, then just removing the file will not help we need to remove the file from git history. Steps to check the largest files: - cd my-repo - git rev-list --objects --all | git cat-file --batch-check='%(objecttype) %(objectname) %(objectsize) %(rest)' | awk '/^blob/ {print substr($0,6)}' | sort --numeric-sort --key=2 --reverse | head | gcut --complement --characters=13-40 | gnumfmt --field=2 --to=iec-i --suffix=B --padding=7 --round=nearest Steps to remove the largest files: - Download the bfg.jar file(https://rtyley.github.io/bfg-repo-cleaner/). - git clone --mirror git@github.com:saravanan/my-repo.git - java -jar bfg.jar --delete-files large-file my-repo.git - cd my-repo.git - git reflog expire --expire=now --all && git gc --prune=now --aggressive - git push

docker.errors.APIError: 400 Client Error client is newer than server

You can face this issue when you are trying to use docker python library. Error: docker.errors.APIError: 400 Client Error: Bad Request for url: http+docker://localunixsocket/v1.26/images/json?only_ids=0&all=0 ("client is newer than server (client API version: 1.26, server API version: 1.24)") Solution: client= docker.from_env(version='auto') Use parameter "version" when instantiating a client.

Frequently update the AMI or Instance-Type in AWS Autoscaling group

If your instances running under AWS autoscaling, then instances will always use the same AMI or Instance-Type which configured in launch configuration. If you installed new packages or did any changes in instance root volume, then you need to take new AMI from that instance[root volume] and need to update that AMI in Autoscaling group[using launch configuration]. But it will require many manual steps. So I created a Bash script and it will do that task automatically. Example 1:- If you want to just update the AMI in Autoscaling group, run the script like below. #./script.sh <Autoscaling Group Name>  <Launch Configuration Name>  <AWS Region> Example 2:- If you want to  update both instance type and AMI in Autoscaling group, run the script like below. #./script.sh <Autoscaling Group Name>  <Launch Configuration Name>  <AWS Region>  <Instance Type> ==================================== #!/bin/bash Autos...

Very Good Tutorial To Learn Git

https://www.atlassian.com/git/ tutorials/what-is-version- control

Bash Script to find the exact application or command which consumed high CPU and Memory

#!/bin/bash #loadavg=$(awk '{ print $1 "\t" $2 "\t" $3 }' /proc/loadavg ) #To find the exact memory used percentage[Used-(Cached+Buffer)] Memusage=$(printf '%.0f\n' `free | awk 'FNR == 3{print $3/($3+$4)*100}'`) now=$(date +"%Y-%m-%d %H:%M:%S") #To find the cpu load average cpu_load=$(printf '%.0f\n' `uptime | awk '{print $(NF-2)}' | cut -c1-4`) if [ "$cpu_load" -ge "25" ] || [ "$Memusage" -ge "50" ];then   echo -e "\n\n=================\t$now\t========================" >> /var/log/top_report   top -b -c -n1 >> /var/log/top_report fi

Mongod is not stopping properly on Centos6.5

Just change the following line on function mongo_killproc() in mongod init script[line 94], # vim /etc/init.d/mongod local pid=`pidof ${procname}` instead of local pid=`pidofproc -p "${pid_file}" ${procname}`

How to install and configure the GitLab in Ubuntu 14.04

Step 1:- Check the Hardware requirements. ====== http://doc.gitlab.com/ce/install/requirements.html Here I going to use Ubuntu 14.04 LTS version. Step 2:- Choose an installation method [ Installation from source or Omnibus package installer ]. ====== But they recommend installing the Omnibus package instead of installing GitLab from source. Omnibus GitLab takes just 2 minutes to install and is packaged in the popular deb and rpm formats. Compared to an installation from source, the Omnibus package is faster to install and upgrade, more reliable to upgrade and maintain, and it shortens the response time for our subscribers' issues. A package contains GitLab and all its dependcies (Ruby, PostgreSQL, Redis, Nginx, Unicorn, etc.), it can be installed without an internet connection. But I am going to try both methods now. Step 3:- Install GitLab from Omnibus package. ====== Enterprise Edition is license-based, so I going to install Community Edition which is free. 1. Install and config...

P2V Migration in VMware

Question:- --------------- i need to migrate a Linux box with no downtime to ESXi and with VMware tool that is not possible or i don't know how because you can't install the VCenter Converter into a linux OS and to the live migration with synchronization. Does any 1 know a way to do this? Solution:- --------------- Of course, we can migrate the Linux physical machine to ESXi host using vsphere converter tool. Yes, we can't install converter tool in linux but we can install it in windows machine then we can migrate the physical machine to ESXi through windows VM. Recently I did this task. Please follow the below steps. 1. Install a windows machine in ESXi host where you want to migrate the physical machine. 2. Install the vsphere converter tool in that windows machine. 3. But please choose the source machine as linux physical machine instead of "localhost". If you select localhost then that win...

KVM & Qemu

QEMU is a powerful emulator, which means that it can emulate a variety of processor types. Xen uses QEMU for HVM guests, more specifically for the HVM guest's device model. The Xen-specific QEMU is called qemu-dm (short for QEMU device model) QEMU uses emulation; KVM uses processor extensions (intel-VT) for virtualization. Both Xen and KVM merge their various functionality to upstream QEMU, that way upstream QEMU can be used directly to accomplish Xen device model emulation, etc. Xen is unique in that it has paravirtualized guests that don't require hardware virtualization. Both Xen and KVM have paravirtualized device drivers that can run on top of the HVM guests. The QEMU hypervisor is very similar to the KVM hypervisor. Both are controlled through libvirt, both support the same feature set, and all virtual machine images that are compatible with KVM are also compatible with QEMU. The main difference is that QEMU does not support native virtualization. Consequently, QEMU has ...

Packet Drop issue

Packet drops can happen at two layers. one at the NIC level or at the Network stack level. 1. Nic level:-  Check 'ifconfig' output. RX packets:297126179 errors:0 dropped:3981 overruns:0 frame:0 TX packets:233589229 errors:0 dropped:0 overruns:0 carrier:0 That means packets drops at the NIC level. These are most likely caused by exhaustion of the RX ring buffer. Increase the size of the Ethernet device ring buffer. First inspect the output of "ethtool -g eth0". If the "Pre-set maximums" are more than the what's listed in the current hardware settings it's recommend to increase this number. As an example: # ethtool -g eth0 Ring parameters for eth0: Pre-set maximums: RX: 1020 RX Mini: 0 RX Jumbo: 16320 TX: 255 Current hardware settings: RX: 255 RX Mini: 0 RX Jumbo: 0 TX: 255 To increase the RX ring buffer to 4080 you would run "ethtool -G eth0 rx 1020". 2. Network stack level:- Need  to tweak sysctl kernel...

Active vs Passive ftp

Active and passive are the two modes that FTP can run in. FTP uses two channels between client and server, the command channel and the data channel, which are actually separate TCP connections. The command channel is for commands and responses, the data channel is for actually transferring files. It's a nifty way of sending commands to the server without having to wait for the current data transfer to finish. In active mode, the client establishes the command channel (from client port X to server port 21 (b) ) but the server establishes the data channel (from server port 20 (b) to client port Y , where Y has been supplied by the client). In passive mode, the client establishes both channels. In that case, the server tells the client which port should be used for the data channel. Passive mode is generally used in situations where the FTP server is not able to establish the data channel. One of the major reasons for this is network firewalls. While you may have...

Create a simple webserver container on Docker

1. Start the container with a shell, #docker run -i -t -p 8080:80 214a4932132a /bin/bash Mapped the host port 8080 to container port 80, so that you can access the container webserver contents through apache. [It is similar as DNAT] 2. Install the web server package inside the container, At first, the containers have only internal IP addresses. To access the Internet, SNAT (Source Network Address Translation, also known as IP masquerading) should be configured on theHardware Node. [root@proxy ~]# iptables -t nat -A POSTROUTING -s 172.17.0.0/24 -o eth0 -j SNAT --to 31.x.x.x 172.17.0.0 - Container's ip range 31.x.x.x - Host public IP Now you can able to access the internet from container, then we can install the apache server. [root@c576532e21ab /]# yum install httpd -y 3. Create a custom content in document root, [root@c576532e21ab /]# echo "Hello world" > /var/www/html/index.html 4. Test the web server, We cant start the httpd service through ...

Install and Configure the Docker on Centos

We are going to install and configure the Docker on Centos 1. Enable the epel repository. 2. yum install docker-io 3. Start the docker daemon [root@proxy ~]# service docker start Starting cgconfig service:                                 [  OK  ] Starting docker:                                       [  OK  ] [root@proxy ~]# chkconfig docker on 4. Download any public container images and store them in a local repository, [root@proxy ~]# docker pull ubuntu ubuntu:latest: The image you are pulling has been verified 511136ea3c5a: Pull complete f3c84ac3a053: Pull complete a1a958a24818: Pull complete 9fec74352904: ...

How to install and configure GlusterFS server on centos

We are going to setup glusterfs storage server with four nodes. 1. On all of your four nodes install the glusterfs and xfs packages, wget -P /etc/yum.repos.d http://download.gluster.org/pub/gluster/glusterfs/LATEST/CentOS/glusterfs-epel.repo yum install glusterfs-server xfsprogs chkconfig glusterd on service glusterd start 2. On all of your cluster nodescreate a new 2Gb LV called brick1 in vgsrv VG and format this LV with an XFS filesystem with 512byte inodes. lvcreate -L 2G -n brick1 vgsrv mkfs.xfs -i size=512 /dev/vgsrv/brick1 mkdir /server1_export1 echo "/dev/vgsrv/brick1 /server1_export1 xfs defaults 0 1" >> /etc/fstab mount -a 3. From server1, add the other three nodes as trusted peers. [root@proxy ~]# gluster peer probe server2{ip} [root@proxy ~]# gluster peer probe server3{ip} [root@proxy ~]# gluster peer probe server4{ip} [root@proxy ~]# gluster peer status Number of Peers: 3 Hostname: server2 Uuid: a381532b-81a0-41c7-9adb-cd29f9f38158 State: Peer in Cluster (C...

Create XFS filesystem on Centos

Server : Test1 Step 1: ------------ Create two logical volumes with size 2Gb & 512MB respectively. [root@sara2 ~]# lvcreate -n xfsdata -L 2G vgsrv -->xfs filesystem   Logical volume "xfsdata" created [root@sara2 ~]# lvcreate -n xfsjournal -L 512M vgsrv -->external journal setup   Logical volume "xfsjournal" created [root@sara2 ~]# yum install xfsprogs Create a new xfs filesystem with external journal. [root@sara2 ~]# mkfs -t xfs -l logdev=/dev/vgsrv/xfsjournal /dev/vgsrv/xfsdata meta-data=/dev/vgsrv/xfsdata     isize=256    agcount=4, agsize=131072 blks          =                       sectsz=512   attr=2, projid32bit=0 data     =                  ...

Reverting LVM changes

On test1 create a 2GB Logical volume resizeme in vgsrv volume group and create a ext4 file-system, then mount your file-system and create some test files. [root@test1 ~]# vgs   VG    #PV #LV #SN Attr   VSize  VFree   vgsrv   1   1   0 wz--n- 20.00g 16.00g [root@test1 ~]# lvcreate -n resizeme -L2G vgsrv   Logical volume "resizeme" created [root@test1 ~]# mkfs -t ext4 /dev/vgsrv/resizeme [root@test1 ~]# mount /dev/vgsrv/resizeme /mnt [root@test1 ~]# touch /mnt/file{0..9} [root@test1 ~]# umount /mnt You want to resize your filesystem to 1GB, but accidently you forget to resize the file system first[resize2fs]. [root@test1 ~]# lvresize -L1G /dev/vgsrv/resizeme   WARNING: Reducing active logical volume to 1.00 GiB   THIS MAY DESTROY YOUR DATA (filesystem etc.) Do you really want to reduce resizeme? [y/n]: y   Size of logical volume vgsrv/resizeme changed from 2.00 GiB (512 extents) to 1.00 GiB (...