Skip to content

Important linux commands

The article gives insight into the most important commands

System basic commands

Scenarios Commands
Inspect the host name hostname
Inspect the host name and version uname -a
Inspect the Linux information lsb_release -a
Inspect the CPU information cat /proc/cpuinfo
Inspect the memory free -m
Inspect the user 1. w
2. who
3. whoami
4. tty
Inspect the hard drive 1. df -h
2. df -aTh
3. du -h --max-depth=1 .
Inspect the ports 1. netstat -ano | findstr 7001
2. sudo netstat -tunlp
3. netstat -aonp | grep 5050
Inspect the firewall sudo iptables -L -n
Inspect the dns to host and port status nmap -Pn -p 27018
Inspect the path env for current user echo $PATH
Inspect the process by pid tasklist | findstr 23416
Inspect the process by name ps -ef | grep nginx
Kill the process by pid taskkill -f -pid 9788
Curl and save to txt curl --trace-ascii dump.txt
Get a root shell 1. sudo -s
2. sudo bash -c su -
Clean Linux temp files sudo rm -rf /var/log/*
Clean Linux temp files cursor and processes 1. sudo lsof | grep delete
2. sudo kill *
MacOS kill process using specific port 1. lsof -i tcp:8443
2. kill 86166
Clean Linux swap and move into memory then restart and modify threshold 1. sudo swapoff -a && sudo swapon -a
2. sudo sysctl vm.swappiness=10
3. sudo vi /etc/sysctl.conf -> vm.swappiness=10

File commands

Scenarios Commands
Find a file with the given name under current path find . -name "*"
Find a archive file containing the given xml file find . -name "*.jar" | xargs grep "*.xml"
Find a file containing the given content under current path 1. grep -s -r "*" ./
2. find . -name "*.xml" | xargs grep "*"
Find a file containing the given content under current path and only print the file name to the output file find . -name "*.*" | xargs grep -ri "APXMTMRR" -l > grep_APXMTMRR.log
Find the top size files 1. du -a . | sort -n -r | head -n 10
2. find . -type f -size +20000k -exec ls -lh {} \; | awk '{ print $9 ": " $5 }'
List the folders' size in current folder du -h --max-depth=1 .
List the latest changed files under current folder ls -lrth
Find the files under specific folder and append the timestamp find /var -maxdepth 2 -type d -exec stat -c "%n %y" {} \;
Paste text with # in the VIM and keep the format :set paste
Zip folder to a file sudo tar -zvcf /data/bak.tar.gz /data/db
Unzip file to a folder sudo tar -zvxf /data/bak.tar.gz
Copy file to remote by scp scp agriculture-platform/ ec2-user@
Copy Folder to remote by scp scp -r agriculture-platform ec2-user@
Copy file from remote to local by scp 1. scp -i sshkey .
2. scp -i "~/sshkey.pem" -r data-test
Clean the content in a file sed -i "d" Dockerfile

Software commands

Scenarios Commands
Install software by yum 1. sudo yum -y install docker
2. sudo yum install java-1.8.0-openjdk.x86_64
Uninstall software by yum sudo yum -y remove docker
List installed software by yum 1. yum list installed | grep docker
2. yum -y list docker*
Search software by yum yum search java | grep 'java-'
Install software by apt-get 1. apt-get update
2. apt-get install vim
List installed software by RPM rpm -qa | grep docker

Service commands

Scenarios Commands
Restart vsftpd systemctl restart vsftpd
Restart nginx /usr/nginx/sbin/nginx -s reload
Check nginx configuration /usr/nginx/sbin/nginx -t

JVM commands

Scenarios Commands
Inspect JVM default heap flag java -XX:+PrintFlagsFinal -version | grep HeapSize
Inspect the java process arguments jinfo -flags 1
Inspect the java process GC information jmap -heap 1
List java process jps
List the basic class memory information by pid jmap -histo 1 | head
List the class loaders memory information by pid jmap -clstats 1
Dump the java process heap information by pid jcmd 1 GC.heap_dump heap_dump.hprof


Maven and SpringBoot commands

#Generate project
mvn archetype:generate \ \
 -DartifactId=tutorial1 \
 -Dversion=1.0-SNAPSHOT \ \
 -DarchetypeArtifactId=archetype-test-quickstart \ \
 -DarchetypeVersion=1.0 \

#Build project without testing
mvn clean package -Dmaven.test.skip=true

#Run SpringBoot with specific profile
java -Xms256m -Xmx1024m -jar calculate-1.0.jar
mvn spring-boot:run -Drun.profiles=dev
mvn spring-boot:run -Drun.jvmArguments=""

#Run in debug mode
java -agentlib:jdwp=transport=dt_socket,server=y,address=5050,suspend=y -jar calculate-1.0.jar
mvn spring-boot:run -Drun.jvmArguments="-Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=y,address=5005”
mvn spring-boot:run -Drun.jvmArguments="-Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=5005"

#Create SpringBoot project using
curl -d dependencies=web,data-jpa,devtools,h2 -d -d artifactId=example -d name=example -d description="Spring Boot Example Application" -d baseDir=example -o && unzip && rm -f

SSH from server1 to server2 without password

#Generate private/public key in server1, and copy the public key
ssh-keygen -t rsa
sudo cat ~/.ssh/

#Append the public key to authorized keys in server2
vi ~/.ssh/authorized_keys

#SSH from server1 to server2:
ssh ec2-user@

#Allow or deny IPs: (block the ssh request or not)
sudo vi /etc/hosts.deny
#(remove the client IP)
sudo systemctl restart sshd.service

Change Timezone(Ubuntu)

date -R
sudo timedatectl set-timezone Australia/Brisbane
sudo timedatectl set-local-rtc 1
sudo timedatectl set-local-rtc 0

Modify the fstab system file to automatically mount a volume after rebooting

sudo vi /etc/fstab
# modify the content as bellow
/dev/vdb        /mnt/data       auto    defaults,nofail,comment=cloudconfig     0       2
#reboot the instance

Establish the tunnel from local port to online website by ngrok (the header setting is because of angular app Invalid Host Header issue)

ngrok http 4200 -host-header="localhost:4200"
ngrok http --host-header=rewrite 4200

Pandoc usage

pandoc -s -o README.docx -M title:architecture  #(markdown -> docx)
pandoc --pdf-engine=xelatex -o README.pdf  #(markdown -> pdf)

Recover rm -rf files

#Install extundelete
sudo yum search extundelete
sudo yum install extundelete.*

#Find filesystem and type(support ext3/ext4) to recover
df -T

#Find nodes and node to recover:
sudo extundelete --inode 2 /dev/vda1
sudo extundelete --inode 393217 /dev/vda1

sudo extundelete --restore-inode 542480 /dev/vda1

#Recover all deleted after time
sudo yum search extundelete
sudo yum install extundelete.*
df -T
sudo umount /dev/vdc
date -d "2018-06-02 23:00:00" +%s
sudo extundelete /dev/vdc --after 1527980400 --restore-all
sudo cp -R /data .
sudo mount /dev/vdc /data -t auto
sudo cp -R ./data/* /data/

HTTPS certificate

#Generate key
keytool -genkey -alias tomcat -storetype PKCS12 -keyalg RSA -keysize 2048 -keystore keystore.p12 -validity 3650

#Install certbot and certbot-auto
git clone
cd certbot
./certbot-auto --help

#Generate certificates and a private key
#Remember to stop springboot process first
sudo ./certbot-auto certonly -a standalone -d -d

#Convert PEM files to PKCS12 files
sudo -s
cd /etc/letsencrypt/live/
openssl pkcs12 -export -in fullchain.pem -inkey privkey.pem -out keystore.p12 -name tomcat -CAfile chain.pem -caname root
#copy keystore.p12 file to ~/agriculture-platform folder, exit su, then start springboot process

#Update HTTPS key(
cd ~/certbot
docker stop agriculture
sudo ./certbot-auto certonly -a standalone -d
sudo -s
cd /etc/letsencrypt/live/
openssl pkcs12 -export -in fullchain.pem -inkey privkey.pem -out keystore.p12 -name tomcat -CAfile chain.pem -caname root
cp keystore.p12 /home/ec2-user/agriculture-platform/
cd ~/agriculture-platform/
vi Dockerfile #modify the date to recopy the key

#Generate generic HTTPS key(*
./certbot-auto certonly  -d * --manual --preferred-challenges dns --server
./certbot-auto certonly -d * --manual --preferred-challenges dns --server
#create txt record in dns

#Generate p12 by crt and key
openssl pkcs12 -export -name alpha -in alpha_v2.crt -inkey alpha.key -out alpha.p12

AWS volume

#Extend volume
lsblk #Check if the volume need to extent in Linux
df -h
sudo growpart /dev/xvda 1 #xvda1 in lsblk result means change the partition 1 here
sudo resize2fs /dev/xvda1
df -h

#Mount/Unmount volume
df -h
sudo mount /dev/xvdf1 /data
sudo umount /dev/xvdf1
df -h

NectarCloud volume

#ttach and mount volume(/dev/vdc)
sudo fdisk -l
sudo mkfs.ext4 /dev/vdc
sudo mount /dev/vdc /data -t auto
sudo mount /dev/vdd /var/lib/docker -t auto
sudo chmod -R 777 /data

sudo umount /dev/vdd

#Increase the volume size(/dev/vdc)
sudo umount /dev/vdc
sudo e2fsck -f /dev/vdb
sudo resize2fs /dev/vdc
sudo mount /dev/vdc /data -t auto

Singularity in HPC

singularity pull docker://mongo:3.4
singularity shell --bind data:/data mongo_3.4.sif
singularity instance start mongo_3.4.sif mongodb
mongod &

LVM management

#Install lvextend
yum whatprovides */lvextend
sudo yum install lvm2

#Enlarge the / disk
df -h
sudo lvextend -L +50G /dev/vda1

Openstack API

#Install openstack client in Ubuntu
sudo apt update
sudo apt-get install python-pip python-dev
sudo apt  install python-openstackclient
#Replace the queue with Queue
##import queue
#import Queue as queue
vi /home/ubuntu/.local/lib/python2.7/site-packages/openstack/
vi /home/ubuntu/.local/lib/python2.7/site-packages/openstack/cloud/
openstack server list

MongoDB operation

mongo --port 27017 -u "database" -p "password" --authenticationDatabase "admin"
use admin

#Clear plan cache

#Inspect database/collection size

#Check aggregation plan
    { $match: { datasetId: "5a6fddf188fba650b83b94c9", validated: true, "timestamp.minuteStr": { $lte: "2018-09-09 00:00", $gte: "2017-12-31 00:00" } } },
    { $project: { date: "$timestamp.minuteStr", toStat: 1 } },
    { $group: { _id: "$date", toStat: { $push: "$toStat" } } },
    { $sort: { _id: 1 } }
], {cursor: {}, allowDiskUse: true, explain: true})

#Check aggregation execution plan
    { $match: { datasetId: "5a6fddf188fba650b83b94c9", validated: true, "timestamp.minuteStr": { $lte: "2018-09-09 00:00", $gte: "2017-12-31 00:00" }, "filter.Tunnels_to_survey~Plants_to_survey~Variety": { $in: [ "Eureka" ] } } },
    { $project: { date: "$timestamp.minuteStr", toStat: 1 } },
    { $group: { _id: "$date", toStat: { $push: "$toStat" } } },
    { $sort: { _id: 1 } }

Crontab service

#Start/stop service
sudo /sbin/service crond start
sudo /sbin/service crond stop
sudo /sbin/service crond restart
sudo /sbin/service crond reload

#Current services
crontab -l

#Edit services
crontab -e

#Delete services
crontab -r

Less commands

#less generate min.css
lessc --clean-css AdminLTE.less ../../dist/css/AdminLTE.min.css
Less generate

Git commands

#Git clone
git clone***.git

#Configure new git remote url and push
git remote add origin***.git
git push --set-upstream origin master

#Modify git remote url
git remote set-url origin***.git
git remote -v

#Find the size of git repository
git count-objects -v

#Prune all of the reflog references from this point back (unless you’re explicitly only operating on one branch) and repack the repository
git -c -c gc.autodetach=false -c gc.autopacklimit=1 -c gc.garbageexpire=now -c gc.reflogexpireunreachable=now gc --prune=all

#Push all your changes back to the Bitbucket repository
git push --all --force && git push --tags --force

#Force push the local to remote
git push -u origin master -f

MongoDB commands

#Find by id in mongoldb compass

#start/shutdown commands:
mongod -f /etc/mongodb.conf
mongod --shutdown -f /etc/mongodb.conf

#MongoDB on mac
#The databases are stored in the /usr/local/var/mongodb/ or /usr/local/opt/mongodb/ directory
#The mongod.conf file is here: /usr/local/etc/mongod.conf
#The mongo logs can be found at /usr/local/var/log/mongodb/
#The mongo binaries are here: /usr/local/Cellar/mongodb/[version]/bin

Install tools


tar xvf apache-maven-3.5.4-bin.tar.gz
sudo mv apache-maven-3.5.4 /usr/local/apache-maven
# Add the env variables to your ~/.bashrc file
    export M2_HOME=/usr/local/apache-maven
    export M2=$M2_HOME/bin
    export PATH=$M2:$PATH
source ~/.bashrc
mvn -version
rm -rf apache-maven-3.5.4-bin.tar.gz

Ping and telnet

apt-get update
yes | apt-get install iputils-ping
yes | apt-get install telnet


#Install plugin to grunt and update the package.json dependency
npm install grunt --save-dev     (for testing framework, like grunt..)
npm install grunt --save     (for production purpose.)