Commonly needed Linux commands

At WPoets we manage all our servers in house, so here we have listed some of the most common commands that we use.

Disk space

Here is the command that gives us exact disk space.

This will give you the size of each folder within the folder path you have run the command

If you  to count the directories in a folder you do so by

and, if you need the list of files just do

To count files (even files without an extension) recursively from the root of the current directory, use:

Here is an alternative

Dealing with Zip and Tar

Creating a Tar.gz file from a folder

where wordpoets_com.tar.gz is the file name and  /_data/htdocs/ is the folder path we want to compress.
If you want to create tar file, with a relative path instead of full absolute url you can do something like this

It will create the tar file relative to 20190807 instead of  /var/www/backups/db/fullbackup/2019/07/20190807 which is what it generally does.
Let’s say the above folder you want to extract within /htdocs/ folder, without creating another /htdocs/ within, use the following command

the –strip-components allows you to set the depth while extracting the tar file.
If you need to compress a file or uncompress it we can use the following command, we generally use it to compress SQL files

it will create the compressed version of the file as abc.sql.gz and delete the abc.sql, and if you want to uncompress the abc.sql.gz file just type the following command to get the original file back.

 

Fail2Ban

On servers managed by us, we use Fail2ban actively to block IPs and threats, but sometimes we end up banning IPs wrongly in those times from command easily allows you to check which IP was blocked by which rule, making it easier to identify

and, if just want to see list of jails active run following

If you know the name of specific jail, you also use the following command to check the list of banned IP addresses

where wordpress2 is the name of the jail specified in fail2ban jail.conf file.
Finally, to unban an IP, you can use the following command

just remember to replace the wordpress2 and the IP address with your own.

WGET

Once in a while, we need to run wget in a cron, and one the trouble was ignoring the output here is the setting that does that

The -O - option makes sure that the fetched content is sent to stdout.
another alternative is

Here -O sends the downloaded file to /dev/null and -o logs to /dev/null instead of stderr. That way redirection is not needed at all.

Rsync & SCP

When moving files between two servers,  we currently prefer either rsync or scp depending on the situation.

RSYNC options:
-r to recurse into directories,
–ignore-existing to ignore existing files in the destination,
-P if you want to watch progress,
–list-only if you want to see what it would copy without actually copying, and a
-t if you want to preserve the timestamps.

SCP options:
–r Recursively copy entire directories. Note that this follows symbolic links encountered in the tree traversal.
-C Compression enable. Passes the -C flag to ssh to enable compression.
-l limit – Limits the used bandwidth, specified in Kbit/s.
-o ssh_option – Can be used to pass options to ssh in the format used in ssh_config.
-P port – Specifies the port to connect to on the remote host. Note that this option is written with a capital ‘P’.
-p Preserves modification times, access times, and modes from the original file.
-q Quiet mode: disables the progress meter as well as warning and diagnostic messages from ssh.
-v Verbose mode. Print debugging messages about progress. This is helpful in debugging connection, authentication, and configuration problems.