Dealing with “No space left on device” when partition is not full

When you get the “No space left on device” error on linux, first thing you do it to check your disk space usage.  Often times you notice you still have space, so what could be wrong?

Although your partition is not nearly full, most likely you have too many small or zero-sized files on your disk. So while you have enough disk space, all your available inodes have been exhausted. 

To check this run:

df -ih

If IUse percentage is at or near 100%, then huge number of small files is the reason for “No space left on device” errors.

To find out where inodes are being used run:

echo "Detailed Inode usage for: $(pwd)" ; for d in `find -maxdepth 1 -type d |cut -d\/ -f2 |grep -xv . |sort`; do c=$(find $d |wc -l) ; printf "$c\t\t- $d\n" ; done ; printf "Total: \t\t$(find $(pwd) | wc -l)\n"

After finding the directory with largest number of files, delete any unwanted files to free up some inodes. 

NOTE : If a process is using the files you are deleting, you will need a reboot to free the inodes used.

How to check if a port is open on Linux

Sometimes when writing a script/program you need to know if a port your are interested in has an established connection to it or it's free.

My favorite way to do this is to run the following command ( replace PORT with the port number you want to check for ):

netstat -ln | grep :PORT

-l = Show only listening ports

-n = Show port number

 If this command doesn't return anything, it shows that your PORT is available to use.
In your script you can check $? to see the return code of the above command. If it's non-zero, the PORT is available.

One thing to note though is that, this doesn't tell you the status of your network e.g. if PORT is accessible from outside of your notwork.

How to fix Cannot open your terminal '/dev/pts/#' - please check.

simply run :

script /dev/null

and try again.

How to setup SSH private and public keys

• 1)
  % ssh-keygen -t rsa -b 4096 Generating public/private rsa key pair.
  Enter file in which to save the key (~/.ssh/id_dsa): (just press Enter) 
  Enter passphrase (empty for no passphrase): (enter a passphrase and then press Enter or if you don't want want one just press Enter) 
  Enter same passphrase again: (repeat last action) 
  Your identification has been saved in ~/.ssh/id_rsa
  Your public key has been saved in ~/.ssh/id_rsa.pub
  The key fingerprint is:
  A long string appears here
  %
• 2)
  Paste the content of the  ~/.ssh/id_dsa.pub file that just got generated on your local host into the file ~/.ssh/authorized_keys on the remote host and save.
• 3)
  Set proper permissions on your local host keys and .ssh directory:
  chmod 700 ~/.ssh
  chmod 644 ~/.ssh/id_rsa.pub
  chmod 600 ~/.ssh/id_rsa
  set proper permissions on your remote host authorized_keys and .ssh directory
  chmod 700 ~/.ssh
  chmod 644 ~/.ssh/authorized_keys
  

How to find the 10 largest file or directories in Linux

This finds the largest 10 files :

find . -type f -print0 | xargs -0 du -s | sort -n | tail -10 | cut -f2 | xargs -I{} du -sh {}

This finds the largest 10 directories:

find . -type d -print0 | xargs -0 du -s | sort -n | tail -10 | cut -f2 | xargs -I{} du -sh {}

You can easily change -10 to -n where n is the number of files/directories you are trying to find.

How to extract values from a java properties files in a bash/shell script

Let's say you have a java properties (sample.prop ) file that contains this :

ORACLE_HOST=myhost.us.oracle.com
MW_HOME=/scratch/codrguy/sandbox
DB_HOST=dbhost.us.oracle.com

Now, if we want to extract values from this properties file and use it in a shell/bash script for let's say an automation task, how do we go about that.

There are 3 parts to this task: 1) Getting the line that contains the value we want 2) extract the value from that line 3) store it in a variable to be used later in the script

To do part 1 we can simply grep for the value of the property we need :
For example :

grep ORACLE_HOST sample.prop
this returns
ORACLE_HOST=myhost.us.oracle.com

To do part 2 we pipe the output of part 1 into awk for extraction of the value we need

grep ORACLE_HOST sample.prop |  awk -F= '{print $2}'
this returns
myhost.us.oracle.com

the -F option of awk sets the delimiter to be '=' and then we print the second record which is what we need

Finally to store this value in a variable we do :

my_value=$(grep ORACLE_HOST sample.prop |  awk -F= '{print $2}')

Removing passphrase from a SSH key

If you have a SSH key and are tired of typing it over and over, there is an easy way to remove it.

Open a *nix terminal and type :

bash-3.2$ ssh-keygen -p

Enter file in which the key is (/home/myuser/.ssh/id_rsa):
Enter old passphrase:
Key has comment '/home/myuser/.ssh/id_rsa'
Enter new passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved with the new passphrase.

How to receive or forward root emails in Linux

I find the following way the easiest way to manage who gets root's emails.

Open the file /etc/aliases for editing

vi /etc/aliases

Go to the bottom of the file where you see something similar to :

 # Person who should get root's mail
 #root:          marc

Change marc to the email address of the user who should get the email and finally uncomment the line. So for example your last two lines of /etc/aliases should look like:

# Person who should get root's mail
root:          you@yourcompany.com

Last step is to issue the following command ro reload /etc/aliases

newaliase

How to take a Java thread dump from a shell/bash script

We already know how to get a Java thread dump manually. But what if we want to take a Java thread dump from a script to help us automate things.

To do this we need to come up with a ps and grep combination that will only return one result. so grep as many times as neccessary and/or use keywords that are only found in your process name so you only get one result.

Remember to add [] somewhere in your first grep so grep doesn't matches itself.

output=$(ps -ef | grep m[y]ProcesseName )

Then all you need to do is sending the QUIT signal to $2 which stores the PID of your process.

kill -QUIT $2

So just add the above two lines to your script after modifying the first line to fit your needs and you should be good to go.

Using md5 checksum to check for integrity of your large uploaded/downloded file

Have you ever downloaded/uploaded a file from/to another computer and wanted to verify the integrity of the file to make sure it is not corrupted. The obvious way is to compare the file size of the original final and the copied file. However there are many ways that this number could mislead you. 

Just to give two examples, in some cases the program you use to download/upload the file can fail to fill in all the data and just pad the file to the correct size.  Hardware errors such as disk and memory issues can cause corruptions too that won't effect the file size. So how do you make sure your file is not corrupted and check the integrity of your downloaded/uploaded file.

On Linux/Unix you can use md5sum.  The way to use it is to simply run :

md5sum [yourfile]

This prints out something like :

16295afa0087ef75f33751cf003da993  [yourfile]

You will need to run the above command on both the origin of the file and it's destination and compare the digital fingerprint of the file printed. For unique files that fingerprint will be the same.

A lot of websites that let users download files ( specially large files which have a higher chance of getting corrupted in transit ) will provide checksums for their files which you can compare against your when you download is done.

Fixing "authentication is required to set the network proxy ..." on Redhat/REL/OEL

Whenever i started  a vncsession i would get an popup window stating :

authentication is required to set the network proxy used for downloading packages.  An
application is attempting to perform an action that requires privileges.
Authentication as the super user is required to perform this action" and asking
for the root password.

I didn't have the root password for this machine and hitting cancel would just bring back the pop-up in a few minutes. After googling here is the best way i found to fix this problem:

From a terminal window run "gnome-session-properties" and un-check "PackageKit
Update Applet"

Finally restart your vncserver and the issue should be gone.