Monday, May 8, 2017

touch: cannot touch `a': No space left on device

##########################
## Error
##########################

[oracle@oralinux audit]$ touch a
touch: cannot touch `a': No space left on device

##########################
#  Error Occurred
##########################

Database is giving alerts that audit files cant be created. Tried to create a file and gave above error

##########################
## Command Executed
##########################

touch a

**************************************** Step By Step Analysis ******************************************************

#########################################
# 1) Check the File system Usage
#########################################

[oracle@oralinux audit]$ df -h .
Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/vg_oracle_data-lv_orasid01
                      9.9G  6.8G  2.6G  73% /ofa/ORASID01
[oracle@oralinux audit]$
#### Create a new file
[oracle@oralinux audit]$ touch a
touch: cannot touch `a': No space left on device
[oracle@oralinux audit]$

=====================================================================================================================

#########################################
# 2) Check Inode of the file system
#########################################

cat /proc/sys/kernel/sem

ipcs -ls

[oracle@oralinux audit]$ cat /proc/sys/kernel/sem
250     524288  256     2048
[oracle@oralinux audit]$ ipcs -ls

------ Semaphore Limits --------
max number of arrays = 2048
max semaphores per array = 250
max semaphores system wide = 524288
max ops per semop call = 256
semaphore max value = 32767

df -i .

[oracle@oralinux audit]$ df -i .
Filesystem            Inodes   IUsed   IFree IUse% Mounted on
/dev/mapper/vg_oracle_data-lv_orasiD01
                     1310720 1310720       0  100% /ofa/ORASID01
[oracle@oralinux audit]$

=====================================================================================================================

#########################################
# 3) Reason for Failure
#########################################

From above output, its clear the Inode of that particular file system has reached 100%, even though file system is having sufficient space, the Inode has reached 100%, so new files are not allowed.

=====================================================================================================================

##########################
## Solution
##########################

#### Remove the files which are older than n number of days.

rm aud*

As i dont need audit files i have cleared everything.

=====================================================================================================================
Post removal of files, new audit files can be created.
=====================================================================================================================

Its better to set up a monitoring to check Inodes before it reaches 100%

=====================================================================================================================
 Comments Are Always welcome
=====================================================================================================================