We have observed a strange issue where we have deleted few GBs files and the space is not reduced on filesystem.
du command started showing less usage, however df still shows the same usage.
Lets see below:
#df -h /var/oracle/backup
/dev/dsk/emcpower1g 2.4T 2.3T 206M 100% /var/oracle/backup------------
#du -h /var/oracle/backup
1.3T /var/oracle/backup
The first thing to check here is inodes, if the system is out of inodes.
We have checked and found the inode usage normal.
And then to identify the issue is whether with : 1. the OS where this disk is mounted, or 2. With this EMC disk,
We have mounted this partition on another server and found the same issue there.
Hence we understood the issue is with the partition, and hence we ran FSCK on the partition after taking approval from customer.
And found the issue is resolved and many corrections on disk.
# fsck -y /var/oracle/backup
CORRECTED SUPERBLOCK SUMMARIES FOR CG 5574
CORRECTED SUPERBLOCK SUMMARIES FOR CG 5575
CORRECTED SUPERBLOCK SUMMARIES FOR CG 5576
CORRECTED SUPERBLOCK SUMMARIES FOR CG 5577
CORRECTED SUPERBLOCK SUMMARIES FOR CG 5578
CORRECTED SUPERBLOCK SUMMARIES FOR CG 5579
CORRECTED SUPERBLOCK SUMMARIES FOR CG 5580
CORRECTED SUPERBLOCK SUMMARIES FOR CG 5581
CORRECTED SUPERBLOCK SUMMARIES FOR CG 5582
CORRECTED SUPERBLOCK SUMMARIES FOR CG 5583
CORRECTED SUPERBLOCK SUMMARIES FOR CG 5584
CORRECTED SUPERBLOCK SUMMARIES FOR CG 5585
CORRECTED SUPERBLOCK SUMMARIES FOR CG 5586
CORRECTED SUPERBLOCK SUMMARIES FOR CG 5587
CORRECTED SUPERBLOCK SUMMARIES FOR CG 5706
CORRECTED SUPERBLOCK SUMMARIES FOR CG 5707
728 files, 178464170 used, 3187097 free (0 frags, 3187097 blocks, 0.0% fragmentation)
***** FILE SYSTEM WAS MODIFIED *****
ORPHANED DIRECTORIES REATTACHED; DIR LINK COUNTS MAY NOT BE CORRECT.
***** PLEASE RERUN FSCK *****
You have new mail in /var/mail//root
bash-3.2#
bash-3.2#
bash-3.2# fsck -y /dev/rdsk/emcpower1g
** /dev/rdsk/emcpower1g
** Last Mounted on /var/oracle/backup
** Phase 1 - Check Blocks and Sizes
** Phase 2 - Check Pathnames
** Phase 3a - Check Connectivity
** Phase 3b - Verify Shadows/ACLs
** Phase 4 - Check Reference Counts
** Phase 5 - Check Cylinder Groups
728 files, 178464170 used, 3187097 free (0 frags, 3187097 blocks, 0.0% fragmentation)
You have new mail in /var/mail//root
bash-3.2# mount /var/oracle/backup
You have new mail in /var/mail//root
bash-3.2# df -h /var/oracle/backup
Filesystem size used avail capacity Mounted on
/dev/dsk/emcpower1g 2.4T 1.3T 1.0T 58% /var/oracle/backup
bash-3.2#
So this is quick resolution that in case the inode numbers are fine, then we need to run FSCK, and in case it also not resolves your issue, just reboot the system to address the issue.
