commit: 3ca46d3a9ebbee316522963ff90cc7c24bee1967
parent: fddafd5b2b0c8959722c105ab9010e2674326d11
author: Chris Noxz <chris@noxz.tech>
date: Sun, 14 Aug 2022 15:33:17 +0200
add new article about disk being full and empty
5 files changed, 95 insertions(+)
diff --git a/noxz.tech/articles/your_disk_is_full_and_empty/.assemble b/noxz.tech/articles/your_disk_is_full_and_empty/.assemble
@@ -0,0 +1,2 @@
+index.html
+cat.png
diff --git a/noxz.tech/articles/your_disk_is_full_and_empty/.buildignore b/noxz.tech/articles/your_disk_is_full_and_empty/.buildignore
diff --git a/noxz.tech/articles/your_disk_is_full_and_empty/.metadata b/noxz.tech/articles/your_disk_is_full_and_empty/.metadata
@@ -0,0 +1,5 @@
+.ds YEAR 2022
+.ds MONTH August
+.ds DAY 14
+.ds DATE \*[MONTH] \*[DAY], \*[YEAR]
+.ds TITLE Your disk is full and empty
diff --git a/noxz.tech/articles/your_disk_is_full_and_empty/cat.png b/noxz.tech/articles/your_disk_is_full_and_empty/cat.png
diff --git a/noxz.tech/articles/your_disk_is_full_and_empty/index.www b/noxz.tech/articles/your_disk_is_full_and_empty/index.www
@@ -0,0 +1,88 @@
+I recently had a Schrödinger's cat experience with the root drive of my NAS.
+Suddenly, I wasn't able to apply any system updates to the system while getting
+errors indicating my root drive was full, which was strange as a 32G drive
+should be more than sufficient for storing the system. When checking the file
+allocation using
+.ICD "du -cksh --exclude /strg --exclude /fbkp /"
+I could see that, yes, the system was only allocating about 1.7G:
+
+.CDS
+.COS
+1.7G /
+1.7G total
+.COE
+.CDE
+
+To confirm that my disk indeed had unallocated space, I also checked the
+allocation using
+.ICD "df"
+and got a conflicting result:
+
+.CDS
+.COS
+Filesystem 1K-blocks Used Available Use% Mounted on
+/dev/root 30386396 29719476 0 100% /
+\[u2026]
+.COE
+.CDE
+
+What was going on?
+
+.PIMG cat.png
+
+The next thing I tried was to delete file caches and logs from various places,
+and even after that,
+.ICD "df"
+still showed 100% full. At this point, I was very confused and tried
+many useless things before realizing what was going on. When checking the file
+allocation using
+.ICD "du" ,
+I excluded two file paths, one for the main NAS storage partition and one for
+backing up the SD card of the Raspberry Pi, as they were mounted on different
+disks and partitions than the root one. After checking the log, I could see
+that before the backup of the SD card, the disk used for storing the backup
+had, for unknown reasons, been unmounted. After restarts, that same disk had
+been mounted again, effectively hiding the problem.
+
+After unmounting that disk and listing the contents of the directory acting as
+the mount point, I saw what was allocating the disk space.
+
+.CDS
+.COS
+# umount /fbkp
+# ls -l /fbkp
+total 28031472
+-rw-r--r-- 1 root root 28704194560 Aug 11 06:52 sdcard.img
+.COE
+.CDE
+
+The way I was backing up the SD card had a major flaw considering that the
+destination of the backup was on a mounted disk, or rather a memory stick. I
+hadn't taken into account that the memory stick could either fail or, for some
+other reason, not be mounted while the backup was taking place. My solution was
+to first make sure that the disk was in fact mounted before performing the
+backup. I changed the cron job from this
+
+.CDS
+.COS
+\[u2026]
+# create backup of sdcard
+0 6 * * * dd if=/dev/mmcblk0 of=/fbkp/sdcard.img
+\[u2026]
+.COE
+.CDE
+
+to this
+
+.CDS
+.COS
+\[u2026]
+# create backup of sdcard
+0 6 * * * mount | grep '/dev/sd.1 on /fbkp' && dd if=/dev/mmcblk0 of=/fbkp/sdcard.img
+\[u2026]
+.COE
+.CDE
+
+Since then, the cron job has done what it was supposed to do. I was once again
+reminded of how a simple solution to a simple, but seemingly illogical problem,
+can seem to be impossible to find from an initial look.
