All posts

5 min read

How I Deleted the Minecraft Server

keerokeero
devopslinuxlessons-learned

I ran a small Minecraft server for friends on a Google Cloud VM. Nothing fancy: a Compute Engine instance, a screen session, a cron job that copied the world folder every night. It worked for months.

Then one evening I opened a terminal to clean up an old backup, and about ten seconds later I was staring at this:

Terminal showing rm -rf /home returning 'cannot remove /home/Minecraft: Device or resource busy', then a corrected rm command with the full path
gagi muntek ko na mabura buong server HAHA

Roughly: "damn, I almost wiped the whole server HAHA." ("gagi" is a softened "g*go"; it lands about as hard as "f**k.")

That is the message I sent the group chat. It is not the whole story. I did not almost wipe the server. I wiped it. I just did not understand that yet, and then I let everyone keep believing the version where I got away with it.

The setup that set the trap

I had been lazy about where backups lived. Instead of putting them on a separate disk or a bucket, the nightly job copied everything into a backup/ folder inside the server directory. And because I used a plain recursive copy, the backup preserved the full path from the root. So the layout looked like this:

the backup layout on the gcp vm

/home
└── Minecraft/                     ← live server (running)
    ├── world/  server.jar  ...
    └── backup/                    ← nightly rsync mirror
        └── home/
            └── Minecraft/         ← stale copy I wanted gone
                └── world/  server.jar  ...

working directory:  /home/Minecraft/backup/home/Minecraft

what I meant  →  rm -rf ./         (just this stale copy)
what I ran    →  rm -rf /home      (the VM's real /home. everything.)

Read that path again: /home/Minecraft/backup/home/Minecraft. The backup contained a folder called home, which contained a folder called Minecraft, which contained another world/ and another server.jar. A little russian doll of my own filesystem.

I wanted to delete that inner stale copy to free up space. So I cd'd all the way down into it, looked at the contents, confirmed it was the junk copy, and typed what my brain had already decided was "delete this folder":

rm -rf /home

Why that is the wrong command

I meant here. I typed /home, which is not here. The leading slash makes it absolute: it does not mean "the home folder I am standing in", it means the machine's real /home, the one that also holds the live server I was standing inside.

After an hour of navigating backup/home/Minecraft/..., my eyes had completely stopped distinguishing between the mirror and the original. Every path looked like every other path.

Why I thought it survived

rm chewed through /home and then printed this:

rm: cannot remove '/home/Minecraft': Device or resource busy

The server process was running with the world files open, so that one directory could not be unlinked while it was in use. I saw "cannot remove", exhaled, and read it as "nothing happened." That is why I sent the group chat a laughing caption. As far as I knew, I had caught it in time.

What actually happened

I had not caught anything. The "busy" message was about a single locked directory that could not be removed right now. Everything else under /home was already gone: the server jar, the plugins and their configs, the cron scripts, the outer backups, most of the world tree. The running process was still writing into a directory that the filesystem no longer had a real path to.

I spent the rest of the night trying to bring it back. Stopping the server cleanly and hoping the open files would flush somewhere recoverable. Pulling the GCP snapshot I had (weeks stale, missing every build people actually cared about). extundelete, photorec, the whole desperate toolbox. Nothing got the world back in a state worth running.

A few days later I shut the server down "because nobody was really playing anymore." That part was half true. People had drifted off. But the honest reason is that the world they built was gone, and I did not want to be the one to announce that I ended it with one command.

What I changed afterwards

  • Backups live somewhere the server can never be. A separate disk, or better, a Cloud Storage bucket with gsutil rsync. If a backup shares a parent directory with the thing it is backing up, it is not a backup, it is a second copy waiting to be deleted in the same command.
  • Backups do not mirror the full path. rsync -a /home/Minecraft/ backup/ copies the contents, not /home/Minecraft/backup/home/Minecraft. The trailing slash matters. No more russian doll.
  • rm -rf never takes a path that starts with / unless I have said the whole path out loud first. For cleanup I delete relative paths, from a directory I pwd'd one second ago.
  • rm -rf --one-file-system when a script has to recurse, so it stops at mount boundaries instead of walking into the whole machine.
  • Scheduled GCP snapshots, and I actually test restoring from one. The stale snapshot that failed me was not a backup, it was a false sense of one.
  • Destructive scripts run with set -euo pipefail and echo the exact target before touching it.

The actual lesson

The command was not the bug. The bug was a structure where "the folder I want to delete" and "the entire system" could be typed almost identically, at 11pm, after an hour of clicking through nested directories.

Good structure is not decoration. It is the thing that makes the dangerous command look dangerous. When your backup is nested inside your production directory, rm -rf in the wrong pane is not a mistake you might make, it is a mistake the layout is quietly asking for.

Put backups out of reach. Copy contents, not paths. Take the snapshot, and test that you can actually restore from it, before you need to.

All posts

© 2026Made withbykeero1· All rights reserved.