Run a mock round

Measure latency and utilisation, then find the culprit.

1. Space versus inodes: use df -h and df -i. A full inode table also breaks writes.
2. Utilisation and latency:

iostat -xz 1 5
iotop -o
pidstat -d 1 5

Look at %util, await, and queue size. High await with low utilisation suggests slow storage, while high %util suggests saturation.

1. Find heavy writers: iotop shows per-process I/O, and lsof +D can reveal open files.
2. Check for errors: dmesg for I/O resets, smartctl -a /dev/sda for disk health, and mount options such as noatime.
3. Filesystem: a full or fragmented filesystem, or a runaway log, can bottleneck. Check and rotate logs.

dstat -d --top-io

Remedies: tune the I/O scheduler, adding mq-deadline or none for SSD and NVMe, add IOPS, move data, or fix the application's write pattern.

The boot sequence moves from firmware to the kernel and then to userspace.

1. Firmware, BIOS or UEFI, runs POST and then loads the bootloader from disk or network.
2. GRUB reads its configuration, presents the menu, and loads the kernel and initramfs into memory.
3. The kernel initialises hardware, mounts a temporary root from initramfs, then switches to the real root filesystem.
4. The kernel starts PID 1, systemd. systemd reads the default target and starts units with their dependencies.
5. systemd brings up services, mounts, and getty or login, reaching multi-user.target or graphical.target.

systemd-analyze
systemd-analyze blame
journalctl -b -p err

Troubleshooting: check firmware boot order, GRUB configuration, kernel parameters, and fsck. A corrupt initramfs or missing root device usually drops to an emergency shell. Recovery uses a rescue target or live media to chroot in and repair.

Linux permissions have three classes (user, group, other) and three bits each: read (4), write (2), and execute (1).

• ls -l shows them as rwxr-xr--. The leading character is the type: d for directory, - for file, l for symlink.
• chmod sets permissions with symbolic (u+x) or octal (755) notation.
• chown changes owner and group. Directories also have setuid, setgid, and sticky bits; the sticky bit on /tmp means only owners can delete their files.

chmod 640 config.yml      # rw-r-----
chmod u+x deploy.sh
chown app:app /var/www
chmod +t /shared

Special bits: setuid runs a file as its owner, used by passwd, and setgid on a directory makes new files inherit the group. Also check ACLs with getfacl. Always apply least privilege, especially on secrets.

Identify the consumer before acting.

CPU:

top -o %CPU
uptime              # load average
pidstat 1 5         # per-process CPU
perf top            # hot functions

Check load average against core count. High load with low CPU often means I/O wait. Look at runnable versus blocked processes, and check for CPU steal on virtual machines.

Memory:

free -h
vmstat 1 5
ps -eo pid,rss,cmd --sort=-rss | head
cat /proc/<pid>/status | grep -i vm

Distinguish buff and cache from used memory; Linux uses free RAM for caching. Check for OOM kills in dmesg or journalctl -k. Look for swap thrashing, visible as high si and so in vmstat.

Then profile the application, check for leaks, tune limits, or right-size the instance. Use cgroups to cap runaway processes.

cron runs commands on a schedule defined by five time fields: minute, hour, day of month, month and day of week.

# m h dom mon dow command
30 2 * * * /usr/local/bin/backup.sh >> /var/log/backup.log 2>&1

Common pitfalls:

• cron uses a minimal environment, so PATH and variables from your shell are absent. Use absolute paths or set PATH at the top of the crontab.
• Redirect stdout and stderr or you lose the output.
• Jobs run concurrently if the previous run overruns; guard with flock.
• The percent sign is special and must be escaped.
• Timezone depends on the system or CRON_TZ.

For complex schedules, prefer systemd timers, which offer logging through journald, dependency handling and better observability.

Both point to files, but at different levels.

• Hard link: another directory entry for the same inode. The file data is shared, and the link count increases. Deleting one name does not remove the data until all links are gone. Hard links cannot cross filesystems and usually cannot point to directories.
• Symbolic (soft) link: a separate file containing the path to the target. It can cross filesystems, point to directories, and become dangling if the target is removed. Permissions on the link itself are ignored; the target's permissions apply.

ln file.txt hard.txt      # hard link
ln -s /var/www site       # symbolic link
ls -li                    # shows inode and link count
readlink site

Use hard links for deduplication and backups within a filesystem, such as rsync --link-dest, and symlinks for flexible references such as versioned releases and config pointers.

systemd is the init system on most modern distributions. It starts services and manages dependencies, logging, and lifecycle.

• Units describe resources: .service, .socket, .timer, .target, and .mount.
• Unit files live in /etc/systemd/system for custom units, /usr/lib/systemd/system for packaged ones, or ~/.config/systemd/user for user units.
• systemctl controls units: start, stop, restart, reload, enable, disable, and status.
• Dependencies use After, Wants, and Requires. Targets group units for boot states.
• journald collects logs, and journalctl reads them.

[Unit]
Description=My API
After=network.target
[Service]
ExecStart=/usr/bin/myapi --port 8080
Restart=on-failure
User=app
[Install]
WantedBy=multi-user.target

sudo systemctl daemon-reload
sudo systemctl enable --now myapi
systemctl status myapi

Run daemon-reload after editing unit files, and enable for boot persistence.