So we are all familiar at some point with the advice to use the command
dd if=image.iso of=/dev/sdx
or similar. People almost religiously think of dd as the tool to do this. It certainly can do the job, but not only is it really bad at it (mostly because it doesn't let the kernel automatically select a good block size), but it is also infamous for having a very fragile command syntax that is easy to typo and destroy your data with. dd wasn't actually written as a disk copying tool and is meant to be used as a data conversion tool. People using it to copy images to disks without doing any sort of conversion are kind of just abusing the fact that it reads one file in and writes another out. A lot of people don't know this, but on Linux/UNIX systems you can interact with block devices like they are normal files. As a result it's generally faster and safer to do something like
cp image.iso /dev/sdx
instead. People frequently uselessly risk using dd in places where ordinary file commands like cat and cp work fine. Some of you nerds probably already knew this but I want to spread it because people recommending dd is way too common.
Also, regardless of what commands you are using, when you finish doing anything involving writing to block devices always remember to run the command sync to make sure the kernel has actually written the data to the disk and isn't waiting to later.
Friendship ended with
dd if=image.iso of=/dev/sdx && sync, nowcp image.iso /dev/sdx && syncis my best friend.man when they say everything is a file they really mean it huh
You can also use the
gzipcommand directly on block devices (as input) to make backups, because everything is a file.plan9 actually took that idea all the way to its conclusion but it doesn't actually pan out on unixes. but it does way more often than people realize. like here's a one-liner to test network connectivity with nothing but a bash shell:
timeout 5 bash -c "</dev/tcp/${HOSTNAME/://}"
there's probably a cleaner way to do that with exec but you get the idea. super useful when you're in a container that doesn't have many useful network utilities.
been using linux for a few years now and I still have no idea what anything in terminal means. people told me I'd learn but I can never make heads or tails of it
It's really not too hard and makes a lot of tasks much faster, but you won't just magically learn it.
Ubuntu has a tutorial here for basics: https://ubuntu.com/tutorials/command-line-for-beginners#1-overview
And you can find out the options for most commands by running them with -h or --help. You can also use the man command followed by the name of another command to view a manual for it. Pressing the tab key as you type a command will also suggest autocompletions on a lot of modern shells.
Yeah, ive got a grip on the basics (cd, ls, nano, man, cp, mv) but thats about it. Ive been wanting to try more terminal based applications, but have struggled just getting started. My idea was to use a matrix client to start, but installing hasnt quite worked for me
using an app that needs to take over the terminal won't help as you won't really get a feel for it. this is old school, but pick a distro that requires you to set it up through the command line and bash your way through it. you'll learn a lot about the many pieces that work together to make a modern operating system actually work and you put yourself in an environment where learning is the only choice.
you put yourself in an environment where learning is the only choice
That's been my experience with learning linux in general. I've learned most of what I know about linux either by breaking things and needing to fix them, or by wanting to set up something I'd never done before and diving into wiki pages and config files.
I can not tell you how much I learned about linkers a couple of days ago beating my head into the "statically compile rust with c++ libs" wall for like 12 hours while barely making progress, lol. dear god symbol resolution is complicated.
Days like that make you appreciate both how nice it is that linking works seamlessly 99% of the time, and also what an absolute mess your library setup usually is.
unfortunately, the library set up wasn't that unreasonable, it's just an unsupported and untested combination.
I appreciate the tip. Maybe i can find a junk laptop to fiddle with on craigslist or something. Any suggestions on the distro?
gentoo is much, much easier these days, lol. it's a mostly scripted install process you can bail out of to customize whatever you like. the only reason I suggest arch over it is that emerge is much more complex to learn than pacman and the archwiki is the best linux documentation on the internet (from a user perspective)
I mean, I'm comparing it to compiling on a first gen athlon 64 from 2004 so not really? lol. there's also build caches for most everything in the basic install process already.
I wouldn't bother, there's literally no advantage. nixos is a much more interesting source based distribution because it largely delivers on its promise of a fully reproducible installation - meaning you configure stuff once and (if you've pinned versions correctly), you will get the exact same results on any other computer you take that configuration to.
Guix is usable on something like a thinkpad IME. It's really fun to learn Guile Scheme too and I ended up submitting patches.
yea, I second arch. way back in the day, a stage 1 gentoo install was the most thorough way to learn but it took literally two weeks of beating your head into a wall and watching code fail to compile. your biggest problem now is that arch actually automates a lot of the install process - but enough of the core is still exposed that there's plenty to learn.
also, I suggest a VM. easier to work with than a whole separate laptop.
I'd say go with arch, it has a really detailed guide on the wiki for setting everything up.
If you really want to learn, try the Bandit OverTheWire game, https://overthewire.org/wargames/bandit/
/dev/sdxis bad even if you're not using good old disk destroyer, use/dev/disk/by-label/whateverto avoid trying to remember if/is sd1 or sd2 or if your init was racey this morning and they switched.Been using Linux for a decade and never knew this, thank you!
I don't understand, how exactly is
cpmore secure thanddto typos? I've been using dd all my lifedd has a very unusual syntax that doesn't comply with other UNIX commands (because of the if= of=). If you screw this syntax up, dd will not care and will do exactly as you typed the command. This last part is also true of cp to some extent (though cp has a lot more safeguards), but cp only takes the input and output file paths in, uses a very ordinary syntax, works properly with shell tab completion. If you're careful it shouldn't matter, but cp is just simpler, more familiar, and better integrated with most shells.
Regardless of that though, dd is very bad at this task for the other specified reasons, and there isn't any real reason to use it over them unless you need something specific to dd like conversion, or for whatever reason need a specific uniform block size (very uncommon on modern disk interfaces).
If you need a progress indicator use
pvinstead, it at least use correct file copy syscalls.
Off the top of my head, was it mounted when you were trying to do that? The kernel doesn't like you writing directly to a mounted disk's block device because the filesystem layer is still running.
Huh this is actually Debian's recommended method. I just use a GUI utility these days in true scrub fashion, because using disk destroyer always feels janky unless I'm just doing a quick flush on a disk (otherwise GNU
shredis my best friend).Holy shit i'm considered by some people to be a professional with this kind of thing. Thank you for this information. I don't know how the entire internet decided to ignore this.
oh god I thought I was the only one to do this. Rufus consistently creates more reliable drives than whatever I do - and no I will not investigate why.
Dd has nice progress stats though, and sometimes it's useful to know when the data is being flushed to disk, and not just being "copied" into a virtual buffer for later copying.
Sure you can poll the cp command with a progress tool, but that's an extra window
