Which Linux tool or command is surprisingly simple, powerful, and yet underrated?"
Which Linux command or utility is simple, powerful, and surprisingly unknown to many people or used less often?
This could be a command or a piece of software or an application.
For example I'm surprised to find that many people are unaware of Caddy, a very simple web server that can make setting up a reverse proxy incredibly easy.
Another example is fzf. Many people overlook this, a fast command-line fuzzy finder. It’s versatile for searching files, directories, or even shell history with minimal effort.
nmap 192.168.1.43/24 will show you what devices are connected to the local network, and what ports are open there. really useful, for example, when you forgot the address of your printer or raspi yet again.
you can also use it to understand what ports on your computer are open from an attacker's perspective, or simply to figure out what services are running (ssh service).
it's useful for dealing with virtual disk images. like a real physical hard disk, but it's a file on the computer. you can mount it, format it, and write it to a real physical disk.
it's sometimes used with virtual machines, with iso images, or when preparing a bootable disk.
I once wrote a bc script that calculated parameters for the Blackman window for a FIR filter. (Had formulas already so not that impressive) Upped the precision until it needed like 30 sec to calculate, completely unnecessarely :).
I love jq, but I wouldn't call it "surprising simple" for anything but pretty-formatting json. It has a fairly steep learning curve for doing anything with all but the simplest operations on the simplest data structures.
Run it normally and it just spams 'y' from the keyboard. But when one of the commands above is piped to it, then it will respond with 'y'. Not every command has a true -y to automate acceptance of prompts and that's what this is for.
Sorry, I should have explained that. it's command | yesyes|command - Eg, yes|apt-get update (Not a great example since apt-get has -y, but sometimes that fails when prompting for new keys to accept)
The first incantation - ip address (you can abbreviate whilst it is unambiguous) gets you a quick report of interfaces, MAC, IPs and so on. The second command assigns another IP address to an interface. Handy for setting up devices which don't do DHCP out of the box or already have an IP and need a good talking to.
Oh and you can completely set up your IP stack, interfaces and routing etc with it. Throw in nft or iptables (old school these days - sigh!) for filtering and other network packet mangling shenanigans.
There are minor feature differences there's also a convenience factor: youtube-dl people for some reason stopped doing releases, so you can't get a fresh version from pypi (only installing from github or their site). Yt-dlp is on pypi, including nightly builds.
After spending years dealing with shady freeware and junk software on windows, I was floored by how easy and nonchalantly I was able to set up a simple security camera on my PC
I heard about helix from you and I've used it for a year and a half or so now, it's by far the best editor I've used so far and I can definitely vouch for it
I've actually been testing with fish recently coming from zsh, though I might wait until 4.0 fully releases before I make a more conclusive decision to move or not.
With that said, I remember looking through omf themes and stumbled onto Starship that branched off one of the themes and really liked the concept.
One thing that holds people back sometimes is that bash scripts that set environment variables don't work by default. https://github.com/edc/bass is an easy solution
Helix is a terminal based text editor. It’s much like vim / neovim, but unlike those editors it’s good to go right out of the box, no configuration or plugins needed to make it work well.
Topgrade is one I haven’t used, but it looks like its intended purpose is to let you upgrade your apps with one command, even if you use multiple different package managers (I.e. if you were on Ubuntu, you could use it to upgrade your apt packages, at the same time as your snap packages, as well as flatpak, nix, and homebrew if you’ve added those.)
Fish is a replacement of bash that's a bit more user friendly (has some cool auto completion features out of the box and more sane behaviour like handling of spaces when expanding variables). I personally started to use nutshell recently but unlike fish it's very different from bash.
Starship is a "prompt" for various shells (that bit of text in terminal before you enter the command that shows current user and directory in bash). I haven't used it but AFAIK it has many features like showing current time, integration with git, etc.
zoxide. It's a fabulous cd replacement. It builds a database as you navigate your filesystem. Once you've navigated to a directory, instead of having to type cd /super/long/directory/path, you can type zoxide path and it'll take you right to /super/long/directory/path.
I have it aliased to zd. I love it and install it on every system
You can do things like using a partial directory name and it'll jump you to the closest match in the database. So zoxide pa would take you to /super/long/directory/path.
And you can do partial paths. Say you've got two directories named data in your filesystem.
One at /super/long/directory/path1/data
And the other at /super/long/directory/path2/data
You can do zoxide path2 data and you'll go to /super/long/directory/path2/data
ddccontrol... it looks complicated on the surface but it's really not and being able to control monitor brightness without fcking around in some garbage monitor OSD is a god sent and should be the standard
I don't see anyone mentions htop. So, I will:)
Just works, could be installed in any distro. Much more friendly than top but isn't bloated with features as some other alternatives are.
Came here to say both of these things. (Awk and "> simple".)
To be totally honest, I don't think awk is any more complicated than something like grep, it's just that regular expressions get used more often so they're typically more familiar. In the same way that programming languages with c-like syntax (like Java and C#) often feel easier than ones that don't (like Haskell and Clojure).
I'd recommend tmux for that particular use. Screen has a lot of extras that are interesting but don't really follow the GNU mentality of "do one thing and do it well."
When tmux was first released I was already so used to screen that I never really considered switching. What would some convincing arguments be for me to make the effort to switch now?
I know everyone likes tmux but screen is phenomenal. I have a .screenrc I deploy everywhere with a statusbar at the bottom, a set number of pre-defined tabs, and logging to a directory (which is cleaned up after 30 days) so I can go back and figure out what I did. Great tool.
Woah screen is seeing active development again? There was like a decade where it stagnated. So much so that different distros were packaging different custom feature patches (IIRC only Ubuntu had a vertical split patch by default?) Looking at it now, the new screen maintainers had to skip a version to not conflict with forks that had become popular.
When tmux stabilized I jumped ship immediately and never looked back.
I've had nohup fail to keep things running after my session ended quite frequently. It's like it just goes to the next step in the process then gives up.
I use it occasionally but every time I need to do something a tiny bit more complex than "extract field from an object" I have to spend half an hour studying its manual, at which point it's faster to just write a Python script doing exactly what I need it to do.
Underrated? I'd say lftp is the best FTP command line client there is. And Midnight Commander is a very very good file browser. I don't see either praised enough.
The terminal-based file browser space is so filled today but for my part I love what vifm has done for the dual-pane midnight commander concept - it's the same basic idea, uses (somewhat) vim-like bindings by default and is super extensible.
Great call out! I first used ftp about 30 years ago, and lftp has been my go to for about the last decade. I rarely need it anymore, but I still use it for quickly transferring files with my homebrew switch.
I know tmux is incredibly popular, but a good use case for it that isn’t common is teaching people how to do things in the terminal. You can both be attached to the same tmux session, and both type into the same shell.
Tmux is so much better than screen, and yes that is the hill I will die on
Specially when confined with tmuxp , it's how I handle Game servers that can run headless to start at boot without losing access to giving commands to the server via its server console
vim isn't required for any files, you just followed online tutorials for how to edit those files instead of RTFM
terminal text editing is convoluted because it has to strike a balance between figuring out when a keypress is part of the text you're typing, vs when it's a command you're using, and making sure that all the editor commands the designer wanted are accessible.
vim is great because it allows for thousands more editing commands and macros, and much more customization of the editor, up to allowing plugins that emulate other functionality. As it stands, my setup basically functions as a full, lightweight-ish, multi-language IDE that rivals Emacs or Visual Studio.
On top of all that, I don't have to move my hands away from the homerow of keys to navigate or edit, which may not seem like much, but adds up to a lot of avoid typos and time saved from moving my hands to reach the arrows/delete/home/end/pgup/pgdn.
Some examples:
h, j,k,l move left, down, up, and right respectively, but they can be combined with a number to move that many rows or columns; e.g. 6j will move down 6 rows
dd deletes a line, but using a number + d + a movement will delete that many characters/lines in the path of the cursor: e.g. 34dl will delete 34 characters to the right of the cursor, 12dk will delete 12 lines up.
gg will take you to the first line, G will take you to the last, and number + either will take you to that line: e.g. 3275gg or 3275G will take you to line 3275
and finally you can use /text or regex pattern you want to search for and Enter to search the document for the first occurence below your current location, and then use n to search for the next occurence, or N to search for the previous
That doesn't even scratch the surface (that's just the cheatsheet, which only scratches the surface), but if you can get a handle on only what I've said, and switching between input and command mode (i and Esc respectively), the speedup to navigation alone will make it seem more sensible.
And as always, don't forget to :wq (write to file and quit)
It's for people to memorize hundreds of arcane shortcuts and shit so they can feel like a smug hacker and gloat over the rest of us using other editors and getting just as much done as they are.
Also for graybeards that haven't realized it's not 1985 anymore.
For the average user you're definitely right, but I will say for the sysadmin of headless systems, having a powerful cli editor is a godsend. While it may seem arcane and unnecessary, learning vim is easier than managing remote x or sshfs or copying files to and from a system.
I didn't learn vim to be a contrarian; I learned it because it seemed (and still seems to be) the path of least resistance for many workflows.
It's for people that don't want a big bulky IDE and are willing to put a little work in to get used to it. I do all my coding in the terminal with vim and tmux and I like the simplicity and that with two dotfiles I can migrate my whole development environment to whatever PC, server or RaspberryPi that I need.
I've used Vim for some pretty non-nerdy stuff. Like ripping my DVD collection, when I got to the TV section I had a lot of file names to modify in bulk, and Vim let me do that. Also guitar tablature, the ability to edit plaintext both horizontally and vertically is surprisingly handy. Just having a macro to be able to add a bar line saves a shocking amount of time.
seems like you need to try micro. It's like nano, but with more sensible standard keybinds imho, as well as syntax highlighting and global clipboard use.
It's weird but VIM is so powerful and I love it but i also agree it wouldn't be the default just an option if you needed it. It's like with notepad ++ on windows it's wonderful but not everyone needs it from day one notepad will work just fine for basic typing.
I used nano when I started but now I am using vim for one year already. I'd recommend taking a few days where you only use vim and I think you will see why people like it. With a few motions you can be much faster than you would be in Nano.
One of the big reasons I switched to nixos is that I mostly need to use the console only for updating my system by editing the configuration file using nano. I do very little besides that thankfully while the GUI side of linux gets better everyday.
As for why: arbitrary choice, they just needed a printable character they could show on screen, for when people pressed it and the terminal echoed it back out to them.
We've been using tsp at my work for years and it works well. It is just a very basic queueing system so if you can run the job from the command line then you can run it via tsp.
Our workflow is to have concurrent jobs run on the remote servers with cron and tsp but you should be able to trigger remote jobs over SSH also if you prefer to have a single machine in charge of task allocation.
yq is crazy cool for converting between different text-based data formats such as yaml, json, xml, csv and others, and it has a super nice pretty-printing function as well. I use it all the time!
Just be aware that your distroy might come with a yq variant too, but possibly one that isn't as powerful as the one I linked. I know this to be true at least for Ubuntu.
I used jq for something similar before, recently I've discovered Nu Shell and have been using that for converting and analyzing data since a full shell is a lot more powerful than a command (e.g. open a yaml, for each element on key X grab the first element of list Y and export to a CSV)
dd is probably well known, but one of the simplest and most powerful ways to accidentally delete all data on your hard drive. dd if=/dev/random of=/dev/sda
@bamboo@mfat , DD, great tool. Utilize it so often, but it is powerful and dangerous. I always double, triple, quadruple check my target disks with multiple programs to avoid destroying my production workstation. Might be best if I just designated a RPi for the job. 😅
I'm not sure how underrated it is but the exec feature in find is so useful, there are so many bulk tasks that would just be incredibly difficult otherwise but instead are just one line
vd (VisiData) is a wonderful TUI spreadsheet program. It can read lots of formats, like csv, sqlite, and even nested formats like json. It supports Python expressions and replayable commands.
I find it most useful for large CSV files from various sources. Logs and reports from a lot of the tools I use can easily be tens of thousands of rows, and it can take many minutes just to open them in GUI apps like Excel or LibreOffice.
I frequently need to re-export fresh data, so I find myself needing to re-process and re-arrange it every time, which visidata makes easy (well, easier) with its replayable command files. So e.g. I can write a script to open a raw csv, add a formula column, resize all columns to fit their content, set the column types as appropriate, and sort it the way I need it. So I can do direct from exporting the data to reading it with no preprocessing in between.
Use less for checking contents of files. Many people use cat all the time, but I don't like it, because if you do that often, your terminal window quickly gets flooded with stuff, and then you have to scroll up and down if you wanna see a previous output. With less, your file opens in a different "frame", which you can close when you're done.
I'd like to interject for a moment.
There is also a tool called bat that is just cat with extra features. It prints out and works just like cat, but when the contents get too big, it works like less. The is syntax highlighting and works with git.
Because sudo is best suited for server administration, way overengineered (with the occasional critical vulnerability) for desktop use.
Alternatives can fit the required functionality for desktop use in 150 loc C code while sudo has what, something over 100'000 loc?
the (usually rust-based) coreutils "alternatives" like bat, fd, eza, procs
trash-put (rm with trash integration. But beware that it also operates on directories by default, which rm only does with -r. There should be an option to change that behavior but there isn't. Don't alias rm to this)
wl-copy/paste (or the older one for X11, 'xclip' IIRC. Enables you to do stuff like "cat image.jpg | wl-copy" to copy it to the clipboard. Best alias it to something shorter)
xdg-open (open the file using your associated program for that file type. Alias to "o" or so)
pass (awesome password manager, when you have a GPG key pair. Even better in combination with e.g. wofi)
notify-send (to send GUI notifications from shell scripts)
ledger (plain-text accounting software. If you use Emacs you should take a look at this as it's written by an Emacs dev, and has good integration of course)
I like https://github.com/aristocratos/btop personally. It's way prettier than the normal top command which you use to watch processes to find the one that's hogging all of the CPU or whatever. And it's not so much that it's underrated so much as it's not very well known or distributed by default.
..for parsing the output of other commands quickly and simply. Then that parsed output can be used to create simple log messages or be passed as args to other scripts. Powerful.
I agree with your sentiment regarding confusing syntax, however I think that confusion simply requires a calculated approach to dispell it.
It's a prime example of why I use scripts as reminders as much as I use them functionally. I work out the syntax once.. save it to an example script, then save myself 20 minutes of remembering by just $ cat ./path/to/script.sh and copying said syntax.
So if you can change your workflow such that learned things stay around as examples, I feel that you will pick it up much more quickly :)
An OS used by IBM on their mainframes. I have to work with it because some big orgs, especially financial and gov, still use it and run our apps on it.
I love lazygit and I'm still surprised at how many people are shocked when they see it for the first time. Not exactly a command, but a very handy text UI tool.
For more elementary tools, I can't believe how many people know about ! and ctrl+r who don't also know about fc and edit-and-execute-command.
degit is a tool that will check out a git repo (or a specific branch or commit), but not set it up as a git repo. Basically just downloading a specific commit to a directory.
I immediately had flashbacks of diagnosing bad I/O performance on CentOS 5 servers. That was the week when I learned what updatedb is and why it was always running in the background (there was a lot of files)
No sorry, I should have elaborated. The package name is mlocate but the command is locate. Occasionally run updatedb as it populates an sqlite db with every file on your system that you can then list out using locate followed by the filename you want to locate.
EDIT: Lol. Sorry barely read your reply. Yes, you should wear a fedora while installing mlocate.
Executing a command, capturing all terminal formatting and escape codes so I can do some light manipulation on leading whitespace before dumping it back to the terminal.
+1 to caddy. There are some services that set safe headers following the recommendations outlined by Mozilla but others don't control headers as strictly. Caddy is the only web server that I found that supports loose default header values. These values will be selected unless the upstream application specifies their own values.
You can do something similar in nginx but it requires playing with maps and has a little more indirection than I'd like.
Just wish caddy was capable of starting as root and stepping down permissions like Nginx. I have certs being managed by other tools and have to make sure they are installed and chowned for caddy's use when they are cycled.
I just started the process of switching from nginx Proxy Manager to Caddy yesterday, and even before setting up a single rule, I'm enjoying it more than NPM. Really wish I would have heard about it sooner!
I'm currently using NPM and don't have any problems with it for at least my use case. Is there something I'm missing out on never having tried Caddy, or is it one of those no need to switch if there's Nothing bugging you situations? That last bit is how I feel about Bazzite on the Steam Deck when people ask of they should switch.
I use node as a calculator a lot. It can be dangerous, because it suffers from floating point errors, but it’s generally more powerful than a calculator if you know the Math lib well.
Wezterm has made this easy : shift + ctrl + x and your in vim motion moving all over with yank line or word or visual mode. It's super awesome! I never touch my mouse. You can also remap the keymap of course.