Useful GNU/Linux Commands for Developers — Part 2

We cordially invite you to delve into a compilation of convenient GNU/Linux commands tailored for developers to address everyday challenges. It is important to note that this selection is geared towards individuals who wish to stay within the scope of regular usage without delving deep into the system, yet occasionally require interaction with a Linux server for work purposes.

In the preceding article, we explored commands pertaining to user and system information queries, file manipulation, process handling, and text manipulation. In this sequel, our focus shifts to exploring the capabilities of the bash shell, networking, and ssh functionalities. The content of this article is drawn from an IT gathering during which we shared valuable insights and tips.

XKCD: 1168

Bash Commands

Most Linux distributions are using bash shell, which harbors numerous useful features. It’s worth noting that bash offers auto-completion, enabling you to initiate command names and have bash automatically complete them if they exist within the path variable. Autocompletion is facilitated using the TAB key. The underlying rules for autocompletion are slightly intricate, and you can modify them as per your preferences. For commonly used commands, predefined autocompletion rules are generally included in your distribution.

You can refresh the terminal interface by removing previous clutter using the command:

clear

Alternatively, you have the option to restore the terminal interface’s client component to its initial state in the event of unexpected issues:

reset

A snapshot of previously executed commands on a specific machine for a given user can be obtained using:

history

This command supports several valuable variables to customize its behavior:

• HISTCONTROL: By default, commands beginning with a space character are omitted from history recording; this variable permits you to disable this behavior.
• HISTIGNORE: You can define one or more patterns to exclude specific commands from being recorded in history.
• HISTTIMEFORMAT: This variable facilitates the inclusion of timestamps in command history, proving particularly useful for server usage.
• HISTSIZE / HISTFILESIZE: These variables offer control over the history’s size; the default is typically set to 1000 entries.

History search functionality is accessible through the Ctrl+R combination. Upon activation, you can commence typing a command, and bash will automatically retrieve a matching entry from history. The right and left cursor keys allow editing of the selected command. Pressing Enter executes the chosen command, while Ctrl+G exits search mode.

Repeating the last two or three commands does not necessitate search usage. Instead, you can navigate history using the up and down arrow keys to sequentially cycle through previous commands. However, this approach becomes unwieldy for extensive command histories.

Further useful options:

• !!: Executes the last command from history.
• !N: Executes command N from history.
• !*: Substitutes arguments from the last history command, facilitating reuse of the same file as an argument in multiple commands.

Let’s delve into bash-specific output redirection (note that these commands are exclusive to bash; other shells feature distinct constructs):

• > file: Redirects STDOUT to a file, either overwriting or creating the file. Example usage involves redirecting command output to a file: history > ./222.
• >> file: Similar to the previous, but appends without overwriting.
• &>> / &> file: Redirects both STDERR (error messages) and STDOUT to a file. These options are employed when a command may generate errors.

Piping the output of one command as input to another can be achieved through the following structure:

command1 | command2

For instance, consider the command:

tail -f access.log | grep 502

This command combination opens the access.log file (let’s say it’s an nginx log) and searches for occurrences of “502”. This mechanism supports the construction of intricate pipelines.

Executing multiple commands sequentially can be achieved using these constructs, among others (we’ll highlight the most commonly used ones):

• cmd1 && cmd2: Executes cmd2 only if cmd1 succeeds.
• cmd1; cmd2: Executes both commands sequentially, regardless of cmd1’s outcome.

For convenient reference, a collection of helpful bash shortcuts is available here: https://www.tecmint.com/linux-command-line-bash-shortcut-keys/ .

Bash Aliases

An often-used feature of bash is aliases, which permit the assignment of custom names to commands alongside specific parameters or arguments. Some aliases are preconfigured in bash, including the “ll” alias that equates to “ls -l”. Notably, even “ls” itself is an alias (displaying a file list with color parameters). You can independently create additional aliases or install them from external packages.

An extensive selection of aliases geared towards efficient git usage is accessible here: https://github.com/Bash-it/bash-it/blob/master/aliases/available/git.aliases.bash. This resource serves as a versatile package for bash enhancements, encompassing not only aliases but also useful functions tailored for git operations. Noteworthy is the functionality that provides enhanced information when navigating directories containing downloaded Git repositories. This information includes the current branch, presence of changes, and their nature, simplifying Git-related workflows.

To incorporate these enhancements, copy the resource to your home directory and append the following lines to your .bashrc file:

if [ -f ~/git.aliases.bash ]; then
 . ~/git.aliases.bash
fi

Existing aliases within the system can be located in the /etc/profile.d directory.

Classic Networking Techniques

In this section, we present a compilation of commands that are excluded from modern Linux distributions. To utilize them, additional installation is sometimes necessary.

First and foremost, we have the netstat command. This command is integral to networking exploration and analysis. Key usages include:

• netstat -tunlp [seconds]: Displays an inventory of open TCP and UDP ports within the system and the corresponding listening processes. Refer to the man page for explanations of each character.
• netstat -es [seconds]: Presents a concise overview of Ethernet and TCP statistics, invaluable during network diagnostics.
• netstat -ontp [seconds]: Provides information about active connections, both with and without sockets.
• netstat -rn / netstat -rn: These keys facilitate route management and allow viewing of the routing table.

An alternative to netstat with real-time visualization is the utility iptraf-ng (previously known as iptraf2), which also accommodates UDP tracking. Although its presence may not be universal across servers, its usefulness renders it worthy of recommendation.

To inspect network interface configurations, the ifconfig command remains invaluable. While its inclusion has been omitted from modern distributions, it can be obtained alongside netstat from the net-tools package.

For viewing MAC addresses instead of IP addresses, the arp command is employed, particularly beneficial when dealing with older equipment featuring duplicate MAC addresses.

For checking open ports on a specific machine, the following commands are available:

• nc <ip> <port>
• telnet <ip> <port>

Exiting telnet mode can be achieved using Ctrl+] or the “quit” command.

Networking with iproute2

Let’s now delve into a more contemporary suite of utilities known as iproute2. This comprehensive toolkit serves as a versatile solution for managing the network subsystem of the kernel. It comprises three principal commands: ip, ss, and tc. The latter, tc, is primarily utilized for traffic shaping and is of greater relevance to network administrators working with Linux on border routers. Let’s shift our focus to the first two commands.

ip addr and ip link are analogous to ifconfig, providing the means to view and modify network settings, albeit with a slightly different approach. Similarly, ip route mirrors the functionalities of route and netstat.

For ARP table management, akin to arp, we have the ip neigh command. Parameter ‘neigh’ is an abbreviation for ‘neighbors’.

The commands ip -s and ss offer functionality akin to netstat. While they display similar statistics for connections, ss provides additional information. Notably, ss -ontup showcases open connections, including packet counts. On the other hand, ss -tunlp presents open TCP/UDP ports along with the corresponding listening processes.

It’s noteworthy that ip supports a cisco-like interface, enabling the use of abbreviated parameter spellings, for instance, ‘ip a’, akin to the practice on Cisco switches.

A valuable networking cheat sheet from RedHat can be accessed here: https://access.redhat.com/sites/default/files/attachments/rh_ip_command_cheatsheet_1214_jcs_print.pdf. This document contains practical examples for configuration and troubleshooting.

Using Curl for Diagnostics

Curl serves as a versatile tool for diagnosing website-related issues. Here are some useful options:

To perform a health check on a website without affecting hosts or DNS, you can utilize the following command:

curl -v — resolve example.com:443:127.0.0.1 https://example.com

This method proves helpful, particularly when testing a local copy of a virtual machine’s engine. While the conventional approach involves registering an IP address and domain name in hosts, curl offers an alternative solution. By employing the ‘resolve’ parameter, you can specify the domain name, port, and IP for evaluation.

You also have the option to create a text file, format.txt, in which you define the desired output format. This enables you to observe how curl processes the network connection:

curl -w “@format.txt” -o /dev/null -s “https://google.com”

The format file can be configured to display various time-related metrics, such as DNS lookup time, connection time, app connection time, pretransfer time, redirect time, start transfer time, and total time.

time_namelookup: %{time_namelookup}s\n
time_connect: %{time_connect}s\n
time_appconnect: %{time_appconnect}s\n
time_pretransfer: %{time_pretransfer}s\n
time_redirect: %{time_redirect}s\n
time_starttransfer: %{time_starttransfer}s\n
\n
time_total: %{time_total}s\n

This insight into the timing of different stages is valuable for diagnostic purposes.

Working with SSH

This section holds relevance not only for Linux users but also for owners of SSH clients on other platforms (including Mac and Windows). SSH is often considered a versatile tool for remote server interactions.

It’s worth noting that SSH can be employed in non-interactive mode, enabling direct execution of commands. For instance, by employing the syntax ‘ssh <server address> <command>’, tasks can be accomplished efficiently. This approach also supports the use of pipes to remotely execute commands, like ‘executecat log’ followed by ‘make itgrip’ through a pipe.

The command ‘scp’ allows the secure copying of files via SSH.

SSH can be utilized as a proxy, provided this feature is enabled in the SSH server configuration. For those hesitant to set up openVPN, SSH can serve as an alternative by using the command ‘ssh -D 8080’. Configuring your browser accordingly allows you to navigate the internet through the remote machine.

When the need arises to run a graphical program on a remote machine, the ‘ssh -X’ command can be utilized with SSH. This enables the execution of applications such as Firefox. Notably, the browser window opens on the remote machine, but it’s displayed on the local machine through x11. This approach works optimally when the ping to the remote machine is minimal.

For port forwarding, the ‘ssh -L’ command offers flexibility. Suppose you have a MySQL server on a remote machine. You can establish a connection that makes it seem as if MySQL is locally available on port 12345.

Furthermore, if you possess an SSH server with access to another server, you can forward ports to your advantage. For instance, utilizing SSH and the second server’s internal IP, you can make port 12345 appear locally as if it’s directed to the second server. If your network features multiple servers and resources, the ‘sshuttle’ utility offers a transparent solution. It proxies a designated subnet through SSH, resulting in seamless and localized access.

The reverse approach, involving port redirection, can be likened to a “poor man’s dyndns.” This technique proves invaluable when forwarding a local port to an external destination, granting remote access.

To configure SSH conveniently, you can create a file (typically ‘~/.ssh/config’ on Linux) to specify user preferences, such as usernames, ports, keys, reconnect intervals, and more. A sample file might resemble:

Host my-server
 Hostname 1.2.3.4
 User user
 IdentityFile ~/.ssh/my_sever_rsa
 LocalForward 5432 localhost:5432
 LocalForward 8000 localhost:8000
 ServerAliveInterval 300
 ServerAliveCountMax 2

Once configured, connecting to the designated server is as simple as executing ‘ssh my-server’.

That’s all. What Linux commands do you use the most?

Credit goes to Igor Ivanov, Anton Dmitrievsky, Denis Palaguta, Alexander Metelkin, and Nikolai Eremin (Maxilect) for compiling this valuable selection.

PS. Subscribe to our social networks: Twitter, Telegram, FB to learn about our publications and Maxilect news.