¶ Linux Tools
This is a collection about various Linux tools and applications that are mainly used in a Linux/UNIX environment or inside a bash terminal.
¶ Table of Contents
- Visual Studio Code
- Terminal Duplicator: Screen
- Tor Browser
- Sherlock - Find Usernames Across Social Networks
- cURL
- Wget
- Cron
- Rsync
¶ Tor Browser
The Tor Browser is a internet browser that allows users to surf the web anonymously (If used the right way). It also gives you access to the dark web.
Tor Browser can be installed very easily by using apt.
Open a terminal, then run the following commands:
sudo apt update
sudo apt install -y tor torbrowser-launcher
After the process completes, run the following command as non-root user:
torbrowser-launcher
The first time, it will download and install Tor Browser, including the signature verification.
Next time, it will be used to update and launch Tor Browser.
Warning: It is true that you are more anonymous when using the Tor Browser instead of a normal browser like Chrome or Firefox.
But to be even more
secureandanonymouswhile browsing the web with theTor Browser, get a good and secure VPN.A good VPN will encrypt your data and hide your IP address as well. The possibility of being tracked will be very low and harder to accomplish.
¶ Visual Studio Code
Visual Studio Code is a free, powerful code editor that runs on every machine.
¶ Screen
Screen is a terminal multiplexer. In other words, it means that you can start a screen session and then open any number of windows (virtual terminals) inside that session. Processes running in Screen will continue to run when their window is not visible, even if you get disconnected.
¶ Install Screen
You can install Screen by using the apt manager
sudo apt update
sudo apt install screen
¶ Start Screen
To start a Screen session, simply type screen in your terminal.
Now that you have a screen session open, you can get a list of commands by typing:
Ctrl + a (Shift) ?
¶ Start Named Session
You can as well start a named session, for example if you are running multiple screen sessions. To create a named session:
screen -S session_name
¶ Useful Screen Commands
Tip: You can use
manto see a detailed documentation aboutscreen.
To create a new window with shell
Ctrl + a + c
To list all windows
Ctrl + a + "
Switch to a window
Ctrl + a + 0..9
Rename a current window
Ctrl + a + A
¶ Window splitting
Split current region horizontally into two regions
Ctrl + a + S
Split current region vertically into two regions
Ctrl + a + |
Switch the input focus to the next regions
Ctrl + a + Tab
Toggle between the current and previous windows
Ctrl + a Ctrl + a
Close all regions except the current one
Ctrl + a + Q
Close the current region
Ctrl + a + X
¶ Detach from Screen session
You can detach from a screen session anytime by typing
Ctrl + a + d
The program running in the screen session will continue to run after you detach from the session.
¶ Reattach to a Screen session
To resume your screen session, use the following command
screen -r
If you have multiple screen sessions running on your machine, you will need to append the screen session ID after the r switch.
To find the session ID, list the current running screen sessions with
screen -ls
The output will look like this
user@hostname:~# screen -ls
There are screens on:
20131.pts-0.hostname (05/30/2022 12:43:27 AM) (Detached)
29062.pts-0.hostname (05/30/2022 12:41:46 AM) (Detached)
2 Sockets in /run/screen/S-root.
If you want to restore screen 29062.pts-0, then type the following command
screen -r 29062
¶ Sherlock
Sherlock is a python tool, that uses a username to find social media accounts across various social networks.
¶ Install Sherlock
Sherlock can be installed very easily by using the following commands:
Clone the official repository into a desired location first:
git clone https://github.com/sherlock-project/sherlock.git
After that, change the working directory to sherlock:
cd sherlock
python3andpipfirst, before you can install the necessary requirements
Update your system:
apt update
Then install python3 and pip:
apt install python3
apt install python3-pip
The last step is installing the python requirements:
python3 -m pip install -r requirements.txt
¶ Sherlock Usage
A detailed description about all possible arguments sherlock provides can be looked up with:
python3 sherlock --help
And on the official GitHub repository.
To search only for one user:
python3 sherlock user123
To search for more than one user:
python3 sherlock user1 user2 user3 ...
If any accounts are found, they will be stored in an individual text file with the corresponding username (e.g.
user1.txt,user2.txt).
¶ cURL
cURL (Client for URL) is a command-line tool used to transfer data to or from a server, using various protocols such as HTTP, HTTPS, FTP, SMTP, and more. It’s primarily used to interact with APIs, download/upload files, or test network services. cURL supports many features, like proxy support, user authentication, file upload, SSL connections, and more.
In addition to the command-line tool, there is a libcurl library that developers can use in their applications to perform the same operations programmatically.
¶ Basic Syntax of cURL
curl [options] [URL]
URL: The web address (or file address) you’re trying to interact withoptions: Flags or options to modify the request is sent (e.g., specify HTTP headers, method type, etc.)
¶ Common Use Cases of cURL
¶ Making Simple GET Requests
curl https://example.com
The command fetches the content of the specified URL and outputs it to the terminal.
¶ Downloading a File
curl -O https://example.com/file.txt
-O: Saves the file with its original name
¶ Saving Output to a File
curl https://example.com -o output.html
-o: Saves the output of the request to a specified file
¶ Sending POST Requests
curl -X POST -d "param1=value¶m2=value2" https://example.com/api
-X POST: Specifies that the request should be a POST request-d: Sends form data (URL-encoded by default) with the request
You can also send JSON data
curl -X POST -H "Content-Type: application/json" \
-d '{"param1":"value1","param2":"value2"}' \
https://example.com/api
Or even HTTP Headers
curl -H "Authorization: Bearer YOUR_TOKEN" https://example.com/api
¶ File Upload
curl -F "file=@/path/to/your/file.txt" https://example.com/upload
-F: Specifies form data for file uploads. The@sign is used to upload a local file
¶ SSL Certificate Handling
By default, cURL verifies SSL certificates. If you want to bypass it, use the following:
curl -k https://example.com
-kor--insecure: Allows curl to perform “insecure” SSL connections and transfers without verifying the certificate.
¶ Sending a Request with Basic Authentication
curl -u username:password https://example.com/login
-u: Provides the username and password for basic HTTP authentication
¶ Follow Redirects
By default, cURL does not follow HTTP redirects. You can use -L to enable it
curl -L https://example.com
¶ Wget
Wget is a command-line utility like cURL used for downloading files from the web. It supports protocols like HTTP, HTTPS and FTP. Wget is primarily used for downloading files non-interactively (i.e., it can be run in the background or via scripts) and has features like resuming interrupted downloads, mirroring entire websites, and recursive downloading.
¶ Basic Syntax of Wget
wget [options] [URL]
URL: The web address (or file address) you’re trying to downloadoptions: Flags or options to customize the behavior of the download (e.g., resuming downloads, limiting download speed)
¶ Common Use Cases of Wget
¶ Downloading a Single File
wget https://example.com/file.txt
This command downloads the specified file from the server.
¶ Saving a Download with a different Filename
wget -O newfile.txt https://example.com/file.txt
-O: Saves the file asnewfile.txtinstead of its original name
¶ Resume an Interrupted Download
wget -c https://example.com/largefile.zip
-c: Allows wget to continue an interrupted download, resuming where it left off
¶ Recursive Download (Download Entire Website)
wget -r https://example.com
-r: Enables recursive downloading, allowing you to download entire websites, including their files and directories
¶ Limit Download Speed
wget --limit-rate=100k https://example.com/largefile.zip
--limit-rate: Limits the download speed (e.g.,100kfor 100 KB/s)
¶ Downloading Files in Background
wget -b https://example.com/largefile.zip
-b: Downloads the file in the background
¶ Mirroring a Website
wget -m https://example.com
-m: This is a shortcut for mirroring a site. It automatically turns on recursive downloading, timestamping, and other useful features for copying websites.
¶ Ignore SSL Certificate Errors
wget --no-check-certificate https://example.com
--no-check-certificate: Bypasses SSL certificate verification (useful for testing or in situations where the certificate is invalid).
¶ Downloading with HTTP Authentication
wget --user=username --password=yourpassword https://example.com/securefile.zip
--userand--password: Used to download files from servers requiring HTTP authentication.
¶ Download Files from an FTP Server
wget ftp://username:password@ftp.example.com/file.zip
Downloads a file from an FTP server with login credentials.
¶ Cron
cron is a Unix-based job scheduler that automates the execution of scripts or commands at specific intervals. It runs in the background and executes tasks (known as “cron jobs”) at predetermined times or intervals.
Each user can define their own cron jobs, and system administrators can also set system-wide cron jobs.
¶ What is Crontab?
crontab (short for cron table) is a configuration file where cron jobs are defined. You can schedule commands or scripts to run at specific times by adding entries to your crontab file.
¶ How to use Crontab?
Opening Crontab for Editing
crontab -e
This opens your crontab file in the default editor (usually vim or nano).
Crontab Syntax: Each line in the crontab represents a scheduled job with the following syntax
* * * * * command-to-execute
┬ ┬ ┬ ┬ ┬
│ │ │ │ │
│ │ │ │ │
│ │ │ │ └── Day of the week (0 - 7) (0 or 7 is Sunday)
│ │ │ └──── Month (1 - 12)
│ │ └────── Day of the month (1 - 31)
│ └──────── Hour (0 - 23)
└────────── Minute (0 - 59)
Special characters:
*: Any value (wildcard),: List of values-: Range of values/: Step values (e.g.,*/5for every 5 minutes)
¶ Some Crontab Examples
- Run a command every minute
* * * * * /path/to/command
- Run a script every day at midnight
0 0 * * * /path/to/script.sh
- Run a backup script every Sunday at 2 AM
0 2 * * 0 /path/to/backup.sh
- Run a script every 5 minutes
*/5 * * * * /path/to/script.sh
- Run a job on weekdays at 1 PM
0 13 * * 1-5 /path/to/weekday-job.sh
- Log output to a file: You can log both standard output and errors to a log file
* * * * * /path/to/script.sh >> /path/to/logfile.log 2>&1
¶ How to manage Crontab Jobs
- List crontab jobs: To view your current cron jobs, run
crontab -l
- Remove crontab jobs: To remove all your cron jobs, run
crontab -r
¶ Cron Timing Syntax Examples
| Expression | Meaning |
|---|---|
* * * * * |
Every minute |
0 * * * * |
Every hour at minute 0 |
0 0 * * * |
Every day at midnight |
0 0 * * 0 |
Every Sunday at midnight |
*/15 * * * * |
Every 15 minutes |
0 9-17 * * 1-5 |
Every hour from 9 AM to 5 PM on weekdays |
30 2 * * 6 |
At 2:30 AM every Saturday |
¶ Advanced Features of cron
- Running jobs under a different user: System administrators can specify a user for a job in
/etc/crontabor/etc/cron.dfiles
0 0 * * * root /path/to/script.sh
This runs the job as root.
¶ Cron Logs and Debugging
To verify if your cron job is running as expected, you can check the cron logs
cat /var/log/syslog | grep CRON
If your cron job isn’t running as expected, it may be because:
- Incorrect PATH: Add the full path to the executable (e.g.,
/usr/bin/python3instead of justpython3). - Permission issues: Make sure the cron job has sufficient permissions to execute the command.
¶ Rsync
rsync is a fast and versatile file copying and synchronization tool used to efficiently transfer and synchronize files between local directories or between local and remote systems over a network. It’s commonly used for backups, mirroring, and copying large sets of files because it minimizes data transfer by copying only the differences between source and destination.
¶ Key Features of rsync
- Incremental transfers: Only transfers files that have changed or are new, minimizing bandwidth usage
- Network efficiency: Can transfer data locally or over a network via SSH or rsync protocol
- Preservation of file permissions: Supports preserving file attributes such as permissions, ownership and timestamps
- Compression: Can compress data during transfer using the
-zoption to reduce bandwidth usage - Resume interrupted transfers: You can resume partial file transfers, reducing the need to start from scratch after an interruption
- SSH integration: It can transfer files securely over SSH, making it ideal for remote backups