Using Pipes and Command Chaining

1. Piping Commands

Command	Description	Example
command1 \| command2	Pipes output of command1 to input of command2	`ls -l \| grep .txt`
command1 \| command2 \| command3	Chain multiple pipes for complex operations	`ps aux \| grep node \| wc -l`
cat file.txt \| command	Use pipe to feed file content to a command	`cat data.log \| grep "ERROR"`

Example: Filtering Log Files

tail -f /var/log/syslog | grep "error"

Note: Pipes are unidirectional and process data sequentially. The first command's output becomes the second command's input.

2. Using tee for Splitting Output

Option	Values	Description
tee	`tee [file]`	Reads from standard input and writes to both standard output and file(s)

Example: Splitting Output to File and Terminal

ls -l | tee output.txt

Example: Appending to a File with tee

echo "New line" | tee -a log.txt

Note: The tee command is useful for debugging and logging, allowing you to see output while also saving it to a file.

3. Chaining with AND Operator

Operator	Description	Usage	Example
&&	Executes the next command only if the previous one succeeds (exit status 0)	`command1 && command2`	`ls /home && echo "Directory exists"`
;	Executes commands sequentially regardless of success or failure	`command1 ; command2`	`ls /home ; echo "Command executed"`

Example: Using && to conditionally execute commands

mkdir /tmp/testdir && cd /tmp/testdir && touch file.txt

Note: The && operator is useful for chaining commands where each step depends on the success of the previous one.

4. Chaining with OR Operator

Operator	Description	Usage	Example
\|\|	Executes the next command only if the previous one fails (exit status is non-zero)	`command1 \|\| command2`	`ls /nonexistent_directory \|\| echo "Directory not found"`

Example: Using || to provide a fallback

grep "pattern" file.txt || echo "Pattern not found in file"

Note: The || operator is useful for error handling and providing fallback actions in shell scripts.

5. Running Commands Sequentially

Command	Description	Example
;	Runs commands sequentially, ignoring errors	`ls -l; pwd`
&&	Runs next command only if previous succeeded (exit code 0)	`mkdir testdir && cd testdir`
\|\|	Runs next command only if previous failed (non-zero exit code)	`ls nonexistent \|\| echo "File not found"`

Example: Chaining with && and ||

mkdir backup && cp file.txt backup/ || echo "Backup failed"

Note: The && and || operators are useful for conditional execution based on command success or failure.

6. Using Command Substitution

Method	Description	Example
Backticks `` `command` ``	Older syntax for command substitution	ls -l `pwd`
$(command)	Preferred modern syntax for command substitution	`echo $(date)`

Example: Using command substitution to get current directory size

du -sh $(pwd)

Example: Combining command substitution with other commands

echo "Today is $(date +%A)"

Note: Command substitution allows embedding the output of one command directly into another command or script.

7. Process Substitution

Process substitution allows you to treat the output of a command as a file. It's particularly useful for comparing files or passing command output to tools that expect file inputs.

Feature	Description	Example
Syntax	Using `<(command)` to substitute command output as a file	`diff <(ls /etc) <(ls /tmp)`
Use Case	Comparing command outputs without creating temporary files	`diff <(sort file1.txt) <(sort file2.txt)`
File Path	Generated by shell, typically under `/dev/fd/`	`echo <(ls /etc) \| cat`

Example: Comparing Directory Contents

diff <(ls -R /etc) <(ls -R /tmp)

Example: Using with `sort`

sort <(cat file1.txt) <(cat file2.txt)

Note: Process substitution is supported in Bash and Zsh but not in POSIX sh.

8. Using xargs for Command Building

Option	Values	Description
xargs	`command`	Reads items from standard input and executes command with those items as arguments
-I	`replacement_string`	Replace occurrences of replacement_string in command with input data
-n	`number`	Limit number of arguments passed to command per invocation
-0	`flag`	Treat input as null-separated (instead of newline-separated)

Example: Using xargs to process files

find . -name "*.txt" | xargs wc -l

Example: Using xargs with -I to rename files

echo "old_file.txt" | xargs -I {} mv {} new_{}.txt

Example: Using xargs with -0 for null-separated input

find . -name "*.log" -print0 | xargs -0 rm

Note: xargs is particularly useful when working with commands that have limits on the number of arguments they can accept. It helps in breaking down large inputs into manageable chunks.

9. Parallel Command Execution

Parallel command execution allows running multiple commands simultaneously, which can significantly improve performance for CPU-intensive tasks.

Command	Description	Example
command1 && command2	Runs command2 only if command1 succeeds (exit status 0)	`ls /etc && echo "Directory exists"`
command1 \|\| command2	Runs command2 only if command1 fails (non-zero exit status)	`ls /nonexistent \|\| echo "File not found"`
command1 & command2 &	Runs both commands in background concurrently	`sleep 5 & echo "Started sleep"`
{ command1; command2; } &	Groups commands and runs them in background	`{ echo "Start"; sleep 2; echo "End"; } &`

Example: Running Multiple Commands in Parallel

echo "Starting tasks..." &
sleep 3 &
date &
wait
echo "All tasks completed."

Note: The wait command ensures that all background processes finish before continuing.

Key Points

Use & to run commands in the background
Use wait to synchronize completion of background jobs
Combine commands with && or || for conditional execution

10. Grouping Commands

Grouping commands allows you to combine multiple commands into a single unit, controlling their execution flow and scope.

Command	Description	Example
()	Groups commands to be executed in a subshell	`(ls -l; pwd)`
{ }	Groups commands to be executed in the current shell	`{ echo "Hello"; echo "World"; }`

Example: Using braces to group commands

{ date; whoami; }

Note: Braces do not create a new subshell, so variables set inside the group will persist in the current shell.