Working with Regular Expressions
1. Understanding Basic Regex Syntax
| Element | Description | Examples |
|---|---|---|
| Literal Characters | Match exact characters | a matches "a" |
| Metacharacters | Special characters with meaning in regex | . matches any character except newline |
| Character Classes | Match any one of a set of characters | [abc] matches "a", "b", or "c" |
| Quantifiers | Specify how many times to match | * matches zero or more times |
| Anchors | Match position in string | ^ matches start of string |
| Groups | Group parts of regex together | (abc) groups "abc" |
2. Using Character Classes
| Class | Description | Example | Matches |
|---|---|---|---|
| [abc] | Matches any one of the characters inside brackets | [abc] |
a, b, or c |
| [^abc] | Matches any character except those in brackets | [^abc] |
Any character except a, b, or c |
| [a-z] | Matches any lowercase letter from a to z | [a-z] |
Any lowercase letter |
| [A-Z] | Matches any uppercase letter from A to Z | [A-Z] |
Any uppercase letter |
| [0-9] | Matches any digit from 0 to 9 | [0-9] |
Any digit |
| [a-zA-Z0-9] | Matches any alphanumeric character | [a-zA-Z0-9] |
Any letter or digit |
| . | Matches any single character except newline | . |
Any character (except newline) |
| \d | Matches any digit (equivalent to [0-9]) | \d |
Any digit |
| \D | Matches any non-digit character | \D |
Any non-digit character |
| \w | Matches any word character (letters, digits, underscore) | \w |
Any word character |
| \W | Matches any non-word character | \W |
Any non-word character |
| \s | Matches any whitespace character (space, tab, newline) | \s |
Any whitespace character |
| \S | Matches any non-whitespace character | \S |
Any non-whitespace character |
Note: In some regex flavors,
. matches newline characters. Use [^<newline>] to match any character except newline.
3. Using Quantifiers
| Quantifier | Description | Example | Matches |
|---|---|---|---|
| a* | Zero or more of 'a' | baaa* |
'b', 'ba', 'baa', 'baaa', etc. |
| a+ | One or more of 'a' | ba+ |
'ba', 'baa', 'baaa', etc. |
| a? | Zero or one of 'a' | ba? |
'b', 'ba' |
| a{3} | Exactly 3 of 'a' | ba{3} |
'baaa' |
| a{2,4} | Between 2 and 4 of 'a' | ba{2,4} |
'baa', 'baaa', 'baaaa' |
| a{2,} | Two or more of 'a' | ba{2,} |
'baa', 'baaa', 'baaaa', etc. |
4. Using Anchors
| Anchor | Description | Example | Match |
|---|---|---|---|
| ^ | Matches the start of a line | ^Hello |
Matches "Hello" at the beginning of a line |
| $ | Matches the end of a line | world$ |
Matches "world" at the end of a line |
| \b | Matches a word boundary | \bcat\b |
Matches "cat" as a whole word, not part of "catch" |
| \B | Matches a non-word boundary | \Bcat\B |
Matches "cat" only when it's part of a larger word like "catch" |
Note: In extended regex mode (ERE), anchors work the same way. For basic regex (BRE), you need to escape
^ and $ with backslashes.
5. Using Groups and Capturing
| Feature | Description | Example |
|---|---|---|
| () | Groups parts of a regex pattern for capturing and reuse. | /(a)(b)(c)/ |
| \(n\) | Matches the nth captured group from a previous set of parentheses. | /(\w+) \1/ matches "hello hello" |
| Non-capturing group | Use (?:...) to group without capturing. |
/(\w+)(?:\s+)(\w+)/ captures only first and last word |
Example: Extracting Date Components
date_string="2024-12-25"
echo "$date_string" | grep -oE "([0-9]{4})-([0-9]{2})-([0-9]{2})" | sed -E "s/([0-9]{4})-([0-9]{2})-([0-9]{2})/\1 year, \2 month, \3 day/"
Example: Reformatting Phone Numbers
phone="(555) 123-4567"
echo "$phone" | sed -E "s/\(([^)]+)\) ([0-9]+)-([0-9]+)/\1:\2:\3/"
Note: Group numbers start at 1, not 0. Groups can be nested; inner groups are numbered based on their opening parenthesis order.
6. Using Extended Regex
| Feature | Description | Example |
|---|---|---|
| Extended Regular Expressions (ERE) | Enhanced regex syntax with more powerful features than basic regex | grep -E '^(root|daemon):' /etc/passwd |
| Alternative (|) | Matches either the expression before or after the pipe | echo "cat" | grep -E 'cat|dog' |
| Grouping (()) | Groups parts of the regex for applying quantifiers or alternatives | echo "ab" | grep -E '(a|b)+c' |
| Quantifiers (+, *, ?) | Specify how many times a pattern should occur | echo "aaabbb" | grep -E 'a{3}b{3}' |
| Character Classes | Match any character from a set | echo "abc" | grep -E '[a-z]' |
| Anchors (^, $) | Match start or end of line | echo "hello world" | grep -E '^hello' |
Note: Extended regex is supported by
grep -E, egrep, and sed.
7. Using Perl-Compatible Regex
| Feature | Description | Usage |
|---|---|---|
| PCRE | Perl-Compatible Regular Expressions | grep -P 'pattern' file |
| Extended Syntax | Supports advanced regex features like lookaheads and lookbehinds | grep -P '(?<=word)pattern' file |
| Case Insensitive | Match patterns ignoring case | grep -Pi 'pattern' file |
Example: Matching with PCRE
grep -P '^\d{4}-\d{2}-\d{2}
8. Using Lookahead and Lookbehind
| Pattern | Description | Example | Match |
|---|---|---|---|
| (?=pattern) | Positive lookahead - asserts that the pattern matches ahead | /(?=.*[A-Z])(?=.*[0-9])[a-zA-Z0-9]{8,}/ |
String with at least one uppercase letter and digit |
| (?!pattern) | Negative lookahead - asserts that the pattern does NOT match ahead | /^(?!.*[aeiou])[a-zA-Z]+$/ |
String with no vowels |
| (?<=pattern) | Positive lookbehind - asserts that the pattern matches behind | /(?<=USD|EUR)\d+\.?\d*/ |
Amounts preceded by currency codes |
| (?<!pattern) | Negative lookbehind - asserts that the pattern does NOT match behind | /(?<!http|https):\/\/\S+/ |
URLs not preceded by protocol |
Example: Positive Lookahead for Password Validation
const passwordRegex = /(?=.*[A-Z])(?=.*[a-z])(?=.*\d)(?=.*[@$!%*?&])[A-Za-z\d@$!%*?&]{8,}/;
const testPassword = "MySecure123!";
console.log(passwordRegex.test(testPassword)); // trueExample: Positive Lookbehind for Currency Parsing
const priceRegex = /(?<=USD|EUR)\s*(\d+\.?\d*)/g;
const text = "The price is USD 100 and EUR 50";
const matches = text.match(priceRegex);
console.log(matches); // [" 100", " 50"]
Note: Lookahead and lookbehind are zero-width assertions. They do not consume characters in the string but only assert conditions.
9. Using Non-Capturing Groups
| Pattern | Description | Example | Result |
|---|---|---|---|
| (?:pattern) | Defines a non-capturing group | cat (?:and|or) dog |
Matches "cat and dog" or "cat or dog" |
| (?<name>pattern) | Named capturing group (not non-capturing) | (?<animal>cat|dog) and (?<animal>bird) |
Groups named "animal" with values "cat" and "bird" |
Example: Using Non-Capturing Groups in Replace Operations
echo "The price is $100 and tax is $10" | sed -E "s/\$([0-9]+)(?: and tax is \$[0-9]+)?/\1/g"
Note: Non-capturing groups are useful for grouping without creating a backreference, which can improve performance and simplify regex logic.
10. Testing Regex Patterns
| Command | Description | Example |
|---|---|---|
| grep | Searches for patterns in files or input | grep 'pattern' file.txt |
| egrep | Extended grep with regex support | egrep '^\w+' file.txt |
| sed | Stream editor for pattern substitution | sed 's/pattern/replacement/g' file.txt |
| awk | Pattern scanning and processing language | awk '/pattern/ {print $0}' file.txt |
8. Using Lookahead and Lookbehind
| Pattern | Description | Example | Match |
|---|---|---|---|
| (?=pattern) | Positive lookahead - asserts that the pattern matches ahead | /(?=.*[A-Z])(?=.*[0-9])[a-zA-Z0-9]{8,}/ |
String with at least one uppercase letter and digit |
| (?!pattern) | Negative lookahead - asserts that the pattern does NOT match ahead | /^(?!.*[aeiou])[a-zA-Z]+$/ |
String with no vowels |
| (?<=pattern) | Positive lookbehind - asserts that the pattern matches behind | /(?<=USD|EUR)\d+\.?\d*/ |
Amounts preceded by currency codes |
| (?<!pattern) | Negative lookbehind - asserts that the pattern does NOT match behind | /(?<!http|https):\/\/\S+/ |
URLs not preceded by protocol |
Example: Positive Lookahead for Password Validation
const passwordRegex = /(?=.*[A-Z])(?=.*[a-z])(?=.*\d)(?=.*[@$!%*?&])[A-Za-z\d@$!%*?&]{8,}/;
const testPassword = "MySecure123!";
console.log(passwordRegex.test(testPassword)); // trueExample: Positive Lookbehind for Currency Parsing
const priceRegex = /(?<=USD|EUR)\s*(\d+\.?\d*)/g;
const text = "The price is USD 100 and EUR 50";
const matches = text.match(priceRegex);
console.log(matches); // [" 100", " 50"]
Note: Lookahead and lookbehind are zero-width assertions. They do not consume characters in the string but only assert conditions.
9. Using Non-Capturing Groups
| Pattern | Description | Example | Result |
|---|---|---|---|
| (?:pattern) | Defines a non-capturing group | cat (?:and|or) dog |
Matches "cat and dog" or "cat or dog" |
| (?<name>pattern) | Named capturing group (not non-capturing) | (?<animal>cat|dog) and (?<animal>bird) |
Groups named "animal" with values "cat" and "bird" |
Example: Using Non-Capturing Groups in Replace Operations
echo "The price is $100 and tax is $10" | sed -E "s/\$([0-9]+)(?: and tax is \$[0-9]+)?/\1/g"
Note: Non-capturing groups are useful for grouping without creating a backreference, which can improve performance and simplify regex logic.
10. Testing Regex Patterns
| Command | Description | Example |
|---|---|---|
| grep | Searches for patterns in files or input | grep 'pattern' file.txt |
| egrep | Extended grep with regex support | egrep '^\w+' file.txt |
| sed | Stream editor for pattern substitution | sed 's/pattern/replacement/g' file.txt |
| awk | Pattern scanning and processing language | awk '/pattern/ {print $0}' file.txt |
Note: The
-E flag enables extended regular expressions in grep.
Warning: Be cautious with
emails.txt
sed -i as it modifies files in place.