> ## Documentation Index
> Fetch the complete documentation index at: https://docs.ahmadraza.in/llms.txt
> Use this file to discover all available pages before exploring further.

# RegexPattern

# 📘 The DevOps Regex Handbook

**Subtitle:** *Turning Unstructured Text into Structured Data*

## 1. Introduction: Why do we need this?

In DevOps, data is rarely clean. It comes as streams of text: log lines, CLI outputs, or messy config files.

* **Without Regex:** A log is just a long string: `"Error: 500 occurred at 10:00"`
* **With Regex:** It becomes data: `{"level": "Error", "code": 500, "time": "10:00"}`

Regex is the tool that allows you to define a **pattern** to extract this data.

***

## 2. The Basics: Matching Characters

Before we build complex logic, we must learn the alphabet of Regex.

### 2.1 Literals vs. Metacharacters

Most characters are **Literals**. They match themselves.

* Regex: `firefox`
* Matches: "firefox"

Some characters have special powers. These are **Metacharacters**.

* `^ $ . * + ? { } [ ] \ | ( )`

### 2.2 The Wildcard (Dot)

The dot `.` matches **any single character** (except a new line).

* Regex: `user.id`
* Matches: "user\_id", "user.id", "user9id"

### 2.3 Character Classes (The "Types" of text)

Instead of matching a specific letter, you often want to match "any number" or "any letter".

| Shorthand        | Meaning                                   | DevOps Use Case                           |
| ---------------- | ----------------------------------------- | ----------------------------------------- |
| `\d`             | Any **D**igit (0-9)                       | HTTP Status codes, PIDs, Ports            |
| `\w`             | Any **W**ord char (A-Z, a-z, 0-9, \_)     | Usernames, container IDs, log levels      |
| `\s`             | Any **W**hitespace (space, tab)           | Separators between log fields             |
| `\D`, `\W`, `\S` | The **Opposite** (Not digit, Not word...) | finding everything *except* the timestamp |

**Example:**

* Regex: `\w\w\w` matches "App", "Log", "Err"
* Regex: `\d\d` matches "00", "99"

### 2.4 Anchors (Location, Location, Location)

Anchors don't match text; they match a **position**.

| Symbol | Meaning       | Example                                        |
| ------ | ------------- | ---------------------------------------------- |
| `^`    | Start of line | `^Error` (Matches lines *starting* with Error) |
| `$`    | End of line   | `.log$` (Matches lines *ending* in .log)       |

***

## 3. The Logic: Quantifiers & Groups

Now we make it dynamic.

### 3.1 Quantifiers (How many?)

These symbols apply to the character *immediately before* them.

* `*` = **0 or more** (Maybe it's there, maybe it repeats).
* `+` = **1 or more** (Must be there at least once).
* `?` = **0 or 1** (Optional).
* `{3}` = Exactly 3 times.
* `{3,5}` = Between 3 and 5 times.

**DevOps Example:**

* Matching an Error Code (3 digits): `\d{3}`
* Matching a URL (http or https): `https?` (The 's' is optional)

### 3.2 Sets `[ ]`

Match **one** character from a specific list.

* `[abc]` = Match "a" OR "b" OR "c".
* `[0-9]` = Same as `\d`.
* `[A-Z]` = Any uppercase letter.

### 3.3 Negated Sets `[^ ]` (The Secret Weapon)

This is crucial for logs. `[^ ]` means "Match anything that is **NOT** in this list".

* `[^ ]` = Match anything that isn't a space.
* `[^"]` = Match anything that isn't a quote.

**Why is this powerful?**
If you want to match a URL inside quotes `"http://google.com"`, don't try to guess the characters in the URL. Just say: **"Match everything until you hit a quote."**
Regex: `"[^"]+"`

***

## 4. Advanced: Capture Groups

This is how Fluent Bit, Vector, and Promtail work. They don't just "find" text; they **extract** it into variables.

### 4.1 Basic Grouping `( )`

Groups logic together.

* Regex: `(Error|Warning)` matches "Error" OR "Warning".

### 4.2 Named Capture Groups `(?<name>...)`

This is the standard for modern logging tools. It assigns a name to the match.

**Syntax:** `(?<field_name>PATTERN)`

**Example:**
Log: `User: admin`
Regex: `User: (?<username>\w+)`
Result: `username = "admin"`

***

## 5. Real-World Case Studies (The Labs)

Here we will dissect the three most common scenarios you will face.

### Lab 1: The Standard Nginx Access Log

**The Goal:** Parse a standard web server log into JSON.

**Data:**

```text theme={null}
192.168.1.5 - - [10/Oct/2023:13:55:36] "GET /app/main.js HTTP/1.1" 200 512

```

**The Regex:**

```regex theme={null}
^(?<remote_ip>[^ ]*) - - \[(?<timestamp>[^\]]*)\] "(?<method>\w+) (?<url>\S+) (?<protocol>[^\"]*)" (?<status>\d{3}) (?<bytes>\d+)

```

**Detailed Breakdown:**

1. `^` : Start of line.
2. `(?<remote_ip>[^ ]*)` : Capture everything that **isn't a space**. (Grabs `192.168.1.5`).
3. `- -` : Match the junk separators (space dash space dash space).
4. `\[` : Match literal opening bracket.
5. `(?<timestamp>[^\]]*)` : Capture everything that **isn't a closing bracket**. (Grabs the date).
6. `\] "` : Match closing bracket, space, quote.
7. `(?<method>\w+)` : Capture word characters (Grabs `GET`).
8. `     ` : Space.
9. `(?<url>\S+)` : Capture everything that **isn't whitespace**. (Grabs `/app/main.js`).
10. `     ` : Space.
11. `(?<protocol>[^\"]*)` : Capture everything that **isn't a quote**. (Grabs `HTTP/1.1`).
12. `"` : Closing quote.
13. ` (?<status>\d{3})` : Space, then capture exactly 3 digits. (Grabs `200`).

### Lab 2: Python Application Log (Custom Format)

**The Goal:** Extract log level and service name.

**Data:**

```text theme={null}
2024-01-20 14:00:01 [auth-service] ERROR: Connection failed

```

**The Regex:**

```regex theme={null}
^\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2} \[(?<service>[\w-]+)\] (?<level>[A-Z]+): (?<message>.*)

```

**Key Concepts Used:**

* `[\w-]+` : Matches words that might contain dashes (like `auth-service`).
* `[A-Z]+` : Ensures we only capture uppercase log levels (`ERROR`, `INFO`).
* `.*` : The "Greedy match". Matches everything until the end of the line for the message.

### Lab 3: Filtering Docker Images (Terraform/CI)

**The Goal:** Validate that a docker image tag is a valid Semantic Version (e.g., `v1.0.2`).

**Data:**
`v1.2.5`

**The Regex:**

```regex theme={null}
^v\d+\.\d+\.\d+$

```

**Breakdown:**

* `^v` : Must start with 'v'.
* `\d+` : One or more digits (Major version).
* `\.` : Literal dot (You **must** escape the dot with `\`, otherwise it means "any char").
* `\d+` : Minor version.
* `\.` : Literal dot.
* `\d+$` : Patch version, anchored to the end of the line.

***

## 6. Cheatsheet & Best Practices

| Character | Name      | Meaning                                      |
| --------- | --------- | -------------------------------------------- |
| `.`       | Dot       | Any character (except newline)               |
| `*`       | Asterisk  | 0 or more times                              |
| `+`       | Plus      | 1 or more times                              |
| `?`       | Question  | Optional (0 or 1)                            |
| `^`       | Caret     | Start of line                                |
| `$`       | Dollar    | End of line                                  |
| `\`       | Backslash | Escape character (Turn `.` into literal dot) |
| `[]`      | Brackets  | List of allowed chars                        |
| `[^]`     | Negated   | List of FORBIDDEN chars                      |

### 🛑 DevOps Safety Warnings

1. **Escape your Dots:** If you want to match an IP `1.1.1.1`, write `1\.1\.1\.1`. If you write `1.1.1.1`, it will match `1a1b1c1` because `.` is a wildcard.
2. **Avoid `.*` in the middle:** `.*` is "greedy". It eats as much as it can.

* Bad: `Tag: (.*) suffix`
* Good: `Tag: (\w+) suffix`

3. **Test Before Deploying:** Regex errors in Fluent Bit can crash your logging pipeline.

***

## 7. How to Practice (Your Homework)

You cannot learn Regex by reading. You must do.

1. Go to **[Regex101.com](https://regex101.com/)**.
2. Set the "Flavor" to **Golang** (Fluent Bit uses a Go-like regex engine usually) or **PCRE** (Standard).
3. Paste the **Nginx Data** from Lab 1 into the "Test String" box.
4. Try to write the pattern from memory.
5. Look at the "Explanation" panel on the right—it tells you exactly what matches what.