HTTPQL Query Language
HTTPQL is a powerful query language for filtering and searching HTTP requests and responses in SpiderSuite.
Overview
HTTPQL allows you to write expressive queries to find exactly the pages you're looking for. It was originally designed by the team at Caido and has been adapted for SpiderSuite.
SpiderSuite uses a slightly modified version of HTTPQL to match the application's features and data structure. See Caido's HTTPQL documentation at https://docs.caido.io/app/reference/httpql.html.
Syntax
The constructing primitives of an HTTPQL query statement, in order of position, are the:
Namespaces
Info: Namespaces are project-specific.
| Namespace | Description |
|---|---|
req | All proxied HTTP requests. |
resp | All proxied HTTP responses. |
Fields
req
| Field | Description | Value Type |
|---|---|---|
created_at | The date and time the request was sent. | Date/Time: RFC3339 (2024-06-24T17:03:48+00:00) / ISO 8601 (2024-06-24T17:03:48+0000) / RFC2822 (Mon, 24 Jun 2024 17:03:48 +0000) / RFC7231 (Mon, 24 Jun 2024 17:03:48 GMT) / ISO9075 (2024-06-24T17:03:48Z) |
ext | The extension of the requested file. | String/Byte |
host | The value of the request's Host header. | String/Byte |
method | The HTTP method used for the request. | String/Byte |
path | The URL path (includes files). | String/Byte |
port | The port of the target server. | Integer |
query | The URL query string (excludes the leading ?). | String/Byte |
header | The request's headers (returns true if even one header matches) | String/Byte |
cookie | The request' cookies (returns true if even one cookie matches). | String/Byte |
body | The request body data. | String/Byte |
raw | The full raw data of the request (includes request line, headers, and body data). | String/Byte |
len | The request size in bytes (includes request line, headers, and body data). | Integer |
body_len | The request body's size in bytes | Integer |
resp
| Field | Description | Value Type |
|---|---|---|
code | The status code of the response. | Integer |
raw | The full raw data of the response (includes response line, headers, and body data). | String/Byte |
roundtrip | The total request/response cycle time (in milliseconds). | Integer |
header | The response's headers (returns true if even one header matches) | String/Byte |
cookie | The response' cookies (returns true if even one cookie matches). | String/Byte |
len | The response size in bytes (includes response line, headers, and body data). | Integer |
body_len | The response body's size in bytes | Integer |
Operators
| Operator | Description | Value Type | Additional Details |
|---|---|---|---|
eq | Equal to the supplied value. | String/Byte, Integer | Case sensitive. Requires a leading . character for the ext field. |
gt | Greater than the supplied value. | Date/Time, Integer | |
gte | Greater than or equal to the supplied value. | Integer | |
lt | Less than the supplied value. | Date/Time, Integer | |
lte | Less than or equal to the supplied value. | Integer | |
ne | Not equal to the supplied value. | String/Byte, Integer | Case sensitive. Requires a leading . character for the ext field. |
cont | Contains the supplied value. | String/Byte | Case insensitive. |
ncont | Does not contain the supplied value. | String/Byte | Case insensitive. |
regex | Matches to the regular expression. | String/Byte | Rust-flavored syntax. |
nregex | Does not match to the regular expression. | String/Byte | Rust-flavored syntax. |
Combining Statements
Query statements can be combined together using logical operators and logical grouping.
Logical Operators
| Operator | Description |
|---|---|
AND | Both the left and right clauses must be true. |
OR | Either the left or right clause must be true. |
Operators are case insensitive. Both AND and OR have the same default priority, but AND takes precedence over OR in evaluation order (see Logical Grouping below).
Logical Grouping
SpiderSuite supports operator precedence: AND has a higher priority than OR.
| Expression | Equivalent To |
|---|---|
<Clause1> AND <Clause2> OR <Clause3> | ((<Clause1> AND <Clause2>) OR <Clause3>) |
<Clause1> OR <Clause2> AND <Clause3> | (<Clause1> OR (<Clause2> AND <Clause3>)) |
<Clause1> AND <Clause2> AND <Clause3> | ((<Clause1> AND <Clause2>) AND <Clause3>) |
While parentheses are optional, it is recommended to use them to make your logical grouping explicit and clear.
Comments
SpiderSuite doesn't support comments in HTTPQL queries.