Skip to main content

Transforms & Functions

Are an array of functions which are used to transform your data after it is received. To use transform functions you must declare a post_transforms block in your specified action

name: json-tests
metrics_enabled: true
docs: true

interfaces:
tests/add_attribute:
output: http
method: GET

actions:
- name: AddAttribute
http:
url: https://jsonplaceholder.typicode.com/todos/10
post_transforms:
- extract_value: ".body"
- add_attribute:
env: "production"

Add Attribute

To add an attribute to your action you can use the add_attribute function. This will add an extra attribute to your data. You can use data from your input or add your own information for example an environment attribute.

Usage

post_transforms:
- add_attribute:
env: "Production"
pageId: a|body::id|
Add Attribute Example

Adding an exists attribute to a succesful user check

name: user_api
version: 0.0.1
description: Retrieve User Details
metrics_enabled: true

global:
databases:
main:
driver: postgres
conn_string: |
postgresql://a|env::POSTGRES_USER|:a|env::POSTGRES_PASS|@a|env::POSTGRES_HOST|?connect_timeout=10"

interfaces:
user:
output: http
route: details
method: POST

actions:
- name: CheckBody
input: a|body
run_when_succeeded:
- previous
assert:
tests:
- value: firstname
is_not_null: true

- name: CheckUser
run_when_succeeded:
- previous
database: main
query: |
SELECT firstname, lastname, address1 FROM customers WHERE firstname LIKE $1 LIMIT 2;
params:
- a|CheckBody::firstname|
assert:
error_message: "User Does Not Exist"
tests:
- value: count()
is_equal_to: 1
error_message: "No User Found"
post_transforms:
- add_attribute:
exists: true

This will result in the below output when the API is called.

{
"data": {
"CheckBody": {
"data": {
"firstname": "VKUUXF"
},
"message": "success",
"time.ms": 0
},
"CheckUser": {
"data": [
{
"address1": "4608499546 Dell Way",
"exists": "true",
"firstname": "VKUUXF",
"lastname": "ITHOMQJNYX"
}
],
"message": "success",
"time.ms": 670
}
},
"metrics": {
"user_api.config.ms": 670
}
}

Add JWT

To add a JWT token to your action you require the below fields.

Options

NameTypeDefaultDescription
keystringthe key or attribute name in the payload i.e jwt
secretstringthe secret to use to sign the JWT
expstring*expiry is set with friendly human time formats
  • Parses durations in free form like 15days 2min 2s
  • Parses and formats timestamp in rfc3339 format: 2018-01-01T12:53:00Z
  • Parses timestamps in a weaker format: 2018-01-01 12:53:00
dataarrayput the variables to be included in the token

Usage

post_transforms:
- add_jwt:
key: jwt
secret: a|var::JWT_SECRET|
exp: 1d
data: [email]
Add JWT Example

Add a JWT to a succesful user login.


name: login_user # unique name within your organization
version: 0.0.1
metrics_enabled: true # enables aggregate metrics

global:
variables:
JWT_SECRET: a|env::JWT_SECRET|

databases:
main:
driver: postgres
conn_string: |
postgresql://a|env::POSTGRES_USER|:a|env::POSTGRES_PASS|@a|env::POSTGRES_HOST|?connect_timeout=10"

interfaces:
user/login:
output: http
method: POST

response:
http_code_on_error: 403

actions:
- name: LoginBody
input: a|body
hide_data_on_success: true
hide_data_on_error: true
assert:
tests:
- value: email
is_not_null: true
- value: pass
is_not_null: true

- name: GetUserDetails
run_when_succeeded:
- LoginBody
database: main
query: SELECT email, password, verified FROM users WHERE email = $1;
params: [a|LoginBody::email|]
assert:
error_message: "Authentication failed"
tests:
- value: count()
is_equal_to: 1
- value: "[0]verified"
is_equal_to: true
error_message: user not verified
post_transforms:
- return_row: 0

- name: VerifyPassword
input: a|LoginBody
hide_data_on_error: true
run_when_succeeded:
- GetUserDetails
assert:
error_message: "Authentication failed"
tests:
- value: pass
bcrypt_verify: a|GetUserDetails::password|
post_transforms:
- remove_keys:
- pass
- add_jwt:
key: jwt
secret: a|var::JWT_SECRET|
exp: 1d
data: [email]

This will result in a JWT being added to the api response.

{
"data": {
"VerifyPassword": {
"data": {
"jwt": "eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJleHAiOjE3MTI0MTAwNDMsImlzcyI6IkFpciBQaXBlIiwiZGF0YSI6e319.I4NDj8XrLU0Ojk0HIxkZCMpClFgtDUoj4sIpqiY23uw",
......
},
"time.ms": 302
}
},
"metrics": {
"jwt.config.ms": 302
}
}

Bcrypt

To easily hash and verify passwords Air Pipe implements the Bcrypt Crate. Requires the below fields.

Options

NameTypeDefaultDescription
keystringthe key or attribute name in the payload i.e password
valuestringthe password to hash
costintincrease the amount of time cpu/memory needs to compute the has

Usage

post_transforms:
- bcrypt:
key: password_hash
value: password
cost: 12
Bcrypt Example

Hash a users password during registration.


# unique name within your organization
name: register_user
version: 0.0.2
description: Register a user
metrics_enabled: true

global:
variables:
company_name: a|env::COMPANY_NAME|
app_url: a|env::APP_URL|

templates:
RegisterEmail: |
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>a|var::company_name| Email Registration Verification</title>
</head>
<body>
<div style="background-color: #f9f9f9; padding: 20px;">
<div style="max-width: 600px; margin: 0 auto; background-color: #ffffff; padding: 20px; border-radius: 10px; box-shadow: 0px 0px 10px rgba(0, 0, 0, 0.1);">
<h2 style="text-align: center;">Email Registration Verification with a|var::company_name|</h2>
<p>Hi $1,</p>
<p>Glad to see you are joining the a|var::company_name|! To complete your account registration, please click the button
button below to verify your email address.</p>
<p style="text-align: center;">
<a href="a|var::app_url|/verify?token=$2" style="display: inline-block; background-color: #007bff; color: #ffffff; text-decoration: none; padding: 10px 20px; border-radius: 5px;">Verify Email</a>
</p>
<p>If the button above doesn't work, you can also copy and paste the following link into your browser's address
bar:</p>
<p style="text-align: center;">a|var::app_url|/verify?token=$3</p>
<p>If you didn't register a a|var::company_name| account, please ignore this email.</p>
<p>Best regards,</p>
<p>The a|var::company_name| Team</p>
</div>
</div>
</body>
</html>

databases:
main:
driver: postgres
conn_string: |
postgresql://a|env::POSTGRES_USER|:a|env::POSTGRES_PASS|@a|env::POSTGRES_HOST|?connect_timeout=10"

interfaces:
user/register:
output: http
method: POST
show_error_detail: true
actions:
- name: UserShouldNotExist
database: main
query: SELECT email FROM users WHERE email = $1;
params: [a|body::email|]
timeout: 2000
data_hide_on_error: true
run_when_succeeded:
- previous
assert:
error_message: "Registration failed"
tests:
- value: count()
is_equal_to: 0
error_message: "User already exists"

- name: Input
input: a|body
hide_data_on_empty: true
run_when_succeeded:
- UserShouldNotExist
assert:
error_message: "Password not strong enough"
tests:
- value: first
is_not_null: true
is_greater_than: 2
is_less_than: 20
regex: \S+
- value: last
is_not_null: true
is_greater_than: 3
is_less_than: 20
regex: \S+
- value: email
is_not_null: true
is_greater_than: 3
is_less_than: 100
regex: \S+
- value: pass
is_not_null: true
is_greater_than: 9
is_less_than: 30
- value: pass
regex: "[A-Z]"
error_message: "Password should contain one uppercase character"
- value: pass
regex: "[a-z]"
error_message: "Password should contain one lowercase character"
- value: pass
regex: "[0-9!@#]"
error_message: "Password should contain at least one number (0-9) or symbol (!,@,#)"
post_transforms:
- add_attribute:
user_uuid: a|uuid|
verification_token: a|uuid|
- bcrypt:
value: pass
cost: 12

- name: InsertUser
data_hide:
on_empty: true
database: main
query: INSERT INTO users(uuid, email, first, last, verification_token, password) VALUES($1::uuid, $2, $3, $4, $5::uuid, $6);
params:
- a|Input::user_uuid|
- a|Input::email|
- a|Input::first|
- a|Input::last|
- a|Input::verification_token|
- a|Input::pass|
run_when_succeeded:
- previous
assert:
success_message: "User successfully added"
error_message: "Failed to register user"
tests:
- value: count()
is_equal_to: 0

- name: SendEmail
data_metrics:
on_error: true
data_hide:
on_empty: true
email:
to: a|Input::first| <a|Input::email|>
from: a|env::COMPANY_NAME| <admin@omiconvo.com>
html: |
a|template::RegisterEmail|
params:
- a|Input::first|
- a|Input::verification_token|
- a|Input::verification_token|
subject: Welcome to a|env::COMPANY_NAME|
success_message: "Email successfully sent"
error_message: "Failed to send email"
smtp:
user: a|env::SMTP_USER|
pass: a|env::SMTP_PASS|
server: a|env::SMTP_SERVER|
port: 465
run_when_succeeded:
- previous

Extract Value

This function is used to extract values from your input.
extract_value uses javascript like notation:

  • body
  • [0].name
  • body.some.value
  • data[2].value
  • data.value[1]

Usage

post_transforms:
- extract_value: "body"
Extract Value Example

Extract the body from an API Response

name: http-tests
metrics_enabled: true
docs: true

interfaces:
tests/extracts:
output: http
method: GET

actions:
- name: ExtractData
http:
url: https://jsonplaceholder.typicode.com/todos/10
post_transforms:
- extract_value: body

This will result in the below output when the API is called.

{
"data": {
"ExtractData": {
"data": {
"completed": true,
"id": 10,
"title": "illo est ratione doloremque quia maiores aut",
"userId": 1
},
"time.ms": 307
}
},
"metrics": {
"http-tests.config.ms": 307
}
}

Generate Password

This function generates a password to use temporarily. For example inviting users or resetting passwords.

Options

NameTypeDefaultDescription
keystringthe key or attribute name in the payload i.e password
lengthintpassword length
numbersbooleantrue
lowercase_lettersbooleantrue
uppercase_lettersbooleantrue
symbolsbooleantrue
spacesbooleanfalse
exclude_similar_charactersbooleanfalse
strictbooleantrue

Usage

The generate_password function only targets objects.

post_transforms:
- generate_password:
key: password
length: 10
Generate Password Example

Check if a user exists before generating a password.
Note that we extract the object out of the array before generating a password.

# unique name within your organization
name: user_api
version: 0.0.1
description: Retrieve User Details
metrics_enabled: true

global:
databases:
main:
driver: postgres
conn_string: |
postgresql://a|env::POSTGRES_USER|:a|env::POSTGRES_PASS|@a|env::POSTGRES_HOST|?connect_timeout=10"

interfaces:
user/reset-password:
output: http
route: details
method: POST

actions:
- name: CheckBody
input: a|body
run_when_succeeded:
- previous
assert:
tests:
- value: firstname
is_not_null: true

- name: ResetUser
run_when_succeeded:
- previous
database: main
query: |
SELECT firstname, lastname, address1 FROM customers WHERE firstname LIKE $1 LIMIT 2;
params:
- a|CheckBody::firstname|
assert:
error_message: "User Does Not Exist"
tests:
- value: count()
is_equal_to: 1
error_message: "No User Found"
post_transforms:
- extract_value: "[0]"
- generate_password:
key: password
length: 10

This will result in the below output when the API is called.

{
"data": {
"CheckBody": {
"data": {
"firstname": "VKUUXF"
},
"message": "success",
"time.ms": 0
},
"ResetUser": {
"data": {
"address1": "4608499546 Dell Way",
"firstname": "VKUUXF",
"lastname": "ITHOMQJNYX",
"password": "8/3];*P2sy"
},
"message": "success",
"time.ms": 673
}
},
"metrics": {
"user_api.config.ms": 674
}
}

Group By

This function will group your data by a particular field(s)

Usage

post_transforms:
- group_by:
- customerid

OR

post_transforms:
- group_by: [customerid, orderid]
Group By Example

Group data by specified field(s)

name: user_api
version: 0.0.1
description: Retrieve User Details
metrics_enabled: true

global:
databases:
main:
driver: postgres
conn_string: |
postgresql://a|env::POSTGRES_USER|:a|env::POSTGRES_PASS|@a|env::POSTGRES_HOST|?connect_timeout=10"

interfaces:
group/orders:
output: http
method: GET
actions:
- name: GroupData
database: main
query: |
SELECT * FROM orders ORDER BY customerid LIMIT 20;
post_transforms:
- group_by:
- customerid

This will result in the below output when the API is called.

{
"data": {
"GroupData": {
"data": {
"12": [
{
"customerid": 12,
"netamount": "350.8700",
"orderdate": "2004-04-20",
"orderid": 3710,
"tax": "28.9500",
"totalamount": "379.8200"
}
],
"13": [
{
"customerid": 13,
"netamount": "227.4500",
"orderdate": "2004-01-14",
"orderid": 379,
"tax": "18.7600",
"totalamount": "246.2100"
},
{
"customerid": 13,
"netamount": "83.3100",
"orderdate": "2004-10-10",
"orderid": 9447,
"tax": "6.8700",
"totalamount": "90.1800"
}
]
},
"time.ms": 307
}
},
"metrics": {
"http-tests.config.ms": 307
}
}

Numerics

This function is used to convert strings to numerics.

Usage

post_transforms:
- numerics: [percentage]
Numerics Example

Consider data that is returning numbers as strings. For example a payload containing {"percentage":"50%"}

name: http-tests
metrics_enabled: true
docs: true

interfaces:
string-numeric:
output: http
route: details
method: POST

actions:
- name: ConvertPercentage
input: a|body
run_when_succeeded:
- previous
assert:
tests:
- value: percentage
is_not_null: true
post_transforms:
- numerics: [percentage]

This would output

{
"data": {
"ConvertPercentage": {
"data": {
"percentage": 50
},
"message": "success",
"time.ms": 0
}
},
"metrics": {
"http-tests.config.ms": 0
}
}

Remove Attributes

Remove attributes allows you to remove attributes in an object, or multiple attributes in an array of objects.

This leverages JSONPath for selection and intends to adhered to the RFC 9535 standard.

For examples see:

Usage

# remove username attribute in a returned object
post_transforms:
- rename_attributes:
- $.username
# wildcard match to remove username attribute in an array of objects
post_transforms:
- remove_attributes:
- $[*].username

Single object example | remove

Single Object Example

Input

  {
"id": 1,
"name": "Leanne Graham",
"username": "Bret",
"email": "Sincere@april.biz"
}
Rename Attributes Object
loading...

Result

  {
"id": 1,
"name": "Leanne Graham",
"email": "Sincere@april.biz"
}

Array of objects example | remove

Array of objects example

Input

[
{
"id": 1,
"name": "Leanne Graham",
"username": "Bret",
"email": "Sincere@april.biz"
},
{
"id": 2,
"name": "Ervin Howell",
"username": "Antonette",
"email": "Shanna@melissa.tv"
}
]
Remove Attributes Array
loading...

Result

[
{
"id": 1,
"fullName": "Leanne Graham",
"email": "Sincere@april.biz"
},
{
"id": 2,
"fullName": "Ervin Howell",
"email": "Shanna@melissa.tv"
}
]

Rename Attributes

Rename attributes allows you to rename attributes in an object, or multiple attributes in an array of objects.

This leverages JSONPath for selection and intends to adhered to the RFC 9535 standard.

For examples see:

Usage

# rename firstname attribute in a returned object
post_transforms:
- rename_attributes:
$.firstname: first
# wildcard match to rename username attribute in an array of objects
post_transforms:
- rename_attributes:
$[*].username: USER

Single object example | rename

Single Object Example

Input

  {
"id": 1,
"name": "Leanne Graham",
"username": "Bret",
"email": "Sincere@april.biz"
}
Rename Attributes Object
loading...

Result

  {
"id": 1,
"name": "Leanne Graham",
"USER": "Bret",
"email": "Sincere@april.biz"
}

Array of objects example | rename

Array of objects example

Input

[
{
"id": 1,
"name": "Leanne Graham",
"username": "Bret",
"email": "Sincere@april.biz"
},
{
"id": 2,
"name": "Ervin Howell",
"username": "Antonette",
"email": "Shanna@melissa.tv"
}
]
Rename Attributes Array
loading...

Result

[
{
"id": 1,
"fullName": "Leanne Graham",
"username": "Bret",
"email": "Sincere@april.biz"
},
{
"id": 2,
"fullName": "Ervin Howell",
"username": "Antonette",
"email": "Shanna@melissa.tv"
}
]

Nested Transforms

Nested transforms allow you to perform further transformation functions at any level of your data structure using JSONPath.

This leverages JSONPath for selection and intends to adhered to the RFC 9535 standard.

For examples see:

Usage

post_transforms:
- rename_attributes:
$[*].username: USER
- nested_transforms:
$[*].address:
- remove_attributes:
- $.geo
- $.suite
Remove unwanted attributes from array of objects

Input

[
{
"id": 1,
"name": "Leanne Graham",
"username": "Bret",
"email": "Sincere@april.biz",
"address": {
"street": "Kulas Light",
"suite": "Apt. 556",
"city": "Gwenborough",
"zipcode": "92998-3874",
"geo": {
"lat": "-37.3159",
"lng": "81.1496"
}
},
"phone": "1-770-736-8031 x56442",
"website": "hildegard.org",
"company": {
"name": "Romaguera-Crona",
"catchPhrase": "Multi-layered client-server neural-net",
"bs": "harness real-time e-markets"
}
},
{
"id": 2,
"name": "Ervin Howell",
"username": "Antonette",
"email": "Shanna@melissa.tv",
"address": {
"street": "Victor Plains",
"suite": "Suite 879",
"city": "Wisokyburgh",
"zipcode": "90566-7771",
"geo": {
"lat": "-43.9509",
"lng": "-34.4618"
}
},
"phone": "010-692-6593 x09125",
"website": "anastasia.net",
"company": {
"name": "Deckow-Crist",
"catchPhrase": "Proactive didactic contingency",
"bs": "synergize scalable supply-chains"
}
}
]
Nested Transforms
loading...

Result

[
{
"id": 1,
"name": "Leanne Graham",
"company": {
"name": "Romaguera-Crona",
"catchPhrase": "Multi-layered client-server neural-net",
"bs": "harness real-time e-markets"
},
"email": "Sincere@april.biz",
"address": {
"street": "Kulas Light",
"zipcode": "92998-3874",
"city": "Gwenborough"
},
"phone": "1-770-736-8031 x56442",
"website": "hildegard.org",
"USER": "Bret"
},
{
"id": 2,
"name": "Ervin Howell",
"company": {
"name": "Deckow-Crist",
"catchPhrase": "Proactive didactic contingency",
"bs": "synergize scalable supply-chains"
},
"email": "Shanna@melissa.tv",
"address": {
"street": "Victor Plains",
"zipcode": "90566-7771",
"city": "Wisokyburgh"
},
"phone": "010-692-6593 x09125",
"website": "anastasia.net",
"USER": "Antonette"
}
]

Remove Keys

!!! See remove attributes as your first choice, this is potentially planned for deprecation !!!

This function is used to rename keys for better readability, usage and consistency.

Usage

post_transforms:
- remove_keys:
- address1
Remove Keys Example

Consider the following data:

[
{
firstname: "blah"
lastname: "blahblah"
address1: "blah address"
}
]

Using the remove keys function we can remove unnecssary data from your response i.e address1.

# unique name within your organization
name: user_api
version: 0.0.1
description: Retrieve User Details
metrics_enabled: true

global:
databases:
main:
driver: postgres
conn_string: |
postgresql://a|env::POSTGRES_USER|:a|env::POSTGRES_PASS|@a|env::POSTGRES_HOST|?connect_timeout=10"

interfaces:
user/remove-keys:
output: http
route: details
method: POST

actions:
- name: CheckBody
input: a|body
run_when_succeeded:
- previous
assert:
tests:
- value: firstname
is_not_null: true

- name: CheckUser
run_when_succeeded:
- previous
database: main
query: |
SELECT firstname, lastname, address1 FROM customers WHERE firstname LIKE $1 LIMIT 2;
params:
- a|CheckBody::firstname|
assert:
error_message: "User Does Not Exist"
tests:
- value: count()
is_equal_to: 1
error_message: "No User Found"
post_transforms:
- remove_keys:
- address1

This will result in the below output when the API is called.

{
"data": {
"CheckBody": {
"data": {
"firstname": "VKUUXF"
},
"message": "success",
"time.ms": 0
},
"CheckUser": {
"data": [
{
"firstname": "VKUUXF",
"lastname": "ITHOMQJNYX"
}
],
"message": "success",
"time.ms": 669
}
},
"metrics": {
"user_api.config.ms": 670
}
}

Rename Keys

!!! See rename attributes as your first choice, this is potentially planned for deprecation !!!

This function is used to rename keys for better readability, usage and consistency.

Usage

post_transforms:
- rename_keys:
colour: col
Rename Keys Example

Consider the following data:

[
{
firstname: "blah"
lastname: "blahblah"
address1: "blah address"
}
]

Using the rename keys function we can map new keys for better readability

# unique name within your organization
name: user_api
version: 0.0.1
description: Retrieve User Details
metrics_enabled: true

global:
databases:
main:
driver: postgres
conn_string: |
postgresql://a|env::POSTGRES_USER|:a|env::POSTGRES_PASS|@a|env::POSTGRES_HOST|?connect_timeout=10"

interfaces:
user/rename-keys:
output: http
route: details
method: POST

actions:
- name: CheckBody
input: a|body
run_when_succeeded:
- previous
assert:
tests:
- value: firstname
is_not_null: true

- name: CheckUser
run_when_succeeded:
- previous
database: main
query: |
SELECT firstname, lastname, address1 FROM customers WHERE firstname LIKE $1 LIMIT 2;
params:
- a|CheckBody::firstname|
assert:
error_message: "User Does Not Exist"
tests:
- value: count()
is_equal_to: 1
error_message: "No User Found"
post_transforms:
- rename_keys:
address1: addr
firstname: fname
lastname: lname

This will result in the below output when the API is called.

{
"data": {
"CheckBody": {
"data": {
"firstname": "VKUUXF"
},
"message": "success",
"time.ms": 0
},
"CheckUser": {
"data": [
{
"addr": "4608499546 Dell Way",
"fname": "VKUUXF",
"lname": "ITHOMQJNYX"
}
],
"message": "success",
"time.ms": 669
}
},
"metrics": {
"user_api.config.ms": 670
}
}