Generation

Getting Started

Smoke Test

Smoke testing

This guide describes generating smoke tests using the Skyramp CLI. Throughout this guide, we are using Skyramp’s Demo Shop API as an example REST API. You can find all relevant information on the Demo Shop here.

If you haven’t already installed Skyramp, follow the instructions here.

1. Overview

Smoke testing is a method of testing that determines whether the endpoint being tested returns a response. A smoke test establishes whether the endpoint exists and can be contacted. With Skyramp, you can generate smoke tests either for a single method or all the methods of a REST API endpoint.

2. Generate a smoke test for REST APIs

This section explains how you can use Skyramp to generate a smoke test for a specific method of a REST API. To reliably generate test cases, we require at least one of the following inputs:

OpenAPI schema (JSON or YAML file)
Sample request data (JSON blob or JSON file)

These instructions are for using an API schema as input. If you want more control over the generated body values, you can also do test generation from sample data [LINK].

Smoke testing

If you haven’t already installed Skyramp, follow the instructions here.

1. Overview

2. Generate a smoke test for REST APIs

This section explains how you can use Skyramp to generate a smoke test for a specific method of a REST API. To reliably generate test cases, we require at least one of the following inputs:

OpenAPI schema (JSON or YAML file)
Sample request data (JSON blob or JSON file)

These instructions are for using an API schema as input. If you want more control over the generated body values, you can also do test generation from sample data [LINK].

Smoke testing

If you haven’t already installed Skyramp, follow the instructions here.

1. Overview

2. Generate a smoke test for REST APIs

This section explains how you can use Skyramp to generate a smoke test for a specific method of a REST API. To reliably generate test cases, we require at least one of the following inputs:

OpenAPI schema (JSON or YAML file)
Sample request data (JSON blob or JSON file)

These instructions are for using an API schema as input. If you want more control over the generated body values, you can also do test generation from sample data [LINK].

Python

Typescript

2.1 Single Method

To create a smoke test for a single method, specify the method you want to test against in the command. In this example, we are using the https://demoshop.skyramp.dev/api/v1/products as the URL to our service. When testing your service, replace it with the URL to the endpoint you want to test.

You can find the used API specification here.

skyramp generate smoke rest https://demoshop.skyramp.dev/api/v1/products \
-X POST \
--language python \
--framework pytest \
--api-schema openapi.json

This command generates a smoke test file: products_POST_smoke_test.py.
The content of this file is explained Section 4 below.

Adjustments

These flags will help you tune the basic smoke test. Additional flags are explained here.

--response-status-code: Specify the expected status code. By default, Skyramp either asserts against the defined status code in the API specification or defaults to 20X.
--output: Specify the name of the generated test file. Useful if you need it to meet a certain format.
--output-dir: Allows you to specify the directory to store the generated test file in.

3. Execute Smoke Test

You can execute the generated tests without any additional adjustments to the code. However, based on the application you want to test, you can pass your authentication token to Skyramp Tests via an environment variable.

3.1 Set environment variable for authentication (if applicable)

Skyramp’s sample application doesn't require any authentication.

To test against an application that does require authentication, pass your token using an environment variable. By default, Skyramp expects a Bearer Token but we support additional authentication methods.

export SKYRAMP_TEST_TOKEN=$your_auth_token

3.2 Run the test

Run the tests using Pytest. If you don’t have Pytest, install it with pip by running the following command in your terminal:

# Prerequisites 
pip install pytest

# Execution of smoke test for products/POST 
python3 -m pytest products_POST_smoke_test.py

3.3 Review Test Results

We are using Pytest’s default test output in this guide, printing a line for each test that is being run and listing all failures at the end. You can adjust the output behavior following this documentation.

3.3.1 Successful Test

Using the single POST method example with the /v1/products endpoint, the test passes.

3.3.2 Test Failure

To confirm that the test is functioning correctly, let’s make it fail. For this section, we intentionally changed the expected status code from 20X to 40X in the products_POST_smoke_test.py test file (line 43).

assert skyramp.check_status_code(api_v1_products_POST_response, "40X")

This results in a test failure, since the expected status code does not match the response status code. The expected output looks like the following:

Congratulations, you have now successfully generated a smoke test, executed it against a REST API, and reviewed the results!

4. Skyramp Test File

This section explains the key elements of the generated test file, allowing you to quickly make adjustments, if needed.

The header of each test file shows when the test was generated and what command was used to generate it.
The body of the test file imports all relevant libraries and specifies the URL for all test requests.
Each method being tested has a function that consists of:
- Invocation of Skyramp Client
- Definition of the authentication header
- Definition of the request body
- Formation of the request
- Assertion against expected status code

4.1 Single Method

# Generated by Skyramp v0.5.7 on 2025-03-05 16:26:17.266387 -0500 EST m=+0.217684209
# Command: skyramp generate smoke rest https://demoshop.skyramp.dev/api/v1/products \
# 		--api-schema openapi.json \
# 		--framework pytest \
# 		--language python \
# 		--method POST \

# Import of required libraries
import skyramp
import os
import time
# URL for test requests
URL = "https://demoshop.skyramp.dev"

# smoke test for /api/v1/products POST
def test_api_v_1_products_post():
    # Invocation of Skyramp Client
    client = skyramp.Client()
    # Definition of authentication header
    headers = {}
    if os.getenv("SKYRAMP_TEST_TOKEN") is not None:
        headers["Authorization"] = "Bearer " + os.getenv("SKYRAMP_TEST_TOKEN")

    # Request Body
    api_v1_products_POST_request_body = r'''{
            "category": "string",
            "description": "string",
            "image_url": "string",
            "in_stock": false,
            "name": "string",
            "price": 0
        }'''
    
    # Execute Request
    api_v1_products_POST_response = client.send_request(
        url=URL,
        path="/api/v1/products",
        method="POST",
        body=api_v1_products_POST_request_body,
        headers=headers
    )
    # Generated Assertions
    assert skyramp.check_status_code(api_v1_products_POST_response, "20X")

if __name__ == "__main__":
    test_api_v_1_products_post()

3. Execute Smoke Test

3.1 Set environment variable for authentication (if applicable)

Skyramp’s sample application doesn't require any authentication.

export SKYRAMP_TEST_TOKEN=$your_auth_token

3.2 Run the test

Run the tests using Pytest. If you don’t have Pytest, install it with pip by running the following command in your terminal:

# Prerequisites 
pip install pytest

# Execution of smoke test for products/POST 
python3 -m pytest products_POST_smoke_test.py

3.3 Review Test Results

3.3.1 Successful Test

Using the single POST method example with the /v1/products endpoint, the test passes.

3.3.2 Test Failure

assert skyramp.check_status_code(api_v1_products_POST_response, "40X")

This results in a test failure, since the expected status code does not match the response status code. The expected output looks like the following:

Congratulations, you have now successfully generated a smoke test, executed it against a REST API, and reviewed the results!

4. Skyramp Test File

This section explains the key elements of the generated test file, allowing you to quickly make adjustments, if needed.

The header of each test file shows when the test was generated and what command was used to generate it.
The body of the test file imports all relevant libraries and specifies the URL for all test requests.
Each method being tested has a function that consists of:
- Invocation of Skyramp Client
- Definition of the authentication header
- Definition of the request body
- Formation of the request
- Assertion against expected status code

4.1 Single Method

# Generated by Skyramp v0.5.7 on 2025-03-05 16:26:17.266387 -0500 EST m=+0.217684209
# Command: skyramp generate smoke rest https://demoshop.skyramp.dev/api/v1/products \
# 		--api-schema openapi.json \
# 		--framework pytest \
# 		--language python \
# 		--method POST \

# Import of required libraries
import skyramp
import os
import time
# URL for test requests
URL = "https://demoshop.skyramp.dev"

# smoke test for /api/v1/products POST
def test_api_v_1_products_post():
    # Invocation of Skyramp Client
    client = skyramp.Client()
    # Definition of authentication header
    headers = {}
    if os.getenv("SKYRAMP_TEST_TOKEN") is not None:
        headers["Authorization"] = "Bearer " + os.getenv("SKYRAMP_TEST_TOKEN")

    # Request Body
    api_v1_products_POST_request_body = r'''{
            "category": "string",
            "description": "string",
            "image_url": "string",
            "in_stock": false,
            "name": "string",
            "price": 0
        }'''
    
    # Execute Request
    api_v1_products_POST_response = client.send_request(
        url=URL,
        path="/api/v1/products",
        method="POST",
        body=api_v1_products_POST_request_body,
        headers=headers
    )
    # Generated Assertions
    assert skyramp.check_status_code(api_v1_products_POST_response, "20X")

if __name__ == "__main__":
    test_api_v_1_products_post()

3. Execute Smoke Test

3.1 Set environment variable for authentication (if applicable)

Skyramp’s sample application doesn't require any authentication.

export SKYRAMP_TEST_TOKEN=$your_auth_token

3.2 Run the test

Run the tests using Pytest. If you don’t have Pytest, install it with pip by running the following command in your terminal:

# Prerequisites 
pip install pytest

# Execution of smoke test for products/POST 
python3 -m pytest products_POST_smoke_test.py

3.3 Review Test Results

3.3.1 Successful Test

Using the single POST method example with the /v1/products endpoint, the test passes.

3.3.2 Test Failure

assert skyramp.check_status_code(api_v1_products_POST_response, "40X")

This results in a test failure, since the expected status code does not match the response status code. The expected output looks like the following:

Congratulations, you have now successfully generated a smoke test, executed it against a REST API, and reviewed the results!

4. Skyramp Test File

This section explains the key elements of the generated test file, allowing you to quickly make adjustments, if needed.

The header of each test file shows when the test was generated and what command was used to generate it.
The body of the test file imports all relevant libraries and specifies the URL for all test requests.
Each method being tested has a function that consists of:
- Invocation of Skyramp Client
- Definition of the authentication header
- Definition of the request body
- Formation of the request
- Assertion against expected status code

4.1 Single Method

# Generated by Skyramp v0.5.7 on 2025-03-05 16:26:17.266387 -0500 EST m=+0.217684209
# Command: skyramp generate smoke rest https://demoshop.skyramp.dev/api/v1/products \
# 		--api-schema openapi.json \
# 		--framework pytest \
# 		--language python \
# 		--method POST \

# Import of required libraries
import skyramp
import os
import time
# URL for test requests
URL = "https://demoshop.skyramp.dev"

# smoke test for /api/v1/products POST
def test_api_v_1_products_post():
    # Invocation of Skyramp Client
    client = skyramp.Client()
    # Definition of authentication header
    headers = {}
    if os.getenv("SKYRAMP_TEST_TOKEN") is not None:
        headers["Authorization"] = "Bearer " + os.getenv("SKYRAMP_TEST_TOKEN")

    # Request Body
    api_v1_products_POST_request_body = r'''{
            "category": "string",
            "description": "string",
            "image_url": "string",
            "in_stock": false,
            "name": "string",
            "price": 0
        }'''
    
    # Execute Request
    api_v1_products_POST_response = client.send_request(
        url=URL,
        path="/api/v1/products",
        method="POST",
        body=api_v1_products_POST_request_body,
        headers=headers
    )
    # Generated Assertions
    assert skyramp.check_status_code(api_v1_products_POST_response, "20X")

if __name__ == "__main__":
    test_api_v_1_products_post()

Smoke Test

Smoke testing

1. Overview

2. Generate a smoke test for REST APIs

Smoke testing

1. Overview

2. Generate a smoke test for REST APIs

Smoke testing

1. Overview

2. Generate a smoke test for REST APIs

2.1 Single Method

Adjustments

3. Execute Smoke Test

3.1 Set environment variable for authentication (if applicable)

3.2 Run the test

3.3 Review Test Results

3.3.1 Successful Test

3.3.2 Test Failure

4. Skyramp Test File

4.1 Single Method

Related topics

2.1 Single Method

Adjustments

3. Execute Smoke Test

3.1 Set environment variable for authentication (if applicable)

3.2 Run the test

3.3 Review Test Results

3.3.1 Successful Test

3.3.2 Test Failure

4. Skyramp Test File

4.1 Single Method

Related topics

2.1 Single Method

Adjustments

3. Execute Smoke Test

3.1 Set environment variable for authentication (if applicable)

3.2 Run the test

3.3 Review Test Results

3.3.1 Successful Test

3.3.2 Test Failure

4. Skyramp Test File

4.1 Single Method

Related topics