2.1 Single Method

To create a fuzz test for a single method, specify the method you want to test against in the command. In this example, we are using the https://demoshop.skyramp.dev/api as the URL to our service. When testing your service, replace it with the URL to the endpoint you want to test.

You can find the used API specification here.

skyramp generate fuzz rest https://demoshop.skyramp.dev/api/v1/products \
-X POST \
--language python \
--framework robot \
--api-schema openapi.json

This command generates two files:

• products_POST_fuzz_test.py

• products_POST_fuzz_test.robot (simple Robot wrapper file)

The content of the generated test is explained Section 4 below.

Adjustments

Below are a few flags to customize the test generation. Additional flags are explained here.

• --auth-header: This flag allows you to specify the key of your authentication header, e.g. --auth-header X-API-KEY. By default, we assume Bearer.

• --response-status-code: Allows you to specify the expected status code for the fuzzed body values (default=40X)

• --path-params: This flag allows you to override path parameters from your endpoint URL or the pre-defined values in the API schema, e.g. --path-params id=3fa85f64-5717-4562-b3fc-2c963f66afa6

• --output: Allows you to specify the name of the generated test file.

• --output-dir: Allows you to specify the directory to store the generated test file.

3. Execute Fuzz Test

You can execute the generated tests without any additional adjustments to the code.

3.1 Set env variable for authentication

Ensure proper authentication for test execution. By default, we expect a Bearer Token but support additional authentication methods. If your API does not require any authentication, you can skip this step and just run the test.

export SKYRAMP_TEST_TOKEN=$your_auth_token

3.2 Run the test

We now run the test using the Robot Framework. You can call the generated Python file either in an existing robot file of yours or leverage the simple wrapper file we generate.

# Prerequisites
pip install robotframework

# Execution of fuzz test for products/POST 
robot products_POST_fuzz_test.robot

3.3 Results

As we use the Robot Framework to execute the test in this example, you can expect both the typical stdout from Robot and the generated files (output, log, report). You can find more information on the generated outputs in the Robot documentation.

As we have not adjusted the status codes to reflect the expected behavior of the service the test is currently failing. Here, we show how to quickly adjust the code to reflect the expected behavior.
You can find more detailed information about the test failure in the report.html and log.html.

4. Skyramp Test File

This section explains the key elements of the generated test files. This will enable you to make adjustments when needed quickly.

• At the top of each file, we show when the test was generated and what command was used

• Below, we import all relevant libraries and specify the URL for all test requests

• We define a function per method that is tested. It consists of:

  • Invocation of Skyramp Client

  • Definition of the authentication header

  • Definition of the default request body (based on API schema or sample data)

  • Definition of all fuzzed body values

  • Definition of all expected response status codes for fuzzed body values (default = 40X)

  • Loop through all fuzzed values. Each loop:

    • Creates a request with the fuzzed body value

    • Creates a request with the fuzzed value being None

  • Status Code Assertion

Test Execution Behavior

The generated fuzz test will execute in the following way:

• First, it will execute a request with the default body values from the API spec or sample data you provide

• The test then iterates through each body value, changing the selected body value with a fuzzed value and None while keeping the default values for all other keys

• Lastly, it asserts the status codes of all requests. This is done at the end of the loop to avoid premature failure that would lead to unnecessary reruns of the test.

Default Fuzz Strategy

By default, Skyramp generates random data for all values in the request body and stores those in a separate dictionary. Additionally, the generated code contains a dictionary that stores the expected status codes for each fuzzed value. The default value is 40X. Below, we explain how to change those values to ensure your desired fuzz strategy quickly.

• strings: All string values receive the value “0123456789"

• integer/float: Integers and floats are assigned the value -10

• boolean: The boolean value is changed to the opposite, e.g. true to false; if no default value is defined, we assign True.

• enum: A randomly generated string, that is not part of the enum, is assigned.

  
  

4.1 Single Method

# Generated by Skyramp v0.5.0.af68db77 on 2025-02-13 11:59:55.78257 -0500 EST m=+0.237459626
# Command: skyramp generate fuzz rest https://demoshop.skyramp.dev/api/v1/products \
# 		--api-schema openapi.json \
# 		--framework robot \
# 		--language python \
# 		--method POST \

# Import of required libraries
import skyramp
import os
import time
# URL for test requests
URL = "https://demoshop.skyramp.dev"

# fuzz test for /api/v1/products POST
def test_products_post():
    # Invocation of Skyramp Client
    client = skyramp.Client(
        framework="robot"
    )
    # Definition of authentication header
    headers = {}
    if os.getenv("SKYRAMP_TEST_TOKEN") is not None:
        headers["Authorization"] = "Bearer " + os.getenv("SKYRAMP_TEST_TOKEN")

    # Request Body
    products_POST_request_body = r'''{
            "category": "string",
            "description": "string",
            "image_url": "string",
            "in_stock": false,
            "name": "string",
            "price": 0
        }'''
    
    # Fuzz strategies
    products_post_fuzzed_body = {
        "category": "0123456789",
        "description": "0123456789",
        "image_url": "0123456789",
        "in_stock": True,
        "name": "0123456789",
        "price": -10
    }
    # Fuzz status codes
    expected_products_post_status_code = {
        "category": "40x",
        "description": "40x",
        "image_url": "40x",
        "in_stock": "40x",
        "name": "40x",
        "price": "40x"
    }

    # Execute Request
    products_POST_response = client.send_request(
        url=URL,
        path="/api/v1/products",
        method="POST",
        body=products_POST_request_body,
        headers=headers,
        expected_code="20x"
    )

    for key in skyramp.iterate(products_post_fuzzed_body):
        # Execute Request
        products_POST_response = client.send_request(
            url=URL,
            path="/api/v1/products",
            method="POST",
            body=products_POST_request_body,
            headers=headers,
            data_override={key: skyramp.get_value(products_post_fuzzed_body, key)},
            expected_code=skyramp.get_value(expected_products_post_status_code, key),
            description=f'Fuzzing request body { key } to { skyramp.get_value(products_post_fuzzed_body, key) }'
        )

        # Execute Request
        products_POST_response = client.send_request(
            url=URL,
            path="/api/v1/products",
            method="POST",
            body=products_POST_request_body,
            headers=headers,
            data_override={key: None},
            expected_code=skyramp.get_value(expected_products_post_status_code, key),
            description=f'Fuzzing request body { key } to None'
        )

    assert client.is_success()


if __name__ == "__main__":
    test_products_post()

4.1.1 Changing fuzzed values

You can easily change the generated fuzz values as well as the expected status codes

Default Generated Fuzz Values (lines 37-54)

  products_post_fuzzed_body = {
        "category": "0123456789",
        "description": "0123456789",
        "image_url": "0123456789",
        "in_stock": True,
        "name": "0123456789",
        "price": -10
    }
    
  expected_products_post_status_code = {
        "category": "40X",
        "description": "40X",
        "image_url": "40X",
        "in_stock": "40X",
        "name": "40X",
        "price": "40X"
    }

Manually Adjusted Fuzz Values

  products_post_fuzzed_body = {
        "category": "toys",
        "description": "toys for kids",
        "image_url": "example.com/picture",
        "in_stock": True,
        "name": "warrior",
        "price": -10
    }
    
  expected_products_post_status_code = {
        "category": "20X",
        "description": "20X",
        "image_url": "20X",
        "in_stock": "20X",
        "name": "20X",
        "price": "40X"
    }