Advanced Load Testing Generation

Introduction

This guide will show you how to generate load tests for complex scenarios with Skyramp using Skyramp-collected traces. We'll demonstrate using Skyramp’s Demo Shop API, a simple e-commerce API for product and order management. Learn more about the Demo Shop API.

Prerequisites

• Skyramp CLI installed

• Relevant libraries installed (ie Python / Java libraries)

• (for trace collection only) Docker installed and running

Refer to the Installation Guide if you haven't installed Skyramp yet.

Overview

To generate load tests that span across multiple endpoints or services, Skyramp requires trace information to generate the test case reliably.

What is a trace?

A trace is a log of a set of actions executed in a particular window or scenario in your system. Skyramp can function as a “Man-in-the-Middle” proxy that monitors your traffic and allows you to simply record the scenario you want to test. By following these simple steps, you'll be able to collect a trace that you can directly supply for the Skyramp test generation.

Following, we will generate a Skyramp trace and corresponding load test for a simple scenario across endpoints.

1. POST of a new product (via cURL)

2. GET of products (via UI)

3. POST of a new order containing the new product (via cURL)

4. GET of all orders (UI)

5. GET of the new order (UI)

6. DELETE of the new order (UI)

Start Trace Collection

Want to skip straight to test generation? Download this sample trace and skip to “Generate Load Test from Trace"

This page is dedicated to collecting only network traffic triggered via the CLI or browser. In additional guides, we will cover how to collect full UI traces.

To start collecting traces, you need to run the following command. This command will spawn a new shell configured to collect all relevant network traffic and a web browser that is configured to use the “Man-in-the-Middle” as a proxy. All visited web links traffic will also be part of the collected trace. Browser can be closed if only CLI traffic is of interest. The first time you run this command, it might take a while to bring up Skyramp.

skyramp generate trace \
--include "https://demoshop.skyramp.dev/*" \
--runtime docker \
--output

Explanation of Command

• --include: Specify a comma-separated list of fully qualified domain names (FQDNs) for Skyramp to filter for during trace collection or in a trace file.

• --runtime: Select the environment in which the Skyramp trace collector gets deployed. [Docker, Kubernetes]

• --output: Specify the name of the generated trace file.

A Note on Fully Qualified Domain Names (FQDNs)

FQDNs need to be specified either as an exact path to the endpoint or in combination with a wildcard:

• demoshop.skyramp.dev/api/v1/products will only filter for POST and GET of the products endpoint.

• demoshop.skyramp.dev/products* will filter for all products endpoints + any nested endpoints.

• demoshop.skyramp.dev/api/v1/products/* will filter only for any nested endpoints that require a product ID. It will not collect the traces of POST and GET products.

For endpoints that require path parameters, please replace the path parameter with the wildcard:

• Convert demoshop.skyramp.dev/products/{product_id}/reviews to demoshop.skyramp.dev/products/*/reviews

Additional Flags:

• --exclude: Specify a comma-separated list of FQDNs for which you do not want Skyramp to filter for. This flag takes precedence over --include if a FQDN is specified in both.

• --docker-network: Specify the name (string) of your Docker network.

• --docker-skyramp-port: Specify a port number for Skyramp (default=35142).

Record Scenario

Now that we’ve started the trace collection process, we can record our scenario from earlier:

1. POST of a new product (via cURL)

2. GET of products (via UI)

3. POST of a new order containing the new product (via cURL)

4. GET of all orders (via UI)

5. GET of the new order (via UI)

6. DELETE of the new order (via UI)

We are using cURL to demonstrate the network calls, however, any network call that is made from the newly spawned shell will be collected by Skyramp (e.g. executing old tests).

• Due to the proxy certificate handling, please ensure you use the -k flag with curl

1. POST of a new product (via cURL)

• To start, let’s create a new product with the Demo Shop API

curl -X POST https://demoshop.skyramp.dev/api/v1/products \
  -H "Content-Type: application/json" \
  -k \
  -d '{
    "name": "SkyPhone 1",
    "description": "An iPhone powered by Skyramp",
    "price": 2023.99,
    "image_url": "https://images.app.goo.gl/jGPHo3ZEzEbHG8o2A",
    "category": "Phones",
    "in_stock": true
  }'

2. GET of products (via UI)

• In the newly spawned Chromium browser, go to https://skyramp.dev/demoshop/products. This will open the Demo Shop UI.

• Navigating to this page will open a webpage and load a list of products. During page load, the GET https://demoshop.skyramp.dev/api/v1/products API will be called.

3. POST of a new order containing the new product (via cURL)

• From the original POST products API call, copy the product_id field that is returned in the response

• We will use this product_id to create a new order. Replace <placeholder> in the below command with your product_id:

curl -X POST https://demoshop.skyramp.dev/api/v1/orders \
  -H "Content-Type: application/json" \
  -k \
  -d '{
    "customer_email": "sahil@skyramp.dev",
    "items": [
      {
        "product_id": <placeholder>,
        "quantity": 2,
        "unit_price": 2023.99
      }
    ]
  }'

4. GET of all orders (via UI)

• From the Demo Shop UI in the Chromium browser, click on “Orders” at the top right of the page.

• Navigating to this page will open a webpage and load a list of orders. During page load, the GET https://demoshop.skyramp.dev/api/v1/orders API will be called.

5. GET of the new order (via UI)

• From the Demo Shop UI in the Chromium browser, find the order you placed via the cURL, and click on “View Details.”

• Navigating to this page will open a webpage and load the details of the order you just placed. During page load, the GET https://demoshop.skyramp.dev/api/v1/orders/{order_id} API will be called, with order_id being the ID of the order you just created.

6. DELETE of newly placed order (via UI)

• From the Demo Shop UI in the Chromium browser, click on “Delete Order”

• Clicking on that button will delete your order and redirect you back to the Orders page. The following APIs will be called:

  • DELETE https://demoshop.skyramp.dev/api/v1/orders/{order_id}

  • GET https://demoshop.skyramp.dev/api/v1/orders

End Trace Collection

Once the scenario is completed, you can end the trace collection with Ctrl + D. You will see for which endpoints, Skyramp was able to collect traces:

You can find a sample trace for the scenario above here.

Generate Load Test from Trace