Generate Integration Test from Trace

Introduction

This guide will show you how to generate integration tests for complex scenarios with Skyramp using Skyramp-collected traces. We'll demonstrate using Skyramp’s Demo Shop API, a simple e-commerce API for product and order management. Learn more about the Demo Shop API.

Prerequisites

• Skyramp CLI installed

• Relevant libraries installed (ie Python / Java libraries)

• (for trace collection only) Docker installed and running

Refer to the Installation Guide if you haven't installed Skyramp yet.

Overview

To generate integration tests that span across multiple endpoints or services, Skyramp requires trace information to generate the test case reliably.

What is a trace?

A trace is a log of a set of actions executed in a particular window or scenario in your system. Traces can be used to simulate real-world scenarios of product usage, and those traces can in turn be used to generate tests that give you confidence that said scenario will be resilient in production traffic.

Skyramp can function as a “Man-in-the-Middle” proxy that monitors your traffic and allows you to simply record the scenario you want to test. For integration tests, Skyramp can collect traffic from 2 different sources:

• HTTP traffic from a browser UI

• Backend traffic through the CLI

By following these simple steps, you'll be able to collect a trace that you can directly supply for the Skyramp test generation.

Start Trace Collection

Want to skip straight to test generation? Download this sample trace and skip to “Generate Integration Test from Trace”

To start collecting traces, you need to run the following command. This command will spawn a new shell configured to collect all relevant network traffic and a web browser that is configured to use the “Man-in-the-Middle” as a proxy. All visited web links traffic will also be part of the collected trace. The browser can be closed if only CLI traffic is of interest. The first time you run this command, it might take a while to bring up Skyramp.

skyramp generate trace \
--include "https://demoshop.skyramp.dev/*" \
--output

Explanation of Command

• --include: Specify a comma-separated list of URLs for Skyramp to filter for during trace collection or in a trace file.

• --output: Specify the name of the generated trace file.

Adjustments

• --exclude: Specify a comma-separated list of URLs for which you do not want Skyramp to filter for. This flag takes precedence over --include if a URL is specified in both.

• --docker-network: Specify the name (string) of your Docker network.

• --docker-skyramp-port: Specify a port number for Skyramp (default=35142).

A Note on Including/Excluding URLs

When using the --include or --exclude flag, URLs need to be specified either as an exact path to the endpoint or in combination with a wildcard:

• demoshop.skyramp.dev/api/v1/products will only filter for POST and GET of the products endpoint.

• demoshop.skyramp.dev/products* will filter for all products endpoints + any nested endpoints.

• demoshop.skyramp.dev/api/v1/products/* will filter only for any nested endpoints that require a product ID. It will not collect the traces of POST and GET products.

For endpoints that require path parameters, please replace the path parameter with the wildcard:

• Convert demoshop.skyramp.dev/products/{product_id}/reviews to demoshop.skyramp.dev/products/*/reviews

Record Scenario

Now that we’ve started the trace collection process, we can record our scenario from earlier:

1. Set up session (via UI)

2. POST of a new product (via cURL)

3. GET of products (via UI) - opening page

4. POST of a new order containing the new product (via cURL)

5. GET of all orders (via UI)

6. GET of the new order (via UI)

7. DELETE of the new order (via UI)

8. DELETE of newly created product (via UI)

We are using cURL to demonstrate the network calls, however, any network call that is made from the newly spawned shell will be collected by Skyramp (e.g. executing old tests).

• Due to the proxy certificate handling, please ensure you use the -k flag with curl

1. Set up session (via UI)

• To start, in the newly spawned Chromium browser, go to https://demoshop.skyramp.dev. This will open the Demo Shop UI.

• At the top of the page, you will see a section called Session ID with three words. Click on the Edit button, and fill in the session ID with any string (for this example, we will use skyramp-integration-demo)

  

• Save the session ID by clicking on the green checkmark.

• Next, click “Clear State” to reset and begin your session recording. NOTE: You must hit this button to ensure your test runs against a clean Demo Shop database state.

• This session ID will be used as your “Authorization” token for every API call made to the Demo Shop. You can save your session ID in an environment variable for re-use across the different network requests from the terminal:

  export SKYRAMP_TEST_TOKEN=skyramp-integration-demo

2. POST of a new product (via cURL)

• Let’s create a new product with the Demo Shop API

curl -X POST https://demoshop.skyramp.dev/api/v1/products \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $SKYRAMP_TEST_TOKEN" \
  -k \
  -d '{
    "name": "SkyPhone 1",
    "description": "An iPhone powered by Skyramp",
    "price": 2023.99,
    "image_url": "https://images.app.goo.gl/jGPHo3ZEzEbHG8o2A",
    "category": "Phones",
    "in_stock": true
  }'

3. GET of products (via UI)

• Refresh your Chromium browser. This will cause your newly created product to appear on the Product Catalog.

  

4. POST of a new order containing the new product (via cURL)

• From the original POST products API call, copy the product_id field that is returned in the response (product_id should be 24 if this entire recorded flow is done from a cleared session)

• We will use this product_id to create a new order:

curl -X POST https://demoshop.skyramp.dev/api/v1/orders \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $SKYRAMP_TEST_TOKEN" \
  -k \
  -d '{
    "customer_email": "sahil@skyramp.dev",
    "items": [
      {
        "product_id": 24,
        "quantity": 2
      }
    ]
  }'

5. GET of all orders (via UI)

• From the Demo Shop UI in the Chromium browser, click on “Orders” at the top right of the page.

• Navigating to this page will open a webpage and load a list of orders. During page load, the GET https://demoshop.skyramp.dev/api/v1/orders API will be called.

  

6. GET of the new order (via UI)

• From the Demo Shop UI in the Chromium browser, find the order you placed via the cURL, and click on “View Details.”

• Navigating to this page will open a webpage and load the details of the order you just placed. During page load, the GET https://demoshop.skyramp.dev/api/v1/orders/{order_id} API will be called, with order_id being the ID of the order you just created. There will also be one API call to https://demoshop.skyramp.dev/api/v1/products/{product_id} for each product ordered.

  

7. DELETE of newly placed order (via UI)

• From the Demo Shop UI in the Chromium browser, click on “Delete Order”

• Clicking on that button will delete your order and redirect you back to the Orders page. The following APIs will be called:

  • DELETE https://demoshop.skyramp.dev/api/v1/orders/{order_id}

  • GET https://demoshop.skyramp.dev/api/v1/orders

  

8. DELETE of newly created product (via UI)

• From the Demo Shop UI in the Chromium browser, click on “Products”

• Find the product you created and click on “View Details”

• Click on “Delete Product”

• Clicking on that button will cancel your order and redirect you back to the Products page. The following APIs will be called:

  • DELETE https://demoshop.skyramp.dev/api/v1/products/{product_id}

  • GET https://demoshop.skyramp.dev/api/v1/products

End Trace Collection

Once the scenario is completed, you can end the trace collection with Ctrl + D. You will see for which endpoints, Skyramp was able to collect traces:

You can find a sample trace for the scenario above here.

Generate Integration Test from Trace