Announcing Skyramp Agentic Experience Beta: AI-Powered Deterministic Testing

Today, we're excited to announce beta release of the Skyramp Agentic Experience, a new way to create, manage, and execute comprehensive test suites through natural language conversations in your development environment.

Whether you're building traditional applications, shipping AI-generated code, or somewhere in between, testing shouldn't require mastering yet another complicated framework. Skyramp Agentic Experience makes enterprise-grade testing accessible through a user-friendly interface that is already part of most developers' workflows: AI collaboration in coding assistants and agentic IDEs. Skyramp Agentic Experience is available in Cursor and VS Code today, with support for Claude Code and Windsurf coming soon.

The Design Philosophy

Large Language Models are remarkable at understanding context and reasoning about problems, but they struggle with software testing because they are inherently non-deterministic. Ask an LLM to generate tests for the same code multiple times, and you will likely get different results each time. Further, they can generate plausible-looking code that may contain subtle errors, hallucinate non-existent specifications, and generate unnecessary and inconsistent scaffolding for test execution. Testing, on the other hand, demands exactly the opposite: deterministic, reproducible results that developers can trust. When a test passes, it needs to mean something concrete and reproducible. When it fails, developers need to trust that the failure represents a real problem, not random variation in how the test executed.

These requirements seem fundamentally incompatible. But what if we could combine the natural language accessibility and contextual understanding of LLMs with the reliability and precision of purpose-built, deterministic testing tools? Skyramp Agentic Experience positions AI as an intelligent orchestrator in a three-way collaboration between the human, a large language model, and a comprehensive suite of deterministic testing tools. Each party plays to its strengths:

Deterministic tools provide structure and reliability by defining clear specifications, required inputs and optional parameters: API schemas, authentication methods, endpoint URLs, expected behaviors. This provides necessary guardrails for AI behavior.

The LLM analyzes code patterns, maps dependencies, facilitates communication, and translates user requirements into tool specifications. This brings powerful autonomous analytical and investigative capabilities to the mix.

The human guides priorities, validates assumptions, and ensures tests align with real-world usage and business goals.

Built on Deterministic Foundations

At the center of the solution sits the Skyramp Core Library: deterministic capabilities designed to deliver reliability in test generation and execution.

Test Generation

Creates tests from explicit specifications, schemas, and traces using proven algorithms where the same inputs always produce the same tests, making version control meaningful and debugging straightforward. The system implements strict typing and constraint patterns that channel AI creativity through validated interfaces. These constraints don't limit capability, they ensure that when tests are generated, they're well-formed and valid.

Trace Collection

Captures full-stack, real application behavior that represents how your systems actually operate across user interactions, API calls, service communications, and data flows. Synchronized frontend and backend traces help address application behavior uncertainty from dynamic elements, async calls, and timing issues.

Containerized Execution

Runs tests in completely isolated containerized environments where dependencies are locked, state is reset between runs, and results are reproducible. Locked dependency versions ensure test frameworks and libraries remain constant across executions. Isolated environments prevent environment, state contamination and enable parallel, interference-free test execution.

Intelligent Orchestration

The Agentic Experience layer transforms how you interact with testing infrastructure. When you describe your testing needs, it maps those requirements to deterministic tools from the Skyramp core library. Each deterministic tool has specific input needs, and the orchestrator fulfills these using the artifacts and patterns it discovers by analyzing your repository and based on reasonable inferences from codebase context. If it's unable to identify any inputs or parameters with certainty, it queries the user to clarify requirements and provide any missing information.

Beyond facilitating test creation, the orchestration layer generates maintainable test code with proper modularization and reuse, recommends test types based on actual architecture and codebase analysis, and keeps tests current through automated maintenance as your application evolves. This intelligence enhances efficiency without sacrificing the deterministic reliability that testing requires.

How It Works

Skyramp Agentic Experience is designed for self-guided exploration. Start with broad discovery questions to understand capabilities, then dive into specific test types, and finally generate and execute tests for your applications.

Discover

Ask exploratory questions to understand what Skyramp can do for your specific situation:

"What types of tests can Skyramp generate?"

"How does trace-based testing work with Skyramp?"

"Can Skyramp recommend tests for this repository?"

The assistant analyzes your codebase context and explains capabilities relevant to your actual project structure, not just generic testing theory.

Explore

Dive deeper into specific test types and approaches:

"How can I create integration tests from traces using Skyramp"

"What's the best way to test my API endpoints using Skyramp?"

"How do I create E2E tests for user workflows?"

The AI explains and suggests approaches while investigating what's already in your codebase—existing tests, API specifications, authentication patterns—to provide contextualized guidance.

Act

Generate and execute tests with natural language commands:

"Generate contract tests for https://api.example.com/users using the OpenAPI spec at /specs/api.json"

"Create integration tests for my orders API that reuse code from existing tests"

"Execute the test using Skyramp in an isolated environment with all dependencies configured"

The assistant guides you through each step, prompts for any missing information it couldn't infer from codebase analysis, and delivers tests that are ready to run and easy to maintain.

Early Preview: Advanced Intelligence Capabilities

As part of the Beta release, we're introducing the Test Recommendations and Test Maintenance tools. These features are currently in 'Early Preview' stage and will continue to evolve based on real-world usage and feedback. Please review and validate the results of these tools against your specific requirements.

Intelligent Test Recommendations

One of the hardest questions in software testing is knowing where to focus limited time and resources. Which test types provide the most value for your specific architecture? Where are the highest-risk areas demanding rigorous validation?

Our test recommendation system addresses this through comprehensive repository analysis. It examines your project structure to understand whether you're building a REST API, full-stack application, or microservices system. It inventories your technology stack because framework choices influence what testing approaches work well. It discovers artifacts like OpenAPI specifications and trace files that enable specific test generation types. It analyzes authentication mechanisms, evaluates infrastructure configuration, and assesses existing test coverage to identify gaps. The result is actionable recommendations with specific test scenarios grounded in your actual code.

Automated Test Maintenance

Test suites tend to decay over time as applications evolve. Tests break as APIs and codebase change. Accumulated failures cause developers to lose confidence and effectively abandon automated testing.

Our test maintenance system addresses this by treating tests as living code requiring regular attention. It discovers Skyramp-generated tests in your repository, performs drift analysis comparing current code against the baseline that shaped original tests, executes tests to verify actual behavior, and automatically updates tests for manageable changes while providing detailed recommendations for complex breaking changes.

Join the Beta!

We're launching the Skyramp Agentic Experience for teams and individuals ready for testing that is more accessible, reliable, and maintainable. If that's you, join the beta in a few easy steps

• Access Skyramp Agentic Experience in Cursor or VS Code

• Join Skyramp Community Public Slack workspace for support and feedback.

• Get early access to new capabilities as they ship

• Share feedback, observations, and issues

• Have direct input into product direction and priorities

Whether you're testing traditional applications or AI-generated code, deterministic test generation and execution through natural language interactions can transform how your team ships software. Try it out today!

Happy Testing!