Authoring Scenarios

This page explains how to write testing metadata for SolidX.

Where Scenarios Live

Testing definitions live inside a module metadata JSON file under the testing key.

At a high level, the shape looks like this:

{
  "testing": {
    "specs": ["path/to/register-test-specs.js"],
    "roles": [],
    "users": [],
    "data": [],
    "scenarios": []
  }
}

Top-Level Testing Keys

Quick reference:

• scenarios: Executable scenario definitions for API, UI, or mixed testing flows.
• data: Fixture records loaded before execution and referenced through ${data:...} interpolation.
• users: Test user definitions created during solidctl test data --load.
• roles: Test role definitions with permission bindings.
• specs: Custom spec registration modules used by test.spec.

specs

Paths to custom spec registration modules.

Use these when you want to invoke test.spec from a scenario.

roles

Optional role definitions that solidctl test data --load creates in the database.

Each entry names a role and lists the permissions to bind to it:

{
  "name": "Editor",
  "permissions": [
    "BookController.*",
    "LoanController.*",
    "DashboardController.findMany",
    "DashboardController.findOne"
  ]
}

Fields:

• name (required): Role name, created if it does not already exist.
• permissions (optional): List of permission names to bind to this role.

Permission syntax:

• Exact: ControllerName.methodName
• Wildcard: ControllerName.*
• Global: *

Roles are seeded idempotently: created if absent, left unchanged if they already exist.

users

Optional user definitions that solidctl test data --load creates in the database.

Each entry provides credentials and an optional list of roles to assign:

{
  "username": "libTestEditor",
  "email": "libTestEditor@test.local",
  "password": "Test@1234",
  "fullName": "Library Test Editor",
  "roles": ["Editor"]
}

Fields:

• username (required): Unique username.
• email (required): Email address.
• password (required): Login password.
• fullName (optional): Display name.
• mobile (optional): Mobile number.
• roles (optional): List of role names to assign. Declare these in testing.roles first.

Users are skipped if a user with the same username already exists. They are not deleted during teardown.

A typical module defines one user per role category to support access-level scenario coverage:

"users": [
  { "username": "libTestEditor", "email": "libTestEditor@test.local", "password": "Test@1234", "roles": ["Editor"] },
  { "username": "libTestViewer", "email": "libTestViewer@test.local", "password": "Test@1234", "roles": ["Viewer"] },
  { "username": "libTestNoRole", "email": "libTestNoRole@test.local", "password": "Test@1234", "roles": ["NoRole"] }
]

data

Test fixture records to load before execution.

Each record typically contains:

• modelUserKey
• recUserKeyValue
• data

Real project pattern:

• Use testing.data as a reusable fixture library.
• Express relations through ...UserKey fields such as stateUserKey, cityUserKey, or templateMasterUserKey.
• Keep recUserKeyValue stable so scenarios can reference the fixture by name.

scenarios

The executable scenarios for the module.

This is the core of the testing system.

Scenario Shape

{
  "id": "api-authenticate-success",
  "name": "Authenticate succeeds",
  "type": "api",
  "params": {
    "username": "alice"
  },
  "tags": ["smoke"],
  "timeoutMs": 30000,
  "retries": 1,
  "steps": []
}

Important Fields

• id: Stable scenario identifier.
• name: Optional human-readable label.
• type: api, ui, or mixed.
• params: Free-form scenario parameters.
• tags: Labels for filtering.
• timeoutMs: Scenario timeout override.
• retries: Scenario retry count.
• steps: The executable flow.

Step Styles

Steps can be written in two ways.

{
  "given": { "op": "ui.goto", "with": { "url": "/login" } }
}

{
  "op": "util.log",
  "with": { "message": "Starting scenario" }
}

The engine normalizes both forms before execution, so there is no runtime difference between them.

Use given for setup steps, when for the action being tested, then for assertions, and and to continue the previous phase without repeating it.

then also accepts an array, which is useful when you want to group multiple assertions after a single action:

{
  "then": [
    { "op": "assert.httpStatus", "with": { "is": 201 } },
    { "op": "assert.jsonPath", "with": { "from": "${res:created}", "path": "$.name", "equals": "Test" } }
  ]
}

Step Fields

Each executable step can include:

• op: Required operation name.
• with: Operation-specific input.
• saveAs: Save the step result into the resource store.
• name: Optional reporting label.
• spec: Custom spec id for test.spec.
• timeoutMs: Per-step timeout override.

Interpolation

Before each step runs, the engine resolves interpolation tokens.

Supported token families include:

• ${env:NAME} for environment variables.
• ${params.foo} for scenario params.
• ${res:path.to.value} for saved runtime resources.
• ${data:modelUserKey["recUserKeyValue"].field} for test data lookups.

Examples:

{
  "params": {
    "state": "${data:stateMaster[\"Maharashtra\"].name}"
  }
}

{
  "when": {
    "op": "api.request",
    "with": {
      "method": "POST",
      "url": "${env:API_BASE_URL}/api/example",
      "json": {
        "stateName": "${params.state}",
        "city": "${data:cityMaster[\"New Delhi\"].name}"
      }
    }
  }
}

Referencing Test Data

Test data is indexed as:

data:<modelUserKey>["<recUserKeyValue>"]

Useful patterns:

• .fieldName to access a single field.
• ._rec to access the whole underlying object.

Example:

"${data:cityMaster[\"New Delhi\"]._rec}"

The venue module uses this pattern heavily:

• Master fixtures such as stateMaster["Maharashtra"].
• Relation-aware fixtures such as cityMaster["Mumbai"].
• File-upload fixtures such as lead["LeadWithFile"]._rec.

That keeps scenarios short and readable because large request bodies stay in testing.data instead of being repeated inline.

Using saveAs

When a step returns a value you want later, use saveAs.

The standard pattern is to save the full login response as loginSuccess:

{
  "given": {
    "op": "api.request",
    "with": {
      "method": "POST",
      "url": "${env:TEST_API_BASE_URL}/api/iam/authenticate",
      "json": {
        "email": "libTestEditor@test.local",
        "username": "",
        "password": "Test@1234"
      }
    },
    "saveAs": "loginSuccess"
  }
}

Later steps read the token via:

"Authorization": "Bearer ${res:loginSuccess.bodyJson.data.accessToken}"

Saving the full response rather than only the token preserves the rest of the payload for later assertions or follow-up calls.

Scenario Chaining With util.require

A common SolidX pattern is:

1. Create a reusable bootstrap scenario, usually authentication.
2. Save its result with saveAs.
3. Start later scenarios with util.require.
4. Fail early with a helpful message if the prerequisite resource is missing.

Example:

{
  "given": {
    "op": "util.require",
    "with": {
      "resource": "loginSuccess",
      "message": "Run scenario api-authenticate-success first to create loginSuccess."
    }
  }
}

The venue module uses this pattern throughout its authenticated API scenarios.

Custom Specs

When built-in operations are not enough, use test.spec.

{
  "when": {
    "op": "test.spec",
    "spec": "example.customHealth",
    "with": {
      "input": {
        "url": "${env:API_BASE_URL}/health"
      }
    },
    "saveAs": "custom.health"
  }
}

Custom specs are registered through testing.specs.

Real project pattern:

{
  "specs": ["testing/register-test-specs.js"]
}

That registrar then maps ids such as venue.customHealth to concrete implementations.

The venue example also shows a helpful convention where with.input includes both:

• Direct input values, such as a health URL.
• A resource path, such as authResourcePath.

This lets a custom spec combine metadata input with previously saved runtime state.

Authoring Recommendations

Recommended practices:

• Keep scenario ids stable and descriptive.
• Use tags such as smoke, regression, or auth.
• Keep API and UI scenarios small and composable.
• Use generate module or seed workflows consistently before execution.
• Prefer saveAs plus interpolation over hard-coded chained values.
• Reserve test.spec for genuine escape-hatch cases.
• Prefer reusable fixture libraries in testing.data over repeating large payloads inline.
• Make scenario prerequisites explicit with util.require.
• Keep one small authentication bootstrap scenario per module when many scenarios need auth.

When To Use API vs UI vs Mixed

• Use api when you want fast, direct, backend-facing verification.
• Use ui when you want browser-level user-flow verification.
• Use mixed when your flow crosses both layers and it would be artificial to separate them.