Testing AI Positron Actions with XSpec in Oxygen XML Editor

Introduction

As AI-driven functionality becomes more common in DocBook, DITA, and custom XML workflows, the need for reliable automated testing grows with it. Oxygen XML Editor addresses this by adding XSpec-based AI action testing support. The New Document Wizard now includes an AI Action Test template, designed to help developers write and validate AI-driven XML modification, enrichment, and transformation workflows.

This post covers how to use the AI Action Test template, where to find sample XSpec tests, and practical examples of AI actions you can test.

AI Action Test Template in Oxygen XML Editor

The New Document Wizard now includes a dedicated AI Action Test file template, found under the AI Positron Templates folder. It provides a ready-made starting point for writing XSpec tests specific to AI actions, combining standard XML testing practices with the specifics of AI-driven processing.

Key Features

• Guided XSpec test creation: The template includes scaffolding with comments and structure, making it easier to get started for those new to XSpec or AI-specific testing.
• Rich document context: When creating a new test, you can add a summary describing your intention and attach reference files in multiple formats (text, XML, Markdown, Word, PDF, images) using copy-paste or drag-and-drop.
• Initial structure generation: The AI component generates a skeleton for your XSpec document, with steps and comments to guide the test-writing process.
• Sample library: The AI Positron Assistant sample repository contains ready-to-adapt XSpec test scenarios covering a range of AI actions.

XSpec and AI Actions

XSpec is a BDD-style testing framework for the XML world. It lets you declare expected outcomes and verify that your XSLT, XQuery, or Schematron logic behaves as intended. As AI actions begin transforming XML content, writing tests to confirm these automations produce predictable results becomes essential.

The AI Action Test template integrates this testing approach directly into Oxygen, covering both traditional XML processing and AI-powered transformations.

Real-World Examples

The following scenarios are based on examples from the sample AI action tests provided by the Oxygen team.

Generate Short Description

Action: Generate a DITA <shortdesc> element for a topic using an AI action.

Sample XSpec test:

<x:description xmlns:x="http://www.jenitennison.com/xslt/xspec"
    xmlns:xs="http://www.w3.org/2001/XMLSchema"
    xmlns:ai="http://www.oxygenxml.com/ai/function"
    stylesheet="http://oxygenxml/empty.xsl">

    <x:scenario label="Test 'Generate short description'">
        <x:call function="ai:invoke-action">
            <x:param>action.short.description</x:param>
            <x:param select="''"/>
            <x:param select="unparsed-text('resources/apples.dita', 'UTF-8')"/>
        </x:call>

        <x:expect label="check XML" test="parse-xml($x:result)/*/name()" as="xs:string">shortdesc</x:expect>

        <x:expect label="Generated 'shortdesc' checked using AI" test="ai:transform-content(
            'Return `yes` if this: \n\n ' ||
            $x:result ||
            '\n\n is exactly the element with the name `shortdesc` containing a short description of \n\n' ||
            unparsed-text('resources/apples.dita', 'UTF-8'),
            '')" select="'yes'"/>
    </x:scenario>
</x:description>

Why it matters: The test first checks that the returned XML element is indeed a shortdesc, then uses ai:transform-content to semantically verify the generated content is an accurate short description of the source topic.

Code Generation

Action: Generate XSLT code from an XML fragment description.

Sample XSpec test:

<x:scenario label="Test #1 'Code generation'">
    <x:call function="ai:invoke-action">
        <x:param>action.generate.code.xslt</x:param>
        <x:param select="''"/>
        <x:param select="unparsed-text('resources/sample-xml-fragm.txt', 'UTF-8')"/>
    </x:call>

    <x:expect label="Similarity between generated and expected code checked using AI" test="ai:transform-content(
        'Return `yes` if this: \n\n ' ||
        $x:result ||
        '\n\n is similar to the following text \n\n' ||
        unparsed-text('resources/expected-generated-code.txt', 'UTF-8'),
        '')" select="'yes'"/>
</x:scenario>

Why it matters: The AI-generated XSLT is compared against a reference output using ai:transform-content, which provides semantic similarity checking rather than strict text matching.

Refactoring Suggestions

Action: Suggest refactoring improvements for an existing XSLT stylesheet.

Sample XSpec test:

<x:scenario label="Test #2 'Refactoring suggestions'">
    <x:call function="ai:invoke-action">
        <x:param>action.suggest.refactoring.xslt</x:param>
        <x:param select="''"/>
        <x:param select="unparsed-text('resources/invoice-receipt.xsl', 'UTF-8')"/>
    </x:call>

    <x:expect label="Similarity between generated and expected refactoring suggestions checked using AI" test="ai:transform-content(
        'Return `yes` if this: \n\n ' ||
        $x:result ||
        '\n\n is similar to the following text \n\n' ||
        unparsed-text('resources/expected-refactor-suggestion.txt', 'UTF-8'),
        '')" select="'yes'"/>
</x:scenario>

Why it matters: Ensures that refactoring advice stays relevant and consistent as the underlying AI model or action configuration evolves.

Explain Code

Action: Produce a human-readable explanation of an XSLT stylesheet, including a ## Summary section.

Sample XSpec test:

<x:scenario label="Test #1 'Explain code' - checked on a fragment">
    <x:call function="ai:invoke-action">
        <x:param>action.explain.code</x:param>
        <x:param select="''"/>
        <x:param select="unparsed-text('resources/product-requests.xsl', 'UTF-8')"/>
    </x:call>

    <x:expect label="Similarity between generated and expected explanation checked using AI" test="ai:transform-content(
        'Return `yes` if this: \n\n ' ||
        $x:result ||
        '\n\n contains a section called `## Summary` that contains a message that is similar to the following text \n\n' ||
        '- You define a starting template.\n- You create a variable that holds a web request.\n- You pass that request to another template for further processing.',
        '')" select="'yes'"/>
</x:scenario>

Why it matters: Verifies that the explanation covers the key structural points of the stylesheet and is organized with the expected ## Summary section.

Document Code

Action: Auto-generate documentation prose for an XSLT stylesheet.

Sample XSpec test:

<x:scenario label="Test #2 'Document code'">
    <x:call function="ai:invoke-action">
        <x:param>action.document.code</x:param>
        <x:param select="''"/>
        <x:param select="unparsed-text('resources/product-requests.xsl', 'UTF-8')"/>
    </x:call>

    <x:expect label="Similarity between generated and expected documentation checked using AI" test="ai:transform-content(
        'Return `yes` if this: \n\n ' ||
        $x:result ||
        '\n\n is similar to the following text \n\n' ||
        'Defines an XSLT that sends an HTTP GET request to a specified URL. Then processes the request and outputs its content using templates.',
        '')" select="'yes'"/>
</x:scenario>

Why it matters: Confirms the generated documentation accurately summarizes the purpose of the stylesheet.

Annotate Code

Action: Insert inline XML comments into an XSLT stylesheet, annotating each meaningful block.

Sample XSpec test:

<x:scenario label="Test #3 'Annotate code'">
    <x:call function="ai:invoke-action">
        <x:param>action.annotate.code</x:param>
        <x:param select="''"/>
        <x:param select="unparsed-text('resources/product-requests.xsl', 'UTF-8')"/>
    </x:call>

    <x:expect label="The number of comments generated should be greater than 5 (i.e. 'all modules annotated')" test="count(tokenize($x:result, '&lt;!--')[position() > 1]) ge 6"/>
</x:scenario>

Why it matters: Instead of semantic similarity, this test uses a quantitative assertion — at least 6 XML comments must be present — to confirm thorough annotation coverage.

Running the Tests

XSpec tests for AI actions use the ai:transform-content() XPath function under the hood. This function sends the input content to the configured AI model and returns the transformed result, which XSpec then compares against your expected output.

The ai:invoke-action function invokes a named AI Positron action by its action ID. It accepts an optional context string as the second parameter and the main input content as the third parameter, and returns the AI-generated result as a string.

The ai:transform-content function sends a prompt to the configured AI model and returns the response as a string. In XSpec expectations, it is used to perform semantic or similarity checks by asking the AI to return yes or no. For more details, see the AI XPath Functions blog post and the AI Positron XPath functions documentation.

To run the tests directly in Oxygen:

• Open your XSpec file in Oxygen XML Editor.
• Apply the built-in XSpec Test Report transformation scenario.
• Results appear in the XSpec report view, showing which scenarios passed or failed along with a diff for any mismatches.

For a set of working examples to get started with, see the AI Positron Assistant sample repository.

Tips

• Use the summary and attachments: Clearly describe your intention and attach input/output examples to guide both the AI and future collaborators reviewing the tests.
• Follow the template comments: The generated comments include hints for refining your scenarios and expected outputs — read them before editing the scaffold.
• Integrate with CI: Add your XSpec tests to a CI pipeline so they run automatically whenever AI action configurations or underlying models change.

For step-by-step details, see the Oxygen XML Editor XSpec user guide and the XSpec wiki page on writing scenarios.

Conclusion

The AI Action Test template in Oxygen XML Editor gives you a practical way to write and run XSpec tests for AI-driven transformations. Whether you are enriching content, enforcing terminology, or classifying elements, having a test suite in place means you can update your AI actions with confidence.

To get started, open the New Document Wizard, select the AI Action Test template from the AI Positron Templates folder, and refer to the sample repository for working examples.