Tests pass despite GD.PrintErr(...) or GD.PushError(...)

[bjkarasek]

In documentation (https://mikeschulze.github.io/gdUnit4/testing/assert-error/) it seems that assert-error is not supported in C#. Does that mean there is no way to make tests fail when you print GD.PrintErr or GD.PushErr? Is there a workaround where I can "log" something that will make the tests fail?

[MikeSchulze]

Correct the c# API do not have a monitor to scan for Godot error logs.
Using the logging of an error to force a task to fail is very uncommon in C#.
You should use exception or return error codes to enforce error handling in your code.

[bjkarasek]

Exceptions thrown in the Godot main thread are caught by Godot itself regarding the Godot exception handling.

But you can still track exceptions when you test your components that throw an exception.

Can you please describe your use case in more detail, what you want to test.

I want to be able to implement an integration test where I create a gameplay instance, run some logic on it and then maybe do some asserts. This works fine.

But additionally if in gameplay there is some unexpected issue (in my case there was a desync between host and client as I test a multiplayer game with deterministic simulation) I want to be able to log an error or raise en exception that will fail the tests.

I tried catching the exception and somehow failing the tests this way but failed to do so, though maybe I didn't do it correctly.

[MikeSchulze]

Ok, if I understand you correctly, you have a running game server-client interaction and you have an error handling you want to test.

The best way is to test the components as unit and not as an integration test.
So I would recommend testing the error handling directly.
For example, if there is an error in the client that fails due to a server communication, the test should be structured as follows.
You should use a mocked server so that it corresponds to the error image, call the client in the test and check whether the error is handled.

[bjkarasek]

Ok, if I understand you correctly, you have a running game server-client interaction and you have an error handling you want to test.

The best way is to test the components as unit and not as an integration test. So I would recommend testing the error handling directly. For example, if there is an error in the client that fails due to a server communication, the test should be structured as follows. You should use a mocked server so that it corresponds to the error image, call the client in the test and check whether the error is handled.

Instead of testing error handling I simply want the integration test to fail if something unexpected happens. So another example would be let's say a test where I spawn a building on a map (a strategy game) and the building is actually spawned properly and it exists so asserts pass, but there is some null reference exception thrown because some internal building logic failed after it was properly spawned.

I guess it's reasonable that a test should verify a single unit, but I hoped to do maybe an unusual approach where instead of testing a single thing, I can run a complicated integration test and other than asserting some conditions, simply verify that there were no errors and no exceptions were thrown.

[MikeSchulze]

I understand your problem, but unfortunately, Godot's internal exception logic is thwarting my plans.

Therefore, I would need an exception handler in Godot with which I can forward such internally intercepted exceptions in order to be able to react to them.

For GDScript I had achieved this with a lot of effort with a hack via the Godot.log. I didn't want to install such a hack again ;)

[bjkarasek]

I understand your problem, but unfortunately, Godot's internal exception logic is thwarting my plans.

Therefore, I would need an exception handler in Godot with which I can forward such internally intercepted exceptions in order to be able to react to them.

For GDScript I had achieved this with a lot of effort with a hack via the Godot.log. I didn't want to install such a hack again ;)

That's unfortunate. I guess I'll try to somehow hack it for a partial solution so that at least the exceptions that I throw will fail the tests.

Still thank you for your time and for a great project :)

[MikeSchulze]

Update

I was able to hook into the Godot Exception handling by prototyping an exception hook handler.

    [TestCase]
    public void Example()
    {
        var sceneTree = (SceneTree)Engine.GetMainLoop();
        sceneTree.Root.AddChild(new ExceptionTestNode());
    }


    // Test class to verify the interceptor
    public partial class ExceptionTestNode : Node
    {
        public override void _Ready() =>
            // Trigger a test exception
            throw new InvalidOperationException("This is a internal exception.");
    }

it aborts the test now and the report contains the exception source

[MikeSchulze]

Update

You can now test with release candidate 6 the exception hook.
Update your project

    <PackageReference Include="gdUnit4.api" Version="4.4.0-rc*"/>
    <PackageReference Include="gdUnit4.test.adapter" Version="2.1.0-rc*"/>

The hock will now report exceptions naturally hidden by Godot main thread as test failure.
Godot PushError and PrintErr are not affected, you should use exceptions to propagate unexpected behaviors.

Please give me feedback, Thanks

[bjkarasek]

Update

You can now test with release candidate 6 the exception hook. Update your project

    <PackageReference Include="gdUnit4.api" Version="4.4.0-rc*"/>
    <PackageReference Include="gdUnit4.test.adapter" Version="2.1.0-rc*"/>

The hock will now report exceptions naturally hidden by Godot main thread as test failure. Godot PushError and PrintErr are not affected, you should use exceptions to propagate unexpected behaviors.

Please give me feedback, Thanks

Amazing! Great to hear that it can be done! I tested it a bit and it seems to be working well, however it only works when I run tests from JetBrains Rider. If I run them from the Godot Editor, then they won't fail.

[MikeSchulze]

I have reopened the issue because correctly caught exceptions are still reported as errors, but this should not be the case.

 public override void _Process(double delta)
    {
        // we throw an example exception
        if (frameCount == 10)
            throw new InvalidProgramException("Exception during scene processing");
        try
        {
            if (frameCount == 5)
                // is reported too, but should not
                throw new InvalidProgramException("Exception during scene processing inside a catch block");
        }
        catch (InvalidProgramException)
        {
            // ignore
        }

        frameCount++;
    }

[MikeSchulze]

For running inside Godot the Exception hook is not enabled.

[MikeSchulze]

I'm actually struggling, I opened a discussion here

[MikeSchulze]

Update

The exceptions can now be tracked by using these two attributes on test level.
Will be available in the upcoming release 4.4.0

/// <summary>
///     Indicates that a test method or class should monitor exceptions that occur during Godot's main thread execution.
///     This attribute enables monitoring of exceptions that are caught by CSharpInstanceBridge.Call, which handles
///     exceptions from Godot callbacks and scene tree processing.
/// </summary>
/// <remarks>
///     This attribute is particularly useful for:
///     <list type="bullet">
///         <li>Testing exception handling in Godot node callbacks (_Ready, _Process, etc.)</li>
///         <li>Verifying exceptions during scene tree operations</li>
///         <li>Monitoring exceptions that would normally be caught silently by Godot's bridge</li>
///     </list>
///     Example usage for a test method:
///     <code>
/// [TestCase]
/// [GodotExceptionMonitor]
/// public void TestNodeThrowsInReady()
/// {
///     // This will capture the exception thrown in the node's _Ready method
///     var node = new MyNode();
///     AddChild(node);
/// }
/// </code>
///     Example usage for a test class:
///     <code>
/// [TestSuite]
/// [GodotExceptionMonitor]
/// public class MyNodeTests
/// {
///     // All test methods in this class will monitor Godot exceptions
///     [TestCase]
///     public void TestSceneProcessing()
///     {
///         // Exceptions during scene processing will be monitored
///         var scene = SceneLoader.Load("res://my_scene.tscn");
///     }
/// }
/// </code>
/// </remarks>
[AttributeUsage(AttributeTargets.Method | AttributeTargets.Class, Inherited = false)]
public class GodotExceptionMonitorAttribute : Attribute
{
}

and

/// <summary>
///     Specifies that a test method is expected to throw an exception of the given type. The test will pass only if:
///     <list type="bullet">
///         <li>The expected exception type is thrown</li>
///         <li>The exception message matches (if specified)</li>
///         <li>The exception occurs at the expected file and line (if specified)</li>
///     </list>
/// </summary>
/// <remarks>
///     This attribute can be used in several ways:
///     <list type="bullet">
///         <li>Basic exception type verification</li>
///         <li>Exception type and message verification</li>
///         <li>Full exception verification including source location</li>
///     </list>
///     Multiple attributes can be applied to handle different possible exceptions:
///     <code>
/// [TestCase]
/// [ThrowsException(typeof(ArgumentNullException))]
/// [ThrowsException(typeof(InvalidOperationException))]
/// public void TestMethodCanThrowMultipleExceptions()
/// {
///     // Test logic that might throw either exception
/// }
/// </code>
///     The attribute supports Godot-specific exceptions when used with <see cref="GodotExceptionMonitorAttribute" />:
///     <code>
/// [TestCase]
/// [GodotExceptionMonitor]
/// [ThrowsException(typeof(InvalidOperationException), "Node not found", "/scenes/test_scene.cs", 25)]
/// public void TestGodotException()
/// {
///     // Test Godot code that throws during scene processing
/// }
/// </code>
///     Basic usage example:
///     <code>
/// [TestCase]
/// [ThrowsException(typeof(ArgumentNullException))]
/// public void TestNullArgument()
/// {
///     string? text = null;
///     text.Length; // Will throw ArgumentNullException
/// }
/// </code>
///     Message verification example:
///     <code>
/// [TestCase]
/// [ThrowsException(typeof(ArgumentException), "Value cannot be zero")]
/// public void TestSpecificError()
/// {
///     throw new ArgumentException("Value cannot be zero");
/// }
/// </code>
///     Location verification example:
///     <code>
/// [TestCase]
/// [ThrowsException(typeof(InvalidOperationException), "Operation failed", "TestClass.cs", 42)]
/// public void TestExceptionLocation()
/// {
///     throw new InvalidOperationException("Operation failed");
/// }
/// </code>
/// </remarks>
/// <seealso cref="GodotExceptionMonitorAttribute" />
/// <seealso cref="TestCaseAttribute" />
[AttributeUsage(AttributeTargets.Method, AllowMultiple = true, Inherited = false)]
public class ThrowsExceptionAttribute : GodotExceptionMonitorAttribute
{
    private readonly Type expectedExceptionType;
    private readonly string? expectedFileName;
    private readonly int? expectedLineNumber;
    private readonly string? expectedMessage;

    /// <summary>
    ///     Initializes a new instance of the ThrowsExceptionAttribute with the specified exception type.
    /// </summary>
    /// <param name="expectedExceptionType">The Type of exception that is expected to be thrown</param>
    public ThrowsExceptionAttribute(Type expectedExceptionType) => this.expectedExceptionType = expectedExceptionType;


    /// <summary>
    ///     Initializes a new instance of the ThrowsExceptionAttribute with the specified exception type
    ///     and expected exception message.
    /// </summary>
    /// <param name="expectedExceptionType">The Type of exception that is expected to be thrown</param>
    /// <param name="expectedMessage">The expected message of the thrown exception</param>
    public ThrowsExceptionAttribute(Type expectedExceptionType, string expectedMessage)
    {
        this.expectedExceptionType = expectedExceptionType;
        this.expectedMessage = expectedMessage;
    }

    /// <summary>
    ///     Initializes a new instance of the ThrowsExceptionAttribute with the specified exception type
    ///     and expected exception message.
    /// </summary>
    /// <param name="expectedExceptionType">The Type of exception that is expected to be thrown</param>
    /// <param name="expectedMessage">The expected message of the thrown exception</param>
    /// <param name="expectedLineNumber">The expected line of exception is thrown</param>
    public ThrowsExceptionAttribute(Type expectedExceptionType, string expectedMessage, int expectedLineNumber)
    {
        this.expectedExceptionType = expectedExceptionType;
        this.expectedMessage = expectedMessage;
        this.expectedLineNumber = expectedLineNumber;
    }

    /// <summary>
    ///     Initializes a new instance of the ThrowsExceptionAttribute with the specified exception type
    ///     and expected exception message.
    /// </summary>
    /// <param name="expectedExceptionType">The Type of exception that is expected to be thrown</param>
    /// <param name="expectedMessage">The expected message of the thrown exception</param>
    /// <param name="expectedFileName">The expected file of exception is thrown</param>
    /// <param name="expectedLineNumber">The expected line of exception is thrown</param>
    public ThrowsExceptionAttribute(Type expectedExceptionType, string expectedMessage, string expectedFileName, int expectedLineNumber)
    {
        this.expectedExceptionType = expectedExceptionType;
        this.expectedMessage = expectedMessage;
        this.expectedFileName = expectedFileName;
        this.expectedLineNumber = expectedLineNumber;
    }

Real Example

namespace GdUnit4.Tests.Core.Execution.Monitoring;

using System;
using System.Threading.Tasks;

using GdUnit4.Core.Execution.Exceptions;

using Godot;

[TestSuite]
public partial class GodotExceptionMonitorTest
{
    [GodotTestCase]
    [ThrowsException(typeof(InvalidOperationException), "TestNode '_Ready' failed.",
        "/src/core/execution/monitoring/GodotExceptionMonitorTest.cs", 62)]
    public void CatchExceptionOnAddingNodeToSceneTree()
    {
        var sceneTree = (SceneTree)Engine.GetMainLoop();
        sceneTree.Root.AddChild(new TestNode());
    }

    [GodotTestCase]
    [ThrowsException(typeof(InvalidProgramException), "Exception during scene processing",
        "src/core/resources/scenes/TestSceneWithExceptionTest.cs", 22)]
    public async Task CatchExceptionOnSceneTreeProcessing()
    {
        var sceneRunner = ISceneRunner.Load("res://src/core/resources/scenes/TestSceneWithExceptionTest.tscn", true);
        // run scene, it will throw an InvalidProgramException at frame 10
        await sceneRunner.SimulateFrames(10);
    }

    [GodotTestCase]
    [GodotExceptionMonitor]
    public async Task MonitorOnExceptionsButNotThrows()
    {
        var sceneRunner = ISceneRunner.Load("res://src/core/resources/scenes/TestSceneWithExceptionTest.tscn", true);
        // run scene
        await sceneRunner.SimulateFrames(6);
    }

    [GodotTestCase]
    [ThrowsException(typeof(InvalidOperationException), "Test Exception",
        "src/core/resources/scenes/TestSceneWithExceptionTest.cs", 14)]
    public void CatchExceptionIsThrownOnSceneInvoke()
    {
        var runner = ISceneRunner.Load("res://src/core/resources/scenes/TestSceneWithExceptionTest.tscn");

        runner.Invoke("SomeMethodThatThrowsException");
    }

    [GodotTestCase]
    [ThrowsException(typeof(TestFailedException), "Testing Godot PushError",
        "src/core/execution/monitoring/GodotExceptionMonitorTest.cs", 54)]
    public void PushErrorAsTestFailure() => GD.PushError("Testing Godot PushError");


    // Test class to verify the interceptor
    public partial class TestNode : Node
    {
        public override void _Ready() =>
            // Trigger a test exception
            throw new InvalidOperationException("TestNode '_Ready' failed.");
    }
}