How I am setting up my test suite for a Statamic addon

Why should we test our addons or packages in the first place? It does take time and can be complicated from time to time. So why to do it in the first place?

There are a lot of benefits! Besides yourself sleeping better at night, it will give you the confidence to refactor your code and will simply preserve you from shipping addons with bugs, after you made some changes.

For small addons, like a custom fieldtype in Statamic, I wouldn't write tests. But for bigger addons, I would and will. If you will is up to you. In my case, I am working on a Statamic Ecommerce Addon and definitely want a solid test suite ...

I will explain step by step, how to set up your suite (Basic knowledge about Laravel and Statamic packages is required).

Beta Status

At the time of writing, Statamic v3 is still is in Beta. Breaking changes can happen, but in case they do, I will try to update as soon as possible.

Right at this moment, getting your tests up and running is quite a lot of work. I am positive that this state will change soon, but let's see how to get it working right now.

Basic Structure

You can set up your structure in multiple ways, but this one is pretty common and used a lot in the Laraevel community. Let's create this folder structure. We will leave the files empty right now and will fill them will live from step to step.

Let's call our addon "Amazing Statamic Addon". You can choose whatever you want.

statamic-addon
--| src
--- AmazingStatamicAddonServiceProvider.php
--- AmazingStatamicAddon.php
--| tests
----| Feature
----| Unit
---- TestCase.php
composer.json
.gitignore

How to set up composer

Just to be on the same page, let's set up our composer file together and start with, what the Statamic docs are giving us.

{
    "name": "jonassiewertsen/amazing-statamic-addon",
    "description": "Just amazing. What else?",
    "type": "statamic-addon",
    "autoload": {
        "psr-4": {
            "Jonassiewertsen\\AmazingStatamicAddon\\": "src"
        }
    },
    "authors": [
        {
            "name": "Jonas Siewertsen"
        }
    ],
    "support": {
        "email": "support@yourmailgoeshere.com"
    },
    "extra": {
        "statamic": {
            "name": "Amazing Statamic Addon",
            "description": "Just Amazing. What else?"
        },
        "laravel": {
            "providers": [
                "Jonassiewertsen\\AmazingStatamicAddon\\AmazingStatamicAddonServiceProvider"
            ]
        }
    }
}

This is a great starting point, but let's add a few dependencies to run our tests with PHPUnit.

	...
	"support": {
        "email": "support@yourmailgoeshere.com"
    },
	"require": {
        "php": "^7.2",
        "illuminate/support": "^6.8",
        "statamic/cms": "^3.0@beta"
    },
    "require-dev": {
        "orchestra/testbench": "^4.0",
        "phpunit/phpunit": "^8.5"
    },
 	"autoload-dev": {
    	"psr-4": {
      		"Jonassiewertsen\\AmazingStatamicAddon\\Tests\\": "tests"
    	}
  	},
	"extra": {
	...

We want to make sure, that we do have PHP 7.2 at least.

Ilumminate support does pull in Laravel, which we need to run Statamic. To run the tests, we do need the orchestra testbench a swell and of course PHPUnit itself.

Composer install After updating our composer file, let's install all those dependencies by running composer:

composer install

PHP Unit

Let's create a PHPUnit config file with the name "phpunit.xml.dist" inside the root folder of your package. This can be customized for your own needs, but this should get you going:

<?xml version="1.0" encoding="UTF-8"?>
<phpunit bootstrap="vendor/autoload.php"
         backupGlobals="false"
         backupStaticAttributes="false"
         colors="true"
         verbose="true"
         convertErrorsToExceptions="true"
         convertNoticesToExceptions="true"
         convertWarningsToExceptions="true"
         processIsolation="false"
         stopOnFailure="false">
    <testsuites>
        <testsuite name="Unit">
            <directory suffix="Test.php">tests/Unit/</directory>
        </testsuite>
        <testsuite name="Feature">
            <directory suffix="Test.php">tests/Feature/</directory>
        </testsuite>
    </testsuites>
    <filter>
        <whitelist>
            <directory suffix=".php">src/</directory>
        </whitelist>
    </filter>
    <php>
        <env name="APP_ENV" value="testing"/>
        <env name="APP_KEY" value="AckfSECXIvnK5r28GVIWUAxmbBSjTsmF"/>
        <env name="BCRYPT_ROUNDS" value="4"/>
        <env name="DB_CONNECTION" value="sqlite" />
        <env name="DB_DATABASE" value=":memory:" />
        <env name="CACHE_DRIVER" value="array"/>
        <env name="SESSION_DRIVER" value="array"/>
        <env name="QUEUE_DRIVER" value="sync"/>
        <env name="MAIL_DRIVER" value="array"/>
    </php>
</phpunit>

To see if we did set up PHP Unit correctly, we can run it in the command line:

./vendor/bin/phpunit It should say 'No tests executed!', but as well show the path to our configuration file we just created. Great!

Our First Unit Test

Let's create our first Unit test, to see if everything is working as expected.

Inside the Unit folder, let's hook up a dummy test. I did name the class FirstTest.php

<?php

namespace Tests\Unit;


use PHPUnit\Framework\TestCase;

class FirstTest extends TestCase
{
    /** @test */
    public function dummy_test()
    {
        $this->assertTrue(true);
    }
}

Running the test

./vendor/bin/phpunit

and we got it to green.

Laravel

Statamic is a Laravel package itself. So to get the full Laravel magic, we already did pull in the illuminate package. But calling Laravel specific commands, like the artisan command would only throw an error.

Orchestra Testbench

This is where the Orchestra Testbench comes in. Our amazing addon does not live inside a "normal" Laravel applications, so it can't access the Laravel magic But living without the Laravel magic ... nahhh. Let's fix it:

First, we will open our empty TestCase.php file and write our own TestCase class which is extending the Orchestra Testbench. Just in case a little more setup is needed. I got the feeling we do ...

<?php

namespace Jonassiewertsen\AmazingStatamicAddon\Tests;

use Orchestra\Testbench\TestCase as OrchestraTestCase;

class TestCase extends OrchestraTestCase
{
    // More setup if needed
}

The Orchestra Testbench is extending the PHP Unit TestCase class, so now the Laravel magic is in place.

Test Case

To use the Testbench, we will extend all our tests with our own TestCase.php from now on. You could extend all your tests with the testbench directly, but then you need to setup up everything manually for every test. If you want to, feel welcome to do it. If not, follow me on the "sunny side of the street"

// Inside our FirstTest.php
...
use Jonassiewertsen\AmazingStatamicAddon\Tests\TestCase;

class FirstTest extends TestCase
{
...

This is the place where you could make use of factories if you should work with a database. This is a great place for some testing helpers as well. Just saying.

Let's run our first test again. That does work. "What a feeling".

Let's set up a web route for our addon and test it. We add another test to our 'FirstTest.php'. The route is not set up yet, so we will run into a 404 error as expected.

/** @test */
public function route_testing()
{
	$this->get('amazing')->assertOk();
}

Amazing Statamic Addon Service Provider

Inside our root folder, we create another folder, named 'routes' and create the web.php inside of it.

<?php

Route::get('amazing', function() {
   return 'How amazing';
});

To load the routes in our addon, we need to tell our Service Provider to do so. This approach is sepcial for Statamic addons.

<?php

namespace Jonassiewertsen\AmazingStatamicAddon;

use Statamic\Providers\AddonServiceProvider;

class AmazingStatamicAddonServiceProvider extends AddonServiceProvider
{
    protected $routes = [
        'web' => __DIR__.'/../routes/web.php',
    ];
}

As well we need to load our ServiceProvider inside our TestCase. Otherwise we couldn't test those things, we do make available via our ServiceProvider inside our Package/Addon. Right now it's important to test our routes.

Our TestCase.php needs to look like this now:

...

class TestCase extends OrchestraTestCase
{
    protected function getPackageProviders($app)
    {
        return [
          \Jonassiewertsen\AmazingStatamicAddon\AmazingStatamicAddonServiceProvider::class,
        ];
    }
}

The interesting part will start now. Get ready to party!

Running the tests will lead us into a BindingResolutionException. Okay. Let's fix it!

Fixing Binding Resolution Exception

We need to pull in the Statamic Service Provider as well, to get the Statamic magic on top of the Laravel magic.

protected function getPackageProviders($app)
    {
        return [
            \Statamic\Providers\StatamicServiceProvider::class,
            \Jonassiewertsen\AmazingStatamicAddon\AmazingStatamicAddonServiceProvider::class,
        ];
    }

To prevent PHPUnit from saying, that he couldn't find the Statamic class, we need to add Statamic as an alias, by adding the following into our TestCase.php

protected function getPackageAliases($app)
    {
        return [
            'Statamic' => Statamic::class,
        ];
    }

One Step closer. Let's fix the Exception handler error which should pop up now.

Exception Handler

This is easily fixed, as pointed out by Duncan to me. Let's find the file, and copy it inside our root test folder (ExceptionHandler.php).

<?php

namespace App\Exceptions;

use Exception;
use Illuminate\Foundation\Exceptions\Handler as ExceptionHandler;

/**
 * Statamic extends the Laravel app's exception handler.
 * In a test environment, there would be no such file, so here it is.
 */
class Handler extends ExceptionHandler
{
    /**
     * A list of the exception types that are not reported.
     *
     * @var array
     */
    protected $dontReport = [
        //
    ];

    /**
     * A list of the inputs that are never flashed for validation exceptions.
     *
     * @var array
     */
    protected $dontFlash = [
        'password',
        'password_confirmation',
    ];

    /**
     * Report or log an exception.
     *
     * @param  \Exception  $exception
     * @return void
     */
    public function report(Exception $exception)
    {
        parent::report($exception);
    }

    /**
     * Render an exception into an HTTP response.
     *
     * @param  \Illuminate\Http\Request  $request
     * @param  \Exception  $exception
     * @return \Illuminate\Http\Response
     */
    public function render($request, Exception $exception)
    {
        return parent::render($request, $exception);
    }
}

Load the file inside the TestCase.php:

protected function getPackageAliases($app)
{
	return [
		'Statamic' => Statamic::class,
	];
}

So no more unexpected errors, but we still don't get green in our tests. Damn ...

Environment Set Up

The last remaining step to get our test to green is to get the environment set up.

After doing that, our complete TestCase.php needs to look like this:

<?php

namespace Jonassiewertsen\AmazingStatamicAddon\Tests;

use Orchestra\Testbench\TestCase as OrchestraTestCase;
use Statamic\Statamic;

class TestCase extends OrchestraTestCase
{
    protected function setUp(): void
    {
        require_once(__DIR__.'/ExceptionHandler.php');

        parent::setUp();
    }

    protected function getPackageProviders($app)
    {
        return [
            \Statamic\Providers\StatamicServiceProvider::class,
            \Jonassiewertsen\AmazingStatamicAddon\AmazingStatamicAddonServiceProvider::class,
        ];
    }

    protected function getPackageAliases($app)
    {
        return [
            'Statamic' => Statamic::class,
        ];
    }

    protected function getEnvironmentSetUp($app)
    {
        parent::getEnvironmentSetUp($app);

        $app->make(\Statamic\Extend\Manifest::class)->manifest = [
            'jonassiewertsen/amazing-statamic-addon' => [
                'id' => 'jonassiewertsen/amazing-statamic-addon',
                'namespace' => 'Jonassiewertsen\\AmazingStatamicAddon\\',
            ],
        ];
    }
}

That's it

Our basic test suite is set up and ready to go! Yippieee.

Thanks for staying with me for soo long. There are one or two more things I would like to cover, but it feels like this post is already to looooooooong. So let's call it a day for now.

Thanks a lot to Jason from the Statamic Gentlemen to get this up and running! Thank you!