The Workshop: The Road to 7.3, Part 1

by Joe Ferguson

Last month as I was writing “The Workshop: Producing Packages (Part 3)” I had a feeling I would end up regretting the line “This will be the third and final installment in this series”. Sure enough I have one more topic I want to cover: upgrading to a new PHP version.

Upgrading PHP versions is enough to be its own “The Workshop” series however since 7.3 was just released last month I found it fitting to take our PHP Easy Math library and see how we can improve upon the code utilizing some new features in PHP 7.3 and any we may have missed in previous 7.x versions. This month we’re going to test drive our library through two different static analyzers specifically for catching bugs in PHP code. Static analyzers are applications that can inspect our source code and catch bugs based on predefined rules and other inspections. Think of static analyzers as an WHeopinionated review of your code.

Current Status of PHP Easy Math

You can review our progress on our example library svpernova09/php-easy-math and looking at the upgrade-php branch we can see the current state of our library. We built out our library to have two classes: Addition and Subtraction. The Addition class has two methods: add($x, $y) will return the sum of values $x plus $y, and sum(float …$numbers) which will return the sum of any number of floating point values passed into it. The Subtraction class has similar methods subtract($x, $y) and sub(float $firstNumber, ?float …$numbers) which provide the expected opposite functionality from our Addition class.

We also have test coverage for both classes to ensure our logic is correct for both methods in each class. This gives us 20 tests and 20 assertions which we automatically run against every branch and pull request via Travis-CI to ensure we know when functionality breaks. We’re currently testing PHP version 7.2 as well as the nightly build for Travis-CI which is the master branch on the PHP source repository. This will give us a heads up if an upcoming change will break our library. Since nightly isn’t a stable release we configure Travis-CI to allow the nightly build to fail considering it is not yet an official release and we want a heads up on failures.

We can run our test suite to ensure all of our tests are passing as expected. This will be our baseline before we start updating code:

$ php vendor/bin/phpunit
PHPUnit 7.3.5 by Sebastian Bergmann and contributors.

.................... 20 / 20 (100%)

Time: 39 ms, Memory: 4.00MB

OK (20 tests, 20 assertions

Fantastic! Our tests are passing and we’re confident things are working exactly as expected.

Static Analysis

Before we dive deep into running our code on PHP 7.3 it would be great if we could run something against our code to help look for compatibility issues or errors . As it turns out there are a few different static analyzer tools out there for PHP. We’re going to cover two of the more popular tools: Phan and PHPStan.

Phan

Phan is a static analyzer tool which attempts to prove incorrectness rather than correctness. Phan won’t tell you if your code is correct but it will find problems in your code such as methods, functions, traits are defined and accessible, PHP7 / PHP5 backwards compatibility, and unused code among many other features.

We’ll add Phan to our PHP Easy Math library via composer:

$ composer require phan/phan --dev
Using version ^1.1 for phan/phan
./composer.json has been updated
Loading composer repositories with package information
Updating dependencies (including require-dev)
Package operations: 11 installs, 0 updates, 0 removals
- Installing symfony/polyfill-mbstring (v1.10.0): Loading from cache
- Installing symfony/contracts (v1.0.2): Loading from cache
- Installing symfony/console (v4.2.1): Loading from cache
- Installing psr/log (1.1.0): Loading from cache
- Installing sabre/event (5.0.3): Downloading (100%)
- Installing microsoft/tolerant-php-parser (v0.0.15): Downloading (100%)
- Installing netresearch/jsonmapper (v1.4.0): Downloading (100%)
- Installing felixfbecker/advanced-json-rpc (v3.0.3): Downloading (100%)
- Installing composer/xdebug-handler (1.3.1): Downloading (100%)
- Installing composer/semver (1.4.2): Loading from cache
- Installing phan/phan (1.1.8): Downloading (100%)
Writing lock file
Generating autoload files

We use --dev in our composer require statement to ensure we don’t install Phan in any production environments since we only need it for development.

To configure Phan we need to create a basic configuration file in our project named .phan/config.php . This file tells Phan where our code is and what folders to ignore, and what plugins to use to check functionality.

File .phan/config.php:

return [
‘backward_compatibility_checks’ => true,
‘target_php_version’ => 7.3,

‘directory_list’ => [
‘src’,
‘tests’,
],

“exclude_analysis_directory_list” => [
‘vendor/‘
],

‘plugins’ => [
‘AlwaysReturnPlugin’,
‘UnreachableCodePlugin’,
‘DollarDollarPlugin’,
‘DuplicateArrayKeyPlugin’,
‘PregRegexCheckerPlugin’,
‘PrintfCheckerPlugin’,
],
];

Our target_php_version is 7.3 and we specify our src and tests folder for the directories we want Phan to scan. We ignore the vendor folder and we’ll leave the default plugins enabled for now.

It’s mentioned in the Phan documentation but the PHP extension php-ast is required for Phan to run. You can easily install this via PECL: pecl install ast.

Now we’re ready to run Phan and see what the good (or bad) news for PHP Easy Math is:

$ ./vendor/bin/phan
tests/AdditionTest.php:7 PhanUndeclaredExtendedClass Class extends undeclared class \PHPUnit\Framework\TestCase
tests/AdditionTest.php:19 PhanUndeclaredMethod Call to undeclared method \EasyMath\Tests\AdditionTest::assertEquals
tests/AdditionTest.php:35 PhanUndeclaredMethod Call to undeclared method \EasyMath\Tests\AdditionTest::assertEquals
tests/SubtractionTest.php:7 PhanUndeclaredExtendedClass Class extends undeclared class \PHPUnit\Framework\TestCase
tests/SubtractionTest.php:19 PhanUndeclaredMethod Call to undeclared method \EasyMath\Tests\SubtractionTest::assertEquals
tests/SubtractionTest.php:35 PhanUndeclaredMethod Call to undeclared method \EasyMath\Tests\SubtractionTest::assertEquals

Phan is complaining about undeclared classes and methods from PHPUnit. This should be easily solved since we told Phan to ignore vendor but we should also allow vendor/phpunit/phpunit/src in our directory_list array in ./phan/config:

‘directory_list’ => [
‘src’,
‘tests’,
‘vendor/phpunit/phpunit/src’
],

Now we can run Phan again and see what it catches:

$ php /vendor/bin/phan

No output. Great! This means we have cleared all the issues Phan found with our code base.

Note: Phan didn’t find much wrong with our library because it’s quite basic since we’re wanting to focus on working with the library as a whole instead of any specific feature. Running Phan in your code base will obviously produce different results.

Now in real world projects Phan might have found a lot more issues with the code. Typically when I start working on an older code base I'll configure Phan to use the most strict settings. You can generate a config with the most strict settings (1 to 5 scale with 1 being most strict, 5 being least strict) by using:

$ php vendor/bin/phan --init --init-level=1 --init-overwrite
Successfully initialized 'php-easy-math/.phan/config.php' with the following contents

The previous command will output the entire PHP config file Phan generated for us. If we open this file we can see that there is much more configured than we previously did. We used --init-overwrite to generate the new configuration. I would advise you start with the most strict issues to get the clearest picture of the pending issues in the code base you're working with.

Many of the issues I see Phan catch in legacy code bases (especially code build during the PHP 5 era) are errors around dead code: methods no longer being called from anywhere in the application. Dead code should be easy to remove once you have searched the code base for any usages that Phan may have missed. Another issue specifically common in PHP5 code bases is catching Exception instead of Throwable. This can be a little more difficult to resolve since error handling can vary drastically in older applications built before PHP7 introduced Throwable. Naturally the official PHP docs have a great document on the change. Another issue I see Phan catch often in legacy applications is from the breaking changes introduced in PHP7 such as removal of the ereg methods for regular expressions in favor of using Perl Compatible Regular Expressions.

PHPStan

PHPStan is another static analyzer tool similar to Phan. Phan and PHPStan offer similar features. You do not have to run both tools. I typically run Phan but I wanted to highlight alternatives as well. One feature I appreciate in PHPStan is the extensibility for it to define and check for “magic” behaviors of classes which may not be defined but created in getters and setters. For example if your application makes use of the Doctrine ORM, you can use phpstan/phpstan-doctrine which gives PHPStan the ability to understand how to process the Doctrine files in your application. Another example would be if you are working on a Symfony application you could use phpstan/phpstan-symfony which will warn you about unregistered services being accessed from the container or when a private service is being accessed from the container.

Most static analyzers perform better when your code is annotated and type hinted (this includes use of https://www.phpdoc.org/ doc blocks). While this isn’t a hard requirement to use these tools, you may be missing out on some of the features if you’re missing annotations and type hints. PHP has come along way over the years and if you’re completely new to these ideas you should check out Your Own Custom Annotations - More than Just Comments! — SitePoint and php.net for information about types: PHP: Function arguments - Manual

We’ll install PHPStan as a development dependency like we did with Phan:

$ composer require --dev phpstan/phpstan
Using version ^0.10.6 for phpstan/phpstan
./composer.json has been updated
Loading composer repositories with package information
Updating dependencies (including require-dev)
Package operations: 13 installs, 0 updates, 0 removals
- Installing ocramius/package-versions (1.3.0): Downloading (100%)
- Installing symfony/finder (v4.2.1): Downloading (100%)
- Installing phpstan/phpdoc-parser (0.3): Downloading (100%)
- Installing nikic/php-parser (v4.1.0): Downloading (100%)
- Installing nette/utils (v2.5.3): Downloading (100%)
- Installing nette/finder (v2.4.2): Downloading (100%)
- Installing nette/robot-loader (v3.1.0): Downloading (100%)
- Installing nette/php-generator (v3.0.5): Downloading (100%)
- Installing nette/neon (v2.4.3): Downloading (100%)
- Installing nette/di (v2.4.14): Downloading (100%)
- Installing nette/bootstrap (v2.4.6): Downloading (100%)
- Installing jean85/pretty-package-versions (1.2): Downloading (100%)
- Installing phpstan/phpstan (0.10.6): Downloading (100%)
nette/bootstrap suggests installing tracy/tracy (to use Configurator::enableTracy())
Writing lock file
Generating autoload files

We do not need to create a configuration file for PHPStan so we are able to jump right into running the command and passing the information needed via the command line:

$ php vendor/bin/phpstan analyse src tests
4/4 [▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓] 100%

------
Line tests/AdditionTest.php
------
Class EasyMath\Tests\AdditionTest was not found while trying to analyse it - autoloading is probably not configured properly.
------
------
Line tests/SubtractionTest.php
------
Class EasyMath\Tests\SubtractionTest was not found while trying to analyse it - autoloading is probably not configured properly.
------

[ERROR] Found 2 errors

So our first run of PHPStan caught two problems, both around our test classes and indicating autoloading is probably not configured properly. This may be a case of PHPStan not being aware of some files we didn’t tell it about (the vendor folder being excluded by default like we saw with Phan). One quick google search later determined PHPStan wasn’t a fan of us creating our test class AdditionTest in our tests namespace EasyMath\Tests so we can solve the problem by adding to our autoload in our composer.json file to add our tests folder:

Before:

“autoload”: {
    “psr-4”: {“EasyMath\\”: “src/“}
}

After:

“autoload”: {
    “psr-4”: {
        “EasyMath\\”: “src/“,
        “EasyMath\\Tests\\”: “tests/“
    }
}

Since we made changes to our composer.json we need to dump our autoload classes via composer:

$ composer dump-auto
Generated autoload files containing 637 classes

Not familiar with composer dump-auto? Check out the CLI docs: Command-line interface / Commands - Composer

With our autoloading updated we can run PHPStan again to verify we have fixed the issues reported earlier:

$ php vendor/bin/phpstan analyse src tests
4/4 [▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓] 100%

[OK] No errors

! [NOTE] PHPStan is performing only the most basic checks. You can pass a higher rule level through the --level option
! (the default and current level is 0) to analyse code more thoroughly.

Great, PHPStan is not reporting any errors from the four files it has processed and it gives us the note informing us we’re running with default options for how thoroughly the code is analyzed.

Now PHPStan’s default options are giving us a clean bill of health, lets crank the level up and see what PHPStan really thinks about our example library:

php vendor/bin/phpstan analyse src tests --level max
4/4 [▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓] 100%

------
Line tests/AdditionTest.php
------
15 PHPDoc tag @param has invalid value ($expected): Unexpected token "$expected", expected TOKEN_IDENTIFIER at offset 52
15 PHPDoc tag @param has invalid value ($x): Unexpected token "$x", expected TOKEN_IDENTIFIER at offset 18
15 PHPDoc tag @param has invalid value ($y): Unexpected token "$y", expected TOKEN_IDENTIFIER at offset 35
31 PHPDoc tag @param has invalid value ($expected): Unexpected token "$expected", expected TOKEN_IDENTIFIER at offset 52
31 PHPDoc tag @param has invalid value ($x): Unexpected token "$x", expected TOKEN_IDENTIFIER at offset 18
31 PHPDoc tag @param has invalid value ($y): Unexpected token "$y", expected TOKEN_IDENTIFIER at offset 35
------

------
Line tests/SubtractionTest.php
------
15 PHPDoc tag @param has invalid value ($expected): Unexpected token "$expected", expected TOKEN_IDENTIFIER at offset 52
15 PHPDoc tag @param has invalid value ($x): Unexpected token "$x", expected TOKEN_IDENTIFIER at offset 18
15 PHPDoc tag @param has invalid value ($y): Unexpected token "$y", expected TOKEN_IDENTIFIER at offset 35
31 PHPDoc tag @param has invalid value ($expected): Unexpected token "$expected", expected TOKEN_IDENTIFIER at offset 52
31 PHPDoc tag @param has invalid value ($x): Unexpected token "$x", expected TOKEN_IDENTIFIER at offset 18
31 PHPDoc tag @param has invalid value ($y): Unexpected token "$y", expected TOKEN_IDENTIFIER at offset 35
------


[ERROR] Found 12 errors

Running PHPStan with level set to max certainly found a few more things with our code! Looks to be the same issue around our doc blocks above our methods in our tests.

Here is one of our doc blocks:

/**
* @param $x
* @param $y
* @param $expected
* @dataProvider mathProvider
*/

Oh! There it is! We have forgotten to specify a type for our params! Which is probably why PHPStan is complaining. Let’s clean up the doc block and re-run with max level again.

Fixed doc block:

/**
* @param float $x
* @param float $y
* @param float $expected
* @dataProvider mathProvider
*/

Running PHPStan again with level set to max for the most strict rules to use when analyzing our project:

$ php vendor/bin/phpstan analyse src tests --level max
4/4 [▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓] 100%

[OK] No errors

There we go, by adding our float types to our doc blocks PHPStan has nothing to complain about with our project! Remember this doesn’t mean we’re good to go; static analyzers do not actually run our code. We should ensure our unit tests are still passing:

$ php vendor/bin/phpunit
PHPUnit 7.3.5 by Sebastian Bergmann and contributors.
.................... 20 / 20 (100%)

Time: 52 ms, Memory: 4.00MB

OK (20 tests, 20 assertions)

Our tests still pass because PHPStan doesn’t modify any of our code, it is only reporting on what it found in our code and what it can determine without running the code. Since we only changed our doc blocks we should expect our tests still pass.

If you prefer to use a configuration file PHPStan does support the use of a phpstan.neon.dist file in the root of the project. You can also pass in the path to this file with the -c options via the CLI. We can code our paths in the phpstan.neon.dist configuration file so we don’t have to pass them in via the CLI:

$ php vendor/bin/phpstan analyse --level max
Note: Using configuration file ~/Code/php-easy-math/phpstan.neon.dist.
4/4 [▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓] 100%

[OK] No errors

Our library still checks out clean. Note however we do still have to pass in the --level option since PHPStan does not support reading the level value out of the configuration file. The PHPStan Documentation has a lot of good information about what you can (and can’t) do via the configuration file. You can point to your autoloading files and locations, build file or folder exclusions, as well as ignore specific errors by adding those options to the phpstan.neon.dist folder.

Because PHP Easy Math is a library for use in other projects we use phpstan.neon.dist filename for PHPStan configuration instead of phpstan.neon which will work, however the practice of adding .dist to files is for the library maintainer to ship distribution level configurations which users can then override. For example to override anything in our phpstan.neon.dist file you can copy the contents and save your changes to phpstan.neon and PHPstan will use this file if it exists before looking for phpstan.neon.dist. Stack Overflow has a good thread about .dist naming: naming conventions - What does .dist used as an extension of some source code file mean? - Stack Overflow

Recap

Reminder about static analyzers: these tools can’t determine intention they can only look at your source code and doc blocks and guess what your application is doing based on the functionality it finds. This could possibly lead to some confusing errors being reported. Make sure if something doesn’t make sense you search through the tool’s documentation and Github issues for answers to common issues.

Now we’ve seen two different static analyzer tools give our PHP Easy Math a clean, passing grade we’re ready to start diving into the code to implement some PHP 7.x features! Join us next month for Part 2 where we will introduce static typing, return type hints, and more!