Tempest version
3.19
PHP version
8.5
Operating system
Linux
Description
This is a follow up of #1834.
I did some benchmarks today to compare tempest against symfony, and unfortunately there are still massive performance issues.
I compared symfony and tempest with the exact same PHP setup (frankenphp with worker mode enabled).
Dockerfile
FROM dunglas/frankenphp:1.12-php8.5
# PHP configuration
RUN install-php-extensions intl
RUN cp "$PHP_INI_DIR/php.ini-production" "$PHP_INI_DIR/php.ini"
COPY docker/php/php.prod.ini "$PHP_INI_DIR/conf.d/app.ini"
# Caddy configuration
COPY docker/php/Caddyfile /etc/frankenphp/Caddyfile
# Install unzip (for `composer install` command)
RUN <<-EOF
apt-get update
apt-get install -y --no-install-recommends unzip
rm -rf /var/lib/apt/lists/*
EOF
# Composer
ENV COMPOSER_HOME="/root/.composer"
COPY --from=composer:2 /usr/bin/composer /usr/local/bin/composer
RUN mkdir -p ~/.composer
WORKDIR /app
php.prod.ini:
; Security
expose_php = Off
; General settings
error_reporting = E_ALL
; Resources
max_execution_time = 60
memory_limit = 128M
; OPcache
opcache.validate_timestamps = Off
; Disable .user.ini file
user_ini.filename =
Caddyfile:
{
skip_install_trust
{$CADDY_GLOBAL_OPTIONS}
frankenphp {
{$FRANKENPHP_CONFIG}
}
}
:80 {
root * /app/public
encode zstd br gzip
php_server {
root /app/public # allows for better caching
try_files {path} index.php
file_server off
worker {
file ./index.php
{$FRANKENPHP_WORKER_CONFIG}
}
}
}
.env file:
SIGNING_KEY=...
# Possible values: local, staging, production, ci, testing, other
ENVIRONMENT=production
# The base URI that's used for all generated URIs
BASE_URI=http://localhost:8890
# Setting to `false` force-disable internal caches.
INTERNAL_CACHES=true
# Enable or disable discovery cache. Can be `true`, `partial` or `false`.
DISCOVERY_CACHE=true
# Overwrite default log paths (null = default)
DEBUG_LOG_PATH=null
SERVER_LOG_PATH=null
pub/index.php:
<?php
declare(strict_types=1);
use Tempest\Router\WorkerModeApplication;
ignore_user_abort(enable: true);
require_once __DIR__ . '/../vendor/autoload.php';
$app = WorkerModeApplication::boot(root: __DIR__ . '/..');
$handler = static function () use ($app): void {
$app->run();
};
$maxRequests = (int) ($_SERVER['FRANKENPHP_MAX_REQUESTS'] ?? $_ENV['FRANKENPHP_MAX_REQUESTS'] ?? 500);
$requests = 0;
do {
$proceed = frankenphp_handle_request($handler);
gc_collect_cycles();
} while ($proceed && ($maxRequests === -1 || ++$requests < $maxRequests));
Controller:
<?php
declare(strict_types=1);
namespace App\Controller;
use Tempest\Router\Get;
use Tempest\Http\Responses\Json;
final class TestController
{
#[Get('/test')]
public function __invoke(): Json
{
return new Json(['success' => true]);
}
}
Composer install command:
composer install --no-dev --optimize-autoloader
Tempest commands:
./tempest cache:clear
./tempest discovery:generate
Podman run command:
podman build -f Dockerfile -t frankenphp-test .
podman run -d --rm --name tempest-php -p 8890:80 -v ".:/app:Z" frankenphp-test
Benchmark (tempest)
Running 10s test @ http://localhost:8890/test
4 threads and 100 connections
Thread Stats Avg Stdev Max +/- Stdev
Latency 91.48ms 149.17ms 603.79ms 86.61%
Req/Sec 236.15 324.76 1.07k 83.33%
1128 requests in 10.02s, 695.09KB read
Requests/sec: 112.54
Transfer/sec: 69.35KB
Benchmark (symfony)
podman run --rm --network host williamyeh/wrk -t4 -c100 -d10s http://localhost:8889/test
Running 10s test @ http://localhost:8889/test
4 threads and 100 connections
Thread Stats Avg Stdev Max +/- Stdev
Latency 3.70ms 1.80ms 30.22ms 69.86%
Req/Sec 6.85k 191.73 7.43k 78.25%
272724 requests in 10.02s, 47.86MB read
Requests/sec: 27228.28
Transfer/sec: 4.78MB
So symfony is 242x faster here 😮
At first I though the main culprit was $kernel->shutdown();, but commenting this line of code didn't improve req/s that much.
So the main culprit is actually the routing:
$responseSender->send(
response: $router->dispatch($psrRequest),
);
When I remove the routing from the run function of the FrameworkKernel:
public function run(): void
{
$router = $this->container->get(Router::class);
$psrRequest = $this->container->get(RequestFactory::class)->make();
$responseSender = $this->container->get(ResponseSender::class);
$kernel = $this->container->get(Kernel::class);
$kernel->shutdown();
}
The performance increases dramatically:
Running 10s test @ http://localhost:8890/test
4 threads and 100 connections
Thread Stats Avg Stdev Max +/- Stdev
Latency 8.09ms 3.34ms 50.70ms 77.16%
Req/Sec 3.14k 371.90 3.69k 68.50%
124984 requests in 10.02s, 19.07MB read
Non-2xx or 3xx responses: 124984
Requests/sec: 12478.72
Transfer/sec: 1.90MB
But it's still slower than symfony, even though routing was removed...
Last test with an empty run function:
public function run(): void
{
}
Now it's of course very fast:
Running 10s test @ http://localhost:8890/test
4 threads and 100 connections
Thread Stats Avg Stdev Max +/- Stdev
Latency 2.42ms 802.21us 49.28ms 95.82%
Req/Sec 10.46k 260.22 10.90k 89.00%
416198 requests in 10.01s, 55.97MB read
Requests/sec: 41589.71
Transfer/sec: 5.59MB
Conclusion:
- Biggest issue: as soon as the router is involved, performance decreases dramatically. The main culprit is probably the container though because the router makes heavy use of it.
- Also problematic: a simple call to
$this->container->get is enough to tank performance by a scale of thousands of reqs/s.
Unfortunately, I didn't have time to investigate further so I couldn't identify what exactly in the stack trace is causing such a performance drop.
Tempest version
3.19
PHP version
8.5
Operating system
Linux
Description
This is a follow up of #1834.
I did some benchmarks today to compare tempest against symfony, and unfortunately there are still massive performance issues.
I compared symfony and tempest with the exact same PHP setup (frankenphp with worker mode enabled).
Dockerfile
php.prod.ini:
Caddyfile:
.env file:
pub/index.php:
Controller:
Composer install command:
Tempest commands:
Podman run command:
Benchmark (tempest)
Benchmark (symfony)
So symfony is 242x faster here 😮
At first I though the main culprit was
$kernel->shutdown();, but commenting this line of code didn't improve req/s that much.So the main culprit is actually the routing:
When I remove the routing from the
runfunction of the FrameworkKernel:The performance increases dramatically:
But it's still slower than symfony, even though routing was removed...
Last test with an empty
runfunction:Now it's of course very fast:
Conclusion:
$this->container->getis enough to tank performance by a scale of thousands of reqs/s.Unfortunately, I didn't have time to investigate further so I couldn't identify what exactly in the stack trace is causing such a performance drop.