High performance Symfony webapps

Kévin Dunglas

CEO at Les-Tilleuls.coop
Symfony core-team
API Platform creator
@dunglas

Sorry about my french flavored english

How to design a web app:

super fast to browse
easy to scale
robust (high availability)
requiring no server maintenance
easy to setup
relying on open standards
and using standard PHP libraries

We want

An architecture usable both for startups and big companies.
To deploy a new app in a few minutes...
but to be able to handle high traffic

The big picture

Two separate components

A static HTML5 webapp (SPA)
A web API

The web API

Centralizes R/W access to data
Holds all the business logic
Is built with PHP/Symfony
Is stateless (PHP sessions makes horizontal scalability harder)

The HTML5 webapp

Holds all the presentation logic
Is downloaded first (Single Page Application)
Queries the API to retrieve and modify data using AJAX
Is 100% composed of HTML, JavaScript and CSS assets

The API and the web app are standalone

2 Git repositories and 2 CI
different servers
2 domain names: example.com, api.example.com
The routing is done client-side using HTML5 push state

Immediate benefits

Speed (even on mobile)

Assets including index.html are downloaded from a CDN
After the first page load: no more download/parse/render bunch of HTML required at each request
Only small chunks of raw data transit on the network
API responses can be cached by the proxy

Scalability and robustness

The front app is just a set of static files: can be hosted in a CDN
Stateless API: push and pop servers/containers on demand

Development comfort

A wizard, an autocomplete box? Do it in JS, no more Symfony forms! (API data validation required)
Reuse the API in other contexts: native mobile, game stations, TV apps, connected devices or heavy desktop clients
Give access to your customers and partners to access to raw data trough the API

Long term benefits

Better structure: thanks to the central API, less business logic duplication when the app grows
Easier refactoring: Touching a component has no impact on the other (spec and test the API format)
Simpler project management: Separate teams can work on each app

Drawbacks

SEO and SMO

Formats, (open) standards, patterns

HTTP + REST + JSON

Work everywhere
Lightweight
Stateless
HTTP has a powerful caching model
Extensible (JSON-LD, Hydra, HAL, JSON Schema...)
High quality tooling

JSON Web Token (JWT)

Lightweight and simple authentication system: good bye OAuth
Stateless: token signed and verified server-side then stored client-side and sent with each request in an Authorization header
Store the token in the browser local storage

Infrastructure

A robust and secure infrastructure

Redundant: if a server crash, another must replace it immediately
Maintained: updates must be applied when available
Monitored: we must know when something goes wrong
DOS/DDOS attacks resistant
Encrypted connections

An infrastructure that scale

Horizontal scalability: push and pop new servers on-demand
Load balanced: share the load between available servers
Able to replicate data near the user (CDN)

Complex and expensive

Requires a lot of:

hardware
skills
time

Delegate the complexity

Infrastructure as a Service (IaaS): provide computing power, storage, bandwidth...
Platform as a Service (PaaS): run applications

Our choice

HTML5 webapp hosting

Uploaded in a Amazon S3 bucket (storage) by the CI
Configure the bucket to act as a website
Configure Amazon CloudFront (CDN) in front of it with a custom domain name
All non-existing URLs load to index.html (client-side routing)

All files of the static app are uploaded in an Amazon S tree bucket by the CI. The bucket is configured to act as a website and use the Amazon CloudFront CDN with a custom domain name. All non-existing URLs loads index.html

Why?

Cheap
Fast (CDN)
Easy to setup
Scale automatically
No maintenance
Easy to support different environments

Alternatives: GitHub pages, Google Cloud Storage, Azure, Akamai

PHP API hosting

Heroku Platform
Database using Heroku Postgres
Amazon CloudFront as cache reverse-proxy in front of Heroku

No maintenance
Run on the Amazon hardware
Scalability: Easy to add more dynos (containers) or larger database servers
Battery included: centralized logs, scheduler, ability to run Symfony commands...
Integrated with GitHub (test env for each PR)
Many quality add-ons (databases, Redis, New Relic, mails, autoscaling...)
HHVM and PHP 7 out of the box support

Alternatives: Platform.sh, Amazon Beanstalk, Google App Engine

Other

DNS: Amazon Route 53
Mail: SendGrid (as Heroku add-on)
Monitoring: New Relic (as Heroku add-on)

Creating the API

API Platform

An easy to use framework dedicated to API development.

Full disclaimer: I'm the author of this framework.

Create an entity (Doctrine + Validator)

// src/AppBundle/Entity/Person.php

namespace AppBundle\Entity;

use Doctrine\ORM\Mapping as ORM;
use Symfony\Component\Validator\Constraints as Assert;

/**
 * @ORM\Entity
 */
class Person
{
    /**
     * @ORM\Column(type="integer")
     * @ORM\Id
     * @ORM\GeneratedValue(strategy="AUTO")
     */
    private $id;
    /**
     * @ORM\Column(nullable=true)
     * @Assert\Type(type="string")
     */
    private $name;
    /**
     * Yes it works with relations too.
     *
     * @ORM\ManyToOne(targetEntity="PostalAddress")
     */
    private $address;

    // Getter and setters
}

Register a service

# app/config/services.yml

services:
    resource.person:
        parent:    'api.resource'
        arguments: [ 'AppBundle\Entity\Person' ]
    tags:      [ { name: "api.resource" } ]

You have:

Full CRUD with support for relations and dates
Data validation
Collections pagination
Serialization and validation groups
Error serialization
Automatic routes registration
Filters on exposed properties
Sorting
Automatically generated API doc
Native hypermedia support (JSON-LD + Hydra)

GET /people

{
    "@context": "/contexts/Person",
    "@id": "/people",
    "@type": "hydra:PagedCollection",
    "hydra:totalItems": 1,
    "hydra:itemsPerPage": 30,
    "hydra:firstPage": "/people",
    "hydra:lastPage": "/people",
    "hydra:member": [
        {
            "@id": "/people/1",
            "@type": "http://schema.org/Person",
            "name": "Kevin",
            "address": "/addresses/28"
        }
    ]
}

A collection by API Platform (JSON-LD + Hydra output)

Here is a sample collection exposed by API Platform.

The at context, at id and at type keys are JSON-LD properties describing the content returned by the API. They enable hypermedia features of the API. As you can see, you have nothing to do to get a level 3 REST API also know as the grail of REST.

The totalItems, firstPage and lastPage keys contain pagination information. It enables the client to iterate over the collection efficiently. Internally it uses the Doctrine paginator.

The member key contains all items of this page of the collection.

Another magic power of API Platform thanks to JSON-LD is its compliance with RDF. Any entity exposed by the API is a valid RDF document. It is possible to request several APIs hosted on different servers with only one SPARQL query. It is possible to insert this document in a triple store and it is possible to transform this simple JSON in XML-RDF or turtle.

And many more features

Powerful event system (ex: send a mail after a POST)
Embed relations (ex: GET products and its related offers)
Custom operations and controllers
Custom filters and sorting options
Content Negotiation (experimental)

Alternatives: FOSRestBundle, LemonRestBundle

CORS with NelmioCorsBundle

The webapp and the API have different domain names.
CORS headers must be set by the API to let the webapp querying it:

# src/app/config.yml

nelmio_cors:
    defaults:
        allow_origin:  ["https://example.com"]
        allow_methods: ["POST", "PUT", "GET", "DELETE", "OPTIONS"]
        allow_headers: ["content-type", "authorization"]
        max_age:       3600
    paths:
        '^/': ~

Security with LexikJwtAuth[...]Bundle

Makes the Symfony form login returning a JWT token instead of setting a cookie (stateless)
Allows to use Symfony firewall rules to secure API endpoints

# app/config/security.yml

security:
    # ...
    firewalls:
        login:
            pattern:  ^/login$
            stateless: true
            anonymous: true
            form_login:
                check_path: /login
                success_handler: lexik_jwt_authentication.handler.authentication_success
                failure_handler: lexik_jwt_authentication.handler.authentication_failure
            require_previous_session: false

        api:
            pattern:   ^/
            stateless: true
            lexik_jwt: ~

    access_control:
        - { path: ^/, roles: ROLE_ANONYMOUS, methods: [GET] }
        - { path: ^/special, roles: ROLE_USER }
        - { path: ^/, roles: ROLE_ADMIN }

Cache with FOSHttpCache


fos_http_cache:
    cache_control:
        rules:
        -
            match:
                path: ^/content$
            headers:
                cache_control:
                    public: true
                    max_age: 64000
                    etag: true

Parameters using environment variable

PaaS (including Heroku) use environment variables for the conf
Symfony parameters can be set using them

heroku config:set SYMFONY__MY_PARAMETER_NAME=abc # Heroku config

export SYMFONY__MY_PARAMETER_NAME=abc # local machine

my_bundle:
    parameter: %my_parameter_name%

A popular alternative is to use Incenteev Parameter Handler to map environment variables:

heroku config:set MY_PARAMETER_NAME=abc # Heroku config

export MY_PARAMETER_NAME=abc # local machine

// composer.json
{
    "extra": {
        "incenteev-parameters": {
            "env-map": {
                "my_param": "MY_PARAMETER_NAME"
            }
        }
    }
}

Specs and tests with Behat

Behat and its Behatch extension make testing and API easy.

# features/put.feature
Scenario: Update a resource
    When I send a "PUT" request to "/people/1" with body:
    """
    {
      "name": "Kevin"
    }
    """
    Then the response status code should be 200
    And the response should be in JSON
    And the header "Content-Type" should be equal to "application/ld+json"
    And the JSON should be equal to:
    """
    {
      "@context": "/contexts/Person",
      "@id": "/people/1",
      "@type": "Person",
      "name": "Kevin",
      "address": null
    }
    """

Tools for the static webapp

Angular, React, Ember, Backbone or anything else
Restangular
Auth0 JWT libraries
Yeoman, Gulp or Grunt, Bower, Jasmine, Karma...

A JavaScript client

'use strict';

angular.module('myApp')
    .controller('MainCtrl', function ($scope, Restangular) {
        var peopleApi = Restangular.all('people');

        function loadPeople() {
            peopleApi.getList().then(function (people) {
                $scope.people = people;
            });
        }

        loadPeople();

        $scope.newPerson = {};
        $scope.success = false;
        $scope.errorTitle = false;
        $scope.errorDescription = false;

        $scope.createPerson = function (form) {
            peopleApi.post($scope.newPerson).then(function () {
                loadPeople();

                $scope.success = true;
                $scope.errorTitle = false;
                $scope.errorDescription = false;

                $scope.newPerson = {};
                form.$setPristine();
            }, function (response) {
                $scope.success = false;
                $scope.errorTitle = response.data['hydra:title'];
                $scope.errorDescription = response.data['hydra:description'];
            });
        };
    });

This is an AngularJS controller displaying the list of people returned by the API and handling a form to create a new person.

The Restangular library is used to retrieve data from the /people route. The base URL of the API should be configured in the application bootstrap file.

In the loadPeople function, Restangular is used to retrieve the first page of the collection. The getList function returns a promise. When it is resolved, the API response is automatically parsed and converted to an array of JS documents. This array is bind to the view using the Angular scope.

The view contains a form bind to the newPerson property of the Angular scope. When the createPerson function is called (for instance when a button is pressed) Restangular serialize the object to JSON and send it to the API.

If an error is returned, the client leverages the API Platform error serialization feature to display a user friendly message.

Deploying the API using Heroku

Create an Heroku account
Install the Heroku toolbelt
Adapt the Symfony configuration following the official doc (easy)
Install my Heroku Postgres/Symfony configuration bridge

I hope these tools will help you to build your Symfony API and your JavaScript client. But we still need to show how to deploy our application on Heroku.

The first step is to create an account on the Heroku website (it should be easy).

Then, you must download and install a command line tool provided by Heroku. This command is simply called... Heroku.

The longer step is to adapt the Symfony configuration following the official Symfony (or Heroku) documentation. You'll see that it's really easy.

Basically you must add a configuration file of one line and configure Monolog to write logs on the standard error output instead of in a file.

Heroku use containers with a temporary file system. Everything stored on the container disk will be lost, including logs.

Writing logs to the standard error output enables Logplex, a tool provided by Heroku, to centralize logs of all containers in a single place.

The last step, to use the Heroku Postgres service, is to install a bridge I've created to transform the Heroku Postgres environment variable set by the addon and containing database credentials in parameters needed by the Doctrine bundle.

Run the following commands in the Git repository of the API:


heroku create # Create an Heroku app
heroku config:set SYMFONY_ENV=prod # Set the Symfony env
heroku addons:create heroku-postgresql:hobby-dev # Add Heroku Postgres
git push heroku master # Deploy the API
heroku run "app/console doctrine:schema:create" # Create Postgres tables

The API is up and running with an URL like
https://shielded-cliffs-1234.herokuapp.com (can be customized)

Add more dynos (containers) to handle more traffic: heroku ps:scale web+5 # Add 5 more web servers in the pool