A Structured HTTP Fields Parser and Serializer for the Go Programming Language

“Structured Field Values for HTTP” is an upcoming RFC defining a set of well-defined data types to use in HTTP headers and trailers. This new format will improve the interoperability and the safety of HTTP by allowing to create generic parsers and serializers suitable for all HTTP headers (currently, most headers need a custom parser) and could also allow to improve the performance of the web. Mark Nottingham, one of the authors of this RFC, has published a very interesting article explaining these aspects in depth.

Headers and trailers using structured fields look like this:

Example-Item: token; param1=?0; param2="a-string"
Example-List: token, "string", ?1; parameter, (42, 42.0)
Example-Dict: foo1=bar, foo2="baz"; param, foo3=(?1 10.1)

Custom HTTP headers (and trailers) can start using structured values right now and most upcoming web standards including the new security headers proposed by the Chrome team are embracing them.

The next version of Vulcain, a popular proposal specification of mine relying on HTTP/2 (and HTTP/3) Server Push to allow creating fast and idiomatic client-driven web APIs, will also leverage Structured Fields Values!

Problem: the reference implementation of Vulcain is written in Go… but until now Go had no parser for Structured Field Values. So I wrote one!

Here comes httpsfv, a Structured Field Values parser and serializer for the Go programming language! The library implements entirely the latest version of the Internet-Draft (19), is fully documented, is tested with the official test suite (and many additional test cases), is quite fast (benchmark included in the repository) and is free as in beer, and as in speech (BSD-3-Clause License)!

As the Structured Field Values spec will become a RFC soon and is planned to be used for most new HTTP headers, I also opened a Pull Request on the Go repository to include this parser (and serializer) directly in the standard library of the language. If this PR is merged, it will be possible to deal with SFV without requiring any third-party package.

This PR also provides convenient helper methods to parse and serialize structured headers directly from Header instances:

h := http.Header{}

// Parsing
i := h.GetItem("Foo-Item")
l := h.GetList("Foo-List")
d := h.GetDictionary("Foo-Dictionary")

// Serializing
d := sfv.NewDictionary()
d.Add("b", sfv.NewItem(false))

bar := sfv.NewItem(sfv.Token("bar"))
bar.Params.Add("baz", 42)
d.Add("a", bar)

h.SetStructured("My-Header", d)

Examples and the documentation are available on the dedicated GitHub repository. Give it a try, and if you want to support this project, also give it a star!

Save the Web: Decentralize!

The web was designed to serve humanity, to enable knowledge sharing, for friendship among people.
But data and services centralization by the Tech Giants have completely perverted it. Centralization causes major threats to our societies: mass surveillance, censorship, vote rigging, opponents arrests , advertising tracking.

The situation is such that the founding fathers of the web, including its creator Tim Berners-Lee, multiply the calls to save it.

The solution: decentralize it again, give users control over their data, fight against GAFAM monopolies, rebuild an independent and non-commercial web based on free software.

Developers, journalists, lawyers, artists and activists are organizing to bring out the “Dweb”, the decentralized and distributed web (again). The Dweb is a corpus of standards (usually published by the W3C) such as ActivityPub, JSON-LD, RDF or HTTP as well as tools including Solid, Mastodon, PeerTube or Mobilizon which aim to put the Web at the service of common good.

As web developers, it is our responsibility to build and promote this new web.

The good news is that the PHP ecosystem now has the tools to easily create decentralized web applications based on these standards. During this conference, we will discover why the Dweb is a crucial issue for the future of our societies, how it works technically, and how we implement it with API Platform!

To do so, I just published a new API Platform component: API Platform ActivityPub! This new libraries component allows:

  • Build apps supporting ActivityPub in minutes!
  • Add ActivityPub support to any existingAPI Platform or Symfony app
  • 100% customizable
  • Compatible with all API Platform features
  • State of the art PHP / API Platform

Check it out!

Say Hello to Mercure 0.10!

I’m very excited to announce the immediate availability of the version 6 of the Mercure Internet Draft as well as of the version 0.10 of the reference implementation!

Mercure is a real-time protocol built on top of Server-sent Events and leveraging HTTP/2+. It allows to push messages to JavaScript webapps, mobile apps or IoT devices with ease. Client-side, it relies on the native EventSource class and doesn’t require any JS dependency. Server-side, it allows to use a Hub to handle persistent connections and dispatch messages sent, for instance, from your REST or GraphQL APIs. An open source Hub designed for performance and high loads is developed in parallel to the specification. Get started with Mercure.

Since the initial release of Mercure, we gather feedback from the ever-growing community of users and contributors. This allowed us to identify the pain points, some limitations and new use cases. It took months of work to get there, but these new versions of the protocol and of the FOSS hub (written in Go) fix all the limitations and design issues we identified so far. They are by far the best and the fastest versions ever released! Let’s discover these changes and new features.

Protocol Changes

Authorization Mechanism

One of the key features of Mercure is its authorization mechanism. It allows to send private updates, which will be received only by subscribers having the appropriate credentials.

Until the last version of the protocol, this mechanism was using the concept of targets, and was hard to understand for newcomers.
In the 6th version of the protocol, the authorization mechanism has been dramatically simplified, while staying as powerful as before.

The authorization mechanism has been totally revamped and the concept of targets is gone. Now, to send a private update, the publisher must only mark it as private when sending the POST request to the hub using the new private parameter.

On the other hand, to receive a private update, a subscriber must present a JWT to the hub containing a list of topic selectors in the mercure.subscribe claim of the token.

A private update will be received by a subscriber only if:

  • the topic of the update is contained in the list of topic selectors
  • the topic matches an URI template present in the list
  • the topic selectors list contains the special value *, which allows to receive all updates.

More advanced use cases such as cherry-picking which updates will be received by a subscriber are also supported thanks to existing features such as alternate topics.

Also, the new concept of target selectors can also be used to subscribe to (public or private) updates, making the whole protocol more consistent.

Learn more about topic selectors and the authorization mechanism.

Presence API

One of the most requested feature was the ability to retrieve the list of currently connected subscribers. This is now possible thanks to a new web API exposed by the hub! It’s also possible for a subscriber to receive Mercure updates when other subscribers connect or disconnect.

Hubs can now expose endpoints returning the list of connected users as JSON-LD documents:

  • /.well-known/subscriptions returns the list of all active subscriptions
  • /.well-known/subscriptions/{topic} returns the list of active subscriptions for a specific topic
  • /.well-known/subscriptions/{topic}/{subscriber} returns details for a single subscription

Also, after having fetched the initial list of connected users, clients can subscribe (using the standard subscription mechanism) to connection status updates by using these URIs as topics.

To access to this presence API, subscribers must be authenticated and have topic selectors matched by these URIs.

Finally, arbitrary data (such as the subscriber’s username or IP address) can be attached to a subscription and retrieved by other subscribers using the new mercure.payload JWT claim.

The debug UI shipped with the reference hub now supports this new feature:

Learn more about active Mercure subscriptions.

Event Sourcing

The native features of Mercure, including its built-in history and its state reconciliation mechanism make it a convenient solution to implement Event Sourcing.

The latest version of the protocol introduces changes making dedicated to this kind of usages.

First, it is now possible to set the value of the Last-Event-ID HTTP header (or of the query parameter with the same name) to earliest to retrieve the all past events. This allows to use the Mercure hub similarly to Apache Kafka, but directly from the browser (or any other kind of HTTP client).

Additionally, when a Last-Event-ID header or parameter is set, the hub will now always return a Last-Event-ID header in the HTTP response. If it matches the requested one, it means that all requested events have been returned, otherwise, it means that the requested event may have been deleted by the hub (if it stores a limited number of events for instance), or that this event ID doesn’t exist.

Learn more about Event Sourcing and state reconciliation capabilities of Mercure.

IRIs, Strings, and Fragment Identifiers

The protocol has been made more consistent by allowing to use IRIs (recommended) or plain strings (discouraged but sometimes practical) for every identifiers: topics, event IDs, subscriber IDs…

Event IDs starting by the # character (URI fragment) are now reserved for generation by the hub. This allows a hub to use offsets generated by systems such as Apache Kafka, Apache Pulsar or Redis Streams as update identifiers while keeping these identifiers globally unique (because they are relative to the Hub URI).

Changes in the Reference Implementation

All the features described previously have already been implemented in the reference implementation (the open source hub written in Go). But that’s not all! The hub has also been dramatically improved during the past few months.

New Engine

Until version 0.10, the reference hub was using an engine working like this: when a new update was posted by a publisher, a new light-weight process (goroutine) was started and the update was pushed to every active subscriber in a loop. If a Last-Event-ID header (or parameter) was passed by a subscriber to request past events, these past events were retrieved from the persistence layer and dispatched from another goroutine.

However, this approach had several drawbacks:

  • If a subscriber was slow to receive an update, it was slowing down the main loop, and so increasing the latency also for other subscribers.
  • If a Last-Event-ID was passed, new events could be received before events coming from the history (for example, when a new event was sent from the main goroutine while another goroutine was still retrieving past events from the persistence layer). The order in which events were received wasn’t 100% guaranteed.

Mercure 0.10 contains a brand new engine that fixes these issues! This new engine leverages thoroughly the iconic features of the Go programming language: channels and goroutines. Now, two buffers are assigned to each subscribers: the first one stores events coming from the history while the second one stores “live” events. When a new event is published, the main goroutine pushes this event in the “live” buffers without waiting for events to be dispatched. Then a dedicated goroutine per subscriber pops events from the buffers associated with it. So every subscriber can receive the events as its own pace, without impacting the rest of the system.

This buffer system also fixes the order issue: goroutines dedicated to specific subscribers wait for all updates coming from the history to be dispatched before starting to dispatch updates stored in the “live” buffer.

Last but not least, this new system allows to disconnect subscribers that are too slow to receive the updates. A new config option, dispatch_timeout, has been introduced to configure the maximum time that a subscriber can take to receive an update before being disconnected to free resources.

If you’re interested in the internals, take a look at this piece of Go code!

A big thanks to Dani Kaplan for his precious help testing and debugging this new engine under high loads!

Prometheus and Health Checks Support

Prometheus is a very popular monitoring system and time series database for Cloud Native applications. Jérémy Decool made a series of great contributions that add support for Prometheus to the Mercure hub. Metrics include (among other things) the number of currently connected subscribers, the number of dispatched updates, the number of served HTTP request segregated by status code, the memory usage, the current number of processes and the number of open file descriptors!

In addition, a new URL (/healthz) is now available to check if the hub is up and running without polluting the HTTP logs with useless entries. This particularly useful for Kubernetes’ liveness and readiness probes.

Version Flag

Jérémy Decool also contributed a new --version flag allowing to know what exact version of the Hub is being used. It looks obvious and priceless, but it was a missing feature. Thanks Jérémy for adding it!

Examples and Demos

We updated the documentation as well as all existing examples (in various programming languages) to be in sync with the latest version of the spec. I also added examples and demos illustrating new features, including a chat written in JavaScript and Python:

This demonstration chat is now available online, try it!

Community

Mercure is becoming more and more popular! It recently reached 2k stars on GitHub and integrations with existing ecosystems are now plenty. For this new version, we worked with the community to keep libraries and other Mercure-related projects in sync with the latest changes in the spec. Most of them are already compatible including (but not limited to):

For some others (for instance, the Java library and the Python library), Pull Requests are open. Finally, some tools still need to be updated including the awesome alternative Hub written in Node created by Nicolas Coutin and the Yii framework integration. Any help is very welcome!

To keep up with Mercure news, you can now follow our brand new Twitter account.

Finally, I’m very happy to announce that Les-Tilleuls.coop now provides official training for Mercure, in English and in French!

Improving Libraries Used by Mercure

One of the most powerful feature of Mercure is the ability to use URI templates to subscribe to topics matching patterns. Under the hood, the reference implementation of Mercure use the uritemplate Go library.

The performance of the Mercure hub highly depends of the performance of this library. So I worked with Kohei Yoshida (the author of this lib) to improve its performance, add benchmarks and fix some bugs! This also dramatically improved the performance of the hub (and of other projects using this library). Thank you Kohei for your quick reviews and releases!

The development of the new version of the hub also allowed me to find an issue in the Go standard library, and to open a Pull Request to fix it.

Hosted and High Availability Versions

Managed and On Premise High Availability (cluster) versions of the Mercure hub are available for a while now. They are already compatible with Mercure 0.10 and are now out of beta!

I want to thank very much our first customers, who trusted us, helped us to battle-test our hosting platform and gave us incomparable feedback allowing us to improve both the open protocol, the Managed and On Premise versions and the Open Source hub! They also allow us to invest more in the protocol’s development and in the Open Source version! So thank you again.

I also want to thank very much my GitHub sponsors, who allow me to work always more on my free software projects, including Mercure but also Vulcain, API Platform and the Symfony components I maintain.

Try and Star Mercure!

If you aren’t using Mercure yet, you can install a hub in a few seconds, or use the online demo, give it a try! If you like the project, show your support by starring it on GitHub!

Vulcain: HTTP/2 Server Push
 and the rise of client-driven REST APIs

Over the years, several formats have been created to fix performance bottlenecks of web APIs: the n+1 problem, over fetching, under fetching…
The current hipster solution for these problems is to replace the conceptual model of HTTP (resource-oriented), by the one of GraphQL.

It’s a smart network hack for HTTP/1… But a hack that comes with (too) many drawbacks when it comes to HTTP cache, logs, security…
Fortunately, thanks to the new features introduced in HTTP/2 and HTTP/3, it’s now possible to create REST APIs fixing these problems with ease and class.

Vulcain is a brand new Internet Draft allowing to create fast, idiomatic and client-driven REST APIs.
To do so, it relies on the Server Push feature introduced by HTTP/2+ and on the hypermedia capabilities of the HTTP protocol.

Better, Vulcain comes with an open source reverse proxy that you can put on top of any existing web API to instantly turn it into a Vulcain-compatible one!

HATEOAS is back, and it’s for the best!

Forum PHP 2019: Mercure – Real-Time for PHP Made Easy

Yesterday at Forum PHP 2019 I presented how easy it is to create real-time apps using PHP (among other languages) and the Mercure protocol.

I also introduced the shiny and new Mercure website (designed by Laury S.)!

A special thanks to Eric Comellas who jumped on stage to explain how iGraal uses Mercure on a large scale to serve simultaneously more than 100,000 of their customers!

API Platform 2.5

API Platform 2.5: revamped Admin, new API testing tool, Next.js and Quasar app generators, PATCH and JSON Schema support, improved OpenAPI and GraphQL support

I’m very excited to announce the immediate availability of API Platform 2.5!

API Platform is a set of standalone server and client components for building and consuming REST (JSON-LD, Hydra, JSON:API…) and GraphQL APIs. The server components use PHP and Symfony while the client-side components (which support any Hydra-enabled web API, even the ones not built using API Platform) are written in JS. If you haven’t tried API Platform yet, it only takes a few minutes to create your first project!

API Platform 2.5 is the best version ever of the framework! Let’s discover its most interesting new features.

API Platform Admin 1.0: the power of React Admin and Material UI unleashed!

API Platform comes with a powerful component to create admin interfaces. It is built on top of React Admin and Material-UI.

Pass the URL of any Hydra-enabled API (including, of course, APIs built with API Platform itself) to the HydraAdmin component, and you instantly get a beautiful, fully-featured admin interface for resources exposed by the API including:

  • CRUD pages;
  • widgets based on the type of the properties;
  • filters;
  • sorting;
  • client-side validation…

The admin interface is dynamically built client-side, by parsing the Hydra metadata of the API. To do so, only a single LLOC is necessary:

import React from "react";
import { HydraAdmin } from "@api-platform/admin";

export default () => <HydraAdmin entrypoint="https://demo.api-platform.com"/>

And you get this (live demo):

demo-admin.api-platform.com

In previous versions, customizing the UI wasn’t straightforward nor idiomatic: you had to post-process the JS object containing the parsed API documentation.

Luckily, the Marmelab (creator of React Admin) and Les-Tilleuls.coop (creator of API Platform) teams got together several times during the summer to create a new version which dramatically improves the DX. Thus API Platform Admin 1.0 was born! While keeping the simplicity of previous versions, it now allows to leverage the power of JSX to customize every part of the UI using React Admin components, Material-UI components, or your custom React components:

import React from "react";
import {
  HydraAdmin,
  ResourceGuesser,
  CreateGuesser,
  InputGuesser
} from "@api-platform/admin";
import { ReferenceInput, AutocompleteInput } from "react-admin";

const ReviewsCreate = props => (
  <CreateGuesser {...props}>
    <InputGuesser source="author" />
    <ReferenceInput
      source="book"
      reference="books"
      label="Books"
      filterToQuery={searchText => ({ title: searchText })}
    >
      <AutocompleteInput optionText="title" />
    </ReferenceInput>

    <InputGuesser source="rating" />
    <InputGuesser source="body" />
  </CreateGuesser>
);

export default () => (
  <HydraAdmin entrypoint="https://demo.api-platform.com">
    <ResourceGuesser
      name="reviews"
      create={ReviewsCreate}
    />
  </HydraAdmin>
);

In the previous example, we add an autocomplete input for a relation by using the dedicated React Admin component while letting API Platform Admin guess which inputs to use for the other properties based on the API docs.

Check out the updated and improved documentation to learn more about these new customization capabilities!

This feature has been contributed by Morgan Auchede, Florian Ferbach, Gildas Garcia, Alexis Janvier, Jean-François Thuillier, François Zaninotto and myself.

Next.js and Quasar support in the Client Generator

API Platform also provides an app generator supporting React, Vue and React Native. And in version 2.5, it is also able to generate high quality Next.js and Quasar apps!

API Platform Client Generator is very similar to Symfony’s MakerBundle and to Rails’ scaffolding, but instead of generating server-side code, it generates a client-side app which uses the API to fetch and persist data. To do so, like the admin component, the generator uses the Hydra specification exposed by the API (built with API Platform or otherwise; support for OpenAPI and GraphQL is in progress).

Next.js is the most popular framework for React, with automatic Server-Side Rendering support, routing, code splitting, CSS-in-JS, static exporting… The Next.js generator, created by Grégory Copin, leverages the 0-config TypeScript support introduced in Next 9: all generated code is written in TS!

To generate a Next app from any API exposing a Hydra specification, run the following commands:

$ npm install next --save # Install next
$ npx @api-platform/client-generator https://demo.api-platform.com src/ --generator next # Scaffold the app

And you get high quality code you can customize to fit your needs:

List screenshot

Refer to the dedicated documentation to learn about all the capabilities of this generator.

Quasar is a framework built on top of Vue.js allowing to write SPAs with Server-Side Rendering support, mobile apps and Electron apps in one go. Its comes with a nice Material Design interface.

To use it, run the following commands:

$ npm install -g @quasar/cli
$ quasar create my-app
$ npx @api-platform/client-generator https://demo.api-platform.com src/ --generator quasar my-app/src # Scaffold the app

A big thanks to Paul Apostol for contributing this generator!

A brand new test client and dedicated API assertions

Currently, there is a lack of satisfactory solutions to write functional tests for web APIs built in PHP (API Platform and Symfony included):

  • Behat+Behatch+Mink is BDD-oriented (which is completely fine, but doesn’t fit with all projects/teams), is a bit complex to set up, has a high barrier to entry and is still not fully compatible with Symfony 4 (you need to rely on a dev version of Behat). It also lacks some utilities, for database testing for instance.
  • BrowserKit/WebTestCase is dedicated to webpage testing through web scraping (DOM Crawler, CSS selectors, simulation of browser actions such as clicking or reloading a page…). Its API doesn’t fit well with API testing. However, it benefits from the large ecosystem of PHPUnit, and gives access to numerous functional testing helpers provided by Symfony.
  • External solutions which don’t manipulate the Symfony Kernel (but are true HTTP clients) such as Postman or Blackfire Player require setting up a web server for the testing environment and don’t provide access to the service container (for instance, to test if the database has been updated, or if a mail has been sent).

It’s time to say hi to ApiTestCase, Test\Client and a bunch of new assertions dedicated to API testing (matching a JSON document or a subset of it, checking the status code and headers of HTTP responses, validating against a  JSON Schema…)! This new set of API testing utilities is built on top of Symfony’s KernelTestCase class, and the Test\Client implements the exact same interface as the brand new Symfony HttpClient component.

This new testing tool also plays very well with Alice (a test fixture generator having an official Symfony/API Platform recipe), which has also gained new powers in the process!

Let’s see how it looks:

namespace App\Tests;

use ApiPlatform\Core\Bridge\Symfony\Bundle\Test\ApiTestCase;
use App\Entity\Book;
use Hautelook\AliceBundle\PhpUnit\RefreshDatabaseTrait;

class BooksTest extends ApiTestCase
{
    // This trait provided by HautelookAliceBundle will take care of refreshing the database content to a known state before each test
    use RefreshDatabaseTrait;
    
    public function testCreateBook(): void
    {
        $response = static::createClient()->request('POST', '/books', ['json' => [
            'isbn' => '0099740915',
            'title' => 'The Handmaid\'s Tale',
            'description' => 'Brilliantly conceived and executed, this powerful evocation of twenty-first century America gives full rein to Margaret Atwood\'s devastating irony, wit and astute perception.',
            'author' => 'Margaret Atwood',
            'publicationDate' => '1985-07-31T00:00:00+00:00',
        ]]);

        $this->assertResponseStatusCodeSame(201);
        $this->assertResponseHeaderSame('content-type', 'application/ld+json; charset=utf-8');
        $this->assertJsonContains([
            '@context' => '/contexts/Book',
            '@type' => 'Book',
            'isbn' => '0099740915',
            'title' => 'The Handmaid\'s Tale',
            'description' => 'Brilliantly conceived and executed, this powerful evocation of twenty-first century America gives full rein to Margaret Atwood\'s devastating irony, wit and astute perception.',
            'author' => 'Margaret Atwood',
            'publicationDate' => '1985-07-31T00:00:00+00:00',
            'reviews' => [],
        ]);
        $this->assertRegExp('~^/books/\d+$~', $response->toArray()['@id']);

        // This new assertions checks that the returned JSON document matches the JSON Schema generated by API Platform for this resource (also included in the OpenAPI file) 
        $this->assertMatchesResourceItemJsonSchema(Book::class);
    }

    public function testUpdateBook(): void
    {
        $client = static::createClient();
        // findIriBy allows to retrieve the IRI of an item by searching for some of its properties.
        // ISBN 9786644879585 has been generated by Alice when loading test fixtures.
        // Because Alice use a seeded pseudo-random number generator, we're sure that this ISBN will always be generated.
        $iri = static::findIriBy(Book::class, ['isbn' => '9781344037075']);

        $client->request('PUT', $iri, ['json' => [
            'title' => 'updated title',
        ]]);

        $this->assertResponseIsSuccessful();
        $this->assertJsonContains([
            '@id' => $iri,
            'isbn' => '9781344037075',
            'title' => 'updated title',
        ]);
    }

    public function testDeleteBook(): void
    {
        $client = static::createClient();
        $iri = static::findIriBy(Book::class, ['isbn' => '9781344037075']);

        $client->request('DELETE', $iri);

        $this->assertResponseStatusCodeSame(204);
        $this->assertNull(
            // Through the container, you can access all your services from the tests, including the ORM, the mailer, remote API clients...
            static::$container->get('doctrine')->getRepository(Book::class)->findOneBy(['isbn' => '9781344037075'])
        );
    }
}

Check out the full documentation, or the screencast which has been published on SymfonyCasts on this topic.

Improved GraphQL support

The GraphQL subsystem has been dramatically improved in version 2.5. Kudos to Alan Poulain who has done an awesome job maintaining and improving this part, and to Lukas Lücke and Mahmood Bazdar for their great contributions.

The main features of this new version are the customization of queries, mutations and types.

Whenever you need to add your own logic in your schema, you can now add custom properties in your resource’s annotations to do it. For instance to add a custom query:

/**
 * @ApiResource(graphql={
 *     "myShinyQuery"={
 *         "item_query"=MyQueryItemResolver::class,
 *         "args"={
 *             "id"={"type"="ID"},
 *             "isShiny"={"type"="Boolean!"}
 *         }
 *     }
 * })
 */

You only need to define the corresponding resolver and it’s done!

<?php

namespace App\Resolver;

use ApiPlatform\Core\GraphQl\Resolver\QueryItemResolverInterface;

final class MyQueryItemResolver implements QueryItemResolverInterface
{
    public function __invoke($item, array $context)
    {
        // Do what you want!

        return $item;
    }
}

Please refer to the documentation for customizing the mutations and the types.

The new version also comes with:

  • a better pagination mechanism (including backwards pagination)
  • a new “stage” mechanism allowing to easily customize the built-in resolvers, and an easier way to customize behaviors using decoration
  • support for GraphQL Playground, in addition to GraphiQL
  • a new command to export the GraphQL schema automatically generated by API Platform
  • support for file uploads
  • support for name converters
  • improved overall performance

All these new features have been documented, check it out!

JSON Merge Patch Support

Until now, API Platform had only supported using the PUT HTTP method for replacing a resource. Supporting proper partial updates through the PATCH method is known to be hard.

In version 2.5, a large refactoring work has been done to properly support the PATCH method, and support for JSON Merge Patch (RFC 7386) has been added (to do so, we’ve contributed the required low-level brick to the Symfony Serializer component). The new infrastructure is designed to support other PATCH formats as well. In the future, other formats such as JSON Patch (RFC 6902) may be added.
The JSON:API PATCH format, which was already supported, is of course still working.

Read the dedicated documentation entry to learn more about API Platform’s PATCH support (including how to enable it for existing projects).

JSON Schema Support

JSON Schema is a popular vocabulary to describe the shape of JSON documents. A variant of JSON Schema is also used in OpenAPI specifications. As of API Platform 2.5, JSON Schema is a first-class citizen. A new infrastructure has been created to be able to generate JSON Schemas for any resource, represented in any format (including JSON-LD). A command has also been added to export these schemas:

$ bin/console api:json-schema:generate 'App\Entity\Review'

The generated schema can be used with libraries such as react-json-schema-form to build forms for the documented resources, or to be used for validation.

To generate JSON Schemas programmatically, use the new api_platform.json_schema.schema_factory service.

Improved OpenAPI support

Thanks to the improvements made in the OpenAPI v3 specification, and to the new JSON Schema infrastructure in API Platform, the generated OpenAPI documentation is now better. It generates a specific JSON Schema per supported format:

JSON-LD specific keys such as @id and @context are now documented.

Grégoire Hebert added the ability to configure the versions of OpenAPI to support, while also  allowing to specify the default version being used:

api_platform:
    swagger:
        versions: [3, 2] # OpenAPI v3 is now the default

Frédéric Barthelet and Ryan Weaver improved the OpenAPI documentation of the built-in order and property filters. Thank you!

Screencasts and improved docs

We are very proud to announce an official partnership between API Platform and SymfonyCasts! SymfonyCasts has published more than 7 hours of high quality and funny video tutorials on API Platform (and counting!). The screencasts are now available directly from the main menu of the website, and referenced in the appropriate documentation entries.

Also, for version 2.5, API Platform’s docs have been dramatically improved. All new features presented in this post (and more) are now documented; many parts of the docs have been fixed, improved and modernized; sections have been reordered to provide a better learning experience; and a new “extending API Platform” entry has been added to centralize how to hook your custom logic in a way which is compatible with both the REST and GraphQL subsystems. But there is still a lot to do, and your help is very welcome!

Community milestones and Hacktoberfest

The main strength of API Platform is its vibrant and diverse community! Recently, the project reached great milestones thanks to the many contributors, evangelists and users improving and promoting the framework: the Slack channel reached 2 000 users a few days ago; the core package of the framework has been downloaded more than 2 000 000 times; and workshops, meetups and talks about the project are being organized all over the world (next big events with API Platform talks: Forum PHP Paris and SymfonyCon Amsterdam)!

The framework is close to reaching 5 000 stars on GitHub! It will be the next big milestone, and we’ll organize a big party to celebrate! If you have not starred API Platform yet, please do! It will help us reach a wider audience. If you love the project, tell your friends about it!

Speaking about community, free software and contribution, Hacktoberfest is about to begin! Hacktoberfest is an event helping free software projects by encouraging contributions (code but also docs and marketing materials). As thanks for your contribution, Digital Ocean offers a limited-edition t-shirt, and nice stickers. This year, Les-Tilleuls.coop will also send you some surprise gifts if you contribute at least 3 Pull Requests on the API Platform repositories. Thanks Les-Tilleuls!

Hacktoberfest is a very good opportunity to start contributing to free software, and to API Platform in particular. To help you get started, we’ve marked some code and docs issues with a specific “Hacktoberfest” label:

If you need help working on this, don’t hesitate to ask us directly on GitHub, or in the API Platform Slack channel!

https://lh3.googleusercontent.com/-6RjHzvBuxFo/XZH8jV3YDtI/AAAAAAAADWE/kMKfpexyNWMGC3JMgv0Gp4XfDZxghfe-ACK8BGAsYHg/s0/Image%2Bfrom%2BiOS%2B%252812%2529.jpg

Mercure: Real-Time APIs for Serverless and Beyond

Here is the slide deck I presented during API Days SF 2019:

Mercure is a protocol allowing to push data updates to web browsers and other HTTP clients in a convenient, fast, reliable and battery-efficient way. It is especially useful to publish real-time updates of resources served through web APIs, to reactive web and mobile apps. The protocol is designed for serverless, HTTP/2+, hypermedia and GraphQL, and is fully-featured: auto-discoverable, authorization, re-connection, state reconciliation…

Using Next.js and Material UI Together

Next.js is a convenient and powerful framework for React. Its main benefit over using React directly is its transparent support for Server-Side Rendering.
Material UI is a very popular set of React components implementing Google’s Material Design guidelines.

Both libraries are impressive, but there are some tricks to know to make them playing well together.

Bootstrapping

Setting up MUI in a Next project requires some non-trivial tweaks to Next’s initialization process. Conveniently, Material UI provides a skeleton containing a working Next.js project with Material UI already properly configured. It’s the easiest way to kickstart a new project using both tools, don’t miss it!

# Download the skeleton
$ curl https://codeload.github.com/mui-org/material-ui/tar.gz/master | tar -xz --strip=2  material-ui-master/examples/nextjs
$ cd nextjs
# Install the deps
$ yarn install
# Start the project
$ yarn dev

To learn how to integrate Material UI in an existing project, take a look to pages/_document.js and pages/_app.js they contain most the wiring logic.

Forms

Material UI is especially useful because of the large set of form components it provides. But handling forms with React (and so with Next) is tedious and verbose. My colleague Morgan Auchedé recently told me about Formik. Formik is a tiny yet super powerful library allowing to easily create forms with React. And good news: it plays very well with Next! Here is how a basic login form looks when using Formik:

import React from 'react';
import { Formik, Form, Field, ErrorMessage } from 'formik';

export default MyForm = () => (
    <Formik
      initialValues={{ email: '', password: '' }}
      validate={values => {
        // Your client-side validation logic
      }}
      onSubmit={(values, { setSubmitting }) => {
        // Call your API
      }}
    >
      {({ isSubmitting }) => (
        <Form>
          <Field type="email" name="email" />
          <ErrorMessage name="email" />
          <Field type="password" name="password" />
          <ErrorMessage name="password" />
          <button type="submit" disabled={isSubmitting}>
            Submit
          </button>
        </Form>
      )}
    </Formik>
);

Nice! However, when switching to Material UI inputs, the high level helper components provided by Formik become almost useless. Our forms are verbose again. Fortunately, a small library intuitively named formik-material-ui makes it easy to bridge both libraries! Here is the same form as before (including error handling), but rendered using Material UI components:

import React from 'react';
import { Formik, Form, Field } from 'formik';
import { TextField } from 'formik-material-ui';
import Button from "@material-ui/core/Button";

export default MyForm = () => (
    <Formik
      initialValues={{ email: '', password: '' }}
      validate={values => {
        // Your client-side validation logic
      }}
      onSubmit={(values, { setSubmitting }) => {
        // Call your API
      }}
    >
      {({ isSubmitting }) => (
        <Form>
          <Field type="email" name="email" component="TextField" />
          <Field type="password" name="password" component="TextField" />
          <Button
           type="submit"
           fullWidth
           variant="contained"
           color="primary"
           disabled={isSubmitting}
         >
           Submit
         </Button>
        </Form>
      )}
    </Formik>
);

This form is even less verbose, and is now looking good!

Buttons and Routing

Next.js comes with a nice routing system working transparently regardless if the app is executed client-side or server-side. It’s one of the biggest strength of the framework. However, in Material UI, the Button component is often used to trigger navigation between pages, and using buttons with the Router isn’t very intuitive. Still, it’s easy to do:

import React from "react";
import Link from "next/link";
import Button from "@material-ui/core/Button";

export default MyLink = () => (
  <Link href="/pricing" passHref>
    <Button component="a">Managed version</Button>
  </Link>
);

First, we set the component prop of Button to a. It tells Material UI to use an anchor for this button, instead of a… button by default. Then, we set the passHref prop of the Link element, it hints the Router to pass the href prop to the child component, even if doesn’t look like an anchor. Actually (because of the component prop we set earlier), the grandchild will be an anchor, and Material UI will forward the href prop to it! The rendered a element now has a proper href attribute, both client-side and in the server-side generated HTML. Good SEO, for free!

The same trick can be used with the Typography component:

import React from "react";
import Link from "next/link";
import Button from "@material-ui/core/Typography";

export default MyLink = () => (
  <Link href="/pricing" passHref>
    <Typography variant="caption" component="a">Managed version</Typography>
  </Link>
);

This time, we created a link looking like a caption!

That’s all for today. Have fun with Next and Material UI! For more tricks about JavaScript (among various other technologies), follow me on Twitter!

Upcoming conferences: AFUP Day, Web2Day, API Platform Meetup and more!

During the spring I’ll speak at several tech events about my projects Mercure (Go), API Platform (PHP, server-side and JS, client-side) and Symfony Panther (PHP/WebDriver):

AFUP Day, May 17th in Lille

Discover Symfony Panther, a brand new end-to-end testing tool using real web browsers for Symfony and PHP.

API Platform pre-Web2day meetup, June 3th in Nantes

Topic to be announced!

Web2Day, May 6th in Nantes

Mercure: UIs always synchronized with data in database

My company Les-Tilleuls.coop will have a booth during every of these events. Come, and let’s have a chat!
Also, I’ll announce my participation to some more events soon, stay tuned!

React ESI: Blazing Fast SSR

React ESI is a super powerful cache library for vanilla React and Next.js applications, that can make highly dynamic applications as fast as static sites. It provides a straightforward way to boost your application’s performance by storing fragments of server-side rendered pages in edge cache servers. It means that after the first rendering, fragments of your pages will be served in a few milliseconds by servers close to your end users! It’s a very efficient way to improve the performance and the SEO of your websites; and to dramatically reduce both your hosting costs and the energy consumption of these applications. Help the planet, use React ESI!

Because it is built on top of the Edge Side Includes (ESI) W3C specification, React ESI natively supports most of the well-known cloud cache providers including Cloudflare Workers, Akamai and Fastly. Of course, React ESI also supports the open source Varnish cache server that you can use in your own infrastructure for free (configuration example).

Also, React ESI allows to specify a different Time To Live (TTL) per React component and to generate the corresponding HTML asynchronously using a secure (signed) URL. The cache server fetches and stores in the cache all the needed fragments (the HTML corresponding to every React component), builds the final page and sends it to the browser. React ESI also allows components to (re-)render client-side without any specific configuration.