Native TYPO3 Headless initiative

Background
More and more CMS solutions integrate some kind of “Headless” approach, be it as a simple rest api (as available for Symfony) or more complex apis (as in Drupal’s GraphQL view).

For some reason, “Headless” has some fashion to it that draws customers magically in and may lead to loss of customer projects for the sole reason of “not being a headless CMS”.

This claim is, of course, plain wrong and without any substance (as proven by the many [1] [2] [3] examples where something like this has been done in the past).

One thing that these solutions all have in common, though, is that as an integrator or a developer, there is some kind of overhead (be it defining the entry point or the data to go into the output) for every table you would like to access.

Idea
The idea of this initiative is to settle on an out-of-the-box solution that can simply be required and then instantly allows the user (probably frontend developer) to access all data by a single entry point (say /api/) and have some kind of basic functionality, like listing all data from a table.

In order to ensure a quick and wide adoption of this initiative, such a solution should at least provide:

  1. Automated setup of entry point or wizard based setup in order to be able to define entry points
  2. Automated ExtbaseJsonView configuration based on TCA with possibility to override the configuration of a single entity inside the domain model or in a separate configuration (based on yaml or or TypoScript) with automated paths (like /api/my_extension/table_name_without_domain_model, e.g. /api/news/news)
  3. Configurable methods, based on TCA or other configuration (e.g. table.allowedMethods = GET, table.allowedMethods = GET,POST,PUT,DELETE)
  4. Possibility to override any path (uniquely by page tree) for entity access (e.g. /api/news/news => /api/news/
  5. Possibility to override the entry point controller
  6. Possibility to override the view, in order to generate something else than json (maybe GraphQL or any other new technology that might come up in the future).
  7. Support for authenticated requests (Thanks to @xperseguers)

I’ve got no idea about the time this initiative may take but I’m pretty sure that any time invested in developing this would be well spent and let the product and community grow by some extent.

Happy to hear your thoughts!

Best Regards
Tizian

4 Likes

I did some tests in that direction, long ago, using EXT:routing (https://github.com/xperseguers/t3ext-routing) however this quickly proved to be too strict and too cumbersome to use in real life so I switched to something much more lightweight with EXT:simple_api (https://github.com/xperseguers/t3ext-simple_api).

An interesting part of EXT:simple_api is the possibility to force an authenticated header to access some endpoints. In addition, other endpoints may be accessed either while being authenticated or not, and the response will differ.

IMHO, having support for authenticated requests (which should be generic enough to allow any business authentication to be used, not forcing e.g., fe_users) is a must have.

1 Like

I’ll add this to the list, thanks for your input!

hi @masterofd
Definitely, the headles cms topic is worth an investigation.
Right now there are a few extensions which add a possibility of delivering rest API for TYPO3. One of them is https://extensions.typo3.org/extension/restler/ the other one is simple_api mentioned by Xaview, and there are few more.

Strategically there is work being done in two connected areas:

  1. Persistence initiative (#cig-persistence on slack), where graphql api is part of the investigation, see module created by Artus https://github.com/TYPO3incubator/graphql

  2. PWA initiative where we’re investigating how to create PWA apps with TYPO3 as backend - meaning JS app which renders in browser and consumes API provided by TYPO3

Definitely there is a need to coordinate these 3 aspects/initiatives - persistence, headless, pwa.

2 Likes

I would propose to go for the jsonapi (https://jsonapi.org/) specification.

As we are already naming some other frameworks here have a look at a blog about the route Drupal took https://dri.es/headless-cms-rest-vs-jsonapi-vs-graphql

QraphQL looks quite powerful, the fact that you can retrieve / traverse the relations of a records in one go, seems very neat. But, besides picking a technology, I understood there is some preliminary work to be done beforehand. The current topic would be the relationship fields in TYPO3 CMS

It is very important to keep in mind that the current GraphQL extension is NOT an API for external clients! Thus it is NOT associated with the idea having a headless TYPO3 CMS.

Instead it is an internal (PHP only) read API for all managed tables, supporting access control, workspaces and localizations. As such it lies on a similar architectural level as an ORM layer.

Of course it should be possible to use this (internal) read API to create a REST/GraphQL/JSON API more easily for external clients than using the DBAL directly; especially when it has to be context aware (e.g. security, workspaces and localizations). And in this area I see some overlap between persistence and the other topics.

4 Likes

A kind of headless approach is done by Easy Admin. There all data needs to be retrieved through an common interface. The according access rights are automatically applied and the logged in (or anonymous) user only sees the content they are allowed to see.

Should the headless approach only allow to read or also to write data? How about the TYPO3-backend? that could also be powered by an API.

No need to reinvent the wheel here.

In fact, TYPO3 is able to run “headless” since years and a day -
it is just a matter of setting up a simple typoscript template and sending the correct and necessary headers.

I have two pagetypes configured for this, “simple” and “ajax”.
This setup could be very easily adapted to send out json or whatever.

We shouldn’t overcomplicate this, and communicate that TYPO3 is not just headless already,
but also easily adaptable to whatever your project needs.

2 Likes

Reminds me of the “blog discussions” of the “two-zero” years. As an outcome we had EXT:timtab back then … :wink:

Issue related: https://forge.typo3.org/issues/82839 (maybe there are more).

1 Like

I’m absolutely on your side about this point, but the fact that you have to build the types yourself and that there is not reliable scheme on which to build on might scare off extension developers, since you don’t have any interoperability between systems and therefore might have to recode everything everytime (in the worst case).

Also, it is true that almost every TYPO3 developer built an eID script at some point which is the basically the same as a headless endpoint and I guess all TYPO3 developers and integrators know about this, but it always comes with a cost that is
a.) sometimes hard to know how much time is needed,
b.) turns oftentimes out to be hard to document correctly,
c.) might differ from developer to developer (this might be a concern when you have a big team) and
d.) it might be difficult to update if any of the higher order internals of TYPO3 change

Therefore an out of the box solution provided by the CMS would be very nice.

@erredeco Susi seems to suggest that this indeed might be intended for v10, so this might indeed be unneeded in the end but I find the discussion important eitherway.

1 Like

My comment was more intended as a “yes, this is something we should do” not as a “it will absolutely be part of 10” - because someone still has to do it and as far as I know no-one is working on that concrete part yet, so the basic idea here is valid (and not worked on directly).

2 Likes

In my opinion, this is not how to understand a true headless software. Tío be truly headless, TYPO3 needs to loose all dependencies to cms-frontend and cms-backend. The core needs to provide an easy API to read and manage data, without the need of the Backend or the TypoScript renderer. Also, all extensions need to work with this, so the system should not crash, because an extension wants to use the fluid engine. I think a real headless TYPO3 ist extremely difficult, because the system is so deeply intertwined at several points with how data is presented and there is no real, obvious API on how to fetch content, without the need of intervention by extension authors or another developer who makes the presentation of data in an API comfortable way possible in the first place (that’s the part you mentioned with TypoScript templates)

1 Like

I don’t see why all extensions would need to be compliant with the new API. Managing the data via HTTP is an addtional way unless we would like to turn TYPO3 in a truly headless CMS. But I fail to see the point in that, so I would go the hybrid way. Which would require absolutely no changes to existing extensions.

Ofc, a headless API endpoint should be configurable from the consuming service in at least this aspects/dimensions:

  • language
  • workspace

It also has to take into account:

  • mountpoints
  • shortcut pages
  • access restrictions

How does that headless approach deal with menu rendering i.e. based on the hierarchy represented in the page-tree in the TYPO3 backend ? If you loose that relations at the consumer end, you degrade the CMS to a simple admin-interface for pages/contentelements.

1 Like

Interesting discussion. Im currently using TYPO3 as a main system and Shopware 5 to provide e-commerce features - especially check-out and the connection to post-checkout processes. Ecommerce / shop systems are specialized in delivering content based on the users behaviour but of course lack the CMS features of a system like TYPO3. I think TYPO3 would benefit from providing a content devlivery API without losing its own content rendering features. This would be a hybrid system. Theres an interesting article - sorry its in german - on that topic:

I am more likely to see a Decoupled CMS approach be more future-proof than a true headless CMS approach. Also, the trend in javascript-based frameworks clearly goes to a server-side pre-rendered initial HTML template that is transmitted to the client and the JS-App (be it vue/nuxt or react/angular-based) starts from there.

I am sorry if I ask something obvious.
But is the Structured Content Initiative a child of or has adopted this discussion?

I have read “Implement new rendering mode capabilities for different data formats (JSON / XML)” on https://typo3.org/community/teams/typo3-development/initiatives/structured-content.
Is this a pure coincidence? Is the Structured Content Initiative taking over from https://github.com/TYPO3-Initiatives/graphql? Am I mixing things up completely?

In which direction is this going? Thanks for giving some pointers.