Core Changelog documentation workflow revised

(Christian Kuhn) #1

Discussion Topic

This topic is about the organization, handling and maintenance of the “core” Changelog files we introduced since v7. We now have the third branch that comes with these docs, and this reveals some details not optimal in current handling.

Current situation and workflow

  • Patches that require .rst files add them to the “master” folder
  • Shortly before release these files are reviewed, various fixes are applied we missed during initial review, needed docs.typo3.org changes (especially to the “Reference” docs) are done. The doc files are then moved from the master folder to the target release (eg. 9.1) and say this way “This is reviewed, and all official reference docs have been adapted”.
  • In v8, we removed the v7 Changelog files
  • In v9, we kept the v8 Changelog files
  • The extension scanner from v9 uses the v8 Changelog files
  • The “View upgrade documentation” implemented since core v8 shows the docs files
  • docs.typo3.org renders:
    • Changelog v7 (from the v7 core branch, showing the v7 changes)
    • Changelog v8 (from the v8 core branch, showing the v8 changes)
    • Changelog master / v9 (from core master branch, showing v9 & v8 & master Changelog files)
  • Sometimes stable LTS branches (eg. 8.7.x) get additional .rst files, for instance if a security or other important patch is back ported that needs documentation. For v7, those were put into an additional folder 7.6.x/, in v8, those have been merged into 8.7/ folder.

Current pitfalls

  • With the “dependency” of extension scanner and “Docs viewer” to the docs files, we can’t delete docs from “previous LTS” in master anymore.
  • The rendering on docs.typo3.org show the same information multiple times (eg. v8 changes in “Changelog 8” and again in “Changlog latest” https://docs.typo3.org/typo3cms/Teams/CoreTeam/Index.html
  • Changes to v8 docs files are not always back ported to the v8 docs files of the v8 branch, but only fixed in master
  • New rest files back ported to “previous LTS” don’t have a clear location and can’t be easily found

Proposal

To simplify this workflow and have a working strategy for upcoming versions, we slightly adapt and clearify:

  • All docs (including v7, v8, …) are always in core master. .rst files are “usually” added to master/ folder.
  • The current pre-release workflow for master is kept: They are re-reviewed, fixed, references updated, then moved to the target folder (eg. 9.1).
  • If a change that needs documentation needs to be back ported to “previous LTS”, the file is put into eg. “8.7.x” folder on both master & v8 branch. A patch with docs back ported to v7, v8 and master would reside in 7.6.x in all branches.
  • v7, v8 and master contain identical files for their versions. For instance, v7 contains 7.0/, 7.1/ … 7.6/, 7.6.x/ folders, core v8 branch contains all these v7 files, too (!) and well as its v8 files. And master contains everything. We never delete “old” .rst files in master. We also take care that the contents of eg. the v7 & v8 folder is identical in v8 branch compared to master. This could be checked / taken care off automatically, or manually some days before releases.
  • docs.typo3.org renders only the core master branch and then contains everything automatically.

Tasks to do if we align on this

  • [DONE] Get the v7 Changelog files back into v8 and master branches
  • [DONE] Find those .rst files that have been added to 8.7/ folder after 8.7.0 release and separate them into 8.7.x/ folder in both v8 branch and master branch.
  • [DONE] Review changes between core master and v8 branch in the 8.x folders and align them
  • (maybe) Create a job to verify there are no differences in content and file number in v8 and master branch for the 8.x/ folders
  • [NOT DONE] Adapt docs.typo3.org rendering to only render current master. This will then show all files. Update: Rendering old branches will be kept and selectable with the version selector.
  • (maybe) Move the core Changelog display out of the docs.typo3.org “teams” folder to a more prominent place in main menu
  • [DONE] Update the workflow in main Howto.rst file of the Documentation folder

Pro

  • Clear and up-to-date structure on docs.typo3.org
  • Doc files merged to LTS releases in patch level version have a clear workflow and can be easily seen by admins
  • Workflow that can survive more LTS versions, no conflicts between files per core branch

Con

  • Yet another workflow we have to follow to when reviewing changes with .rst files that need back ports

Organizational

Topic Initiator: Christian Kuhn

(Ernesto Baschny) #2

Does the documentation needs to be part of the core git repository? Remember the time we wrote documentation separately in the wiki? That was not ideal, but a similar approach using git could probably get rid of most of the problems you are explaining above.

A dedicated separated git repo for these changelogs (similar to https://github.com/TYPO3-Documentation/TYPO3CMS-Reference-CoreApi). The workflow of creating new features, deprecating or breaking stuff for the developer could include creating a review request for the documentation in that repo. This repo could be cleaned up whenever it fits, and the doc team could render a documentation from that repo just like any other core documentation (TCA, CoreApi, etc).

The core repo would not have any of these directories, there is no need to do anything when releasing (rename directories, delete stuff, move etc), there is no need to backport rest files, etc. Improving these ChangeLog could be part of the “documentation” effort and not core code review process. docs.typo3.org rendering would be easier.

The extension scanner would need to have access to this repository for it’s work, which means for example provide the content as a “zip” which the tool downloads on demand.

Going one step further, the information for the extension scanner (Configuration/ExtensionScanner) could also be put on that separate repository, so that the tool could scan for problems on TYPO3 core versions that are not the current ones (i.e. when planning an upgrade) and we can also update these instructions independently from the core itself. So I have a 8.x installation and want to upgrade to 10.x, the tool fetches the “changelist for 9.x and 10.x” (rest + php matchers) and apply these rules.

(Georg Ringer) #3

Please keep the rst files in the core because rst files also change a lot during patchsets and any other place having those would feel wrong.

(Martin Bless) #4

The proposal sounds good to me. For ChangeLog documentation files I very much support the idea of letting them be part of the actual code merge.

Unfortunately the local rendering of the sysext:core docs (ChangeLogs) takes a lot of time. This will very annoying when docs accumulate.

I do have (I think) a rather simple solution for that and submitted a patchset https://review.typo3.org/#/c/55072/
See https://forge.typo3.org/issues/83325

(I forgot the Resolves: #83325 line)

(Riccardo De Contardi) #5

I don’t have a strong opinion on this topic.

(Christian Kuhn) #6

@Ernesto
I thought about extracting the Changelog docs from core, too, but came to the conclusion the current solution with having them in the main git repo is better: I love the workflow to provide and review “fully baked patches including docs” in one go. I think that’s a huge advantage. This works out very well since v7 and we always have a consistent state. I’d rate that so high that i rather think about how-to-sync to other branches than to have a second git repo which can easily lead to inconsistencies or forgotten docs.

(Markus Klein) #7

I fully support the suggestion.
Splitting docs off again would most probably lead to the same situation again we had for a too long time: outdated or no docs at all!

I’m not sure about the accumulation of rst files over the years though. But it’s okay to revisit this particular issue again in 10 years.

(Frank Naegler) #8

agree with Markus, the suggestion sounds good to me. thx.

(Jan Helke) #9

I am with Markus here. The benefits of “complete patches incl. documentation” outweighs the danger of outdated docs.

(Jigal Van Hemert) #10

For the next decade or so we can easily keep the rst files. There’s no telling how the PHP/CMS world looks like by then.

(Stefan Neufeind) #11

I agree as well. We can later split out files if we deem it necessary, maybe once/after they are language-checked etc. for a bigger release. But for reviews the benefit is they are part of the change and merged together with code at that point in time.

(Christian Kuhn) #12

This topic was automatically closed after 14 days. New replies are no longer allowed.

(Christian Kuhn) #13

Thanks for feedback on this.
All important tasks have been done meanwhile and implemented. The “How to handle .rst” documentation has been updated and the single log files aligned.
Topic is closed and nothing more needs to be done at the moment.