Reduce size of extension manager DB table?

Discuss Topics
#1

Discussion Topic

The database table tx_extensionmanager_domain_model_extension holds all extensions with every version information since the beginning of the TER. This means that there are more than 41500 entries.

That leads to 2 performance issues we all know:

  • The update in the extenison manager takes endless because the XML which is fetched from typo3.org is large and the parsing takes also quite a while.
  • The database table is on many installations the biggest ones with about 30mb.

I propose to drop old entries to improve the performance.

Impact

There are different possibilities

  • Drop all entries all entries which have a last update information before 26th January 2011 which is the release of 4.5.0. That would reduce the size by 50%.
  • Drop all entries before 25th march 2014 which is release of 6.2. which would reduce the size by 75%.
  • Provide a dynamic solution: Downloads from a TYPO3 8.7 would only download all versions which would be compatible to current LTS minus one (to able to force download of outdated versions), that would mean 7.6.

Pro

  • Better performance

Con

It would not be possible to download any older versions within the extension manager.

Organizational

Topic Initiator: Georg Ringer

3 Likes
#2

For 8.7 I would vote for option 2.
For master I would vote for 3 or 2 as fallback.

1 Like
#3

what about the possibility to select in the EM-config which main-versions should be fetched?
by default just extensions for the current version are fetched.
if you want information about other versions, you have to select them manually (older and/or newer).

3 Likes
#4

I’m fine with options 2 or 3, maybe like Markus suggested. Also being able to configure EM for a cut-off-date might be an option, which people could use to remove that limit or even raise the bar further. We could then even port it to 7 LTS (disabled by default).

1 Like
#5

The best would be the #3 IMO (dynamic solution). I would ask: are there still plans to rework EM ? (As I have understood, but I could be totally wrong - correct me - there where plans to make EM something like a visual interface for Composer) How this will affect the presence of the db table?

#6

Composer only won’t be any time soon

#7

I like the idea of making that configureable, with some good defaults.

#8

I confirm then my vote for solution #3

#9

I’d suggest an even more flexible solution of the dynamic option 3.
When retrieving or updating the extension list in the EM, either…

  1. Allow the user to choose what to import (e.g. only extensions which are compatible with the current TYPO3 version, or compatible with current and previous LTS, etc.), or…
  2. Re-work EM, so that the extension list does not get retrieved and stored locally at all, but all data from TER accessed in real-time via API calls for example.

Point 2 clearly requires further considerations, e.g. impact if the TYPO3 (TER) infrastructure is not reachable, etc.

#10

i would vote for #3 too.

But if it’s really a rework, a little bit (simple version) of the plugin-manager look of wp is very nice.
Things like:

  • “Last Updated: 1 month ago”
  • “Compatible with your version of WordPress” OR “Untested with your version of WordPress”
  • “More Details” -> link to the docu?
  • a little bit more description text would also be nice (not only if i hover over it)

so that the click on the key or title is not nesassary anymore (in the list view), to have a overview over one extension of the extension list.

1 Like
#11

This topic was automatically closed after 14 days. New replies are no longer allowed.