Further automate Git Rev News

[chriscool]

Historical background:

This is a follow up from: #273

This is also about using AI to help summarize and select discussions on the mailing list as discussed in the issue for edition 120 starting with this comment: #747 (comment)

This is also about managing the existing scripts that are currently managed in: https://github.com/chriscool/getreleases

Next steps:

• Describe recent improvements
• Describe what's left to automate

cc @jnareb @mjaix @sivaraam @gitster @stepnem @dscho @mricon

[chriscool]

Recent improvement:

• Updated https://github.com/chriscool/getreleases/blob/master/README.md to describe the different scripts in the repo.
• New https://github.com/chriscool/getreleases/blob/master/git_ml_converter.py script to get a mailing list thread in text format.
• New https://github.com/chriscool/getreleases/blob/master/search_mailing_list.py script to initially select possibly interesting mailing list threads to read/analyze before selecting a few ones to be summarized for an article.
• The https://github.com/chriscool/getreleases/blob/master/publish_edition.sh script now creates the issue for the next Git Rev News edition.

[chriscool]

What's left to automate:

• Sending emails to the mailing list (draft email and announce email)

Emails are now copy-pasted from one edition to the next. It should be possible to automatically generate the content and the Cc list which depends on the content, and then to automatically send the email. The automation should probably allow the content to be checked and customized a bit before being sent.

This should likely be done mostly in the publish_edition.sh script.

• Ranking the possibly interesting threads

The new https://github.com/chriscool/getreleases/blob/master/search_mailing_list.py script generates a directory where possibly interesting mailing list threads to read/analyze are converted to text (using the https://github.com/chriscool/getreleases/blob/master/git_ml_converter.py script). Then these threads could be analyzed by an AI that could rank them from the most suitable to the least suitable for a Git Rev News article.

This could be done mostly either in the search_mailing_list.py script or in a new script.

• Generating an article from the best ranked threads

After the threads have been ranked an AI could automatically summarize a few threads we select (likely among the best ranked ones) to generate an article. The article could then be polished either from humans or other AIs.

This could be done mostly either in the search_mailing_list.py script or in a new script.

• Putting the content of the Releases section directly into the edition-XXX.md file

The content of the Releases section is generated automatically by the get_releases.py script in a releases.md file. So it currently has to be copy-pasted from that file into the edition-XXX.md file.

[dscho]

Fantastic timing! I am experimenting with a "daily digest"-generating system myself, the code is in a non-publishable state, but I have some very early result. Here is a digest of last year's January 25th (I wanted to backfill from Jan 1st, 2025 until today, and had to restart a couple of times). Not sure whether this is exactly the idea you were thinking about, though...

[jnareb]

Then these threads could be analyzed by an AI that could rank them from the most suitable to the least suitable for a Git Rev News article.

Are you thinking about classical Machine Learning approach, where the training dataset would be the list of past threads together with a label - which of those threads was chosen for Git Rev News, or something based on chatbot / Large Language Model?