Build retention strategy #573

petefoth · 2024-02-17T18:00:14Z

We don't have enough disk space to retain 3 builds for every device that is officially supported by LOS
We should aim to keep the two most recent builds for officially supported devices
When a device is 'promoted' from one Android version to a higher Android version (e.g. 20.0->21.0) we should keep the last build of the lower Android version, alongside the two most recent builds of the higher version, until the higher Android version is deemed to be stable. Definition of 'stable' to be decided :)
It would be good to keep the last build of devices which are no longer supported by LOS for as long as we can without running out of space. So, when a device 'drops off` the list of supported devices we should remove all but the mst recent build for rhat device
Many older builds - for 18.1 & 20.0 devices which have been 'dropped' by LOS (and 19.1 builds for one device monet which was dropped by LOS before a 20.1 build was made - are still present on the build server, even though they have been removed from the download server.
If space runs out on the current download server, builds for 'dropped' devices could be stored in an alternative location. This alternative - and 'archove server'? - does not need to provide OTA updates, sone fhere will be no more builds for these devices.
Initial implementation for this strategy will involve a manual process. Eventually, the strategy should be automated

The text was updated successfully, but these errors were encountered:

ArchangeGabriel · 2024-02-18T06:23:13Z

This seems very good to me. :)

petefoth · 2024-02-18T15:41:48Z

We have a number of old builds available on the build server., for devices which are no longer supported by LineagOS - see below

There are two or three builds per device. We should thin them out so there is only one build per device - the latest.
We should then rsync the directories to a new archive folder at https://download.lineage.microg.org/

Builds

d800/lineage-18.1-20220815-microG-d800.zip
d800/lineage-18.1-20220901-microG-d800.zip
d800/lineage-18.1-20221001-microG-d800.zip
d801/lineage-18.1-20220815-microG-d801.zip
d801/lineage-18.1-20220901-microG-d801.zip
d801/lineage-18.1-20221001-microG-d801.zip
d802/lineage-18.1-20220815-microG-d802.zip
d802/lineage-18.1-20220901-microG-d802.zip
d802/lineage-18.1-20221001-microG-d802.zip
d803/lineage-18.1-20220815-microG-d803.zip
d803/lineage-18.1-20220901-microG-d803.zip
d803/lineage-18.1-20221001-microG-d803.zip
d850/lineage-18.1-20220815-microG-d850.zip
d850/lineage-18.1-20220901-microG-d850.zip
d850/lineage-18.1-20221001-microG-d850.zip
d851/lineage-18.1-20220815-microG-d851.zip
d851/lineage-18.1-20220901-microG-d851.zip
d851/lineage-18.1-20221001-microG-d851.zip
d852/lineage-18.1-20220901-microG-d852.zip
d852/lineage-18.1-20221002-microG-d852.zip
d855/lineage-18.1-20220901-microG-d855.zip
d855/lineage-18.1-20221002-microG-d855.zip
f400/lineage-18.1-20220902-microG-f400.zip
f400/lineage-18.1-20221002-microG-f400.zip
jasmine_sprout/lineage-18.1-20220903-microG-jasmine_sprout.zip
jasmine_sprout/lineage-18.1-20221003-microG-jasmine_sprout.zip
jason/lineage-18.1-20220903-microG-jason.zip
jason/lineage-18.1-20221003-microG-jason.zip
kuntao/lineage-18.1-20220903-microG-kuntao.zip
kuntao/lineage-18.1-20221003-microG-kuntao.zip
lavender/lineage-18.1-20220903-microG-lavender.zip
lavender/lineage-18.1-20221003-microG-lavender.zip
lithium/lineage-18.1-20220704-microG-lithium.zip
lithium/lineage-18.1-20220718-microG-lithium.zip
ls990/lineage-18.1-20220904-microG-ls990.zip
ls990/lineage-18.1-20221004-microG-ls990.zip
m20lte/lineage-18.1-20220904-microG-m20lte.zip
m20lte/lineage-18.1-20221004-microG-m20lte.zip
montana/lineage-18.1-20240119-microG-montana.zip
obiwan/lineage-18.1-20221004-microG-obiwan.zip
obiwan/lineage-18.1-20221015-microG-obiwan.zip
platina/lineage-18.1-20221004-microG-platina.zip
platina/lineage-18.1-20221015-microG-platina.zip
scorpio/lineage-18.1-20220705-microG-scorpio.zip
scorpio/lineage-18.1-20220719-microG-scorpio.zip
suzu/lineage-18.1-20221005-microG-suzu.zip
suzu/lineage-18.1-20221015-microG-suzu.zip
twolip/lineage-18.1-20221005-microG-twolip.zip
twolip/lineage-18.1-20221015-microG-twolip.zip
vs985/lineage-18.1-20221005-microG-vs985.zip
vs985/lineage-18.1-20221016-microG-vs985.zip
wayne/lineage-18.1-20221005-microG-wayne.zip
wayne/lineage-18.1-20221016-microG-wayne.zip
whyred/lineage-18.1-20221005-microG-whyred.zip
whyred/lineage-18.1-20221016-microG-whyred.zip
X00P/lineage-18.1-20220720-microG-X00P.zip
X00P/lineage-18.1-20220819-microG-X00P.zip
X01AD/lineage-18.1-20220720-microG-X01AD.zip
X01AD/lineage-18.1-20220820-microG-X01AD.zip
X01BD/lineage-18.1-20220706-microG-X01BD.zip
X01BD/lineage-18.1-20220720-microG-X01BD.zip
YTX703F/lineage-18.1-20220905-microG-YTX703F.zip
YTX703F/lineage-18.1-20221016-microG-YTX703F.zip
YTX703L/lineage-18.1-20220905-microG-YTX703L.zip
YTX703L/lineage-18.1-20221016-microG-YTX703L.zip
monet/lineage-19.1-20230916-microG-monet.zip  
monet/lineage-19.1-20231007-microG-monet.zip

petefoth · 2024-02-18T16:39:56Z

We should thin them out so there is only one build per device - the latest.
We should then rsync the directories to a new archive folder at https://download.lineage.microg.org/

This has now been done :)

We still have 92GB of disk space available,of the total 590G. This should easily be enough room for the remaining 18.1 builds still to be completed this month

ArchangeGabriel · 2024-02-18T17:15:20Z

I think you can move caprip and Spacewar to the archive (they already have only one build however).

petefoth · 2024-02-18T19:07:24Z

I think you can move caprip and Spacewar to the archive (they already have only one build however).

Done! Thanks

petefoth · 2024-03-12T18:31:49Z

Notes

Build Retention

Strategy

When a device is 'new', only one build is available, on both Download server (DS) and Build server (BS)

While a device is supported by LOS, two builds will available on DS

Normally, both builds will be of the currently supported branch
When a device is first 'promoted' to a higher branch (e.g.c19.1 -> 20.0),
- one build on the DS is of the higher branch, and one is of the lower branch
- the final build of the old branch (e.g. 19.1) is available in the archive directory on the BS (in case of 'emergencies'). It is kept until the device is promoted again (e.g. 20.0 -> 21.0), when it will be replaced by the final build of the higher branch e.g. 20.0

When support is dropped for a device

the final two builds are available on the DS, in the normal device directory, for a limited time (i.e. until the next build run is performed).
after that time, one build (the final build for the device) is available in the archive directory on both the DS and the BS

Use cases to handle when building

Device is new this build
- create newlastbuild marker
- rsync with delete args, leaving one build on both servers
Device was new last build
- rsync without delete args, leaving two builds on both servers
- remove newlastbuild marker
Current build branch is the same as the previous build branch - the normal, default case
- rsync with delete args,
Device is promoted this build
- rsync with delete args,leaving two builds - one of each branch - on both servers
- after rsync
  - remove any existing files from the BS archive directory
  - move (copy & delete) old branch build to BS archive directory, leaving only one build in BS zips directory
Device was promoted last build
- rsync without delete args, leaving two builds on both servers
Device is dropped this build
- remove any existing files from the BS archive directory
- on both servers
- remove the penultimate build from the zips directory
- move the zips directory to the archive directory

Cases 1-6 can be handled per device in 'pre-' or post-build.sh

Case 6 must be handled differently

New devices logic

For device new this build, everything works: we have one build on the BS, that will get synced to DS - all is good
It's the next build we need to worry about. Then, there will be two builds on the BS and only one on the DS. We need to remove the --delete rsync args, or the previous build will be removed from the BS during rsync
So, on the first build we need a record that thsi was a new device, that can be read on the next round. Easiest to create a file,
- EITHER create an empty file using touch touch newdevice
- OR write the date of first build to the file echo "$builddate" > newdevice
Then, on the next build, we check for the existence of newdevice and set rsync_delete_args accordingly

Promoted Devices logic

For a promoted device, on the first build, clean_up.py will leave $DELETE_OLD_ZIPS builds on the BS:

(``$DELETE_OLD_ZIPS` - 1) builds of the current branch
one build - the final build - of the previous branch

We want to leave all of them available on the DS, so do nothing before we call rsync.

We also want the final build of the old branch in the archive directory 'just in case'

Next time we build, we want $DELETE_OLD_ZIPS builds of the current branch on both the BS and the DS. If we remove the old branch build *before we start the build, (i.e. in pre-build.sh OR after we call rsync on the previos build) , the we will finish the build with the right number on teh BS, and - after calling rsync with the --delete rsync args - on the DS too: the old build will be gone there too.

So, if we do it all in post build .sh

set rsync_delete_args="--delete --delete-excluded
If newdevice file exists, then
- set rsync_delete_args="" so the first build - which is not present on BS - does not get deleted from DS
call rsync
If this is a new device, then
- create the newdevice file
If this is a promoted device, then
- move old branch build to BS archivedirectory

device_promoted=false
new_device=false

there must be at least one ls "$BRANCH_NAME"*.zip file, becasue we've just built it

so how many are there

current_builds=$(ls -1 "$BRANCH".zip | wc -l)
total_builds=$(ls -1 lineage-.zip | wc -l)
if [ $current_builds -lt 2 ] ; then
if [ $total_builds eq $current_builds] ; then
new_device=true
else
device_promoted=true
fi
fi

files not matching a pattern

find . -type f -not -name "*.html"

builddate=$(date +%Y%m%d)
find . -type f -not -name "$BRANCH"-$builddate*" -print

if there's only one, how many builds in total?

builds=$()

does file exist

if test -f lineage-20.0*.zip; then
echo "file exists"
fi

count builds of current branch in current directory
current branch_num
branch_num=$(echo "$BRANCH" | sed 's/lineage-//') | sed/.//)
branch_num=$(echo "$branch_num" | sed 's/.//')
export branchnum=20

if branch_num = 20 : then
previous_branchnum=18
else
previous_branchnum="$((branch_num-1))"

ls lineage-"$branch_num"*.zip | wc -l
1

This was referenced Feb 17, 2024

Keep only the two most recent builds rather than three #506

Closed

Tidy up old 17.1, 18.1 & 19.1 builds on both servers #525

Closed

petefoth mentioned this issue Feb 20, 2024

Low disk space on download server (again) #505

Open

petefoth closed this as completed Mar 22, 2024

petefoth reopened this Mar 22, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Build retention strategy #573

Build retention strategy #573

petefoth commented Feb 17, 2024 •

edited

Loading

ArchangeGabriel commented Feb 18, 2024

petefoth commented Feb 18, 2024 •

edited

Loading

petefoth commented Feb 18, 2024

ArchangeGabriel commented Feb 18, 2024

petefoth commented Feb 18, 2024

petefoth commented Mar 12, 2024

Build retention strategy #573

Build retention strategy #573

Comments

petefoth commented Feb 17, 2024 • edited Loading

ArchangeGabriel commented Feb 18, 2024

petefoth commented Feb 18, 2024 • edited Loading

petefoth commented Feb 18, 2024

ArchangeGabriel commented Feb 18, 2024

petefoth commented Feb 18, 2024

petefoth commented Mar 12, 2024

Notes

Build Retention

Strategy

Use cases to handle when building

Promoted Devices logic

there must be at least one ls "$BRANCH_NAME"*.zip file, becasue we've just built it

so how many are there

files not matching a pattern

find . -type f -not -name "*.html"

if there's only one, how many builds in total?

petefoth commented Feb 17, 2024 •

edited

Loading

petefoth commented Feb 18, 2024 •

edited

Loading