Cannot stop an operation or limit max file size to avoid crashing the server

[dancherb]

As I stream the server to the user's browser via Express, I monitor the ZIP size and call stream.destroy() and res.end() to end the operation if it exceeds the max file size.

This ends the stream to the user's browser and causes them to recieve a corrupt ZIP (no problem, we'll send them an email alert explaining what happened).

However - under the hood, it seems like s3-zip continues its operation and memory continues to fill up. In a production environment on Heroku, our web app crashes way before stream.destroy is even called due to memory limits.

How can we implement a max file size, or stop all operations if the ZIP being built reaches a certain size? Otherwise, any implementation of S3-zip makes a server easy susceptible to overloading and crashing.

Thanks!

import s3_zip from 's3-zip'

const maxSizeBytes = 300 * 1024 * 1024

const stream = s3_zip.archive(
        { s3, bucket: process.env.S3_BUCKET, debug: true }, 
        '', 
        sourcePaths, 
        finalPaths
    );
    
    let currentSize = 0;
    
    stream.on('data', (data) => {
        currentSize += data.length;
        
        if(currentSize > maxSizeBytes) { 
            stream.destroy(new Error('ZIP file size exceeds limit'));
        }
    });

    stream.on('error', (error) => {
        console.error('ZIP generation error:', error);
        
        res.end()
    });

    pipeline(stream, res, (error) => {
        if(error) {
            console.log('pipeline error =', error)
        } else {
            console.log('pipeline success');
        }
    });