Skip to content

Commit

Permalink
documented compression/packing to tar and tar.gz
Browse files Browse the repository at this point in the history
  • Loading branch information
tobozo committed Feb 1, 2025
1 parent e8dc5f3 commit c743e43
Showing 1 changed file with 58 additions and 32 deletions.
90 changes: 58 additions & 32 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -50,8 +50,6 @@ Limitations
- ESP32-targz decompression can only have one **output** filesystem (see *Support Matrix*), and it must be set at compilation time (see *Usage*).
This limitation does not apply to the **input** filesystem/stream.

- Only indirect tar-to-gz compression is available on ESP8266-arduino (an intermediate tar file must be used).



Support Matrix
Expand Down Expand Up @@ -300,8 +298,24 @@ ESP32 Only: Direct Update (no intermediate file) from `.tar.gz.` stream
```
LZPacker::compress() signatures:
-------------------------------
```cpp
// buffer to stream (best compression)
size_t compress( uint8_t* srcBuf, size_t srcBufLen, Stream* dstStream );
// buffer to buffer (best compression)
size_t compress( uint8_t* srcBuf, size_t srcBufLen, uint8_t** dstBufPtr );
// stream to buffer
size_t compress( Stream* srcStream, size_t srcLen, uint8_t** dstBufPtr );
// stream to stream
size_t compress( Stream* srcStream, size_t srcLen, Stream* dstStream );
// stream to file
size_t compress( Stream* srcStream, size_t srcLen, fs::FS*dstFS, const char* dstFilename );
// file to file
size_t compress( fs::FS *srcFS, const char* srcFilename, fs::FS*dstFS, const char* dstFilename );
// file to stream
size_t compress( fs::FS *srcFS, const char* srcFilename, Stream* dstStream );
```

Compress to `.gz` (buffer to stream)
-------------------------------
Expand Down Expand Up @@ -348,82 +362,94 @@ Compress to `.gz` (stream to stream)
```



TarPacker::pack_files() signatures:
-------------------------------
```cpp
int pack_files(fs::FS *srcFS, std::vector<dir_entity_t> dirEntities, Stream* dstStream, const char* tar_prefix=nullptr);
int pack_files(fs::FS *srcFS, std::vector<dir_entity_t> dirEntities, fs::FS *dstFS, const char*tar_output_file_path, const char* tar_prefix=nullptr);
```
Pack to `.tar` (entities to File)
-------------------------------
```C
std::vector<TAR::dir_entity_t> dirEntities;
TarPacker::collectDirEntities(&dirEntities, &LittleFS, "/folder/to/pack", 3);
TarPacker::collectDirEntities(&dirEntities, &LittleFS, "/folder/to/pack");
auto packedSize = TarPacker::pack_files(&LittleFS, dirEntities, &LittleFS, "/my.archive.tar");
```

Pack to `.tar` (entities to Stream)
-------------------------------
```C
std::vector<TAR::dir_entity_t> dirEntities;
TarPacker::collectDirEntities(&dirEntities, &LittleFS, "/folder/to/pack", 3);
TarPacker::collectDirEntities(&dirEntities, &LittleFS, "/folder/to/pack");
File tarOutfile = LittleFS.open("/my.archive.tar", "w");
TarPacker::collectDirEntities(&dirEntities, &LittleFS, "/folder/to/pack", 3);
size_t packedSize = TarPacker::pack_files(&LittleFS, dirEntities, &tarOutfile);
tarOutfile.close();
```
Pack & compress to `.tar.gz` File (using intermediate tar file)
-------------------------------
TarGzPacker::compress() signatures:
-------------------------------
```C
const char* temporaryTarFile = "/.tmp.tar"; // will be deleted after use
std::vector<TAR::dir_entity_t> dirEntities;
TarPacker::collectDirEntities(&dirEntities, &LittleFS, "/folder/to/compress", 3);
size_t compressedSize = TarGzPacker::compress(&LittleFS, dirEntities, temporaryTarFile, &LittleFS, "/my.compressed.archive.tar.gz");
```cpp
int compress(fs::FS *srcFS, const char* srcDir, Stream* dstStream, const char* tar_prefix=nullptr);
int compress(fs::FS *srcFS, const char* srcDir, fs::FS *dstFS, const char* tgz_name, const char* tar_prefix=nullptr);
int compress(fs::FS *srcFS, std::vector<dir_entity_t> dirEntities, Stream* dstStream, const char* tar_prefix=nullptr);
int compress(fs::FS *srcFS, std::vector<dir_entity_t> dirEntities, fs::FS *dstFS, const char* tgz_name, const char* tar_prefix=nullptr);
```


Pack & compress to `.tar.gz` file/stream (no filtering on source files/folders list, recursion applies)
-------------------------------
```C
File TarGzOutFile = LittleFS.open("/my.archive.tar.gz", "w");
size_t compressedSize = TarGzPacker::compress(&LittleFS/*source*/, "/folder/to/compress", &TarGzOutFile);
TarGzOutFile.close();
```

Pack & compress to `.tar.gz` Stream (using intermediate tar file)
Pack & compress to `.tar.gz` file/stream
-------------------------------

```C
const char* temporaryTarFile = "/.tmp.tar"; // will be deleted after use
std::vector<TAR::dir_entity_t> dirEntities;
TarPacker::collectDirEntities(&dirEntities, &LittleFS, "/folder/to/compress", 3);
TarPacker::collectDirEntities(&dirEntities, &LittleFS/*source*/, "/folder/to/compress");
// eventually filter content from dirEntities
File TarGzOutFile = LittleFS.open("/my.archive.tar.gz", "w");
size_t compressedSize = TarGzPacker::compress(&LittleFS, dirEntities, temporaryTarFile, &TarGzOutFile);
size_t compressedSize = TarGzPacker::compress(&LittleFS/*source*/, dirEntities, &TarGzOutFile);
TarGzOutFile.close();
```


:warning: Direct tar-to-gz compression uses FreeRTOS tasks and is unavailable On ESP8266-arduino. Use the version with intermediate file instead.
Pack & compress to `.tar.gz` Stream (no intermediate tar file)
Pack & compress to `.tar.gz` file (no filtering on source files/folders list, recursion applies)
-------------------------------

```C
std::vector<TAR::dir_entity_t> dirEntities;
TarPacker::collectDirEntities(&dirEntities, &LittleFS, "/folder/to/compress", 3);
File TarGzOutFile = LittleFS.open("/my.archive.tar.gz", "w");
size_t compressedSize = TarGzPacker::compress(&LittleFS, dirEntities, &TarGzOutFile);
size_t compressedSize = TarGzPacker::compress(&LittleFS/*source*/, "/folder/to/compress", &LittleFS/*destination*/, "/my.archive.tar.gz");
TarGzOutFile.close();
```

Pack & compress to `.tar.gz` File (no intermediate tar file)

Pack & compress to `.tar.gz` file
-------------------------------

```C
std::vector<TAR::dir_entity_t> dirEntities;
TarPacker::collectDirEntities(&dirEntities, &LittleFS, "/folder/to/compress", 3);
TarPacker::collectDirEntities(&dirEntities, &LittleFS/*source*/, "/folder/to/compress");
// eventually filter content from dirEntities
File TarGzOutFile = LittleFS.open("/my.archive.tar.gz", "w");
size_t compressedSize = TarGzPacker::compress(&LittleFS, dirEntities, &LittleFS, "/my.archive.tar.gz");
size_t compressedSize = TarGzPacker::compress(&LittleFS/*source*/, dirEntities, &LittleFS/*destination*/, "/my.archive.tar.gz");
TarGzOutFile.close();
```
Callbacks
TarGzUnpacker/GzUnpacker/TarUnpacker Callbacks
---------
```C
Expand Down Expand Up @@ -477,7 +503,7 @@ Callbacks
```

Return Codes for TarGzUnpacker/GzUnpacker/TarUnpacker
TarGzUnpacker/GzUnpacker/TarUnpacker Return Codes
------------

`*Unpacker->tarGzGetError()` returns a value when a problem occured:
Expand Down

0 comments on commit c743e43

Please sign in to comment.