ls/ls-url: introduce --size #9854
Conversation
dvc/commands/ls/__init__.py
Outdated
| entries = _prettify(entries, with_color=True) | ||
| ui.write("\n".join(entries)) | ||
| entries = _prettify(entries, with_color=True, with_size=self.args.size) | ||
| ui.table(entries) |
There was a problem hiding this comment.
Kudos to @skshetry for ui.table, feels really nice to use.
There was a problem hiding this comment.
Note that ui.table() can be slow when we are printing a million lines. 😄
So we were using raw prints and "\n".join() to print everything at once.
There was a problem hiding this comment.
@skshetry Good point. I guess we can keep that optimization in the simplest case. I'll take a look.
|
I thought we had |
|
@efiop, do you mind humanizing the sizes by default? |
|
@skshetry I wanted to avoid doing that, because I really need byte sizes at this point, and if we support both, then we'll need to support some kind of size formatting option. Do you have particualr suggestions that you'd like? |
I think it could be just |
|
@skshetry Actually, I don't need bytes in CLI, so we can indeed just humanize by default and stop there (maybe introduce some fmt option in the future if we need it, otherwise there is --json that has byte sizes). Thanks! |
|
Btw, how are we getting sizes here? Would it be slow? |
|
@skshetry We are getting them from Btw, do you have a particular humanize library to recommend? I also thought we had it somewhere already, but indeed couldn't find it. |
I don' know a library for this, but maybe something like this would be enough for us? |
|
@efiop, I just remembered that you can use from tqdm import tqdm
tqdm.format_sizeof(1000 * 1024, divisor=1024)That's what I used in |
efiop
force-pushed
the
ls-sizes
branch
2 times, most recently
from
e8b011e
to
bf1d0ee
Compare
Codecov ReportPatch coverage:
Additional details and impacted files@@ Coverage Diff @@
## main #9854 +/- ##
==========================================
- Coverage 90.79% 90.77% -0.02%
==========================================
Files 471 471
Lines 35891 35900 +9
Branches 5185 5188 +3
==========================================
+ Hits 32587 32588 +1
- Misses 2713 2720 +7
- Partials 591 592 +1
☔ View full report in Codecov by Sentry. |
efiop
force-pushed
the
ls-sizes
branch
3 times, most recently
from
f442dfa
to
bc53d96
Compare
dvc/commands/ls/__init__.py
Outdated
| if size is None: | ||
| size = "" | ||
| else: | ||
| size = tqdm.format_sizeof(size, divisor=1024) |
There was a problem hiding this comment.
Will need to introduce some humanize util at some point when we'll need again.
I really need it only for studio right now, but this is really nice to have in a daily life as well. E.g. ``` $ dvc ls-url . --size 108 Metadata-and-Results.dvc 97 AudioMP3.dvc 97 AudioWAV.dvc 12181 README.md 54 .gitignore 168 .gitattributes 99 VideoFlash.dvc 312 LICENSE.txt 139 .dvcignore 96 AudioMP3 96 Asset 192 .dvc 64 Metadata-and-Results 64 AudioWAV 192 Scripts 64 VideoFlash 384 .git 96 .vscode ``` ``` $ dvc list . --size 139 .dvcignore 168 .gitattributes 54 .gitignore 96 .vscode 96 Asset 96 AudioMP3 97 AudioMP3.dvc 64 AudioWAV 97 AudioWAV.dvc 312 LICENSE.txt 64 Metadata-and-Results 108 Metadata-and-Results.dvc 12181 README.md 192 Scripts 64 VideoFlash 99 VideoFlash.dvc ```
I really need it only for studio right now, but this is really nice to have in a daily life as well.
Caveat: we just throw whatever we have in
sizefield, which means that fordvcfs, you might get a disk-usage size for a directory instead of its physical size that usually is completely useless (see your ls -la). I think that's acceptable for now, but we will needdvc dufordupurposes in the future.E.g.
Related https://github.com/iterative/studio/pull/6541