--copyfrom gets companion parameter what to copy: all, all-dates, comments, tags, date-created, date-modified, date-last-opened

[porg]

User Story

• With ffmpeg I converted my-video.m4v to my-video.mp4 (just a container conversion no re-encoding).
• my-video.mp4 is a fresh file and all its metadata (Finder tags, Spotlight comment, file timestamps) is blank/initialized.
• I would like to keep the context of when I really worked on that file. At least "Date Created" should remain the original one, "Date Modified" could also be today, when the container conversion was performed.

Existing capabilities

• osxmeta --copyfrom my-video.m4v my-video.mp4 copies over Finder tags and Spotlight comments fine,
• but not the timestamps.

Proposed new feature

• -f, --copyfrom SOURCE_FILE stays as is but gets a companion parameter for fine grained control for what filesystem attributes to copy over:
• -F, --copy attr1[,attr2,attr3,…] accepts: all, all-dates, comments, tags, xattr, date-created, date-modified, date-last-opened
  • all is a shorthand to all attributes
  • all-dates a shorthand to all timestamps
  • comments,tags is what --copyfrom currently does from what I could observe

[RhetTbull]

You're right that --copyfrom does not currently copy date/time (or mode/permissions for that matter). It only works on metadata items. I'll think about this. I can see from your user story that this definitely could be useful but I also don't want too much scope creep into osxmetadata which currently is focused on metadata.

There are other ways to achieve this, for example, touch -r:

touch [-A [-][[hh]mm]SS] [-acfhm] [-r file] [-t [[CC]YY]MMDDhhmm[.SS]] file ...

-r      Use the access and modifications times from the specified file instead of the current time of day.

If implemented, it might make sense to simple do what cp -p does as this is "expected" behavior on the command line:

     -p    Cause cp to preserve the following attributes of each source file in the copy: modification time, access time, file flags,
           file mode, user ID, and group ID, as allowed by permissions.  Access Control Lists (ACLs) and Extended Attributes (EAs),
           including resource forks, will also be preserved.

           If the user ID and group ID cannot be preserved, no error message is displayed and the exit value is not altered.

           If the source file has its set-user-ID bit on and the user ID cannot be preserved, the set-user-ID bit is not preserved in
           the copy's permissions.  If the source file has its set-group-ID bit on and the group ID cannot be preserved, the set-group-
           ID bit is not preserved in the copy's permissions.  If the source file has both its set-user-ID and set-group-ID bits on, and
           either the user ID or group ID cannot be preserved, neither the set-user-ID nor set-group-ID bits are preserved in the copy's
           permissions.

By the way, copyfrom currently supports the following attributes though some of these are applicable to only certain file types (e.g. audio) and will have no effect if set on other files.

Click to expand list of copied attributes

kMDItemAppleLoopDescriptors: Specifies multiple pieces of descriptive information about a loop. kMDItemAppleLoopsKeyFilterType: Specifies key filtering information about a loop. Loops are matched against projects that often in a major or minor key. kMDItemAppleLoopsLoopMode: Specifies how a file should be played. kMDItemAppleLoopsRootKey: Specifies the loop's original key. The key is the root note or tonic for the loop, and does not include the scale type. kMDItemAudioChannelCount: Number of channels in the audio data contained in the file. kMDItemAudioEncodingApplication: The name of the application that encoded the data contained in the audio file. kMDItemAudioSampleRate: Sample rate of the audio data contained in the file. The sample rate is a float value representing hz (audio_frames/second). For example: 44100. 0, 22254. 54. kMDItemAudioTrackNumber: The track number of a song or composition when it is part of an album. kMDItemComposer: The composer of the music contained in the audio file. kMDItemIsGeneralMIDISequence: Indicates whether the MIDI sequence contained in the file is setup for use with a General MIDI device. kMDItemKeySignature: The key of the music contained in the audio file. For example: C, Dm, F#m, Bb. kMDItemLyricist: The lyricist, or text writer, of the music contained in the audio file. kMDItemMusicalGenre: The musical genre of the song or composition contained in the audio file. For example: Jazz, Pop, Rock, Classical. kMDItemMusicalInstrumentCategory: Specifies the category of an instrument. kMDItemMusicalInstrumentName: Specifies the name of instrument relative to the instrument category. kMDItemRecordingDate: The recording date of the song or composition. kMDItemRecordingYear: Indicates the year the item was recorded. For example, 1964, 2003, etc. kMDItemTempo: A float value that specifies the beats per minute of the music contained in the audio file. kMDItemTimeSignature: The time signature of the musical composition contained in the audio/MIDI file. For example: "4/4", "7/8". kMDItemAudiences: The audience for which the file is intended. The audience may be determined by the creator or the publisher or by a third party. kMDItemAuthors: The author, or authors, of the contents of the file. kMDItemAuthorAddresses: This attribute indicates the author addresses of the document. kMDItemCity: Identifies city of origin according to guidelines established by the provider. kMDItemComment: A comment related to the file. This differs from the Finder comment, kMDItemFinderComment. kMDItemContactKeywords: A list of contacts that are associated with this document, not including the authors. kMDItemContributors: The entities responsible for making contributions to the content of the resource. kMDItemCopyright: The copyright owner of the file contents. kMDItemCountry: The full, publishable name of the country or region where the intellectual property of the item was created, according to guidelines of the provider. kMDItemCoverage: The extent or scope of the content of the resource. kMDItemCreator: Application used to create the document content (for example "Word", "Pages", and so on). kMDItemDescription: A description of the content of the resource. The description may include an abstract, table of contents, reference to a graphical representation of content or a free-text account of the content. kMDItemDueDate: Date this item is due. kMDItemDurationSeconds: The duration, in seconds, of the content of file. A value of 10. 5 represents media that is 10 and 1/2 seconds long. kMDItemEmailAddresses: Email addresses related to this item. kMDItemEncodingApplications: Application used to convert the original content into it's current form. For example, a PDF file might have an encoding application set to "Distiller". kMDItemFinderComment: Finder comments for this file. kMDItemFonts: Fonts used in this item. You should store the font's full name, the postscript name, or the font family name, based on the available information. kMDItemHeadline: A publishable entry providing a synopsis of the contents of the file. For example, "Apple Introduces the iPod Photo". kMDItemIdentifier: A formal identifier used to reference the resource within a given context. kMDItemInstantMessageAddresses: Instant message addresses related to this item. kMDItemInstructions: Editorial instructions concerning the use of the item, such as embargoes and warnings. For example, "Second of four stories". kMDItemKeywords: Keywords associated with this file. For example, "Birthday", "Important", etc. kMDItemLanguages: Indicates the languages of the intellectual content of the resource. Recommended best practice for the values of the Language element is defined by RFC 3066. kMDItemNumberOfPages: Number of pages in the document. kMDItemOrganizations: The company or organization that created the document. kMDItemPageHeight: Height of the document page, in points (72 points per inch). For PDF files this indicates the height of the first page only. kMDItemPageWidth: Width of the document page, in points (72 points per inch). For PDF files this indicates the width of the first page only. kMDItemParticipants: The list of people who are visible in an image or movie or written about in a document. kMDItemPhoneNumbers: Phone numbers related to this item. kMDItemProjects: The list of projects that this file is part of. For example, if you were working on a movie all of the files could be marked as belonging to the project "My Movie". kMDItemPublishers: The entity responsible for making the resource available. For example, a person, an organization, or a service. Typically, the name of a publisher should be used to indicate the entity. kMDItemRecipients: Recipients of this item. kMDItemRecipientAddresses: This attribute indicates the recipient addresses of the document. kMDItemRights: Provides a link to information about rights held in and over the resource. kMDItemStarRating: User rating of this item. For example, the stars rating of an iTunes track. kMDItemStateOrProvince: Identifies the province or state of origin according to guidelines established by the provider. For example, "CA", "Ontario", or "Sussex". kMDItemTitle: The title of the file. For example, this could be the title of a document, the name of a song, or the subject of an email message. kMDItemVersion: The version number of this file. kMDItemWhereFroms: Describes where the file was obtained from. kMDItemAuthorEmailAddresses: This attribute indicates the author of the emails message addresses. (This is always the email address, and not the human readable version). kMDItemRecipientEmailAddresses: This attribute indicates the recipients email addresses. (This is always the email address, and not the human readable version). kMDItemTheme: Theme of the this item. kMDItemSubject: Subject of the this item. kMDItemCFBundleIdentifier: If this item is a bundle, then this is the CFBundleIdentifier. kMDItemInformation: Information about the item. kMDItemURL: Url of the item. kMDItemPixelHeight: The height, in pixels, of the contents. For example, the image height or the video frame height. kMDItemPixelWidth: The width, in pixels, of the contents. For example, the image width or the video frame width. kMDItemPixelCount: The total number of pixels in the contents. Same as kMDItemPixelWidth x kMDItemPixelHeight. kMDItemColorSpace: The color space model used by the document contents. For example, "RGB", "CMYK", "YUV", or "YCbCr". kMDItemBitsPerSample: The number of bits per sample. For example, the bit depth of an image (8-bit, 16-bit etc. . . ) or the bit depth per audio sample of uncompressed audio data (8, 16, 24, 32, 64, etc. . ). kMDItemFlashOnOff: Indicates if a camera flash was used. kMDItemFocalLength: The actual focal length of the lens, in millimeters. kMDItemAcquisitionMake: The manufacturer of the device used to aquire the document contents. kMDItemAcquisitionModel: The model of the device used to aquire the document contents. For example, 100, 200, 400, etc. kMDItemISOSpeed: The ISO speed used to acquire the document contents. kMDItemOrientation: The orientation of the document contents. Possible values are 0 (landscape) and 1 (portrait). kMDItemLayerNames: The names of the layers in the file. kMDItemWhiteBalance: The white balance setting used to acquire the document contents. Possible values are 0 (auto white balance) and 1 (manual). kMDItemAperture: The aperture setting used to acquire the document contents. This unit is the APEX value. kMDItemProfileName: The name of the color profile used by the document contents. kMDItemResolutionWidthDPI: Resolution width, in DPI, of this image. kMDItemResolutionHeightDPI: Resolution height, in DPI, of this image. kMDItemExposureMode: The exposure mode used to acquire the document contents. kMDItemExposureTimeSeconds: The exposure time, in seconds, used to acquire the document contents. kMDItemEXIFVersion: The version of the EXIF header used to generate the metadata. kMDItemAlbum: The title for a collection of media. This is analagous to a record album, or photo album. kMDItemHasAlphaChannel: Indicates if this image file has an alpha channel. kMDItemRedEyeOnOff: Indicates if red-eye reduction was used to take the picture. kMDItemMeteringMode: The metering mode used to take the image. kMDItemMaxAperture: The smallest f-number of the lens. Ordinarily it is given in the range of 00. 00 to 99. 99. kMDItemFNumber: The diameter of the diaphragm aperture in terms of the effective focal length of the lens. kMDItemExposureProgram: The class of the exposure program used by the camera to set exposure when the image is taken. Possible values include: Manual, Normal, and Aperture priority. kMDItemExposureTimeString: The time of the exposure. kMDItemEXIFGPSVersion: The version of GPSInfoIFD in EXIF used to generate the metadata. kMDItemAltitude: The altitude of the item in meters above sea level, expressed using the WGS84 datum. Negative values lie below sea level. kMDItemLatitude: The latitude of the item in degrees north of the equator, expressed using the WGS84 datum. Negative values lie south of the equator. kMDItemLongitude: The longitude of the item in degrees east of the prime meridian, expressed using the WGS84 datum. Negative values lie west of the prime meridian. kMDItemTimestamp: The timestamp on the item. This generally is used to indicate the time at which the event captured by the item took place. kMDItemSpeed: The speed of the item, in kilometers per hour. kMDItemGPSTrack: The direction of travel of the item, in degrees from true north. kMDItemImageDirection: The direction of the item's image, in degrees from true north. kMDItemNamedLocation: The name of the location or point of interest associated with the item. The name may be user provided. kMDItemAudioBitRate: The audio bit rate. kMDItemCodecs: The codecs used to encode/decode the media. kMDItemDeliveryType: The delivery type. Values are "Fast start" or "RTSP". kMDItemMediaTypes: The media types present in the content. kMDItemStreamable: Whether the content is prepared for streaming. kMDItemTotalBitRate: The total bit rate, audio and video combined, of the media. kMDItemVideoBitRate: The video bit rate. kMDItemDirector: Directory of the movie. kMDItemProducer: Producer of the content. kMDItemGenre: Genre of the movie. kMDItemPerformers: Performers in the movie. kMDItemOriginalFormat: Original format of the movie. kMDItemOriginalSource: Original source of the movie. kMDItemDownloadedDate: Date the item was downloaded. _kFinderColor: Color label in the Finder _kFinderStationeryPad: Whether or not item is a Stationary Pad item in Finder _kMDItemUserTags: Finder tags

[porg]

1. Thanks for confirming that timestamps are not copied and for the list of attributes that currently get copied.
   For readability sake thanks that you have put your long list into a spoiler tag.

2. Thanks for considering it. In combination with --copypairs src1 dst1 src2 dst2 src3 dst3 … and --copylist src1 src2 src3 dst1 dst2 dst3 … #73 it unfolds its batch processing / repairing / fixing potential.

3. Regarding the danger of feature creep: I know there are other tools to copy timestates specifically. But from a user standpoint "osxmeta" is the software which helps me backup/restore all my metadata conveniently (timestamps included in the JSON). In addition the --copyfrom allows me to mirror metadata in cases where no metadata-backup was made in time but disaster or bad software stroke but where the originals are still available to consider them for their metadata. So my proposal is not really exploding scope, but fitting right into that theme. Would love if you could give it a consideration!

[porg]

• touch -r only can copy Date Modified and Date Last Accessed.
• Date Created can currently only be manipulated with SetFile.
  • SetFile is already deprecated for a while, but Apple kept it, b/c it's still the only command line tool supporting to get/set many filesystem flags.
• I found scripts that I authored and co-authored with @gingerbeardman which I could modify to achieve the goals of this issue --copyfrom gets companion parameter what to copy: all, all-dates, comments, tags, date-created, date-modified, date-last-opened #71 and --copypairs src1 dst1 src2 dst2 src3 dst3 … and --copylist src1 src2 src3 dst1 dst2 dst3 … #73
  • They utilize SetFile and osxmeta and have already provisions to get a file list and copy all of the possible timestamps and have a nice output. I would need to adapt the pairing and list-splitting logic.
• Those scripts are:
  • cp-date-modified.sh
  • archive-order-to-metadata.sh developed in this context
• But I'm a UX designer not a developer. That ship sailed a long time ago.
  • Writing a crappy shellscript and offering as a gist is the best I can probably do.
  • With the whole overhead of testing and devops (git or gist) and the context switching of scripting after not having done so for a while, re-evaluated probably takes not 3-4h but realistically 3-4 days 😉 🙁
  • I would like to have this functionality properly available for all interested audiences.
  • Being part of osxmeta, which is professionally developed in comparison to the best I'd kick off, would increase that chances tremendously I would say.
  • The least i could do now is share my scripts with you, which you as an active developer can make more of, than me having had a pause of zero scripting for some months now. Maybe inspirational.

[RhetTbull]

I think that adding the ability to restore time (and maybe permissions) does make sense but that functionality should be added also to the backup/restore. I'll take a look at implementing this but it'll be a while as I've got a number of other projects I'm working on at the moment.

[porg]

I inspected the JSON and now realized that timestamps and permissions were indeed not backed up.

• The plethora of Mac timestamps really tell you a lot and are so useful. Would be worthy to have them in --backup, --restore and also in --copyfrom. I am glad you recognize the backup value of this too.

• The permissions matter a bit less for general purposes, but matter in some use cases (restoring software directories, or shared directories).

For all use cases (--backup, --restore, --copyfrom (possibly in combo with paste and xargs) ) you may want to be able to set that not ALL attributes are included but only a certain subset:

1. --backup

• a) the default is to include all.
• b) but there are possible scenarios where you may not want to include all, e.g. where one may only want to backup tags (which are used uniformly across a team) but not comments (which are individual or potentially private) or not file owner names or numerical-IDs (for privacy concerns).
• c) Ofc opening the possibility that --backup does not include all attributes complicates the exception handling logic if on the --restore or --copyfrom side you request certain attributes which were not included in the --backup. So in doubt --backup shall not have that attribute filtering capabilities.

2. --restore : Likely that you want only certain attributes to be restored (e.g. comments) or certain attributes NOT restored on purpose (e.g. the modification date shall be now, to have an indication of the restoration).
3. --copyfrom: Even more likely that you want to use that very specifically.

So the companion parameter originally suggested as --copy should have a more general name if it can be used together with --restore, --copyfrom and potentially also --backup. I propose --include and --exclude. That way you must not work in an enumerate fashion. But can say "all but A,B" or "only X,Y".