Iterate through commits running conversion on file

[mpias670]

``Long story short, I have a repository with a bunch of commits containing a single .eapx file that I need to convert to .qeax.

Using an .exe called Lemontree Automation, my goal is to go commit by commit converting this .eapx to .qeax. I know how to do the conversion part, what I don't know how to do is iterate through each commit on each branch, dump the old .eapx file, and add the .qeax file. Anyone have insight on the iteration, removing, and adding pieces below?

The goal is this:

for each branch:  
     for each commit:  
          convert eapx -> qeax  
          remove .eapx  
          add .qeax  

[newren]

I don't know anything about yoru Lemontree Automation, but I'm going to assume that your Lemontree Automation command to convert the contents of one eapx file (without renaming it) is the following:

lemontree-automation-convert.exe --flag1 --flag2 -x -y FILENAME.eapx

My four flags are probably wrong, but you can insert whatever the appropriate flags are. If your program hardcodes that the filenames have to change, then write a script around it that calls the conversion script and then renames back. Anyway...

I think I'd probably do this conversion in two steps. First, I'd use the lint-history script to convert the contents (while keeping the filenames the same):

lint-history --relevant 'return filename.endswith(b".eapx")' \
      lemontree-automation-convert.exe --flag1 --flag2 -x -y

Then, after the contents have been converted, do another history conversion that renames the files. That should look something like:

git filter-repo --filename-callback '
    if filename.endswith(".eapx"):
        return filename.replace(".eapx", "qeax")
    else
        return filename
'

[mpias670]

@newren Thanks for the solution, however I run into an error with --filename-callback, everything else seems to work.

Here is my code:

Set-Location $repoLocation

#does the merge with the old eqpx file and empty qeax and renames the subsequent qeax as eapx to work with the below filter-repo command
lint-history --relevant 'return filename.endswith(b".eapx\")' \
	LemonTree.Automation.exe merge --theirs $emptyQEAX --mine ($repoLocation + $oldEAPX) --out ($repoLocation + $newQEAX)
	echo "Exitcode $LASTEXITCODE"
	
	#remove the old eapx
	Remove-Item $oldEAPX

	#we have to rename the created qeax to eapx to get git filter-repo to work
	$newEAPX = (Get-Item $newQEAX).BaseName + '.eapx'
	Rename-Item -force -path $newQEAX -NewName $newEAPX


git filter-repo --force --filename-callback '
	if filename.endswith(b".eapx"):
		return filename.replace(b".eapx", b".qeax")
'

And here is the error:

Traceback (most recent call last):
  File "C:\Users\piasm1\AppData\Local\Programs\Git\mingw64\libexec\git-core\git-filter-repo", line 4009, in <module>
    main()
  File "C:\Users\piasm1\AppData\Local\Programs\Git\mingw64\libexec\git-core\git-filter-repo", line 4006, in main
    filter.run()
  File "C:\Users\piasm1\AppData\Local\Programs\Git\mingw64\libexec\git-core\git-filter-repo", line 3941, in run
    self._parser.run(self._input, self._output)
  File "C:\Users\piasm1\AppData\Local\Programs\Git\mingw64\libexec\git-core\git-filter-repo", line 1409, in run
    self._parse_commit()
  File "C:\Users\piasm1\AppData\Local\Programs\Git\mingw64\libexec\git-core\git-filter-repo", line 1260, in _parse_commit
    self._commit_callback(commit, aux_info)
  File "C:\Users\piasm1\AppData\Local\Programs\Git\mingw64\libexec\git-core\git-filter-repo", line 3443, in _tweak_commit
    self._filter_files(commit)
  File "C:\Users\piasm1\AppData\Local\Programs\Git\mingw64\libexec\git-core\git-filter-repo", line 3359, in _filter_files
    print(type(self._filename_callback(str(change.filename))))
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<string>", line 3, in callback
NameError: name 'b' is not defined
fatal: stream ends early
fast-import: dumping crash report to .git/fast_import_crash_53936

There seems to be some issue with interpreting the bytestring here - the preceding b is being interpreted as a separate string?
here change.filename prints as b'.gitignore'. I'm not super familiar with bytestring usage/interpretation

[newren]

What shell are you using? Any chance you are using powershell? There are some reports (e.g. #435) saying that powershell is broken and cannot handle the outer quote being a ' character. So you may need to change that to ", and then change the inside quotes accordingly.

[newren]

The new --file-info-callback would allow you to do all the filtering in a single invocation of git-filter-repo (and not need to use lint-history any more). Not sure if that's easier or harder for you; just thought I'd mention it as an option.