Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

burnett01/[email protected] hangs transferring specific files #64

Open
arobthearab opened this issue Aug 21, 2024 · 16 comments
Open

burnett01/[email protected] hangs transferring specific files #64

arobthearab opened this issue Aug 21, 2024 · 16 comments
Assignees
@Burnett01
Labels
help wanted Extra attention is needed investigate

Comments

@arobthearab
Copy link

Actual Behavior

Rsync action fails when attempting to use a path of "*.sh"

Expected Behavior

Rsync would treat the path as it would be treated in any rsync command, in this case to synchronize all shell scripts in the root directory of the repository.

Details

I have five rsync steps in my GitHub action, four of which work without a hitch, and are of this general form:

   - name: Deploy backend to ${{vars.TARGET}} ...
        uses: burnett01/[email protected]
        with:
          switches: -avzr --delete
          path: backend/
          remote_path: ${{vars.TARGET}}
          remote_host: ${{vars.HOST}}
          remote_port: ${{vars.PORT}}
          remote_user: ${{vars.USER}}
          remote_key: ${{secrets.KEY}}

but the fifth attempts to copy a pattern from the base directory of the repository:

 - name: Deploy installation scripts to ${{vars.TARGET}} ...
      uses: burnett01/[email protected]
      with:
        switches: -avz --delete
        path: "*.sh"
        remote_path: ${{vars.TARGET}}
        remote_host: ${{vars.HOST}}
        remote_port: ${{vars.PORT}}
        remote_user: ${{vars.USER}}
        remote_key: ${{secrets.KEY}}

The result of this operation is:

Connection reset by [redacted] port [redacted]
rsync: connection unexpectedly closed (0 bytes received so far) [sender]
rsync error: unexplained error (code 255) at io.c(231) [sender=3.2.7]

All the other successful steps use the same parameters.

Since all the other paths are relative I assume that "*.sh" would be the correct path specification to match all shell files in the root directory of the repository.

Based on logs at the receiving end, the first four rsyncs connect and work as expected, but the fifth to copy *.sh does not appear to connect at all.
@Burnett01
Copy link
Owner

Burnett01 commented Aug 21, 2024 •
edited
Loading

Hey @arobthearab thank you for your report.

I was not able to reproduce the issue I am afraid.

When I use *.sh as you did, rsync correctly found my test.sh file and synced it:

image

Can you please conduct the following steps to find out more:

  • Does a previous step in your deployment change the file permissions of the .sh files before using the action?
  • Can you try the verbose debug mode to find out more by adding -vvv to the switches
  • Please use an absolute path like /*.sh since the absolute path / represents your checked out github workspace

Greetings
Steve

@Burnett01 Burnett01 self-assigned this Aug 21, 2024
@Burnett01 Burnett01 added help wanted Extra attention is needed investigate labels Aug 21, 2024
@arobthearab
Copy link
Author

arobthearab commented Aug 21, 2024 via email

@Burnett01
Copy link
Owner

Hi @arobthearab

can you try using an absolute path like /*.sh since the absolute path / represents your checked out github workspace

@arobthearab
Copy link
Author

arobthearab commented Aug 21, 2024 via email

@nils-kt
Copy link

nils-kt commented Aug 22, 2024

@arobthearab could you please post your logs formatted directly on the GitHub page instead of through the email client? It's quite hard to read and not very professional when shared that way.

@arobthearab
Copy link
Author

Requesting the logs in another format is fine but I suggest you keep the editorial comments to yourself. Broadcasting them in a public forum is ABSOLUTELY not professional.

In summary, neither my firewall or my host are ever receiving the fifth connection for the specific files *.sh. I haven't run a pcap to see if I'm getting a half-open. But I can run between 2 and 6 rsync steps in other repositories without problems as long as the "path" is a directory - it's only when I try to sync specific files in the root of the repo that this occurs, and I have tried:

/*.sh
.sh
./.sh
install.sh

@nils-kt
Copy link

nils-kt commented Aug 22, 2024

If you're benefiting from open-source software for free in a commercial setting (which seems evident from your email signature), it's reasonable to put in some effort.

I couldn't replicate the error either, so it's likely an issue with your system. I recommend closing this issue, @Burnett01

@arobthearab
Copy link
Author

Both your response and downvoting are more evidence of unprofessional conduct. Since you are offering open source software for all use cases, and since it is failing in this use case, the problem is NOT with "my system," but with your software, since a manual rsync from a remote host yields correct results.

It is fair enough to say "we can't replicate this and will park the issue" - that would be the professional thing to do - it is another thing to rationalize and self-justify as you have done here. As someone who has been writing software which I have made freely available for over 40 years, I don't have much tolerance for your attitude.

@nils-kt
Copy link

nils-kt commented Aug 22, 2024

Hey @arobthearab,

It’s unfortunate that you’ve chosen to bring the discussion to this level. While feedback is certainly welcome, the tone and manner in which you’re expressing your concerns are far from the constructive collaboration that is so vital in the open-source community.

Your expectation that we can instantly replicate and resolve every potential issue shows a misunderstanding of how complex software projects and their maintenance work. What’s particularly frustrating is that while you expect help, the information you’ve provided is neither structured nor helpful. Instead of clear and precise information, you’ve sent us disorganized logs that are nearly impossible to sift through.

I understand that you’re feeling frustrated, but that doesn’t justify your aggressive accusations and personal attacks. If you’ve been developing software for over 40 years, you should know that respectful communication is key, especially when you’re relying on others for assistance.

I would therefore ask that in the future, you communicate in a more factual and constructive manner if you’re genuinely interested in finding a solution. This will make collaboration much smoother, and we can focus on solving the actual issue together.

Thank you.

@Burnett01
Copy link
Owner

Hello guys,
please do not fight :)
We are all developers trying to help one another ❤️

@nils-kt I don't think it's unprofessional posting the logs here, unless they contain sensitive information, but so far I didn't see this :)
While I agree that it's hard to read the logs.

Let's get back to the root cause of the issue - the rsync connection not being established.

As you mentioned @arobthearab,
the connection is not being made which may be indeed a connection issue from the action runner on Github. Can you try changing the remote path to use the same as for the working ones.
Let's try that.

@arobthearab
Copy link
Author

arobthearab commented Aug 22, 2024 via email

@arobthearab
Copy link
Author

arobthearab commented Aug 22, 2024 via email

@Burnett01
Copy link
Owner

Hey @arobthearab , no need to apologize :)

Yes that is correct. Let's try it with a remote path from a working one.
I just want to make sure that it's not the remote path causing this.

Greetings Steve

@arobthearab
Copy link
Author

I tried remote_paths as follows:
${{vars.TARGET}}/backend - works in another step
${{vars.TARGET}}/install - different explicit directory that does NOT already exist - it is NOT created
${{vars.TARGET}}/ - just adding an explicit ending slash

In each case the rsync step times out. So I moved the install.sh script to a specific install directory - same result (rsync times out).

I then duplicated a previous rsync (the third step in my action) including path, remote_path, and options, and it still hangs and times out.

I then commented out the third step so that the original fifth step (which was still a duplicate of the third step) is now the fourth step - SUCCESSFUL EXECUTION

Finally, I restored the final (now fourth) step with path=install/install.sh and remote_path=${{vars.TARGET}} - SUCCESSFUL EXECUTION

It appears that the fifth rsync step fails in my action no matter how it is formatted. I don't know if this is a networking limitation in the runner container. I verified that from an AWS EC2 instance (NOT a self-hosted runner, but simply running manual rsync commands) that I can establish more than four connections to my target server.

I went through the GitHub public documentation on GitHub-hosted runners and can't find any evidence of such a limit.

@arobthearab
Copy link
Author

PS - This also explains an earlier issue I had with appleboy/[email protected] (which I initially tried because I was only copying a specific file). It failed in the same way.

So I guess this is not a problem with rsync-deployments but with the GitHub-managed runner networking.

@Burnett01
Copy link
Owner

Hello @arobthearab
thank you for your extensive debugging efforts.

I will try to test this type of scenario (5x rsync steps) and will report back.

It could be a Github limitation as you mentioned. If I have more information I will report back.

Best greetings
Steve

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@Burnett01 Burnett01

Labels
help wanted Extra attention is needed investigate
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@Burnett01 @arobthearab @nils-kt