GitHub Actions Workflow Fails to Authenticate Earth Engine API, but Works Locally or Tutorial Request: Github actions for earthengine

[thekester]

Hello, I couldn't find a tutorial to properly configure Earth Engine on GitHub Actions, so I attempted to do it myself, but I'm encountering an error.

Description: The Earth Engine API test script works as expected on a local machine when authenticated, but the workflow fails during execution in GitHub Actions. The error encountered in the workflow suggests an issue with parsing the credentials JSON file, even though the same setup works locally.

https://github.com/thekester/earthengine-api-test

Error Trace:

Traceback (most recent call last):
  File "test.py", line 4, in <module>
    ee.Initialize()
json.decoder.JSONDecodeError: Expecting value: line 2 column 1 (char 1)

The credentials are stored as a GitHub secret (EARTHENGINE_CREDENTIALS) and are passed into the workflow, but something is going wrong during the authentication process in the GitHub Actions environment.

Steps to Reproduce:

Run the GitHub Actions workflow in this repository: [earthengine-api-test.](https://github.com/thekester/earthengine-api-test)
Observe that the workflow fails during the ee.Initialize() step with the error above.
Running the same script locally, with proper authentication, works without issue.

Expected Behavior: The workflow should initialize the Earth Engine API correctly and authenticate using the credentials provided via GitHub secrets, just as it does in a local environment.

Actual Behavior: The workflow fails with a JSONDecodeError during the API initialization step, indicating that the credentials are either not properly passed to the workflow or the JSON is not being decoded correctly.

The GitHub Actions workflow in this repository, earthengine-api-test. encounters an issue during execution, while the same process works locally without problems.

Additional Context: The script works locally, so the issue seems to be specific to the GitHub Actions environment. This may involve how secrets are being handled, or how the credentials file is being written and accessed within the workflow.

[thekester]

@jdbcode , do you have any ideas? I'm trying to include the ee package in a CI/CD pipeline but I'm encountering issues during the authentication phase.

[jdbcode]

Hmm, I wonder if the credentials formatting is causing a problem. Can you provide it without new lines and tabs - just a single line with one space between common and next field:

{"client_id": "value", "client_secret": "value", "refresh_token": "value", "project": "value"}

[thekester]

Hello @jdbcode,

Thank you for your response and suggestion! I went ahead and copied the content of the credentials file into the EARTHENGINE_CREDENTIAL environment variable, ensuring it was all on one line, with a single space between each field, as you recommended.

Unfortunately, I am still encountering the same issue during the authentication phase. You can view the error trace in this job log:

Earth Engine API Test Job Log](https://github.com/thekester/earthengine-api-test/actions/runs/10946792224/job/31575230100)

Here are also the images of the current setup and error trace:

I would appreciate any further advice you might have regarding how to resolve this issue. Thanks again for your help!

[thekester]

Thank you! It seems the issue has been successfully resolved, as indicated by the GitHub Actions log (https://github.com/jdbcode/earth_engine_github_actions/actions/runs/11354079027/job/31580510185). I'll give it a try in my own code as well.

[thekester]

[thekester]

Thank you @jdbcode, I’ve resolved the issue using the following configuration:{"client_id":"yourclientid.apps.googleusercontent.com","client_secret":"clientsecret","refresh_token":"refresh_token","project":"ee-idproject"} to obtain the idproject please used https://console.cloud.google.com/

[thekester]

Python Script to Reformat JSON

This Python script reads a JSON file that is formatted with indentation and line breaks, reformats it into a single line, and saves the result to another file.

`import json

Function to reformat the JSON

def reformat_json(input_file, output_file):
# Read the JSON from the input file
with open(input_file, 'r') as f:
data = json.load(f)

# Write the reformatted JSON to the output file
with open(output_file, 'w') as f:
    json.dump(data, f, separators=(',', ':'), ensure_ascii=False)

Specify the input and output file names

input_file = 'input.json' # Replace with your input file name
output_file = 'output.json' # Replace with your output file name

Call the function to reformat the JSON

reformat_json(input_file, output_file)

print(f"The JSON has been reformatted and saved to '{output_file}'.")`

Input JSON Structure

Create a file named input.json with the following content, using placeholders for your actual credentials:
{ "client_id": "yourclientid.apps.googleusercontent.com", "client_secret": "yourclientsecret", "refresh_token": "yourrefreshtoken", "project": "yourprojectid" }

Expected Output

After running the script, the content of output.json will be reformatted into a single line, like this:

{"client_id":"yourclientid.apps.googleusercontent.com","client_secret":"yourclientsecret","refresh_token":"yourrefreshtoken","project":"yourprojectid"}

[thekester]

Hi @jdbcode,

I've just updated my quick tutorial here for the tutorial request on setting up GitHub Actions for Earth Engine. Feel free to take a look if you need any inspiration or ideas.

Thanks again!

[jdbcode]

I'm glad you've got it figured out now. In the meantime I put together an example and now I see you have one too.

https://github.com/gee-community/ee-initialize-github-actions/blob/main/README.md

In yours, since you're writing the credentials secret to ~/.config/earthengine/credentials, you should not need to use google.oauth2.credentials.Credentials. You can probably just use ee.data.get_persistent_credentials() and pass that to ee.Initialize(credentials=credentials).

We should maybe try to combine these two tutorials. I'd like to add to mine the option to write the credentials file and use ee.data.get_persistent_credentials(). Also the preferred Google option: https://github.com/google-github-actions/auth

[thekester]

Thanks now i'm just using ee in the test.py and i'm using that in the workflow file - name: Configure Earth Engine token env: EARTHENGINE_TOKEN: ${{ secrets.EARTHENGINE_TOKEN }} run: | mkdir -p ~/.config/earthengine echo "$EARTHENGINE_TOKEN" > ~/.config/earthengine/credentials (https://github.com/thekester/earthengine-api-test/actions/runs/11359836842/workflow) I'll check the preferred Google option: https://github.com/google-github-actions/auth as soon as possible