Feat: add Gitlab support (#194)

## Description Added support for GitLab repository mirroring, allowing users to mirror repositories to and from GitLab instances. ### Features added: - GitLab API v4 integration - Support for high-level GitLab group namespaces (no nested subgroups) - Repository mirroring functionality for GitLab - Updated documentation in README and action.yml with GitLab configuration options #143 --------- Signed-off-by: Kareem Zarka <[email protected]>
Yikun · Jan 27, 2025 · 7a3ba42 · 7a3ba42
1 parent 0da6fbf
commit 7a3ba42
Show file tree

Hide file tree

Showing 3 changed files with 141 additions and 27 deletions.
diff --git a/README_en.md b/README_en.md
@@ -2,10 +2,11 @@
 
 English | [简体中文](./README.md)
 
-Action for mirroring repos between Hubs (like Github and Gitee)
+Action for mirroring repos between Hubs (like GitHub, Gitee, and GitLab).
 
 ## Tutorial
 
+### Mirror GitHub to Gitee
 ```yaml
 steps:
 - name: Mirror the Github organization repos to Gitee.
@@ -19,6 +20,21 @@ steps:
     # src_account_type: org
     # dst_account_type: org
 ```
+### Mirror GitHub to Gitlab
+```yaml
+steps:
+- name: Mirror the GitHub organization repos to GitLab.
+  uses: Yikun/hub-mirror-action@master
+  with:
+    src: github/kunpengcompute
+    dst: gitlab/kunpengcompute
+    dst_key: ${{ secrets.GITLAB_PRIVATE_KEY }}
+    dst_token: ${{ secrets.GITLAB_TOKEN }}
+    account_type: group
+    src_account_type: org
+    dst_account_type: group
+```
+
 
 Here is a workflow to mirror the kunpengcompute org repos from Github to Gitee, see more complete workflows in [here](https://github.com/Yikun/hub-mirror-action/tree/master/.github/workflows).
 
@@ -32,13 +48,13 @@ More than [100+](https://github.com/search?p=2&q=hub-mirror-action+%22account_ty
 #### Required
 - `src` source account, such as `github/kunpengcompute`, is the Github kunpengcompute account.
 - `dst` Destination account, such as `/kunpengcompute`, is the Gitee kunpengcompute account.
-- `dst_key` the private key to push code in destination account (default in ~/.ssh/id_rsa), you can see [generating SSH keys](https://docs.github.com/articles/generating-an-ssh-key/) to generate the pri/pub key, and make sure the pub key has been added in destination. You can set Github ssh key in [here](https://github.com/settings/keys)，set the Gitee ssh key in [here](https://gitee.com/profile/sshkeys).
-- `dst_token` the API token to create non-existent repo, You can get Github token in [here](https://github.com/settings/tokens), and the Gitee in [here](https://gitee.com/profile/personal_access_tokens).
+- `dst_key` the private key to push code in destination account (default in ~/.ssh/id_rsa), you can see [generating SSH keys](https://docs.github.com/articles/generating-an-ssh-key/) to generate the pri/pub key, and make sure the pub key has been added in destination. You can set Github ssh key in [here](https://github.com/settings/keys)，set the Gitee ssh key in [here](https://gitee.com/profile/sshkeys) set the Gitlab ssh key in [here](https://gitlab.com/-/user_settings/ssh_keys).
+- `dst_token` the API token to create non-existent repo, You can get Github token in [here](https://github.com/settings/tokens), and the Gitee in [here](https://gitee.com/profile/personal_access_tokens). and for GitLab in [here](https://gitlab.com/-/user_settings/personal_access_tokens) (Required scopes: api, read_api, read_repository, write_repository).
 
 #### Optional
-- `account_type` (optional) default is `user`, the account type of src and dst account, can be set to `org` or `user`，only support mirror between same account type (that is "org to org" or "user to user"). if u wanna mirror difference account type, use the `src_account_type` and `dst_account_type` please.
-- `src_account_type` (optional) default is `account_type`, the account type of src account, can be set to `org` or `user`.
-- `dst_account_type` (optional) default is `account_type`, the account type of dst account, can be set to `org` or `user`.
+- `account_type` (optional) default is `user`, the account type of src and dst account, can be set to `org` or `user`，For GitLab: can be set to `group` or `user`,only support mirror between same account type (that is "org to org" or "user to user" or "group to group"). if u wanna mirror difference account type, use the `src_account_type` and `dst_account_type` please.
+- `src_account_type` (optional) default is `account_type`, the account type of src account, can be set to `org` or `user` or `group`.
+- `dst_account_type` (optional) default is `account_type`, the account type of dst account, can be set to `org` or `user`r `group`.
 - `clone_style` (optional) default is `https`, can be set to `ssh` or `https`.When you are using ssh clone style, you need to configure the public key of `dst_key` to both source end and destination end.
 - `cache_path` (optional) let code clone in specific path, can be used with actions/cache to speed up mirror.
 - `black_list` (optional) the black list, such as “repo1,repo2,repo3”.
@@ -62,6 +78,19 @@ More than [100+](https://github.com/search?p=2&q=hub-mirror-action+%22account_ty
     dst_token: ${{ secrets.GITEE_TOKEN }}
     account_type: org
 ```
+#### GitLab group mirror, mirror from GitHub organization to GitLab group
+```yaml
+- name: GitLab group mirror
+  uses: Yikun/hub-mirror-action@master
+  with:
+    src: github/organization-name
+    dst: gitlab/group-name
+    dst_key: ${{ secrets.GITLAB_PRIVATE_KEY }}
+    dst_token: ${{ secrets.GITLAB_TOKEN }}
+    account_type: group
+    src_account_type: org
+    dst_account_type: group
+```
 
 #### White/Black list, only mirror the Yikun/hub-mirror-action but not Yikun/hashes
 ```yaml
@@ -185,11 +214,24 @@ Note: please configure the public key of `dst_key` to the source (github in here
 
   You can use below steps to add secrets, you can also see more in [Secrets](https://help.github.com/en/actions/configuring-and-managing-workflows/creating-and-storing-encrypted-secrets).
 
-  1. **Get Token and Key**，you can get them in [ssh key](https://gitee.com/profile/sshkeys) and [token](https://gitee.com/profile/personal_access_tokens).
-  2. **Add Secrets**，add settings-secrets in repo，like `GITEE_PRIVATE_KEY`、`GITEE_TOKEN`
+  1. **Get Token and Key**:
+     * For Gitee: Get SSH key from [here](https://gitee.com/profile/sshkeys) and token from [here](https://gitee.com/profile/personal_access_tokens)
+     * For GitLab: Get SSH key from [here](https://gitlab.com/-/user_settings/ssh_keys) and token from [here](https://gitlab.com/-/user_settings/personal_access_tokens)
+  2. **Add Secrets**，add settings-secrets in repo，like `GITEE_PRIVATE_KEY`、`GITEE_TOKEN` or `GITLAB_PRIVATE_KEY`、`GITLAB_TOKEN`
   3. **Add workflow**，add the workflow file into .github/workflows.
 
 ## Reference
 - [Hub mirror template](https://github.com/yi-Xu-0100/hub-mirror): A template repo to show how to use this action. from @yi-Xu-0100
 - [Auto-Sync GitHub Repositories to Gitee](https://github.com/ShixiangWang/sync2gitee): An introduction about how to use this action. from @ShixiangWang
 - [Use Github Action to sync reois to Gitee](http://yikun.github.io/2020/01/17/%E5%B7%A7%E7%94%A8Github-Action%E5%90%8C%E6%AD%A5%E4%BB%A3%E7%A0%81%E5%88%B0Gitee/): The blog for this action.
+
+## Platform-Specific Notes
+
+### GitLab
+- Uses `group` instead of `org` for organizational accounts
+- Only top-level groups are supported for mirroring. Nested subgroups (group/subgroup) are not supported yet
+- Requires API token with appropriate scopes (api, read_api, read_repository, write_repository)
+- When mirroring to GitLab, ensure your group/user has sufficient permissions to create repositories
+- GitLab.com has API rate limits of 2000 requests per minute for authenticated users
+- [GitLab API Documentation](https://docs.gitlab.com/ee/api/): Official GitLab API documentation
+- [GitLab Personal Access Tokens](https://docs.gitlab.com/ee/user/profile/personal_access_tokens.html): Guide for creating GitLab tokens
diff --git a/action.yml b/action.yml
@@ -1,5 +1,5 @@
 name: "Hub Mirror Action."
-description: "Mirror the organization repos between hub (github/gitee)."
+description: "Mirror the organization repos between hub (github/gitee/gitLab)."
 author: "yikun"
 branding:
   icon: "upload-cloud"
@@ -18,13 +18,13 @@ inputs:
     description: "Source name. Such as `github/kunpengcompute`."
     required: true
   account_type:
-    description: "The account type. Such as org, user."
+    description: "The account type. Such as org, user, group."
     default: 'user'
   src_account_type:
-    description: "The src account type. Such as org, user."
+    description: "The src account type. Such as org, user, group."
     default: ''
   dst_account_type:
-    description: "The dst account type. Such as org, user."
+    description: "The dst account type. Such as org, user, group."
     default: ''
   clone_style:
     description: "The git clone style, https or ssh."

diff --git a/hub-mirror/hub.py b/hub-mirror/hub.py
@@ -12,18 +12,25 @@ def __init__(
         src_account_type=None,
         dst_account_type=None,
     ):
-        # TODO: check invalid type
         self.account_type = account_type
         self.src_account_type = src_account_type or account_type
         self.dst_account_type = dst_account_type or account_type
         self.src_type, self.src_account = src.split('/')
         self.dst_type, self.dst_account = dst.split('/')
+        self._validate_account_type(
+            self.src_type, self.src_account_type, 'source'
+        )
+        self._validate_account_type(
+            self.dst_type, self.dst_account_type, 'destination'
+        )
         self.dst_token = dst_token
         self.session = requests.Session()
         if self.dst_type == "gitee":
             self.dst_base = 'https://gitee.com/api/v5'
         elif self.dst_type == "github":
             self.dst_base = 'https://api.github.com'
+        elif self.dst_type == "gitlab":
+            self.dst_base = 'https://gitlab.com/api/v4'
 
         prefix = "https://" if clone_style == 'https' else 'git@'
         suffix = "/" if clone_style == 'https' else ':'
@@ -33,16 +40,41 @@ def __init__(
         elif self.src_type == "github":
             self.src_base = 'https://api.github.com'
             self.src_repo_base = prefix + 'github.com' + suffix
+        elif self.src_type == "gitlab":
+            self.src_base = 'https://gitlab.com/api/v4'
+            self.src_repo_base = prefix + 'gitlab.com' + suffix
         self.src_repo_base = self.src_repo_base + self.src_account
         # TODO: toekn push support
         prefix = "git@" + self.dst_type + ".com:"
         self.dst_repo_base = prefix + self.dst_account
 
+    def _validate_account_type(self, platform_type, account_type, role):
+        if platform_type not in ("gitlab", "github", "gitee"):
+            raise ValueError(
+                f"Unsupported platform_type '{platform_type}' for {role}."
+            )
+        # gitlab ---> user or group
+        if platform_type == "gitlab":
+            if account_type not in ("user", "group"):
+                raise ValueError(
+                    f"For {platform_type}, {role} account_type must be "
+                    "either 'user' or 'group'."
+                )
+        # github/gitee ---> user or org
+        elif platform_type in ("github", "gitee"):
+            if account_type not in ("user", "org"):
+                raise ValueError(
+                    f"For {platform_type}, {role} account_type must be"
+                    "either 'user' or 'org'."
+                )
+
     def has_dst_repo(self, repo_name):
+        # gitlab ---> projects, github/gitee ---> repos
+        repo_field = "projects" if self.dst_type == "gitlab" else "repos"
         url = '/'.join(
             [
                 self.dst_base, self.dst_account_type+'s', self.dst_account,
-                'repos'
+                repo_field,
             ]
         )
         repo_names = self._get_all_repo_names(url)
@@ -52,17 +84,28 @@ def has_dst_repo(self, repo_name):
         return repo_name in repo_names
 
     def create_dst_repo(self, repo_name):
-        suffix = 'user/repos'
-        if self.dst_account_type == "org":
-            suffix = 'orgs/%s/repos' % self.dst_account
-        url = '/'.join(
-            [self.dst_base, suffix]
-        )
-        result = None
-        if self.dst_type == 'gitee':
-            data = {'name': repo_name}
-        elif self.dst_type == 'github':
-            data = json.dumps({'name': repo_name})
+        # gitlab ---> projects, github/gitee ---> repos
+        repo_field = "projects" if self.dst_type == "gitlab" else "repos"
+        if self.dst_type == "gitlab":
+            url = f"{self.dst_base}/{repo_field}"
+            headers = {'PRIVATE-TOKEN': self.dst_token}
+            data = {'name': repo_name, 'visibility': 'public'}
+            # If creating under a group, add namespace_id
+            if self.dst_account_type == "group":
+                group_id = self._get_gitlab_group_id(self.dst_account)
+                data['namespace_id'] = group_id
+        else:
+            suffix = f"user/{repo_field}"
+            if self.dst_account_type == "org":
+                suffix = f"orgs/{self.dst_account}/{repo_field}"
+            url = '/'.join(
+                [self.dst_base, suffix]
+            )
+            result = None
+            if self.dst_type == 'gitee':
+                data = {'name': repo_name}
+            elif self.dst_type == 'github':
+                data = json.dumps({'name': repo_name})
         if not self.has_dst_repo(repo_name):
             print(repo_name + " doesn't exist, create it...")
             if self.dst_type == "github":
@@ -87,6 +130,17 @@ def create_dst_repo(self, repo_name):
                     print("Destination repo creating accepted.")
                 else:
                     print("Destination repo creating failed: " + response.text)
+            elif self.dst_type == "gitlab":
+                response = self.session.post(
+                    url,
+                    data=data,
+                    headers=headers
+                )
+                result = response.status_code == 201
+                if result:
+                    print("Destination repo creating accepted.")
+                else:
+                    print("Destination repo creating failed: " + response.text)
         else:
             print(repo_name + " repo exist, skip creating...")
         # TODO(snowyu): Cleanup 2s sleep
@@ -95,10 +149,12 @@ def create_dst_repo(self, repo_name):
         return result
 
     def dynamic_list(self):
+        # gitlab ---> projects, github/gitee ---> repos
+        repo_field = "projects" if self.src_type == "gitlab" else "repos"
         url = '/'.join(
             [
-                self.src_base, self.src_account_type+'s', self.src_account,
-                'repos',
+                self.src_base, self.src_account_type + 's', self.src_account,
+                repo_field,
             ]
         )
         return self._get_all_repo_names(url)
@@ -118,3 +174,19 @@ def _get_all_repo_names(self, url, page=1):
             names = [i['name'] for i in items]
             return names + self._get_all_repo_names(url, page=page+1)
         return all_items
+
+    def _get_gitlab_group_id(self, group_name):
+        """Helper method to get GitLab group ID"""
+        url = f"{self.dst_base}/groups"
+        headers = {'PRIVATE-TOKEN': self.dst_token}
+        response = self.session.get(url, headers=headers)
+        if response.status_code == 200:
+            groups = response.json()
+            for group in groups:
+                if group['path'] == group_name:
+                    return group['id']
+            print(f"Failed to find group ID for '{group_name}'.")
+        else:
+            print("Failed to get groups list.")
+            print(f"Error message: {response.text}")
+        return None