Skip to content

Home

Jump to bottom
Alex Chi edited this page Mar 17, 2021 · 5 revisions

mirror-docker-unified stores all infrastructure scripts for SJTUG mirror servers.

SJTUG Mirror Service

SJTUG mirror service is made up of two servers and several services provided by SJTUG NIC.

There are two mirror servers, Zhiyuan and Siyuan, both sponsored by SJTU NIC.

The Zhiyuan mirror is set up in 2016. It is a traditional-VM-based server. Zhiyuan mirror is basically in maintenance mode. We do not add new repositories to that server.

The Siyuan mirror is set up in 2020. It is hosted on SJTU jCloud service, as a cloud VM. Siyuan mirror is intended for replacing ftp.sjtu.edu.cn, and is now in active development. We experiment new technologies and add new repositories on Siyuan mirror.

Zhiyuan Siyuan
Since 2016 2020
Domain https://mirrors.sjtug.sjtu.edu.cn https://mirror.sjtu.edu.cn
OS Fedora Debian
Network IPv4 + IPv6 (outbound only) IPv4 (through jCloud router) + IPv6
Type Traditional VM jCloud VM
CPU 16 16
Memory 48GB 128GB
Status Only security patches New repositories
Disk 12TB HDD 10TB + 10TB HDD

Besides mirror servers, SJTUG mirror service leverages existing infrastructure of SJTU NIC to provide service, which includes:

  • SJTU NIC S3 service. We store some repositories on SJTU S3 service. We use mirror-clone and mirror-intel to synchronize data to S3 and serve our users respectively.
  • SJTU Git Service. All git repositories are first cloned to mirror servers, and then pushed to SJTU git. Mirror users still use URLs like https://mirrors.sjtug.sjtu.edu.cn/git/crates.io-index/ to clone. Our web server will redirect users to SJTU git or directly serve from disk, according to their user agent.

Identify Mirror Server

SJTUG mirrors share the same frontend codebase. At the end of frontend page, you will see "Siyuan" or "Zhiyuan" mirror.

When accessing data on server, there is a x-sjtug-mirror-id header, which indicates which server is used.

When using rsync, mirror id is shown in motd.

Generally, https://mirrors.sjtug.sjtu.edu.cn corresponds to Zhiyuan server, and https://mirror.sjtu.edu.cn corresponds to Siyuan server.

Mirror Server Infrastructure

All of our mirror server is deployed and managed with docker-compose. This repo mirror-docker-unified stores all configurations of two mirror servers.

Each server runs the following services.

  • Mirror Serving
    • Caddy Web Server, which serves our user.
    • lug. lug is the mirror synchronizing manager for our mirror services.
    • mirror-intel. mirror-intel is the smart cache proxy service, which will either redirect users to s3, or cache a missing object in the background.
    • rsyncd, which provides rsync endpoint. Our rsync endpoint is for file listing only. I/O is strictly limited.
    • git-backend, which serves git repositories locally. Note that some requests will be redirected to SJTU git.
    • docker registry (DockerHub + gcr.io, only on Siyuan).
  • Logging and Devops
    • logspout, which pipes logs to our monitor server.
    • v2ray, which sets up a secure tunnel between mirror server and monitor server.
    • clash, used for synchronization requiring stable international connection.
    • cadvisor, which reports container status.
    • node_exporter (on host), which reports server status.
  • Utilities
    • ipv6nat, which assigns IPv6 NAT address to containers.

SJTUG Mirror Projects

Besides those services running on server, we also maintain a lot of other projects closely related to SJTUG mirror infrastructure. Here is a list of all repositories related to SJTUG mirror.

Clone this wiki locally