Skip to content
This repository has been archived by the owner on Aug 4, 2018. It is now read-only.

Infrastructure Access

Jump to bottom
Ernest W. Durbin III edited this page Jul 20, 2014 · 3 revisions

Logging in

Web application servers are

  • web0.pypi.io
  • web1.pypi.io
  • web2.pypi.io

Download stats are collected on

  • counter0.pypi.io

Access to these servers is available to the named users and public keys in this file

Sudoers privileges for the psf-admin group are listed here

Running admin scripts

Admin scripts can be run by logging in as your named user, then gaining a shell as the application user.

  • sudo -u pypi -i or

The source checkout is located in the application user's home directory and a virtual environment with all dependencies is built in the home directory as well.

[pypi@counter0 ~]$ pwd
/opt/pypi
[pypi@counter0 ~]$ tree -L 1 -f
.
├── ./env
└── ./src

2 directories, 0 files

So putting it all together, if an admin script named fizzbuzz.py was added to the repository in the tools directory, it could be executed as follows.

[ernestd@counter0 ~]$ sudo -u pypi -i
[pypi@counter0 ~]$ ~/env/bin/python ~/src/tools/fizzbuzz.py
*BANG*

File storage

Files for PyPI are stored at /data/pypi/{packagedocs/packages} via GlusterFS.

The only real rule here is that it's critical that deletions/additions/modifications to files on the shared volumes are performed from a client. Any of the web servers are fine.

Access to databases

The Postgresql instances are not accessible from the public internet, so you must tunnel via SSH from any of the nodes in the infrastructure to access them.

pgpool-II is used to load balance R/W via a Virtual IP as seen here

Direct access to the Primary/Standby servers is easily accomplished by bypassing the VIP and hitting a node directly.

Deployments

Currently, the following deployments exist in the Infrastructure:

The basic parameters of these deployments can be found here.

Secrets for the deployments are stored in a separate git repository, but is in the format and spirit of this data. Access to this repository is granted to the users in the psf-admin group and kept private for obvious reasons, for access information reach out to @ewdurbin or @dstufft.

Running a deploy

From any server in the infrastructure, a Salt highstate is designed to keep everything up to date. This includes system configuration as well as deployments. Invoking sudo salt-call state.highstate should force a deploy of the PyPI code on the web and counter servers.

The systems are configured to run a highstate every 15 minutes.

Clone this wiki locally