Skip to content

Commit

Permalink
Merge branch 'main' into dependabot/github_actions/rojopolis/spellche…
Browse files Browse the repository at this point in the history
…ck-github-actions-0.36.0
  • Loading branch information
kcgthb authored Mar 21, 2024
2 parents e88a604 + 9826ff4 commit 2eae366
Show file tree
Hide file tree
Showing 8 changed files with 171 additions and 92 deletions.
7 changes: 7 additions & 0 deletions .github/workflows/config/spellcheck.wordlist.txt
Original file line number Diff line number Diff line change
Expand Up @@ -70,3 +70,10 @@ GeForce
unsatisfiable
reproducibility
Skylake
inode
inodes
IOPS
Lustre
JSON
NFS
SSD
9 changes: 5 additions & 4 deletions .github/workflows/config/spellcheck.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@ matrix:
aspell:
lang: en
d: en_US
mode: markdown
ignore-case: true
dictionary:
wordlists:
Expand All @@ -15,6 +16,9 @@ matrix:
- open: ':'
content: '[\w-]+'
close: ':'
# ignore attr_list {: .* :}
- open: '{:'
close: ':}'
- pyspelling.filters.markdown:
markdown_extensions:
- pymdownx.superfences
Expand All @@ -26,10 +30,7 @@ matrix:
- title
- alt
ignores:
- ':matches(code, pre)'
- code
- pre
- img
- 'code, pre, img'
- pyspelling.filters.url:
sources:
- '**/*.md'
Expand Down
16 changes: 8 additions & 8 deletions includes/data/facts.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,9 @@ facts:
- name: users
fields:
- name: user accounts
value: 7085
value: 7190
- name: PI groups
value: 1116
value: 1126
desc: "from all Stanford's seven Schools, SLAC, Stanford Institutes, _etc._"
- name: owner groups
value: 201
Expand Down Expand Up @@ -41,7 +41,7 @@ facts:
icon: plug
fields:
- name: kW
value: 548.42
value: 574.39
desc: total power usage
- name: PDUs
value: 57
Expand All @@ -61,8 +61,8 @@ facts:
value: 104
desc: across **2** Infiniband fabrics (EDR, HDR)
- name: Infiniband cables
value: 5739
desc: spanning about **30.23 km**
value: 5733
desc: spanning about **30.14 km**
- name: Ethernet switches
value: 53
- name: scheduler
Expand All @@ -71,11 +71,11 @@ facts:
- name: Slurm partitions
value: 178
- name: CPU.hours/day
value: 45679
desc: over **5 years** of computing in a single day
value: 41542
desc: over **4 years** of computing in a single day
- name: /month
fmt: "${:,.0f}"
value: 3052120.7296
value: 2775676.0256
desc: to run the same workload on t2.large on-demand cloud instances

- name: partitions
Expand Down
32 changes: 29 additions & 3 deletions includes/data/software.yml
Original file line number Diff line number Diff line change
Expand Up @@ -175,14 +175,18 @@ software_modules:
quality figures in a variety of hardcopy formats and interactive environments
across platforms., family: matplotlib, markedDefault: true, versionName: 2.2.2_py27}
- categories: viz, plotting
defaultVersionName: 2.4.1_py27
defaultVersionName: 5.19.0_py312
description: Plotly's Python graphing library makes interactive, publication-quality
graphs online.
package: py-plotly
url: https://plot.ly/python/
versions:
- {description: 'Plotly''s Python graphing library makes interactive, publication-quality
graphs online.', markedDefault: false, versionName: 2.4.1_py27}
- {description: 'Plotly''s Python graphing library makes interactive, publication-quality
graphs online.', markedDefault: false, versionName: 5.19.0_py39}
- {description: 'Plotly''s Python graphing library makes interactive, publication-quality
graphs online.', markedDefault: false, versionName: 5.19.0_py312}
- categories: viz, imaging
defaultVersionName: 9.2.0_py39
description: Pillow-SIMD is an optimized version of Pillow
Expand Down Expand Up @@ -2298,6 +2302,9 @@ software_modules:
- {description: The SciPy library provides many user-friendly and efficient
numerical routines such as routines for numerical integration and optimization.,
family: scipy, markedDefault: false, versionName: 1.10.1_py39}
- {description: The SciPy library provides many user-friendly and efficient
numerical routines such as routines for numerical integration and optimization.,
family: scipy, markedDefault: false, versionName: 1.12.0_py312}
- {description: The SciPy library provides many user-friendly and efficient
numerical routines such as routines for numerical integration and optimization.,
family: scipy, markedDefault: true, versionName: 1.1.0_py27}
Expand Down Expand Up @@ -3706,6 +3713,12 @@ software_modules:
properties:
arch: {gpu: 1}
versionName: 4.7.0
- description: OpenCV (Open Source Computer Vision Library) is an open source
computer vision and machine learning software library.
markedDefault: false
properties:
arch: {gpu: 1}
versionName: 4.9.0
- description: OpenCV (Open Source Computer Vision Library) is an open source
computer vision and machine learning software library.
markedDefault: true
Expand Down Expand Up @@ -4201,14 +4214,16 @@ software_modules:
from different cohorts, or time-series from the same subject.', markedDefault: false,
versionName: '12'}
- categories: biology, genomics
defaultVersionName: 0.3.4
defaultVersionName: 0.5.3
description: Dorado is a high-performance, easy-to-use, open source basecaller
for Oxford Nanopore reads.
package: dorado
url: https://github.com/nanoporetech/dorado
versions:
- {description: 'Dorado is a high-performance, easy-to-use, open source basecaller
for Oxford Nanopore reads.', markedDefault: false, versionName: 0.3.4}
- {description: 'Dorado is a high-performance, easy-to-use, open source basecaller
for Oxford Nanopore reads.', markedDefault: false, versionName: 0.5.3}
- categories: biology, neurology
defaultVersionName: 21.3.00
description: AFNI (Analysis of Functional NeuroImages) is a set of C programs
Expand Down Expand Up @@ -5149,7 +5164,7 @@ software_modules:
- {description: A Python framework for the analysis and visualization of trees.,
markedDefault: false, versionName: 3.0.0_py27}
- categories: biology, genomics
defaultVersionName: 0.12.0
defaultVersionName: 1.10.0
description: Highly-accurate & wicked fast transcript-level quantification from
RNA-seq reads using lightweight alignments.
package: salmon
Expand All @@ -5158,6 +5173,9 @@ software_modules:
- {description: Highly-accurate & wicked fast transcript-level quantification
from RNA-seq reads using lightweight alignments., markedDefault: false,
versionName: 0.12.0}
- {description: Highly-accurate & wicked fast transcript-level quantification
from RNA-seq reads using lightweight alignments., markedDefault: false,
versionName: 1.10.0}
- categories: biology, genomics
defaultVersionName: 1.1.3_py27
description: Fit-Hi-C is a tool for assigning statistical confidence estimates
Expand Down Expand Up @@ -7091,6 +7109,14 @@ software_modules:
properties:
arch: {gpu: 1}
versionName: 12.2.0
- description: CUDA is a parallel computing platform and application programming
interface (API) model created by Nvidia. It allows software developers and
software engineers to use a CUDA-enabled graphics processing unit (GPU)
for general purpose processing.
markedDefault: false
properties:
arch: {gpu: 1}
versionName: 12.4.0
- description: CUDA is a parallel computing platform and application programming
interface (API) model created by Nvidia. It allows software developers and
software engineers to use a CUDA-enabled graphics processing unit (GPU)
Expand Down
117 changes: 45 additions & 72 deletions src/docs/software/updates.xml
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,51 @@
<description>Sherlock software update feed</description>
<link>https://www.sherlock.stanford.edu/docs/software/list</link>
<atom:link href="https://www.sherlock.stanford.edu/docs/software/updates.xml" rel="self" type="application/rss+xml"/>
<item>
<title>New version: biology/salmon version 1.10.0</title>
<description>Highly-accurate & wicked fast transcript-level quantification from RNA-seq reads using lightweight alignments.</description>
<link>https://combine-lab.github.io/salmon</link>
<guid>https://www.sherlock.stanford.edu/docs/software/list/?add:v=1.10.0#salmon</guid>
<category>biology, genomics</category>
<author>kilian@stanford.edu (Kilian Cavalotti)</author>
<pubDate>Wed, 6 Mar 2024 14:17:16 -0800</pubDate>
</item>
<item>
<title>New version: devel/cuda version 12.4.0</title>
<description>CUDA is a parallel computing platform and application programming interface (API) model created by Nvidia. It allows software developers and software engineers to use a CUDA-enabled graphics processing unit (GPU) for general purpose processing.</description>
<link>https://developer.nvidia.com/cuda-toolkit</link>
<guid>https://www.sherlock.stanford.edu/docs/software/list/?add:v=12.4.0#cuda</guid>
<category>devel, language</category>
<author>kilian@stanford.edu (Kilian Cavalotti)</author>
<pubDate>Tue, 5 Mar 2024 14:39:10 -0800</pubDate>
</item>
<item>
<title>New version: viz/py-plotly version 5.19.0_py312</title>
<description>Plotly's Python graphing library makes interactive, publication-quality graphs online.</description>
<link>https://plot.ly/python/</link>
<guid>https://www.sherlock.stanford.edu/docs/software/list/?add:v=5.19.0_py312#py-plotly</guid>
<category>viz, plotting</category>
<author>kilian@stanford.edu (Kilian Cavalotti)</author>
<pubDate>Wed, 21 Feb 2024 10:02:42 -0800</pubDate>
</item>
<item>
<title>New version: viz/py-plotly version 5.19.0_py39</title>
<description>Plotly's Python graphing library makes interactive, publication-quality graphs online.</description>
<link>https://plot.ly/python/</link>
<guid>https://www.sherlock.stanford.edu/docs/software/list/?add:v=5.19.0_py39#py-plotly</guid>
<category>viz, plotting</category>
<author>kilian@stanford.edu (Kilian Cavalotti)</author>
<pubDate>Wed, 21 Feb 2024 10:02:42 -0800</pubDate>
</item>
<item>
<title>New version: biology/dorado version 0.5.3</title>
<description>Dorado is a high-performance, easy-to-use, open source basecaller for Oxford Nanopore reads.</description>
<link>https://github.com/nanoporetech/dorado</link>
<guid>https://www.sherlock.stanford.edu/docs/software/list/?add:v=0.5.3#dorado</guid>
<category>biology, genomics</category>
<author>kilian@stanford.edu (Kilian Cavalotti)</author>
<pubDate>Wed, 21 Feb 2024 09:29:37 -0800</pubDate>
</item>
<item>
<title>New module: biology/bcl-convert version 4.2.7</title>
<description>The BCL Convert App generates demultiplexed FASTQ files from a run as input.</description>
Expand Down Expand Up @@ -176,77 +221,5 @@
<author>kilian@stanford.edu (Kilian Cavalotti)</author>
<pubDate>Mon, 8 Jan 2024 14:23:50 -0800</pubDate>
</item>
<item>
<title>New version: physics/geos version 3.12.1</title>
<description>GEOS is a C/C++ library for computational geometry with a focus on algorithms used in geographic information systems (GIS) software.</description>
<link>https://libgeos.org</link>
<guid>https://www.sherlock.stanford.edu/docs/software/list/?add:v=3.12.1#geos</guid>
<category>physics, geoscience</category>
<author>kilian@stanford.edu (Kilian Cavalotti)</author>
<pubDate>Tue, 12 Dec 2023 11:17:10 -0800</pubDate>
</item>
<item>
<title>New version: biology/kallisto version 0.50.1</title>
<description>kallisto is a program for quantifying abundances of transcripts from RNA-Seq data using high-throughput sequencing reads.</description>
<link>https://pachterlab.github.io/kallisto/</link>
<guid>https://www.sherlock.stanford.edu/docs/software/list/?add:v=0.50.1#kallisto</guid>
<category>biology, genomics</category>
<author>kilian@stanford.edu (Kilian Cavalotti)</author>
<pubDate>Wed, 29 Nov 2023 13:50:00 -0800</pubDate>
</item>
<item>
<title>New version: system/py-matlab-proxy version 0.10.0_py39</title>
<description>matlab-proxy is a Python package which enables you to launch MATLAB and access it from a web browser.</description>
<link>https://github.com/mathworks/matlab-proxy</link>
<guid>https://www.sherlock.stanford.edu/docs/software/list/?add:v=0.10.0_py39#py-matlab-proxy</guid>
<category>system, tools</category>
<author>kilian@stanford.edu (Kilian Cavalotti)</author>
<pubDate>Wed, 29 Nov 2023 10:37:32 -0800</pubDate>
</item>
<item>
<title>New version: system/rclone version 1.65.0</title>
<description>Rclone is a command line program to sync files and directories to and from</description>
<link>https://rclone.org</link>
<guid>https://www.sherlock.stanford.edu/docs/software/list/?add:v=1.65.0#rclone</guid>
<category>system, file transfer</category>
<author>kilian@stanford.edu (Kilian Cavalotti)</author>
<pubDate>Tue, 28 Nov 2023 07:59:13 -0800</pubDate>
</item>
<item>
<title>New module: devel/darshan version 3.4.4</title>
<description>Darshan is a scalable HPC I/O characterization tool.</description>
<link>https://www.mcs.anl.gov/research/projects/darshan/</link>
<guid>https://www.sherlock.stanford.edu/docs/software/list/?add:v=3.4.4#darshan</guid>
<category>devel, profiling</category>
<author>kilian@stanford.edu (Kilian Cavalotti)</author>
<pubDate>Thu, 16 Nov 2023 17:34:11 -0800</pubDate>
</item>
<item>
<title>New module: system/py-matlab-proxy version 0.9.1_py39</title>
<description>matlab-proxy is a Python package which enables you to launch MATLAB and access it from a web browser.</description>
<link>https://github.com/mathworks/matlab-proxy</link>
<guid>https://www.sherlock.stanford.edu/docs/software/list/?add:v=0.9.1_py39#py-matlab-proxy</guid>
<category>system, tools</category>
<author>kilian@stanford.edu (Kilian Cavalotti)</author>
<pubDate>Thu, 16 Nov 2023 13:19:14 -0800</pubDate>
</item>
<item>
<title>New version: system/py-globus-cli version 3.19.0_py39</title>
<description>A command line wrapper over the Globus SDK for Python.</description>
<link>https://github.com/globus/globus-cli</link>
<guid>https://www.sherlock.stanford.edu/docs/software/list/?add:v=3.19.0_py39#py-globus-cli</guid>
<category>system, file transfer</category>
<author>kilian@stanford.edu (Kilian Cavalotti)</author>
<pubDate>Thu, 16 Nov 2023 13:18:58 -0800</pubDate>
</item>
<item>
<title>New version: math/rstudio version 2023.09.1</title>
<description>RStudio is an integrated development environment (IDE) for R. It includes a console, syntax-highlighting editor that supports direct code execution, as well as tools for plotting, history, debugging and workspace management.</description>
<link>http://www.rstudio.com</link>
<guid>https://www.sherlock.stanford.edu/docs/software/list/?add:v=2023.09.1#rstudio</guid>
<category>math, statistics</category>
<author>kilian@stanford.edu (Kilian Cavalotti)</author>
<pubDate>Thu, 16 Nov 2023 13:18:43 -0800</pubDate>
</item>
</channel>
</rss>
7 changes: 7 additions & 0 deletions src/docs/software/using/rclone.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,13 @@ browser, so you will need to connect to Sherlock with **local port forwarding**
(`ssh -L`). You only need to do this when you are configuring `rclone` for the
first time.

!!! Note "Use local terminal for `rclone config`"

**This method will not work in the Sherlock OnDemand shell.** You will need
to use your local machine's terminal to enable local port forwarding and to
allow `rclone` to communicate with your browser. On Linux and macOS, you
can use the Terminal app; on Windows, you can use the PowerShell app.

When running `rclone config` you will be prompted to enter names and values,
indicated by the `>` symbol. To leave it empty, press Enter.

Expand Down
12 changes: 12 additions & 0 deletions src/docs/storage/data-transfer.md
Original file line number Diff line number Diff line change
Expand Up @@ -219,6 +219,18 @@ and then should not be typed in).
$ umount ~/sherlock_home
```

On Windows, once SSHFS is installed, you can mount the `$SCRATCH`
filesystem as a network drive through the windows file explorer. To do
this, go to "This PC", right-click in the "Network Locations" section of
the window and select "Add a Network Drive". Then, in the "Add Network
Location Wizard", you would use the following network address:

```
\\sshfs\<sunetid>@dtn.sherlock.stanford.edu
```

This will mount the `$SCRATCH` partition as a network drive on your PC.

For more information about using SSHFS on your local machine, you can
refer to this [tutorial][url_sshfs_tuto] for more details and examples.

Expand Down
Loading

0 comments on commit 2eae366

Please sign in to comment.