Skip to content

Commit

Permalink
docs: fix grammar and phrasing issues (1, 2, 3) (#62)
Browse files Browse the repository at this point in the history
1. Insert missing article "a":
   - Original: "This repo is work in progress..."
   - Revised:  "This repo is a work in progress..."
   Rationale:
     The article "a" is needed before "work in progress" to make the sentence grammatically correct.

2. Add "as well as" for parallelism:
   - Original: "...scripts to train and evaluate models as well generate synthetic data..."
   - Revised:  "...scripts to train and evaluate models as well as generate synthetic data..."
   Rationale:
     "As well as" is the correct conjunction to link multiple verbs or verb phrases, improving clarity.

3. Clarify GPU resource phrasing:
   - Original: "we used 2 nodes of 8xH100 each one..."
   - Revised:  "we used 2 nodes, each with 8×H100 GPUs..."
   Rationale:
     This rewording removes redundant language ("each one") and more clearly states that each node has eight H100 GPUs.
  • Loading branch information
CharlesCNorton authored Jan 27, 2025
1 parent feb59d2 commit 8d37c5c
Showing 1 changed file with 3 additions and 3 deletions.
6 changes: 3 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@
# Open R1

*A fully open reproduction of DeepSeek-R1. This repo is work in progress, let's build it together!*
*A fully open reproduction of DeepSeek-R1. This repo is a work in progress, let's build it together!*

## Overview

The goal of this repo is to build the missing pieces of the R1 pipeline such that everybody can reproduce and build on top of it. The project is simple by design and mostly consists of:

- `src/open_r1` contains the scripts to train and evaluate models as well generate synthetic data:
- `src/open_r1` contains the scripts to train and evaluate models as well as generate synthetic data:
- `grpo.py`: trains a model with GRPO on a given dataset.
- `sft.py`: simple SFT of a model on a dataset.
- `evaluate.py`: evaluates a model on the R1 benchmarks.
Expand Down Expand Up @@ -234,7 +234,7 @@ Take a look at the sample dataset at [HuggingFaceH4/numina-deepseek-r1-qwen-7b](

### Generate data from DeepSeek-R1

To run the bigger DeepSeek-R1, we used 2 nodes of 8xH100 each one, using the slurm file present in this repo at `slurm/generate.slurm`. First, install the dependencies:
To run the bigger DeepSeek-R1, we used 2 nodes, each with 8×H100 GPUs using the slurm file present in this repo at `slurm/generate.slurm`. First, install the dependencies:

(for now we need to install the vllm dev wheel that [fixes the R1 cuda graph capture](https://github.com/vllm-project/vllm/commits/221d388cc5a836fa189305785ed7e887cea8b510/csrc/moe/moe_align_sum_kernels.cu))
```shell
Expand Down

0 comments on commit 8d37c5c

Please sign in to comment.