From cd620862bcaba1fa4d05240f8e5431329be25e6d Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Olender?= <92638966+TC-MO@users.noreply.github.com> Date: Thu, 16 Jan 2025 11:02:32 +0100 Subject: [PATCH] docs: consolidate README & CONTRIBUTING & remove other READMEs (#1333) Co-authored-by: Adam Kliment <79609+netmilk@users.noreply.github.com> --- .../config/vocabularies/Docs/accept.txt | 2 + CONTRIBUTING.md | 389 ++++++++++++------ README.md | 206 ++-------- apify-api/README.md | 255 ------------ apify-api/openapi/README.md | 13 - apify-api/openapi/components/README.md | 13 - apify-api/openapi/paths/README.md | 108 ----- 7 files changed, 310 insertions(+), 676 deletions(-) delete mode 100644 apify-api/README.md delete mode 100644 apify-api/openapi/README.md delete mode 100644 apify-api/openapi/components/README.md delete mode 100644 apify-api/openapi/paths/README.md diff --git a/.github/styles/config/vocabularies/Docs/accept.txt b/.github/styles/config/vocabularies/Docs/accept.txt index 725428840..fb5719ddd 100644 --- a/.github/styles/config/vocabularies/Docs/accept.txt +++ b/.github/styles/config/vocabularies/Docs/accept.txt @@ -24,6 +24,8 @@ Docker's Docusaurus navbar nginx +npm +camelCase :::caution :::note diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 2b62e50eb..083c95966 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -1,174 +1,315 @@ # Contributing to Apify Documentation -## Architecture +## Before you start + +1. Review this guide completely +2. Setup you development environment +3. Familiarize yourself with our documentation style guide + +## Development setup + +### Prerequisites + +1. Git +2. Node.js +3. GitHub access + +### Installation steps + +1. Clone the repository +2. Configure GitHub access: + + ```bash + npm login --scope@apify-packages -registry=https://npm.pkg.github.com --auth-type=legacy + ``` + +3. Run `npm install` +4. Start development server: `npm start` + +This will be enough to work on Platform, Academy and, OpenAPI. If you want to work on entire documentation set you need to join them using nginx. + +#### Join all repositories with nginx + +1. Clone all the repositories +2. Run `npm start:dev` instead of `npm start` from the main repository +3. Run `npm start -- --port ` to start Docusaurus instance on specific port, refer to the table for each repository port -Apify documentation consists of the following top-level areas: + |Repository|Port| + |:---|:---| + |apify-docs|3000| + |apify-client-js|3001| + |apify-client-python|3002| + |apify-sdk-js|3003| + |apify-sdk-python|3004| + |apify-cli|3005| -- **Academy** - Collection of mostly platform-independent courses on scraping techniques. -- **Platform** - The main docs on how to use Apify as a product. -- **API** - API reference and API client libraries docs, i.e. interacting with the platform from outside. -- **SDK** - SDK libraries docs, i.e. interacting with the platform from code which runs inside it. -- **CLI** - Tool for building code for the platform, and for interacting with the platform from outside. +4. To serve them together, setup the nginx server with the following configuration: -Thus the source code of the docs is spread across six different repositories: + ```nginx + server { + listen 80; + server_name docs.apify.loc; + location / { + proxy_pass http://localhost:3000; + } + location /api/client/js { + proxy_pass http://localhost:3001; + } + location /api/client/python { + proxy_pass http://localhost:3002; + } + location /sdk/js { + proxy_pass http://localhost:3003; + } + location /sdk/python { + proxy_pass http://localhost:3004; + } + location /cli { + proxy_pass http://localhost:3005; + } + } + ``` -- apify-docs (this repository) -- apify-client-js -- apify-client-python -- apify-sdk-js -- apify-sdk-python -- apify-cli +5. Add a record to `/etc/hosts`, which maps the `docs.apify.loc` to a localhost: -The main documentation content for Platform docs and Academy is inside the `./sources` directory. Every project repository has its own [Docusaurus](https://docusaurus.io/) instance and is available on a URL prefix (used as the `baseUrl` in Docusaurus) that's routed via nginx reverse proxy to the main domain. All those Docusaurus instances are deployed to GH pages on push. + ```text + 127.0.01 docs.apify.loc + ``` -### Shared theme +You should be able to open https://docs.apify.loc in your browser and run all the repositories jointly as one project. -We use a shared Docusaurus theme published to npm as `@apify/docs-theme`. It contains custom components and styles to be used in all the Apify Docusaurus instances. -Aside from the regular Docusaurus theme interface, it also exports the common parts of the Docusaurus config, such as the navbar contents, URL, `og:image`, etc. +## Documentation style guide -#### Theme synchronization +### Language guidelines -The theme can be installed in any Docusaurus instance by running `npm install @apify/docs-theme`. It is automatically synced in all the existing repositories via CI. +- Use US English +- Write in inclusive language +- Avoid directional language (like "left" or "right" or instead of "see" use "check out") +- Use active voice whenever possible -A GitHub Action automatically publishes the theme to npm whenever any changes are pushed to the `master` branch. However, this only happens if you update the version in the `package.json` file manually - if the current version already exists on npm, the publish will be skipped. +### Formatting conventions -Additionally, if there are any changes to the `apify-docs-theme` folder detected, the GitHub action will invoke docs builds in all the subprojects to make sure that all the pages are using the latest theme version. This is done in the `rebuild-docs` job. This job utilizes a matrix strategy to run the builds in parallel. The actual rebuild is initiated by the `workflow_dispatch` event in the respective repositories. Because of this, the `GITHUB_TOKEN` env var has to be replaced by the PAT token stored in the `GH_TOKEN` secret - the original token doesn't have the necessary permissions to trigger the workflows in other repositories. +1. Text emphasis: -### Redirects + - use *Bold* for UI elements + - use **Italics** for emphasis + - use `code` for inline code, by using back-ticks (\`\`\) + - use code blocks with language specification + - use [code tabs](https://docusaurus.io/docs/markdown-features/tabs) whenever you want to include examples of implementation in more than one language -In [`./nginx.conf`](./nginx.conf) you can find the production version of the Nginx configuration that handles the serving of content from all the different repositories. It also handles redirects from old URLs to new ones so that we don't lose any SEO juice. +2. Documentation elements: + - Use [admonitions](https://docusaurus.io/docs/2.x/markdown-features/admonitions) to emphasize crucial information, available admonitions are: + - note + - tip + - info + - caution + - danger + - Use code tabs for multiple languages + - Include proper metadata in front matter -### API reference + Example of proper usage and formatting: -The API reference at () is built directly from the OpenAPI specification of our API, versioned at https://github.com/apify/openapi. + ```text + :::note Your Title Here -For contribution details regarding the OpenAPI specification, consult the related [contribution guide](https://github.com/apify/openapi?tab=readme-ov-file#contribution-guide). + Your important message here. -### Homepage + ::: + ``` -The homepage menu card items are in the `docs/homepage_content.json` file. The cards aim to suit three types of use cases: +3. Screenshots: -- **Get started**, **Use Actors and scrapers** - Beginners and people who just want to use the Actors. -- **Reduce blocking**, **Use platform features** - Experienced Actor and platform users. -- **Build Actors**, **Advanced tutorials and debugging** - Actor builders and advanced users. + - Use light theme when taking screenshots + - Include meaningful alt texts + - Use red indicators -Each item has its own JSON object, in which `cardItem` is the title and `href` is the link. If the link leads to outside the Apify Docs site, add the `"isExternalLink": true` property. For local links, just use the article's path. E.g. `"/tutorials/apify-scrapers/web-scraper"`. +### Front matter metadata best practices -In the title (`cardItem`), don't just give the article's name. Phrase the title in a way that answers a question or fulfills a goal the user might have. +- Keep descriptions between 140 and 160 characters +- Use action-oriented phrasing +- Avoid repetitive keywords +- Avoid the word "documentation" in descriptions -For example: +## Repository structure -```text -{ - "cardItem": "How to run an actor", - "href": "https://www.youtube.com/watch?v=BsidLZKdYWQ", - "isExternalLink": true -}, -``` +### Theme management + +- uses `@apify/docs-theme` package +- automatic synchronization via CI +- Theme updates trigger rebuilds across all projects + +### Content organization + +Content lives in the following locations: + +- Main content like Platform documentation & Academy: `/sources` directory +- API reference: Generated from OpenAPI specs within `/apify-api` directory +- SDK docs: separate repositories +- Client docs: separate repositories +- CLI docs: separate repositories + +## API Documentation + +### Overview + +The API reference documentation at [docs.apify.com](https://docs.apify.com/) is built directly from our OpenAPI specification. The source of truth for the API specification lives in the `/apify-api/openapi` directory within [apify-docs](https://github.com/apify/apify-docs) repository. + +### Setup requirements -> In JSON, all entries except booleans (`true/false`) and numbers need to be in double quote marks (""). - -Over time, we should track which items are useful and which don't get any traffic. Also, as Apify Docs expand, we may need to add more cards and update which articles we link to. - -## Installation and setup - -1. Use Git to clone this repository. -2. To install packages prefixed with `@apify-packages`, first go to GitHub and navigate to [Settings / Personal access tokens / Tokens (classic)](https://github.com/settings/tokens). Despite the UI suggesting you should use the new fine-grained tokens, generate a **classic token** with scopes **repo** and **write:packages**. Keep the token somewhere safe. Then run the following command and use your GitHub username together with the token as credentials: - - ```bash - npm login --scope=@apify-packages --registry=https://npm.pkg.github.com --auth-type=legacy - ``` - - Read [#909](https://github.com/apify/apify-docs/issues/909) if you want to understand why this is needed. -3. Run `npm install`. -4. Run `npm start`. The website should open at . - -This is sufficient to work on the main content, i.e. the Academy and Platform docs. If you want to also work on the other parts of the docs, you may want to first join them all together using Nginx. - -### Join all the docs repos together using Nginx - -By default, the parts of the docs sourced from other repositories will give you 404s. If you need to locally run the project with all the other repositories included, clone them all and setup an Nginx server according to the steps below. - -For the setup to work, use `npm start:dev` instead of `npm start` when starting `apify-docs`. This causes all links in the top navigation and the footer to be absolute. They will now use `docs.apify.loc` as a hostname. Clone all the remaining docs repositories, and start their Docusaurus instances. To run Docusaurus on a specific port, use `npm start -- --port XXXX`. - -| repo | port | -|---------------------|------| -| apify-docs | 3000 | -| apify-client-js | 3001 | -| apify-client-python | 3002 | -| apify-sdk-js | 3003 | -| apify-sdk-python | 3004 | -| apify-cli | 3005 | - -To serve them together as a single website, setup an Nginx server with the following config: - -```nginx -server { - listen 80; - server_name docs.apify.loc; - location / { - proxy_pass http://localhost:3000; - } - location /api/client/js { - proxy_pass http://localhost:3001; - } - location /api/client/python { - proxy_pass http://localhost:3002; - } - location /sdk/js { - proxy_pass http://localhost:3003; - } - location /sdk/python { - proxy_pass http://localhost:3004; - } - location /cli { - proxy_pass http://localhost:3005; - } -} +1. Install Node.js +2. Clone the repository +3. Run `npm install` + +### Basic commands + +- `npm start` - Starts docs preview server including API reference +- `npm test` - Validates the definition + +### Adding new documentation + +#### Schema documentation + +1. Navigate to `apify-api/openapi/components/schemas` +2. Create a new file named after your schema +3. Define the schema structure +4. Reference schema using `$ref` in other files + +Example schema + +```yaml +type: object +properties: + id: + description: The resource ID + readOnly: true + allOf: + -$ref: ./ResourceId.yaml ``` -Add a record to `/etc/hosts`, which maps the `docs.apify.loc` hostname to a localhost: +#### Path documentation + +1. Navigate to `apify-api/openapi/paths` +2. Create YAML file following the URL structure replacing `/` with `@` +3. Place path parameters in curly braces (e.g., {queueId}) +4. Add path reference to openapi.yaml -```text -127.0.0.1 docs.apify.loc +Example addition to `openapi.yaml` file: + +```yaml +'/requests-queues': + $ref: './paths/request-queues/request-queues.yaml' +'/requests-queues/{queueId}': + $ref: './paths/request-queues/request-queues@{queueId}.yaml' +``` + +Example YAML file `request-queues@{queueId}.yaml` in the `paths/request-queues` folder : + +```yaml +get: + tags: + - Request Queues + summary: Get a Request Queue + operationId: requestQueues_get + description: | + You can have a markdown description here. + parameters: + responses: + '200': + '401': + x-code-samples: + - lang: PHP + source: + $ref: ../code_samples/PHP/customers/get.php ``` -Now you should be able to open in your browser and see all the documentation projects joined together, with the top navigation allowing you to browse not only Academy or Platform docs, but also CLI, SDK, and all the other docs. +#### Operation ID conventions + +Operation IDs must follow this format: + +- Generated from path structure and HTTP method +- Use camelCase for object names +- Single object for paths with {id}, plural otherwise +- Underscore separator between object name and action +- Method name in lowercase at the end + +Examples: + +- `/requests-queues` GET -> `requestQueues_get` +- `/requests-queues/{queueId}` PUT -> `requestQueue_put` +- `/acts/{actorId}/runs` POST -> `act_runs_post` + +#### Code samples + +1. Navigate to the `openapi/code_samples` folder +2. Navigate to the `` sub-folder +3. Navigate to the `path` folder, and add ref to the code sample + +Add languages by adding new folders at the appropriate path level. + +#### Submitting changes + +1. Make your changes following the guidelines above +2. Test locally using provided npm commands +3. Submit a pull request to the `main` branch +4. Ensure all CI checks pass + +## Development workflow + +### Local development + +1. Basic setup + + ```bash + + npm install + npm start -## Linting + ``` -### Markdown and code +2. Full setup with nginx: -The **apify-docs** repo contains both Markdown and JavaScript/TypeScript files. We have two commands for linting them: + - Clone all documentation repositories + - Configure nginx server + - Update hosts file + - Use `npm start:dev` -- `npm run lint:md` and `npm run lint:md:fix` checks the `*.md` files. -- `npm run lint:code` and `npm run lint:code:fix` checks the `*.{js,ts}` files. +## Quality check -For Markdown, we use the [markdownlint](https://github.com/DavidAnson/markdownlint) package, which also has a handy VSCode [extension](https://marketplace.visualstudio.com/items?itemName=DavidAnson.vscode-markdownlint). +### Linting -For JavaScript, we use the [ESLint Markdown plugin](https://github.com/eslint/eslint-plugin-markdown). +1. Markdown: -### Prose + ```bash + npm run lint:md # Checks for any issues using markdownlint + npm run lint:md:fix # Applies fixes + ``` -Apart from `markdownlint` we also utilize [Vale](https://github.com/errata-ai/vale) as linting solution for prose. You can either use Vale as a CLI tool (for more information how to set it up go [here](https://vale.sh/docs/vale-cli/installation/)) or you can use Vale with a VSCode [extension](https://marketplace.visualstudio.com/items?itemName=ChrisChinchilla.vale-vscode). The rulesets that Vale will utilize while linting can be found within `.github/styles` directory. +2. Code: -#### Exceptions for Vale + ```bash + npm run lint:code # Checks .js & .ts files + npm run lint:code:fix # Applies fixes + ``` -If Vale catches some specific words that you feel that should not be subjected to linting you can add them to the `accept.txt` found within the `.github/styles/Vocab/Docs` directory. For more information how it works visit Vale [docs](https://vale.sh/docs/topics/vocab/). +3. Prose: -## Pull requests + - Use [Vale](https://vale.sh/) for content linting + - Configure exceptions in `accepts.txt` -Follow the semantic commit message format described in the [Conventional Commits](https://www.conventionalcommits.org/en/v1.0.0/) specification. This will help us with automatic changelog generation and versioning. +## Pull request process -When you create a PR, the CI will try to build the project, searching for broken links and linting both the application code and markdown files. +1. Follow [Conventional Commits](https://www.conventionalcommits.org/) +2. Pass all CI checks +3. Include comprehensive documentation updates ## Deployment -On each commit to the `master` branch of this repository, a new version of the Apify documentation gets built and deployed to the appropriate subdomain. +- Automatic deployment on merge to `master` +- Builds deploy to appropriate subdomains +- Updates trigger theme synchronization -## Resources +## Additional resources -- -- -- +- [Docusaurus documentation](https://docusaurus.io/docs) +- [Vale style guide](https://vale.sh/docs) diff --git a/README.md b/README.md index b261bcfd5..416dc5f87 100644 --- a/README.md +++ b/README.md @@ -2,182 +2,62 @@ [![Check & Release](https://github.com/apify/apify-docs/actions/workflows/test.yaml/badge.svg)](https://github.com/apify/apify-docs/actions/workflows/test.yaml) -## Intro +## Overview -This repository is the home of Apify's documentation, which you can find at [docs.apify.com](https://docs.apify.com/). The documentation is written using [Markdown](https://github.com/adam-p/markdown-here/wiki/Markdown-Cheatsheet). Source files of the [platform documentation](https://docs.apify.com/platform) are located in the [/sources](https://github.com/apify/apify-docs/tree/master/sources) directory. However, other sections, such as SDKs for [JavaScript/Node.js](https://docs.apify.com/sdk/js/), [Python](https://docs.apify.com/sdk/python/), or [CLI](https://docs.apify.com/cli/), have their own repositories. For more information, see the [Contributing guidelines](./CONTRIBUTING.md). +This repository is the home of Apify's documentation, available at [docs.apify.com](https://docs.apify.com/). The documentation uses [Markdown](https://github.com/adam-p/markdown-here/wiki/Markdown-Cheatsheet) & [Docusaurus](https://docusaurus.io/). -## Before you start contributing +## Documentation structure -> [!IMPORTANT] -> Before you contribute to Apify documentation with your first pull request, please read the following 2 articles: -> -> - [Contributing guidelines](CONTRIBUTING.md), where you learn about the project structure, local development, testing, and setting up redirects. -> - [Style guide](#style-guide), here below 👇, where you learn how to keep the documentation style consistent. +Our documentation ecosystem consists of: -## Style guide +- **Platform documentation**: Located in the [`/sources`](https://github.com/apify/apify-docs/tree/master/sources) directory +- **Academy**: Platform-independent courses on scraping technique. Located in the [/sources](https://github.com/apify/apify-docs/tree/master/sources) directory +- **API documentation**: + - **OpenAPI documentation**: [OpenAPI reference](https://docs.apify.com/api/v2) + - [JavaScript/Node.js](https://docs.apify.com/api/client/js/) + - [Python](https://docs.apify.com/api/client/python/) +- **SDK documentation**: + - [JavaScript/Node.js](https://docs.apify.com/sdk/js/) + - [Python](https://docs.apify.com/sdk/python/) +- **CLI documentation** - [CLI docs](https://docs.apify.com/cli/) -In addition to the following tips, check existing docs for formatting and style tips when in doubt. +## Getting started -### Language +Before contributing, read these essential resources: -Adhere to US-EN language standards to maintain consistency and clarity in our documentation. +- [Contributing guidelines](CONTRIBUTING.md) +- [Documentation style guide](CONTRIBUTING.md#style-guide) -Use inclusive language, for example: +## Documentation architecture -Instead of _see_, use _view_ or _check out_. +Our documentation consists of these main sections: -Avoid using directional language like: +1. **Academy**: Collection of mostly platform-independent courses on scraping techniques. +2. **Platform**: Main documentation for using Apify as a product. +3. **API**: API reference and client libraries documentation. +4. **SDK**: SDK libraries documentation. +5. **CLI**: Documentation for building code and platform interaction. -> Click the **Home** button on the left to return to the homepage. -> You'll find the account settings on the left-hand side menu. +### Source Repositories + +- apify-docs (this repository) +- apify-client-js +- apify-client-python +- apify-sdk-js +- apify-sdk-python +- apify-cli + +## Architecture -Instead, try to be more descriptive, for example: +Our documentation is built using: -> Click the **Home** button in the menu, to return to the homepage. -> You'll find the account settings under the **Settings** button in the menu. +- **Docusaurus**: Powers our documentation platform +- **Shared Theme**: Custom `@apify/docs-theme` package +- **Automated Deployment**: CI/CD pipeline for continuous updates +- **nginx routing**: Handles subdomain routing and redirects -### Highlighting +## Need help -- For consistency, use **bold** to highlight UI text when dealing with UI-focused documentation (for example, Apify Console). -- For consistency, use _italics_ to emphasize text. -- For inline `code` examples, use **back-ticks** (\` \`). -- For multi-line code examples, use code fences and specify the language. Preferably, specify the title as well. Within platform documentation, always use [Code tabs](README.md#code-tabs) and declare the language. +- For contribution questions, see our [Contributing Guidelines](CONTRIBUTING.md) +- For technical issues, [create a GitHub issue](https://github.com/apify/apify-docs/issues/new) - ``````markdown - ```js showLineNumbers title='foo.js' - - const docsAreCool = require('coolDocs');
- ...
- return docsAreCool;
- - ``` - `````` - -Check out [Markdown features](https://docusaurus.io/docs/markdown-features) in the Docusaurus docs for more information. - -### Admonitions - -In Apify platform documentation, always use admonitions to emphasize crucial information, warnings, tips, or additional context. The available admonitions include: - -- `note` -- `tip` -- `info` -- `caution` -- `danger` - -Consult the [Docusaurus documentation](https://docusaurus.io/docs/2.x/markdown-features/admonitions) for visual distinctions between each type and guidelines on usage. Choose the most relevant type for your content. Maintain proper formatting by adding blank lines following the admonition blocks. Title your admonition next to its type, as shown below: - -```markdown -:::note Your Title Here - -Your important message here. - -::: -``` - -### Code tabs - -View [Docusaurus documentation for tabs](https://docusaurus.io/docs/markdown-features/tabs) for examples. In Apify platform documentation, always use the code tabs to include examples for both JavaScript and Python languages. - -```markdown -import Tabs from '@theme/Tabs'; -import TabItem from '@theme/TabItem'; - - - - This is an apple 🍎 - - - This is an orange 🍊 - - - This is a banana 🍌 - - -``` - -### Metadata - -You must provide the new page metadata as part of the so-called [front matter](https://docusaurus.io/docs/api/plugins/@docusaurus/plugin-content-docs#markdown-front-matter). - -```markdown ---- -id: doc-markdown -title: Docs Markdown Features -hide_title: false -hide_table_of_contents: false -sidebar_label: Markdown -sidebar_position: 3 -pagination_label: Markdown features -custom_edit_url: https://github.com/facebook/docusaurus/edit/main/docs/api-doc-markdown.md -description: How do I find you when I cannot solve this problem -keywords: - - docs - - docusaurus -image: https://i.imgur.com/mErPwqL.png -slug: /myDoc -last_update: - date: 1/1/2000 - author: custom author name ---- - -# Markdown Features - -My Document Markdown content -``` - -### Descriptions - -- Metadata descriptions are super important in making our documentation easy to find using search engines. To maximize our SEO, _keep the descriptions between [140 and 160 characters](https://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&cd=&cad=rja&uact=8&ved=2ahUKEwigg6Og56brAhUNi1wKHULsAHEQFjAGegQIDBAG&url=https%3A%2F%2Fmoz.com%2Flearn%2Fseo%2Fmeta-description&usg=AOvVaw3L26bXhHZTd0wYDM_5xtJ9) whenever possible_. - - Of course, when there just isn't enough to say, don't waffle - it's not a university essay. - - GOOD: - - > "Store anything from images and key-value pairs to structured output data. Learn how to access and manage your stored data from the Apify platform or via API." - - AVOID: - - > "Apify storage docs." - -- Avoid using the word "_documentation_". - - GOOD: - - > "Learn how to develop, run and share serverless cloud programs. Create your own web scraping and automation tools and publish them on the Apify platform." - - AVOID: - - > "Documentation of Apify Actors - the easy way to build serverless cloud programs." - -- Avoid _keyword stuffing_, i.e., repeating the article's name too much; view the [Wikipedia article](https://en.wikipedia.org/wiki/Keyword_stuffing) for more. - - GOOD: - - > Title: Proxy - > - > Description: Learn how to anonymously access websites when running web scraping or automation jobs. Prevent IP address-based blocking using IP address rotation. - - AVOID: - - > Title: Proxy - > - > Description: Learn how to use Apify Proxy. Prevent IP address-based blocking using a proxy. Apify Proxy helps you bypass security. - -- Phrase the descriptions in a way that answers a question that the person using the search engine might have. - - GOOD: - - > Learn how to make your Actor available to the public or keep it private. Prepare your Actor for Apify Store with a description and README file. - - AVOID: - - > Description of the processes regarding optimizing and preparing for publishing of one's Actor in Apify Store. - -### Screenshots - -Documentation has both light and dark themes. However, to keep the screenshots consistent and easily manageable, use the light theme for all screenshots. - -All screenshots should have a meaningful alt text to accommodate screen readers. - -If you add arrows or other indicators to the screenshots, they should be red for high contrast and visibility on the light theme. diff --git a/apify-api/README.md b/apify-api/README.md deleted file mode 100644 index d1da2fee9..000000000 --- a/apify-api/README.md +++ /dev/null @@ -1,255 +0,0 @@ -# Apify API v2 OpenAPI Definition - -[//]: # (TODO rewrite based on the current setup or remove completely and adjust the contributing guide instead) - -## Working on OpenAPI Definition - -### Install - -1. Install [Node JS](https://nodejs.org/). -2. Clone this repo and run `npm install` in the repo root. -3. Install Schemathesis for contract testing against staging: `python -m pip install schemathesis` -4. `cp .env.example .env`, and insert the `APIFY_STAGING_TOKEN` - -### Usage - -#### `npm start` - -Starts the reference docs preview server. - -#### `npm run build` - -Bundles the definition to the dist folder. - -#### `npm test` - -Validates the definition. - -## Contribution Guide - -**TL;DR**: Add your schema to `openapi/components/schemas` and your path to `openapi/paths` and reference them in -`openapi.yaml`. -Open pull request into `main` branch. - -NOTE: The `.redocly.yaml` controls settings for various -tools including the lint tool and the reference -docs engine. Open it to find examples and -[read the docs](https://redoc.ly/docs/cli/configuration/) -for more information. - -### Schemas - -#### Adding Schemas - -1. Navigate to the `openapi/components/schemas` folder. -2. Add a file named as you wish to name the schema. -3. Define the schema. -4. Refer to the schema using the `$ref` (see example below). - -##### Example Schema - -This is a very simple schema example: - -```yaml -type: string -description: The resource ID. Defaults to UUID v4 -maxLength: 50 -example: 4f6cf35x-2c4y-483z-a0a9-158621f77a21 -``` - -This is a more complex schema example: - -```yaml -type: object -properties: - id: - description: The customer identifier string - readOnly: true - allOf: - - $ref: ./ResourceId.yaml - websiteId: - description: The website's ID - allOf: - - $ref: ./ResourceId.yaml - paymentToken: - type: string - writeOnly: true - description: | - A write-only payment token; if supplied, it will be converted into a - payment instrument and be set as the `defaultPaymentInstrument`. The - value of this property will override the `defaultPaymentInstrument` - in the case that both are supplied. The token may only be used once - before it is expired. - defaultPaymentInstrument: - $ref: ./PaymentInstrument.yaml - createdTime: - description: The customer created time - allOf: - - $ref: ./ServerTimestamp.yaml - updatedTime: - description: The customer updated time - allOf: - - $ref: ./ServerTimestamp.yaml - tags: - description: A list of customer's tags - readOnly: true - type: array - items: - $ref: ./Tags/Tag.yaml - revision: - description: > - The number of times the customer data has been modified. - - The revision is useful when analyzing webhook data to determine if the - change takes precedence over the current representation. - type: integer - readOnly: true - _links: - type: array - description: The links related to resource - readOnly: true - minItems: 3 - items: - anyOf: - - $ref: ./Links/SelfLink.yaml - - $ref: ./Links/NotesLink.yaml - - $ref: ./Links/DefaultPaymentInstrumentLink.yaml - - $ref: ./Links/LeadSourceLink.yaml - - $ref: ./Links/WebsiteLink.yaml - _embedded: - type: array - description: >- - Any embedded objects available that are requested by the `expand` - querystring parameter. - readOnly: true - minItems: 1 - items: - anyOf: - - $ref: ./Embeds/LeadSourceEmbed.yaml - -``` - -##### Using the `$ref` - -Notice in the complex example above the schema definition itself has `$ref` links to other schemas defined. - -Here is a small excerpt with an example: - -```yaml -defaultPaymentInstrument: - $ref: ./PaymentInstrument.yaml -``` - -The value of the `$ref` is the path to the other schema definition. - -You may use `$ref`s to compose schema from other existing schema to avoid duplication. - -You will use `$ref`s to reference schema from your path definitions. - -#### Editing Schemas - -1. Navigate to the `openapi/components/schemas` folder. -2. Open the file you wish to edit. -3. Edit. - -### Paths - -#### Adding a Path - -1. Navigate to the `openapi/paths` folder. -2. Add a new YAML file named like your URL endpoint except replacing `/` with `@` into folder named by the collection ( - example `request-queues`) -3. Put path parameters into curly braces like `{example}`. -4. Add the path and a ref to it inside of your `openapi.yaml` file inside of the `openapi` folder. - -Example addition to the `openapi.yaml` file: - -```yaml -'/requests-queues': - $ref: './paths/request-queues/request-queues.yaml' -'/requests-queues/{queueId}': - $ref: './paths/request-queues/request-queues@{queueId}.yaml' -``` - -Here is an example of a YAML file named `request-queues@{queueId}.yaml` in the `paths/request-queues` folder: - -```yaml -get: - tags: - - Request Queues - summary: Get a Request Queue - operationId: requestQueues_get - description: | - You can have a markdown description here. - parameters: - responses: - '200': - '401': - x-code-samples: - - lang: PHP - source: - $ref: ../code_samples/PHP/customers/get.php -``` - -Here is an example of a YAML file named `request-queues.yaml` in the `paths/request-queues` folder: - -```yaml -post: - tags: - - Request Queues - summary: Create a request queue - operationId: requestQueue_post - description: Another markdown description here. - requestBody: - responses: - '201': - description: '' - content: - application/json: - schema: - allOf: - - $ref: ../../components/schemas/request-queues/CreateRequestQueueResponse.yaml - x-code-samples: - - lang: PHP - source: - $ref: ../code_samples/PHP/customers/post.php -``` - -You'll see extensive usage of `$ref`s in this example to different types of components including schemas. - -You'll also notice `$ref`s to code samples. - -#### Operation ID - -The `operationId` is a unique identifier of the operation. It is used by the Redocly API Reference to generate code -samples and link them to the operation from other parts of the documentation. - -There is a format for the `operationId`: - -- The ID is generated from the path structure and the HTTP method. -- The object names should be in camelCase. -- If there is `{id}` in the path, the previous part of the path should be singular, otherwise plural. -- The divider between the object name and the action should be `_`. -- The action should be last in the `operationId` defined by the method name in lowercase. - -Examples: - -- `/requests-queues` Method: GET -> `requestQueues_get` -- `/requests-queues` Method: POST ->`requestQueues_post` -- `/requests-queues/{queueId}` Method: PUT -> `requestQueue_put` -- `/acts/{actorId}/runs` Method: POST -> `act_runs_post` -- `/acts/{actorId}/runs` Method: GET -> `act_runs_get` -- `/acts/{actorId}/versions/{versionNumber}/env-vars/{envVarName}` Method: GET -> `act_version_envVar_get` - -### Code samples - -NOTE: We do not use code samples in the OpenAPI definition for now. -We can use them in the future to generate code samples for the reference docs. - -1. Navigate to the `openapi/code_samples` folder. -2. Navigate to the `` (e.g. PHP) sub-folder. -3. Navigate to the `path` folder, and add ref to the code sample. - -You can add languages by adding new folders at the appropriate path level. - -More details inside the `code_samples` folder README. diff --git a/apify-api/openapi/README.md b/apify-api/openapi/README.md deleted file mode 100644 index 71fc8c67e..000000000 --- a/apify-api/openapi/README.md +++ /dev/null @@ -1,13 +0,0 @@ -# The `openapi` folder - -This folder contains your entrypoint `openapi.yaml`. - -That file contains references to the entire API definition. - -Here are some sections to pay attention to: - -* Top-level **description**: this accepts markdown, and Redoc and Redocly API Reference will render it at the top of the docs. Consider maintaining your markdown in a separate file and [embedding it](https://redoc.ly/docs/api-reference-docs/embedded-markdown/). Note to Redoc community edition users, the special tags are only available to the Redocly API Reference users, but you can still embed markdown. -* Security schemes: you will define the scheme(s) your API uses for security (eg OAuth2, API Key, etc...). The security schemes are used by the Redocly API Reference "Try It" API console feature. -* [Paths](paths/README.md): this defines each endpoint. A path can have one operation per http method. -* Tags: it's a good idea to organize each operation. Each tag can have a description. The description is used as a section description within the reference docs. -* Servers: a list of your servers, each with a URL. diff --git a/apify-api/openapi/components/README.md b/apify-api/openapi/components/README.md deleted file mode 100644 index d1626bd46..000000000 --- a/apify-api/openapi/components/README.md +++ /dev/null @@ -1,13 +0,0 @@ -# Reusable components - -* You can create the following folders here: - * `schemas` - reusable [Schema Objects](https://github.com/OAI/OpenAPI-Specification/blob/master/versions/3.0.2.md#schemaObject) - * `responses` - reusable [Response Objects](https://github.com/OAI/OpenAPI-Specification/blob/master/versions/3.0.2.md#responseObject) - * `parameters` - reusable [Parameter Objects](https://github.com/OAI/OpenAPI-Specification/blob/master/versions/3.0.2.md#parameterObject) - * `examples` - reusable [Example Objects](https://github.com/OAI/OpenAPI-Specification/blob/master/versions/3.0.2.md#exampleObject) - * `headers` - reusable [Header Objects](https://github.com/OAI/OpenAPI-Specification/blob/master/versions/3.0.2.md#headerObject) - * `requestBodies` - reusable [Request Body Objects](https://github.com/OAI/OpenAPI-Specification/blob/master/versions/3.0.2.md#requestBodyObject) - * `links` - reusable [Link Objects](https://github.com/OAI/OpenAPI-Specification/blob/master/versions/3.0.2.md#linkObject) - * `callbacks` - reusable [Callback Objects](https://github.com/OAI/OpenAPI-Specification/blob/master/versions/3.0.2.md#callbackObject) - * `securitySchemes` - reusable [Security Scheme Objects](https://github.com/OAI/OpenAPI-Specification/blob/master/versions/3.0.2.md#securitySchemeObject) -* Filename of files inside the folders represent component name, e.g. `Customer.yaml` diff --git a/apify-api/openapi/paths/README.md b/apify-api/openapi/paths/README.md deleted file mode 100644 index 63620d54d..000000000 --- a/apify-api/openapi/paths/README.md +++ /dev/null @@ -1,108 +0,0 @@ -# Paths - -Organize your path definitions within this folder. You will reference your paths from your main `openapi.yaml` entrypoint file. - -It may help you to adopt some conventions: - -* path separator token (e.g. `@`) or subfolders -* path parameter (e.g. `{example}`) -* file-per-path or file-per-operation - -There are different benefits and drawbacks to each decision. - -You can adopt any organization you wish. We have some tips for organizing paths based on common practices. - -## Each path in a separate file - -Use a predefined "path separator" and keep all of your path files in the top level of the `paths` folder. - -```yaml -# todo: insert tree view of paths folder -``` - -Redocly recommends using the `@` character for this case. - -In addition, Redocly recommends placing path parameters within `{}` curly braces if you adopt this style. - -### Motivations - -* Quickly see a list of all paths. Many people think in terms of the "number" of "endpoints" (paths), and not the "number" of "operations" (paths * http methods). - -* Only the "file-per-path" option is semantically correct with the OpenAPI Specification 3.0.2. However, Redocly's openapi-cli will build valid bundles for any of the other options too. - - -### Drawbacks - -* This may require multiple definitions per http method within a single file. -* It requires settling on a path separator (that is allowed to be used in filenames) and sticking to that convention. - -## Each operation in a separate file - -You may also place each operation in a separate file. - -In this case, if you want all paths at the top-level, you can concatenate the http method to the path name. Similar to the above option, you can - -### Files at top-level of `paths` - -You may name your files with some concatenation for the http method. For example, following a convention such as: `-.yaml`. - -#### Motivations - -* Quickly see all operations without needing to navigate subfolders. - -#### Drawbacks - -* Adopting an unusual path separator convention, instead of using subfolders. - -### Use subfolders to mirror API path structure - -Example: - -```http request -GET /customers - -/paths/customers/get.yaml -``` - -In this case, the path id defined within subfolders which mirror the API URL structure. - -Example with path parameter: - -```http request -GET /customers/{id} - -/paths/customers/{id}/get.yaml -``` - -#### Motivations - -It matches the URL structure. - -It is pretty easy to reference: - -```yaml -paths: - '/customers/{id}': - get: - $ref: ./paths/customers/{id}/get.yaml - put: - $ref: ./paths/customers/{id}/put.yaml -``` - -#### Drawbacks - -If you have a lot of nested folders, it may be confusing to reference your schemas. - -Example: - -```yaml -file: /paths/customers/{id}/timeline/{messageId}/get.yaml - -# excerpt of file - headers: - Rate-Limit-Remaining: - $ref: ../../../../../components/headers/Rate-Limit-Remaining.yaml - -``` - -Notice the `../../../../../` in the ref which requires some attention to formulate correctly. While openapi-cli has a linter which suggests possible refs when there is a mistake, this is still a net drawback for APIs with deep paths.