Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add conan datasource and manager #12009

Merged
merged 120 commits into from
Feb 11, 2022
Merged

Conversation

mhetzel
Copy link
Contributor

@mhetzel mhetzel commented Oct 5, 2021

Changes:

adding a conan datasource to interact with conan v2 api and a conan manager to parse conanfile.txt and conanfile.py files.

Context:

closes #5933

Documentation (please check one with an [x])

  • I have updated the documentation, or
  • No documentation update is required

How I've tested my work (please tick one)

I have verified these changes via:

  • Code inspection only, or
  • Newly added/modified unit tests, or
  • No unit tests but ran on a real repository, or
  • Both unit tests + ran on a real repository

@JamieMagee JamieMagee self-requested a review October 5, 2021 03:08
docs/usage/conan.md Outdated Show resolved Hide resolved
docs/usage/conan.md Outdated Show resolved Hide resolved
lib/datasource/conan/__fixtures__/pocoDigest Outdated Show resolved Hide resolved
lib/manager/conan/readme.md Outdated Show resolved Hide resolved
docs/usage/conan.md Outdated Show resolved Hide resolved
docs/usage/conan.md Outdated Show resolved Hide resolved
@mhetzel mhetzel requested a review from HonkingGoose October 7, 2021 13:35
Copy link
Collaborator

@HonkingGoose HonkingGoose left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Contents of both files is good. Not sure if the current location of the file is correct though.

Do we want the conan.md file under the docs/usage directory?
We have a managers overview page. 1

We could merge the conan.md with the lib/manager/conan/readme.md file in the lib/manager/conan directory structure? 2 That would put everything conan related on it's own page, that can be found via the managers overview page.

Footnotes

  1. https://docs.renovatebot.com/modules/manager/

  2. https://github.com/renovatebot/renovate/tree/main/lib/manager

@rarkins
Copy link
Collaborator

rarkins commented Oct 8, 2021

It would be better to have all information in the manager readme file

@mhetzel
Copy link
Contributor Author

mhetzel commented Oct 8, 2021

@HonkingGoose @rarkins I've merged the two files. Hopefully that's what you were looking for.

@mhetzel mhetzel requested a review from HonkingGoose October 8, 2021 19:07
lib/manager/conan/extract.spec.ts Outdated Show resolved Hide resolved
lib/manager/conan/extract.spec.ts Outdated Show resolved Hide resolved
lib/manager/conan/extract.spec.ts Outdated Show resolved Hide resolved
@mhetzel mhetzel requested a review from viceice February 10, 2022 19:11
@viceice viceice enabled auto-merge (squash) February 10, 2022 23:25
@mhetzel mhetzel disabled auto-merge February 10, 2022 23:32
@mhetzel mhetzel requested a review from rarkins February 10, 2022 23:38
@JamieMagee JamieMagee enabled auto-merge (squash) February 11, 2022 04:50
@rarkins
Copy link
Collaborator

rarkins commented Feb 11, 2022

Amazing PR, @mhetzel!

@JamieMagee JamieMagee merged commit 8f6bf44 into renovatebot:main Feb 11, 2022
@renovate-release
Copy link
Collaborator

🎉 This PR is included in version 31.78.0 🎉

The release is available on:

Your semantic-release bot 📦🚀

Copy link

@memsharded memsharded left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi folks!

ConanCenter at the moment has some issues with the search endpoint that you might have hit. It can be quite slow, and even timeout. We are working on it, but it might take some time. I would like to recommend:

  • Try to limit the number of searches, cache them whenever possible
  • Use always the full exact package name, avoid patterns, specifically avoid a * pattern

You might want to track conan-io/conan#10298 for updates about this.

@HonkingGoose
Copy link
Collaborator

@rarkins seems we might need another limit like we created for repology. We're hammering the Conan servers too much now.

@memsharded
Copy link

Alternatively, there could be a very practical and fast approach: Everything in ConanCenter comes from the conan-center-index github repo: https://github.com/conan-io/conan-center-index. All the information is there, and the structure of the repo is well defined, information about versions is in there. Cloning the repo and getting the information would be a 0 hit on servers, and always super-fast, while the implementation should be straightforward enough?

@rarkins
Copy link
Collaborator

rarkins commented Feb 11, 2022

I think this means we might need to roll back Conan support in the meantime? :-/

@memsharded
Copy link

memsharded commented Feb 11, 2022

I think this means we might need to roll back Conan support in the meantime? :-/

You can measure a bit, and see what you get against the ConanCenter server, but if the number of queries is going to be high, then it might be a problem yes. Until conan-io/conan#10298 is closed, I would probably try to avoid the API search endpoint, and try getting the information from the repo. If cloning a repo locally is an issue for the tool, it can also be very simple to obtain the current versions directly from Github, like from https://github.com/conan-io/conan-center-index/blob/master/recipes/abseil/config.yml

@viceice
Copy link
Member

viceice commented Feb 11, 2022

@rarkins Or just disable it for now via force from app and disable in code for now?

@rarkins
Copy link
Collaborator

rarkins commented Feb 11, 2022

It's not yet live in the app, so let's disable it by default in this repo first, and maybe allow people just opt-in for now. If that still becomes a problem, we can force it disabled.

@rarkins
Copy link
Collaborator

rarkins commented Feb 11, 2022

I've created an issue #14170 to move discussion to

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Mar 14, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add support for Conan
7 participants