Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

#173: use force update mode for ide-urls #155

Merged
merged 57 commits into from
Feb 27, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
57 commits
Select commit Hold shift + click to select a range
4021e08
#6: fixed git pull hangs
jan-vcapgemini Dec 12, 2023
20172d5
#6: moved urls git url to constant
jan-vcapgemini Dec 12, 2023
3ea78c0
#6: added some cleanups
jan-vcapgemini Dec 12, 2023
bfba3f7
Merge branch 'main' into fix/6-git-pull-hangs
hohwille Dec 15, 2023
7f9b949
Update cli/src/main/java/com/devonfw/tools/ide/context/AbstractIdeCon…
jan-vcapgemini Jan 8, 2024
097d2f7
Update cli/src/main/java/com/devonfw/tools/ide/context/AbstractIdeCon…
jan-vcapgemini Jan 8, 2024
512a367
#6: implemented requested changes
jan-vcapgemini Jan 8, 2024
5b4d856
Merge branch 'main' into fix/6-git-pull-hangs
jan-vcapgemini Jan 8, 2024
36383e9
Merge branch 'main' into fix/6-git-pull-hangs
hohwille Jan 8, 2024
d46ec7b
#6: fixed git fetch and reset
jan-vcapgemini Jan 9, 2024
86e874f
#6: adjusted warning
jan-vcapgemini Jan 9, 2024
2d7792a
#6: re-added git pull
jan-vcapgemini Jan 9, 2024
21347a5
#6: removed invalid interactive option
jan-vcapgemini Jan 9, 2024
22936b9
#6: fixed git pull
jan-vcapgemini Jan 9, 2024
198c66c
Merge branch 'main' into fix/6-git-pull-hangs
jan-vcapgemini Jan 11, 2024
2763a73
Merge branch 'main' into fix/6-git-pull-hangs
jan-vcapgemini Jan 22, 2024
ac711b9
Merge branch 'main' into fix/6-git-pull-hangs
jan-vcapgemini Jan 23, 2024
0826be3
Merge branch 'main' into fix/6-git-pull-hangs
jan-vcapgemini Jan 23, 2024
6cc8b14
#173: added more git checks
jan-vcapgemini Jan 23, 2024
a04db03
#173: introduced git utils
jan-vcapgemini Jan 25, 2024
49733fc
#173: more refactorings
jan-vcapgemini Jan 26, 2024
4dc4437
Merge branch 'main' into fix/6-git-pull-hangs
jan-vcapgemini Jan 26, 2024
ef9ae80
#173: more refactorings
jan-vcapgemini Jan 29, 2024
3d4f63a
#173: added/improved warnings
jan-vcapgemini Jan 29, 2024
e862aac
#173: applied non star imports
jan-vcapgemini Jan 30, 2024
391e9f9
Merge branch 'main' into fix/6-git-pull-hangs
jan-vcapgemini Jan 31, 2024
057b495
#173: replaced isOnline
jan-vcapgemini Feb 2, 2024
43f96c8
#173: implemented requested change
jan-vcapgemini Feb 2, 2024
cccc2e7
Merge branch 'main' into fix/6-git-pull-hangs
jan-vcapgemini Feb 2, 2024
bdab005
#173: added new tests and mocks
jan-vcapgemini Feb 5, 2024
dbe5000
#173: added info messages
jan-vcapgemini Feb 5, 2024
f0de981
#173: changed to debug mode
jan-vcapgemini Feb 5, 2024
89bb42e
#173: changed to sysout
jan-vcapgemini Feb 5, 2024
f337621
#173: replaced try with assertThrows
jan-vcapgemini Feb 5, 2024
664f5f8
#173: disabled throw exception test
jan-vcapgemini Feb 5, 2024
2d98ad0
#173: implemented tempDir
jan-vcapgemini Feb 6, 2024
701df78
#173: fixed offline simulation
jan-vcapgemini Feb 9, 2024
e3ac7ab
#173: added git reset test
jan-vcapgemini Feb 9, 2024
10f09e1
#173: cleanup
jan-vcapgemini Feb 9, 2024
ef70b06
Merge branch 'main' into fix/6-git-pull-hangs
jan-vcapgemini Feb 13, 2024
d5da12e
#173: fixed issues after merge
jan-vcapgemini Feb 13, 2024
b520ddc
#173: refactored AbstractIdeContext
jan-vcapgemini Feb 13, 2024
a9f5b68
#173: new GitContext interface
jan-vcapgemini Feb 14, 2024
b1b62bc
#173: more refactorings
jan-vcapgemini Feb 15, 2024
e914e81
#173: more refactorings
jan-vcapgemini Feb 15, 2024
cf77e70
#173: deleted GitUtils
jan-vcapgemini Feb 15, 2024
0d5bb39
Merge branch 'main' into fix/6-git-pull-hangs
jan-vcapgemini Feb 15, 2024
84760ce
#173: fixed git pull hangs #6
jan-vcapgemini Feb 16, 2024
7ebbf2f
Merge branch 'main' into fix/6-git-pull-hangs
jan-vcapgemini Feb 19, 2024
924d13a
#173: implemented requested changes
jan-vcapgemini Feb 20, 2024
51cd592
#173: fixed behind message
jan-vcapgemini Feb 20, 2024
b358885
#173: implemented requested changes
jan-vcapgemini Feb 21, 2024
ef7dc49
#173: fixed javadoc
jan-vcapgemini Feb 21, 2024
df7b883
Merge branch 'main' into fix/6-git-pull-hangs
jan-vcapgemini Feb 22, 2024
c377e10
#173: fixed merge conflict
jan-vcapgemini Feb 22, 2024
400d0d5
Merge branch 'main' into fix/6-git-pull-hangs
jan-vcapgemini Feb 23, 2024
47e9eda
Merge branch 'main' into fix/6-git-pull-hangs
hohwille Feb 27, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
124 changes: 13 additions & 111 deletions cli/src/main/java/com/devonfw/tools/ide/context/AbstractIdeContext.java
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,6 @@
import java.net.InetAddress;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.attribute.FileTime;
import java.time.Duration;
import java.util.HashMap;
import java.util.Iterator;
import java.util.List;
Expand Down Expand Up @@ -39,7 +37,6 @@
import com.devonfw.tools.ide.os.SystemInfoImpl;
import com.devonfw.tools.ide.process.ProcessContext;
import com.devonfw.tools.ide.process.ProcessContextImpl;
import com.devonfw.tools.ide.process.ProcessErrorHandling;
import com.devonfw.tools.ide.process.ProcessResult;
import com.devonfw.tools.ide.property.Property;
import com.devonfw.tools.ide.repo.CustomToolRepository;
Expand All @@ -54,6 +51,8 @@
*/
public abstract class AbstractIdeContext implements IdeContext {

private static final String IDE_URLS_GIT = "https://github.com/devonfw/ide-urls.git";

private final Map<IdeLogLevel, IdeSubLogger> loggers;

private final Path ideHome;
Expand Down Expand Up @@ -120,8 +119,6 @@ public abstract class AbstractIdeContext implements IdeContext {

private UrlMetadata urlMetadata;

private static final Duration GIT_PULL_CACHE_DELAY_MILLIS = Duration.ofMillis(30 * 60 * 1000);

/**
* The constructor.
*
Expand Down Expand Up @@ -186,10 +183,11 @@ public AbstractIdeContext(IdeLogLevel minLogLevel, Function<IdeLogLevel, IdeSubL
Path rootPath = Path.of(root);
if (Files.isDirectory(rootPath)) {
if (!ideRootPath.equals(rootPath)) {
warning("Variable IDE_ROOT is set to '{}' but for your project '{}' would have been expected.");
ideRootPath = rootPath;
warning(
"Variable IDE_ROOT is set to '{}' but for your project '{}' the path '{}' would have been expected.",
root, this.ideHome.getFileName(), ideRootPath);
}
ideRootPath = this.ideHome.getParent();
ideRootPath = rootPath;
} else {
warning("Variable IDE_ROOT is not set to a valid directory '{}'." + root);
ideRootPath = null;
Expand Down Expand Up @@ -266,10 +264,7 @@ public String getMessageIdeHome() {
*/
public boolean isTest() {

if (isMock()) {
return true;
}
return false;
return isMock();
}

/**
Expand Down Expand Up @@ -496,7 +491,7 @@ public UrlMetadata getUrls() {

if (this.urlMetadata == null) {
if (!isTest()) {
gitPullOrCloneIfNeeded(this.urlsPath, "https://github.com/devonfw/ide-urls.git");
this.getGitContext().pullOrFetchAndResetIfNeeded(IDE_URLS_GIT, this.urlsPath, "origin", "master");
}
this.urlMetadata = new UrlMetadata(this);
}
Expand Down Expand Up @@ -566,7 +561,7 @@ public boolean isOnline() {
try {
int timeout = 1000;
online = InetAddress.getByName("github.com").isReachable(timeout);
} catch (Exception e) {
} catch (Exception ignored) {

}
return online;
Expand Down Expand Up @@ -596,108 +591,15 @@ public DirectoryMerger getWorkspaceMerger() {
}

@Override
public ProcessContext newProcess() {
public GitContext getGitContext() {

ProcessContext processContext = new ProcessContextImpl(this);
return processContext;
return new GitContextImpl(this);
}

@Override
public void gitPullOrClone(Path target, String gitRepoUrl) {

Objects.requireNonNull(target);
Objects.requireNonNull(gitRepoUrl);
if (!gitRepoUrl.startsWith("http")) {
throw new IllegalArgumentException("Invalid git URL '" + gitRepoUrl + "'!");
}
ProcessContext pc = newProcess().directory(target).executable("git").withEnvVar("GIT_TERMINAL_PROMPT", "0");
if (Files.isDirectory(target.resolve(".git"))) {
ProcessResult result = pc.addArg("remote").run(true, false);
List<String> remotes = result.getOut();
if (remotes.isEmpty()) {
String message = "This is a local git repo with no remote - if you did this for testing, you may continue...\n"
+ "Do you want to ignore the problem and continue anyhow?";
askToContinue(message);
} else {
pc.errorHandling(ProcessErrorHandling.WARNING);
result = pc.addArg("pull").run(false, false);
if (!result.isSuccessful()) {
String message = "Failed to update git repository at " + target;
if (this.offlineMode) {
warning(message);
interaction("Continuing as we are in offline mode - results may be outdated!");
} else {
error(message);
if (isOnline()) {
error("See above error for details. If you have local changes, please stash or revert and retry.");
} else {
error(
"It seems you are offline - please ensure Internet connectivity and retry or activate offline mode (-o or --offline).");
}
askToContinue("Typically you should abort and fix the problem. Do you want to continue anyways?");
}
}
}
} else {
String branch = null;
int hashIndex = gitRepoUrl.indexOf("#");
if (hashIndex != -1) {
branch = gitRepoUrl.substring(hashIndex + 1);
gitRepoUrl = gitRepoUrl.substring(0, hashIndex);
}
this.fileAccess.mkdirs(target);
requireOnline("git clone of " + gitRepoUrl);
pc.addArg("clone");
if (isQuietMode()) {
pc.addArg("-q");
} else {
}
pc.addArgs("--recursive", gitRepoUrl, "--config", "core.autocrlf=false", ".");
pc.run();
if (branch != null) {
pc.addArgs("checkout", branch);
pc.run();
}
}
}

/**
* Checks if the Git repository in the specified target folder needs an update by inspecting the modification time of
* a magic file.
*
* @param urlsPath The Path to the Urls repository.
* @param repoUrl The git remote URL of the Urls repository.
*/

private void gitPullOrCloneIfNeeded(Path urlsPath, String repoUrl) {

Path gitDirectory = urlsPath.resolve(".git");

// Check if the .git directory exists
if (Files.isDirectory(gitDirectory)) {
Path magicFilePath = gitDirectory.resolve("HEAD");
long currentTime = System.currentTimeMillis();
// Get the modification time of the magic file
long fileMTime;
try {
fileMTime = Files.getLastModifiedTime(magicFilePath).toMillis();
} catch (IOException e) {
throw new IllegalStateException("Could not read " + magicFilePath, e);
}
public ProcessContext newProcess() {

// Check if the file modification time is older than the delta threshold
if ((currentTime - fileMTime > GIT_PULL_CACHE_DELAY_MILLIS.toMillis()) || isForceMode()) {
gitPullOrClone(urlsPath, repoUrl);
try {
Files.setLastModifiedTime(magicFilePath, FileTime.fromMillis(currentTime));
} catch (IOException e) {
throw new IllegalStateException("Could not read or write in " + magicFilePath, e);
}
}
} else {
// If the .git directory does not exist, perform git clone
gitPullOrClone(urlsPath, repoUrl);
}
return new ProcessContextImpl(this);
}

@Override
Expand Down
87 changes: 87 additions & 0 deletions cli/src/main/java/com/devonfw/tools/ide/context/GitContext.java
Original file line number Diff line number Diff line change
@@ -0,0 +1,87 @@
package com.devonfw.tools.ide.context;

import java.nio.file.Path;

import com.devonfw.tools.ide.log.IdeLogger;

/**
* Interface for git commands with input and output of information for the user.
*/
public interface GitContext extends IdeLogger {

/**
* Checks if the Git repository in the specified target folder needs an update by inspecting the modification time of
* a magic file.
*
* @param repoUrl the git remote URL to clone from. May be suffixed with a hash-sign ('#') followed by the branch name
* to check-out.
* @param targetRepository the {@link Path} to the target folder where the git repository should be cloned or pulled.
* It is not the parent directory where git will by default create a sub-folder by default on clone but the *
* final folder that will contain the ".git" subfolder.
*/
void pullOrCloneIfNeeded(String repoUrl, Path targetRepository);

/**
* Attempts a git pull and reset if required.
*
* @param repoUrl the git remote URL to clone from. May be suffixed with a hash-sign ('#') followed by the branch name
* to check-out.
* @param targetRepository the {@link Path} to the target folder where the git repository should be cloned or pulled.
* It is not the parent directory where git will by default create a sub-folder by default on clone but the *
* final folder that will contain the ".git" subfolder.
* @param remoteName the remote name e.g. origin.
* @param branchName the branch name e.g. master.
*/
void pullOrFetchAndResetIfNeeded(String repoUrl, Path targetRepository, String remoteName, String branchName);

/**
* Runs a git pull or a git clone.
*
* @param gitRepoUrl the git remote URL to clone from. May be suffixed with a hash-sign ('#') followed by the branch
* name to check-out.
* @param targetRepository the {@link Path} to the target folder where the git repository should be cloned or pulled.
* It is not the parent directory where git will by default create a sub-folder by default on clone but the *
* final folder that will contain the ".git" subfolder.
*/
void pullOrClone(String gitRepoUrl, Path targetRepository);

/**
* Runs a git clone. Throws a CliException if in offline mode.
*
* @param gitRepoUrl the {@link GitUrl} to use for the repository URL.
* @param targetRepository the {@link Path} to the target folder where the git repository should be cloned or pulled.
* It is not the parent directory where git will by default create a sub-folder by default on clone but the *
* final folder that will contain the ".git" subfolder.
*/
void clone(GitUrl gitRepoUrl, Path targetRepository);

/**
* Runs a git pull.
*
* @param targetRepository the {@link Path} to the target folder where the git repository should be cloned or pulled.
* It is not the parent directory where git will by default create a sub-folder by default on clone but the *
* final folder that will contain the ".git" subfolder.
*/
void pull(Path targetRepository);

/**
* Runs a git reset if files were modified.
*
* @param targetRepository the {@link Path} to the target folder where the git repository should be cloned or pulled.
* It is not the parent directory where git will by default create a sub-folder by default on clone but the *
* final folder that will contain the ".git" subfolder.
* @param remoteName the remote server name.
* @param branchName the name of the branch.
*/
void reset(Path targetRepository, String remoteName, String branchName);

/**
* Runs a git cleanup if untracked files were found.
*
* @param targetRepository the {@link Path} to the target folder where the git repository should be cloned or pulled.
* It is not the parent directory where git will by default create a sub-folder by default on clone but the *
* final folder that will contain the ".git" subfolder.
*/
void cleanup(Path targetRepository);

}
Loading
Loading