This does not appear to catch the problem when:

current_version="2.3-rc1"                                                                                      
new_version="2.3"                                                                                              

I get:

ERROR: The version provided (2.3) has to be greater than the current release (2.3-rc1).

The version "2.3" should, semver-wise, be greater than "2.3-rc1".

Running a tool like semver-cli does catch this:

> semver-cli greater -v "2.3-rc1" "2.3"; echo "returned: $?"
2.3
returned: 1
> semver-cli greater -v "2.3" "2.3-rc1"; echo "returned: $?"
2.3
returned: 0

Consider using semver-cli and changing the logic above to:

if ! semver-cli greater "$new_version" "$current_version"; then
	echo "ERROR: The version provided ($new_version) has to be greater than the current release ($current_version)."
	exit 1                                                                      
fi

@gammazero, that's because 2.3-rc1 is not compliant with the SemVer specification, as noted on Item 2 of the specification:

A normal version number MUST take the form X.Y.Z where X, Y, and Z are non-negative integers, and MUST NOT contain leading zeroes. X is the major version, Y is the minor version, and Z is the patch version. Each element MUST increase numerically. For instance: 1.9.0 -> 1.10.0 -> 1.11.0.

Thus, to be correctly formatted, the version must be separated by three dots.

There are some useful tips on the SemVer's FAQ that I use myself:

Q: Is there a suggested regular expression (RegEx) to check a SemVer string?

A: There are two. One with named groups for those systems that support them (PCRE [Perl Compatible Regular Expressions, i.e. Perl, PHP and R], Python and Go).
See: https://regex101.com/r/Ly7O1x/3/

^(?P<major>0|[1-9]\d*)\.(?P<minor>0|[1-9]\d*)\.(?P<patch>0|[1-9]\d*)(?:-(?P<prerelease>(?:0|[1-9]\d*|\d*[a-zA-Z-][0-9a-zA-Z-]*)(?:\.(?:0|[1-9]\d*|\d*[a-zA-Z-][0-9a-zA-Z-]*))*))?(?:\+(?P<buildmetadata>[0-9a-zA-Z-]+(?:\.[0-9a-zA-Z-]+)*))?$

And one with numbered capture groups instead (so cg1 = major, cg2 = minor, cg3 = patch, cg4 = prerelease and cg5 = buildmetadata) that is compatible with ECMA Script (JavaScript), PCRE (Perl Compatible Regular Expressions, i.e. Perl, PHP and R), Python and Go.
See: https://regex101.com/r/vkijKf/1/

^(0|[1-9]\d*)\.(0|[1-9]\d*)\.(0|[1-9]\d*)(?:-((?:0|[1-9]\d*|\d*[a-zA-Z-][0-9a-zA-Z-]*)(?:\.(?:0|[1-9]\d*|\d*[a-zA-Z-][0-9a-zA-Z-]*))*))?(?:\+([0-9a-zA-Z-]+(?:\.[0-9a-zA-Z-]+)*))?$

Thank you for the detailed explanation. I took an example directly from the specification (item 11-3)

When major, minor, and patch are equal, a pre-release version has lower precedence than a normal version:
Example: 1.0.0-alpha < 1.0.0.

The code provided does not catch that "1.0.0-alpha" is less than "1.0.0"

@gammazero, thank you for your review! You're right! And thanks for commenting on the StackOverflow answer related to this!