Skip to content

Promote exit tests to API #324

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 21 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
93521df
[SWT-NNNN] Exit tests
grynspan Apr 2, 2024
f54c9d5
Remove proposal doc (moving to SE)
grynspan Mar 3, 2025
f48b44e
Try to work around compiler crash
grynspan Mar 3, 2025
76eb534
Create an Exit Testing DocC article
grynspan Mar 24, 2025
1c1a67a
Create an Exit Testing DocC article (no rly)
grynspan Mar 24, 2025
0304bfc
Avoid the word 'exit' in documentation, add more details about exit c…
grynspan Mar 25, 2025
fa5aeed
Nest subtopics of article correctly, note platform availability
grynspan Mar 25, 2025
cfbd85c
Remove macOS availability override in article
grynspan Mar 25, 2025
8b12522
Refine the text around statusAtExit always being set
grynspan Mar 25, 2025
7507cf0
Remove cppreference.com links from public docs (not actually an offic…
grynspan Mar 25, 2025
af9776c
Update Linux manpage links to kernel.org
grynspan Mar 25, 2025
090fdea
Incorporate editioral feedback
grynspan Mar 26, 2025
11b772d
Minor language tweaks
grynspan Mar 26, 2025
324e348
StatusAtExit -> ExitStatus
grynspan Apr 15, 2025
89e1885
Merge branch 'main' into jgrynspan/exit-tests-proposal
grynspan Apr 15, 2025
8a76876
Merge branch 'main' into jgrynspan/exit-tests-proposal
grynspan Apr 15, 2025
5d10fd8
Fix a couple of typos
grynspan Apr 23, 2025
933b6af
Merge branch 'main' into jgrynspan/exit-tests-proposal
grynspan Apr 23, 2025
c5c3d29
Merge branch 'main' into jgrynspan/exit-tests-proposal
grynspan Apr 24, 2025
168f275
Merge branch 'main' into jgrynspan/exit-tests-proposal
grynspan Apr 25, 2025
d6cdac5
exitsWith: -> processExitsWith:
grynspan Apr 29, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion Sources/Testing/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ add_library(Testing
ExitTests/ExitTest.Condition.swift
ExitTests/ExitTest.Result.swift
ExitTests/SpawnProcess.swift
ExitTests/StatusAtExit.swift
ExitTests/ExitStatus.swift
ExitTests/WaitFor.swift
Expectations/Expectation.swift
Expectations/Expectation+Macro.swift
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -10,74 +10,94 @@

private import _TestingInternals

/// An enumeration describing possible status a process will yield on exit.
/// An enumeration describing possible status a process will report on exit.
///
/// You can convert an instance of this type to an instance of
/// ``ExitTest/Condition`` using ``ExitTest/Condition/init(_:)``. That value
/// can then be used to describe the condition under which an exit test is
/// expected to pass or fail by passing it to
/// ``expect(exitsWith:observing:_:sourceLocation:performing:)`` or
/// ``require(exitsWith:observing:_:sourceLocation:performing:)``.
@_spi(Experimental)
/// ``expect(processExitsWith:observing:_:sourceLocation:performing:)`` or
/// ``require(processExitsWith:observing:_:sourceLocation:performing:)``.
///
/// @Metadata {
/// @Available(Swift, introduced: 6.2)
/// }
#if SWT_NO_PROCESS_SPAWNING
@available(*, unavailable, message: "Exit tests are not available on this platform.")
#endif
public enum StatusAtExit: Sendable {
/// The process terminated with the given exit code.
public enum ExitStatus: Sendable {
/// The process exited with the given exit code.
///
/// - Parameters:
/// - exitCode: The exit code yielded by the process.
/// - exitCode: The exit code reported by the process.
///
/// The C programming language defines two [standard exit codes](https://en.cppreference.com/w/c/program/EXIT_status),
/// `EXIT_SUCCESS` and `EXIT_FAILURE`. Platforms may additionally define their
/// own non-standard exit codes:
/// The C programming language defines two standard exit codes, `EXIT_SUCCESS`
/// and `EXIT_FAILURE`. Platforms may additionally define their own
/// non-standard exit codes:
///
/// | Platform | Header |
/// |-|-|
/// | macOS | [`<stdlib.h>`](https://developer.apple.com/library/archive/documentation/System/Conceptual/ManPages_iPhoneOS/man3/_Exit.3.html), [`<sysexits.h>`](https://developer.apple.com/library/archive/documentation/System/Conceptual/ManPages_iPhoneOS/man3/sysexits.3.html) |
/// | Linux | [`<stdlib.h>`](https://sourceware.org/glibc/manual/latest/html_node/Exit-Status.html), `<sysexits.h>` |
/// | Linux | [`<stdlib.h>`](https://www.kernel.org/doc/man-pages/online/pages/man3/exit.3.html), [`<sysexits.h>`](https://www.kernel.org/doc/man-pages/online/pages/man3/sysexits.h.3head.html) |
/// | FreeBSD | [`<stdlib.h>`](https://man.freebsd.org/cgi/man.cgi?exit(3)), [`<sysexits.h>`](https://man.freebsd.org/cgi/man.cgi?sysexits(3)) |
/// | OpenBSD | [`<stdlib.h>`](https://man.openbsd.org/exit.3), [`<sysexits.h>`](https://man.openbsd.org/sysexits.3) |
/// | Windows | [`<stdlib.h>`](https://learn.microsoft.com/en-us/cpp/c-runtime-library/exit-success-exit-failure) |
///
/// @Comment {
/// See https://en.cppreference.com/w/c/program/EXIT_status for more
/// information about exit codes defined by the C standard.
/// }
///
/// On macOS, FreeBSD, OpenBSD, and Windows, the full exit code reported by
/// the process is yielded to the parent process. Linux and other POSIX-like
/// the process is reported to the parent process. Linux and other POSIX-like
/// systems may only reliably report the low unsigned 8 bits (0&ndash;255) of
/// the exit code.
///
/// @Metadata {
/// @Available(Swift, introduced: 6.2)
/// }
case exitCode(_ exitCode: CInt)

/// The process terminated with the given signal.
/// The process exited with the given signal.
///
/// - Parameters:
/// - signal: The signal that terminated the process.
/// - signal: The signal that caused the process to exit.
///
/// The C programming language defines a number of [standard signals](https://en.cppreference.com/w/c/program/SIG_types).
/// Platforms may additionally define their own non-standard signal codes:
/// The C programming language defines a number of standard signals. Platforms
/// may additionally define their own non-standard signal codes:
///
/// | Platform | Header |
/// |-|-|
/// | macOS | [`<signal.h>`](https://developer.apple.com/library/archive/documentation/System/Conceptual/ManPages_iPhoneOS/man3/signal.3.html) |
/// | Linux | [`<signal.h>`](https://sourceware.org/glibc/manual/latest/html_node/Standard-Signals.html) |
/// | Linux | [`<signal.h>`](https://www.kernel.org/doc/man-pages/online/pages/man7/signal.7.html) |
/// | FreeBSD | [`<signal.h>`](https://man.freebsd.org/cgi/man.cgi?signal(3)) |
/// | OpenBSD | [`<signal.h>`](https://man.openbsd.org/signal.3) |
/// | Windows | [`<signal.h>`](https://learn.microsoft.com/en-us/cpp/c-runtime-library/signal-constants) |
///
/// @Comment {
/// See https://en.cppreference.com/w/c/program/SIG_types for more
/// information about signals defined by the C standard.
/// }
///
/// @Metadata {
/// @Available(Swift, introduced: 6.2)
/// }
case signal(_ signal: CInt)
}

// MARK: - Equatable

@_spi(Experimental)
#if SWT_NO_PROCESS_SPAWNING
@available(*, unavailable, message: "Exit tests are not available on this platform.")
#endif
extension StatusAtExit: Equatable {}
extension ExitStatus: Equatable {}

// MARK: - CustomStringConvertible
@_spi(Experimental)
#if SWT_NO_PROCESS_SPAWNING
@available(*, unavailable, message: "Exit tests are not available on this platform.")
#endif
extension StatusAtExit: CustomStringConvertible {
extension ExitStatus: CustomStringConvertible {
public var description: String {
switch self {
case let .exitCode(exitCode):
Expand Down
2 changes: 1 addition & 1 deletion Sources/Testing/ExitTests/ExitTest.CapturedValue.swift
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ extension ExitTest {
/// exit test:
///
/// ```swift
/// await #expect(exitsWith: .failure) { [a = a as T, b = b as U, c = c as V] in
/// await #expect(processExitsWith: .failure) { [a = a as T, b = b as U, c = c as V] in
/// ...
/// }
/// ```
Expand Down
107 changes: 76 additions & 31 deletions Sources/Testing/ExitTests/ExitTest.Condition.swift
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,6 @@

private import _TestingInternals

@_spi(Experimental)
#if SWT_NO_EXIT_TESTS
@available(*, unavailable, message: "Exit tests are not available on this platform.")
#endif
Expand All @@ -19,13 +18,29 @@ extension ExitTest {
///
/// Values of this type are used to describe the conditions under which an
/// exit test is expected to pass or fail by passing them to
/// ``expect(exitsWith:observing:_:sourceLocation:performing:)`` or
/// ``require(exitsWith:observing:_:sourceLocation:performing:)``.
/// ``expect(processExitsWith:observing:_:sourceLocation:performing:)`` or
/// ``require(processExitsWith:observing:_:sourceLocation:performing:)``.
///
/// ## Topics
///
/// ### Successful exit conditions
///
/// - ``success``
///
/// ### Failing exit conditions
///
/// - ``failure``
/// - ``exitCode(_:)``
/// - ``signal(_:)``
///
/// @Metadata {
/// @Available(Swift, introduced: 6.2)
/// }
public struct Condition: Sendable {
/// An enumeration describing the possible conditions for an exit test.
private enum _Kind: Sendable, Equatable {
/// The exit test must exit with a particular exit status.
case statusAtExit(StatusAtExit)
case exitStatus(ExitStatus)

/// The exit test must exit successfully.
case success
Expand All @@ -41,49 +56,71 @@ extension ExitTest {

// MARK: -

@_spi(Experimental)
#if SWT_NO_EXIT_TESTS
@available(*, unavailable, message: "Exit tests are not available on this platform.")
#endif
extension ExitTest.Condition {
/// A condition that matches when a process terminates successfully with exit
/// code `EXIT_SUCCESS`.
/// A condition that matches when a process exits normally.
///
/// This condition matches the exit code `EXIT_SUCCESS`.
///
/// @Metadata {
/// @Available(Swift, introduced: 6.2)
/// }
public static var success: Self {
Self(_kind: .success)
}

/// A condition that matches when a process terminates abnormally with any
/// exit code other than `EXIT_SUCCESS` or with any signal.
/// A condition that matches when a process exits abnormally
///
/// This condition matches any exit code other than `EXIT_SUCCESS` or any
/// signal that causes the process to exit.
///
/// @Metadata {
/// @Available(Swift, introduced: 6.2)
/// }
public static var failure: Self {
Self(_kind: .failure)
}

public init(_ statusAtExit: StatusAtExit) {
self.init(_kind: .statusAtExit(statusAtExit))
/// @Metadata {
/// @Available(Swift, introduced: 6.2)
/// }
public init(_ exitStatus: ExitStatus) {
self.init(_kind: .exitStatus(exitStatus))
}

/// Creates a condition that matches when a process terminates with a given
/// exit code.
///
/// - Parameters:
/// - exitCode: The exit code yielded by the process.
/// - exitCode: The exit code reported by the process.
///
/// The C programming language defines two [standard exit codes](https://en.cppreference.com/w/c/program/EXIT_status),
/// `EXIT_SUCCESS` and `EXIT_FAILURE`. Platforms may additionally define their
/// own non-standard exit codes:
/// The C programming language defines two standard exit codes, `EXIT_SUCCESS`
/// and `EXIT_FAILURE`. Platforms may additionally define their own
/// non-standard exit codes:
///
/// | Platform | Header |
/// |-|-|
/// | macOS | [`<stdlib.h>`](https://developer.apple.com/library/archive/documentation/System/Conceptual/ManPages_iPhoneOS/man3/_Exit.3.html), [`<sysexits.h>`](https://developer.apple.com/library/archive/documentation/System/Conceptual/ManPages_iPhoneOS/man3/sysexits.3.html) |
/// | Linux | [`<stdlib.h>`](https://sourceware.org/glibc/manual/latest/html_node/Exit-Status.html), `<sysexits.h>` |
/// | Linux | [`<stdlib.h>`](https://www.kernel.org/doc/man-pages/online/pages/man3/exit.3.html), [`<sysexits.h>`](https://www.kernel.org/doc/man-pages/online/pages/man3/sysexits.h.3head.html) |
/// | FreeBSD | [`<stdlib.h>`](https://man.freebsd.org/cgi/man.cgi?exit(3)), [`<sysexits.h>`](https://man.freebsd.org/cgi/man.cgi?sysexits(3)) |
/// | OpenBSD | [`<stdlib.h>`](https://man.openbsd.org/exit.3), [`<sysexits.h>`](https://man.openbsd.org/sysexits.3) |
/// | Windows | [`<stdlib.h>`](https://learn.microsoft.com/en-us/cpp/c-runtime-library/exit-success-exit-failure) |
///
/// @Comment {
/// See https://en.cppreference.com/w/c/program/EXIT_status for more
/// information about exit codes defined by the C standard.
/// }
///
/// On macOS, FreeBSD, OpenBSD, and Windows, the full exit code reported by
/// the process is yielded to the parent process. Linux and other POSIX-like
/// the process is reported to the parent process. Linux and other POSIX-like
/// systems may only reliably report the low unsigned 8 bits (0&ndash;255) of
/// the exit code.
///
/// @Metadata {
/// @Available(Swift, introduced: 6.2)
/// }
public static func exitCode(_ exitCode: CInt) -> Self {
#if !SWT_NO_EXIT_TESTS
Self(.exitCode(exitCode))
Expand All @@ -92,22 +129,30 @@ extension ExitTest.Condition {
#endif
}

/// Creates a condition that matches when a process terminates with a given
/// signal.
/// Creates a condition that matches when a process exits with a given signal.
///
/// - Parameters:
/// - signal: The signal that terminated the process.
/// - signal: The signal that caused the process to exit.
///
/// The C programming language defines a number of [standard signals](https://en.cppreference.com/w/c/program/SIG_types).
/// Platforms may additionally define their own non-standard signal codes:
/// The C programming language defines a number of standard signals. Platforms
/// may additionally define their own non-standard signal codes:
///
/// | Platform | Header |
/// |-|-|
/// | macOS | [`<signal.h>`](https://developer.apple.com/library/archive/documentation/System/Conceptual/ManPages_iPhoneOS/man3/signal.3.html) |
/// | Linux | [`<signal.h>`](https://sourceware.org/glibc/manual/latest/html_node/Standard-Signals.html) |
/// | Linux | [`<signal.h>`](https://www.kernel.org/doc/man-pages/online/pages/man7/signal.7.html) |
/// | FreeBSD | [`<signal.h>`](https://man.freebsd.org/cgi/man.cgi?signal(3)) |
/// | OpenBSD | [`<signal.h>`](https://man.openbsd.org/signal.3) |
/// | Windows | [`<signal.h>`](https://learn.microsoft.com/en-us/cpp/c-runtime-library/signal-constants) |
///
/// @Comment {
/// See https://en.cppreference.com/w/c/program/SIG_types for more
/// information about signals defined by the C standard.
/// }
///
/// @Metadata {
/// @Available(Swift, introduced: 6.2)
/// }
public static func signal(_ signal: CInt) -> Self {
#if !SWT_NO_EXIT_TESTS
Self(.signal(signal))
Expand All @@ -131,8 +176,8 @@ extension ExitTest.Condition: CustomStringConvertible {
".failure"
case .success:
".success"
case let .statusAtExit(statusAtExit):
String(describing: statusAtExit)
case let .exitStatus(exitStatus):
String(describing: exitStatus)
}
#else
fatalError("Unsupported")
Expand All @@ -149,19 +194,19 @@ extension ExitTest.Condition {
/// Check whether or not an exit test condition matches a given exit status.
///
/// - Parameters:
/// - statusAtExit: An exit status to compare against.
/// - exitStatus: An exit status to compare against.
///
/// - Returns: Whether or not `self` and `statusAtExit` represent the same
/// exit condition.
/// - Returns: Whether or not `self` and `exitStatus` represent the same exit
/// condition.
///
/// Two exit test conditions can be compared; if either instance is equal to
/// ``failure``, it will compare equal to any instance except ``success``.
func isApproximatelyEqual(to statusAtExit: StatusAtExit) -> Bool {
func isApproximatelyEqual(to exitStatus: ExitStatus) -> Bool {
// Strictly speaking, the C standard treats 0 as a successful exit code and
// potentially distinct from EXIT_SUCCESS. To my knowledge, no modern
// operating system defines EXIT_SUCCESS to any value other than 0, so the
// distinction is academic.
return switch (self._kind, statusAtExit) {
return switch (self._kind, exitStatus) {
case let (.success, .exitCode(exitCode)):
exitCode == EXIT_SUCCESS
case let (.failure, .exitCode(exitCode)):
Expand All @@ -170,7 +215,7 @@ extension ExitTest.Condition {
// All terminating signals are considered failures.
true
default:
self._kind == .statusAtExit(statusAtExit)
self._kind == .exitStatus(exitStatus)
}
}
}
Loading