Skip to content

Commit

Permalink
Normalize whitespace (#210)
Browse files Browse the repository at this point in the history
* Normalize whitespace

* Add changelog entry

---------

Co-authored-by: John Judd <[email protected]>
  • Loading branch information
NathanBaulch and funkyshu authored Nov 3, 2024
1 parent a6121a9 commit a955ee2
Show file tree
Hide file tree
Showing 55 changed files with 45 additions and 245 deletions.
1 change: 0 additions & 1 deletion .github/ISSUE_TEMPLATE/custom.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,4 +7,3 @@ assignees: ''

---


1 change: 0 additions & 1 deletion .golangci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,6 @@ run:
# If false (default) - golangci-lint acquires file lock on start.
allow-parallel-runners: false


# output configuration options
output:
# colored-line-number|line-number|json|tab|checkstyle|code-climate|junit-xml|github-actions
Expand Down
13 changes: 6 additions & 7 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
### Fixed
- Unit Test improvements: report underlying unit tests errors, always run test cases in a sub-test, always use test suite functions, use more specific assert functions where possible.
- General spelling and grammar corrections everywhere.
- Normalize whitespace: Removed trailing whitespace and superfluous new lines.

## [6.20.0] - 2024-10-15
### Added
Expand Down Expand Up @@ -51,7 +52,6 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
### Fixed
- Fixed #187 - Update to latest jlaffaye/ftp library to fix issue where FTPS connections were failing due to a bug in the library. Also updated dataconn to continue even if it fails to MakeDir.


## [6.14.1] - 2024-05-28
### Fixed
- Fixed #185 - location.Exists was checking if a list entry was a directory but it was only checking the first entry.
Expand Down Expand Up @@ -111,7 +111,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

## [6.11.0] - 2024-01-22
### Added
- Added support for hmac-sha1 and hmac-sha1-96 and removed hmac-ripemd160
- Added support for hmac-sha1 and hmac-sha1-96 and removed hmac-ripemd160

## [6.10.0] - 2023-12-28
### Fixed
Expand Down Expand Up @@ -220,11 +220,11 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

### Upgrade steps

With v6.0.0, sftp.Options struct changed to accept an array of Key Exchange algorithms rather than a string. To update, change the syntax of the auth commands.
With v6.0.0, sftp.Options struct changed to accept an array of Key Exchange algorithms rather than a string. To update, change the syntax of the auth commands.
```
"keyExchanges":"diffie-hellman-group-a256"
```
becomes
becomes
```
"keyExchanges":["diffie-hellman-group-a256"]
```
Expand Down Expand Up @@ -319,7 +319,7 @@ not just GCS.

## [5.3.0] - 2019-08-24
### Added
- ACL can now be passed in as an s3.Option value. See
- ACL can now be passed in as an s3.Option value. See
https://docs.aws.amazon.com/AmazonS3/latest/dev/acl-overview.html#canned-acl for values.

## [5.2.3] - 2019-08-07
Expand Down Expand Up @@ -351,7 +351,6 @@ created in the UI or by creating a zero-length object ending with "/". These ob
- updated README to include "Touch()" definition under the File interface rather than the Location interface
- updated README to exclude "in-memory-backend" from the list of ideas


## [5.1.0] - 2019-07-08
### Added
- Added Touch() method to File interface and implemented in each backend.
Expand Down Expand Up @@ -382,7 +381,7 @@ than a brand new file with new location. Since File may represent a non-existen

1. Added the Retry() method to the FileSystem interface, along with a no-op default retryer.
2. Integrated the newly defined retryer interface into the GCS implementation of VFS and now allow the retry method to be injected as a FileSystem option.
3. Integrated the request.Retryer from the s3 client as an injectable implementation for the S3 vfs.Option.
3. Integrated the request.Retryer from the s3 client as an injectable implementation for the S3 vfs.Option.

### Upgrade steps

Expand Down
16 changes: 5 additions & 11 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,6 @@ Package vfs provides a pluggable, extensible, and opinionated set of file system
functionality for Go across a number of file system types such as os, Azure, S3, GCS
and SFTP.


### Philosophy

When building our platform, initially we wrote a library that was something to
Expand All @@ -30,7 +29,7 @@ the effect of
}
```

Not only was ugly but because the behaviors of each "file system" were
Not only was ugly but because the behaviors of each "file system" were
different and we had to constantly alter the file locations and pass a bucket string (even
if the file system didn't know what a bucket was).

Expand All @@ -57,7 +56,6 @@ file system backends.
* mockable file system
* pluggability so that third-party implementations of our interfaces could be used


### Install

Pre 1.17:
Expand Down Expand Up @@ -137,13 +135,12 @@ will first delegate actual copying in the following:
3. finally, if neither 1 or 2, io.Copy will do its own buffered copy

In case 3, and most implementations of cases 1 and 2, if reader is empty, Write() never gets called. What that means for
vfs is there is no way for us to ensure that an empty file does or doesn't get written on an io.Copy(). For instance
vfs is there is no way for us to ensure that an empty file does or doesn't get written on an io.Copy(). For instance
OS always creates a file, regardless of calling Write() whereas S3 must Write() and Close().

As such, vfs cannot guarantee copy behavior except in our own CopyToFile, MoveToFile, CopyToLocation, and MoveToLocation
functions. If you need to ensure a file gets copied/moved with io.Copy(), you must do so yourself OR use vfs's [utils.TouchCopy](docs/utils.md)


### Third-party Backends

* none so far
Expand All @@ -160,7 +157,7 @@ Feel free to send a pull request if you want to add your backend to the list.
* [in-memory backend](docs/mem.md)
* [sftp backend](docs/sftp.md)
* [ftp backend](docs/ftp.md)
* [azure backend](docs/azure.md)
* [azure backend](docs/azure.md)
* [utils](docs/utils.md)

### Ideas
Expand All @@ -169,7 +166,6 @@ Things to add:
* Provide better List() functionality with more abstracted filtering and paging (iterator?) Return File structs vs URIs?
* Add better/any context.Context() support


### Contributors

Brought to you by the Enterprise Pipeline team at C2FO:
Expand Down Expand Up @@ -198,13 +194,11 @@ https://github.com/c2fo/
4. Push to the branch (`git push origin feature/fooBar`)
5. Create a new Pull Request


### License

Distributed under the MIT license. See `http://github.com/c2fo/vfs/License.md
for more information.


### Definitions

###### absolute path
Expand Down Expand Up @@ -305,7 +299,7 @@ type File interface {
//
// For file:///some/path/to/file.txt, it would return file.txt
Name() string

// Touch creates a zero-length file on the vfs.File if no File exists. Update File's last modified timestamp.
// Returns error if unable to touch File.
Touch() error
Expand Down Expand Up @@ -345,7 +339,7 @@ type FileSystem interface {
NewLocation(volume string, absLocPath string) (Location, error)

// Name returns the name of the FileSystem ie: Amazon S3, os, Google Cloud Storage, etc.
Name() string
Name() string

// Scheme returns the uri scheme used by the FileSystem: s3, file, gs, etc.
Scheme() string
Expand Down
1 change: 0 additions & 1 deletion backend/azure/doc.go
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,6 @@ azure can be augmented with the following implementation-specific methods. Back
would have to be cast as azure.FileSystem to use the following:
func DoSomething() {
...
// cast if fs was created using backend.Backend(). Not necessary if created directly from azure.NewFileSystem().
Expand Down
1 change: 0 additions & 1 deletion backend/ftp/dataconn.go
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,6 @@ func (dc *dataConn) Delete(path string) error {
return singleOpInvalidDataconnType
}
return dc.c.Delete(path)

}

// GetEntry attempts to retrieve the ftp entry for
Expand Down
6 changes: 0 additions & 6 deletions backend/ftp/dataconn_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -39,11 +39,9 @@ func (s *dataConnSuite) SetupTest() {
},
path: filepath,
}

}

func (s *dataConnSuite) TestGetDataConn_alreadyExists() {

// dataconn already exists
s.ftpFile.fileSystem.dataconn = &dataConn{
mode: types.OpenRead,
Expand All @@ -54,7 +52,6 @@ func (s *dataConnSuite) TestGetDataConn_alreadyExists() {
}

func (s *dataConnSuite) TestGetDataConn_openForRead() {

// dataconn is nil - open for read
s.client.EXPECT().
RetrFrom(s.ftpFile.Path(), uint64(0)).
Expand All @@ -77,7 +74,6 @@ func (s *dataConnSuite) TestGetDataConn_errorClientSetup() {
}

func (s *dataConnSuite) TestGetDataConn_ReadError() {

// dataconn is nil - error calling client.RetrFrom
someErr := errors.New("some error")

Expand All @@ -92,7 +88,6 @@ func (s *dataConnSuite) TestGetDataConn_ReadError() {
}

func (s *dataConnSuite) TestGetDataConn_WriteLocationNotExists() {

// dataconn is nil - open for write - location doesn't exist - success
s.client.EXPECT().
List("/").
Expand Down Expand Up @@ -178,7 +173,6 @@ func (s *dataConnSuite) TestGetDataConn_writeSuccess() {
}

func (s *dataConnSuite) TestGetDataConn_readAfterWriteError() {

// open dataconn for read after dataconn for write exists - error on dataconn.Close
fakedconn := NewFakeDataConn(types.OpenWrite)
closeErr := errors.New("some close err")
Expand Down
1 change: 0 additions & 1 deletion backend/ftp/doc.go
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,6 @@ These methods are chainable:
(*FileSystem) WithOptions(opts vfs.Options) *FileSystem
func DoSomething() {
// cast if fs was created using backend.Backend(). Not necessary if created directly from ftp.NewFileSystem().
fs := backend.Backend(ftp.Scheme)
fs = fs.(*ftp.FileSystem)
Expand Down
4 changes: 0 additions & 4 deletions backend/ftp/file.go
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,6 @@ type File struct {

// LastModified returns the LastModified property of ftp file.
func (f *File) LastModified() (*time.Time, error) {

entry, err := f.stat(context.TODO())
if err != nil {
return nil, err
Expand Down Expand Up @@ -151,7 +150,6 @@ func getTempFilename(origName string) string {

// Size returns the size of the remote file.
func (f *File) Size() (uint64, error) {

entry, err := f.stat(context.TODO())
if err != nil {
return 0, err
Expand Down Expand Up @@ -181,7 +179,6 @@ func (f *File) MoveToFile(t vfs.File) error {
if f.fileSystem.Scheme() == t.Location().FileSystem().Scheme() &&
f.authority.UserInfo().Username() == t.(*File).authority.UserInfo().Username() &&
f.authority.HostPortStr() == t.(*File).authority.HostPortStr() {

// ensure destination exists before moving
exists, err := t.Location().Exists()
if err != nil {
Expand Down Expand Up @@ -287,7 +284,6 @@ func (f *File) CopyToFile(file vfs.File) (err error) { //nolint:gocyclo

return err
}

}

// CopyToLocation creates a copy of *File, using the file's current path as the new file's
Expand Down
1 change: 0 additions & 1 deletion backend/ftp/fileSystem.go
Original file line number Diff line number Diff line change
Expand Up @@ -120,7 +120,6 @@ func (fs *FileSystem) Client(ctx context.Context, authority utils.Authority) (ty

// WithOptions sets options for client and returns the filesystem (chainable)
func (fs *FileSystem) WithOptions(opts vfs.Options) *FileSystem {

// only set options if vfs.Options is ftp.Options
if opts, ok := opts.(Options); ok {
fs.options = opts
Expand Down
12 changes: 0 additions & 12 deletions backend/ftp/file_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,6 @@ func clientGetterReturnsError(_ context.Context, _ utils.Authority, _ Options) (
}

func (ts *fileTestSuite) TestRead() {

// set up ftpfile
fp := "/some/path.txt"
client := mocks.NewClient(ts.T())
Expand Down Expand Up @@ -98,7 +97,6 @@ func (ts *fileTestSuite) TestRead() {
}

func (ts *fileTestSuite) TestClose() {

// set up ftpfile
fp := "/some/path.txt"
client := mocks.NewClient(ts.T())
Expand Down Expand Up @@ -503,7 +501,6 @@ func (ts *fileTestSuite) TestCopyToFile() {
err = sourceFile.CopyToFile(targetFile)
ts.Error(err, "error is expected")
ts.ErrorIs(err, closeErr, "error is expected kind of error")

}

func (ts *fileTestSuite) TestCopyToLocation() {
Expand Down Expand Up @@ -1016,7 +1013,6 @@ func (ts *fileTestSuite) TestDelete() {
func (ts *fileTestSuite) TestLastModified() {
now := time.Now()
entry := &_ftp.Entry{

Name: ts.testFile.Name(),
Type: _ftp.EntryTypeFile,
Time: now,
Expand Down Expand Up @@ -1081,7 +1077,6 @@ func (ts *fileTestSuite) TestName() {
func (ts *fileTestSuite) TestSize() {
contentLength := uint64(100)
entry := &_ftp.Entry{

Name: ts.testFile.Name(),
Type: _ftp.EntryTypeFile,
Size: contentLength,
Expand Down Expand Up @@ -1193,35 +1188,28 @@ func (f *FakeDataConn) List(p string) ([]*_ftp.Entry, error) {
Type: _ftp.EntryTypeFolder,
},
}, f.singleOpErr

}
return nil, errors.New("550")

}

func (f *FakeDataConn) MakeDir(p string) error {
return f.singleOpErr

}

func (f *FakeDataConn) Rename(from, to string) error {
return f.singleOpErr

}

func (f *FakeDataConn) IsSetTimeSupported() bool {
return false

}

func (f *FakeDataConn) SetTime(p string, t time.Time) error {
return f.singleOpErr

}

func (f *FakeDataConn) IsTimePreciseInList() bool {
return f.mlst

}

func (f *FakeDataConn) Read(p []byte) (int, error) {
Expand Down
2 changes: 0 additions & 2 deletions backend/ftp/location_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -361,7 +361,6 @@ func (lt *locationTestSuite) TestVolume() {
}

func (lt *locationTestSuite) TestPath() {

loc, err := lt.ftpfs.NewLocation("host.com", "/path/")
lt.NoError(err)
lt.Equal("/path/", loc.Path(), "Path() should return the path on location.")
Expand Down Expand Up @@ -481,7 +480,6 @@ func (lt *locationTestSuite) TestExists() {
}

func (lt *locationTestSuite) TestChangeDir() {

loc := &Location{fileSystem: lt.ftpfs, path: "/", Authority: utils.Authority{}}

err1 := loc.ChangeDir("../")
Expand Down
1 change: 0 additions & 1 deletion backend/gs/doc.go
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,6 @@ gs can be augmented with the following implementation-specific methods. Backend
would have to be cast as gs.FileSystem to use the following:
func DoSomething() {
...
// cast if fs was created using backend.Backend(). Not necessary if created directly from gs.NewFileSystem().
Expand Down
Loading

0 comments on commit a955ee2

Please sign in to comment.