-
Notifications
You must be signed in to change notification settings - Fork 5
Home
This does synchronous stuff, the commands are described bellow
This does not use pools and it does not do disk size verification.
All operations (download/upload/replicate/delete) are asynchronous and are handled into dedicated pools.
They are all identified by an operationId
to monitor them.
In the case of file Download (not folder), a size check is done by checking the size to download and the disk free space (in PoolOperation::addOperation
' so add the request time, not at the moment of download).
All the commands coming from the two clients are managed and routed in the Executor
class.
In all cases the actual operations are done in the Operations
implementation (Local or Dirac).
OperationBusiness
is an additional layer on top of Operations
, common to all implementations.
The difference between async / sync is the following:
- in sync,
Executor
calls the Command implementation (GetRemoteFile
/UploadFile
/ etc), which calls theOperationBusiness
methods. Of course all is synchronous, there is no database action. - in async,
Executor
calls theAddOperationsCommand
, which does a disk size check and an operation in database. Then different pools threads (PoolDownload
/PoolUpload
/ etc) handle them and useOperationBusiness
.
TODO
All asynchronous commands are managed here to add an entry in database. In all cases (upload, download, etc), a space check is done on disk (in case of error the operation is inserted on db as Failed). In case of downloads, the downloaded file(s) size are taken into account in the disk size check.
- max parallel downloads configured in grida conf file
- basic use of
OperationBusiness::downloadFile
orOperationBusiness::downloadFiles
onOperationBusiness::downloadFolder
- max parallel uploads configured in grida conf file
- upload using same implementation of the
UploadFile
command, not doing any replication yet, but submitting a new replicate operation that will replicate on all SEs configured in the grida conf file
- max parallel replication configured in grida conf file
- replicates file on all SEs configured in the grida conf file
- max parallel deletes configured in grida conf file
- delete using same implementation of the
Delete
command
- checks if dest file already exists on disk
- If absent or ousdated, calls
Operations::downloadFile
- walk through folder and recursively calls itself or
OperationBusiness::downloadFile
- can zip the original folder (the folder is them deleted)
- calls
OperationBusiness::downloadFile
on all files - zip all the files, and then delete them
- Checks CacheBusiness.
- If absent or outdated, calls
OperationBusiness::downloadFile
NO POOLING
NO DISK SIZE CHECK
- calls
OperationBusiness::downloadFolder
NO CACHE
NO POOLING
NO DISK SIZE CHECK
- using dput
- using SE configured in grida conf file (try in order in order, stop when 1 is ok)
- delete local file
- if SEs were given on the command, do a
drepl
on each one - otherwise, add a Replicate operation in a pool (asynchronous)
NO POOLING
- do a
dirac-dms-clean-directory
if it is a directory - do a
dirac-dms-remove-files
if it is a file
NO POOLING