use temp directory instead of /opt for storing temporary downloaded schema files #17
Comments
|
XMLRESOLUTION code was modified to allocate a system temporary directory for storage of its schemas and collections (a collection in the context of xmlresolution - is a set of xml files associated with a package). The temporary directory and its contents, if there are any will be deleted when xmlresoliution is shut down. A schema is deleted from the schema's directory when no collection references the schema. A collection is deleted when core has requested the tar file from xmlresolution via the GET. The amount of storage xmlresolution consumes is greatly reduced. |
ghost
assigned 
|
What is the new location for collections and schemas? /tmp/folders? |
|
The temp directory is created with Dir.mktmpdir Ruby statement. On the mac that seems to be /var/folder/ml and then the rest of te directory is something random each time xmlresolution is recyled like /hltshyv5241d35d3r904351m0000gp/T/d20120726-60100-1sxyq3r/. On darchive it is /var/daitss/tmp. |
|
Per Ira, in FDA production this will be in /var/daitss/tmp |
|
XMLResolution will store its collections, schemas, dtd's, and PI's in a temporary directory that is based upon the temp_directory setting of the config file. The setting will serve as the partial root directory. The entire directory is determined at runtime by Ruby and can be different each time XMLResolution recycles. The entire directory is deleted when XMLResolution is brought down. Example if temp_directory = /var/daitss/tmp then the temp directory will be something like /var/daitss/tmp/d20120828-15859-19ev0td. Schemas will stored in /var/daitss/tmp/d20120828-15859-19ev0td/schemas and collections in /var/daitss/tmp/d20120828-15859-19ev0td/collections. |
|
Issue has been addressed. |
|
Jen has confirmed that this issue is resolved in production. |
|
Per today's DAITSS meeting, the following will be backed out: "The temporary directory and its contents, if there are any will be deleted when xmlresoliution is shut down. A schema is deleted from the schema's directory when no collection references the schema. A collection is deleted when core has requested the tar file from xmlresolution via the GET. The amount of storage xmlresolution consumes is greatly reduced. " That should resolve #30. |
|
Testing this code change on ripple after code update. 2 SIPs were archived and the collections are retained: [lydiam@ripple tmp]$ sudo -u daitss ls -l d20121120-6166-1bzo8pz/collections Stopping daitss (sudo /etc/init.d/daitss stop) removes the parent directory, which also contains schemas. After daitss was restarted and 3 packages were archived, the following collections exist: [lydiam@ripple tmp]$ sudo -u daitss ls -l d20121120-10998-f4h3it/collections Jen will check tomorrow afternoon to confirm that these are gone. |
|
What exact mechanism causes the deletion of collections after 24 hours? |
|
Lydia, When the client request that a new collection be started, xmlresolution On 11/20/12 3:38 PM, Lydia Motyka wrote:
|
|
Per Ira, if there's no DAITSS activity collections won't be deleted. |
|
The following collections are no longer on ripple Neither is the d20121120* directory, which contained the schemas. DAITSS has not been restarted since: 2012 Nov 20 15:40:39 ripple DAITSS: Starting apache services... Why would the top-level directory be deleted? |
|
Apparently xmlresolution went down at 4:02 Nov 25, and this caused the top-level directory to be deleted and subsequently a new one was created this morning when a package was archived: 2012 Nov 25 04:02:34 ripple XmlResolution[11001]: INFO xmlresolution.ripple.fcla.edu: Temporary directory /var/daitss/tmp/d20121120-10998-f4h3it deleted (Note: xmlresolution went down at the same time on darchive. Both of these were due to a weekly 'service httpd reload'.) |
|
Based on Ira's log of the filesystem, the collections were deleted after 24 hours and xmlresolution running against a new file (apparently xmlresolution has to be active for collections to be deleted): [[email protected]]/var/daitss/tmp>date [[email protected]]/var/daitss/tmp>file system after 24 hours and a new put collection E56I60HGF_5KFKTC notice EVITHB8OW_TLPZL8 is not on the file system anymore |
As xml resolution parse and download external links, it stores the downloaded files into its installed directory. Those downloaded files should go to the system temp directory as defined by TMP_DIR environment. It is to be consistent with other DAITSS software and also it's a better practice since those downloaded files are just temporary files.
For background information, see https://prb.fcla.edu/rt3/Ticket/Display.html?id=15929
The text was updated successfully, but these errors were encountered: