Skip to content
GitLab
Explore
Sign in
Primary navigation
Search or go to…
Project
C
cjarchiver
Manage
Activity
Members
Labels
Plan
Issues
Issue boards
Milestones
Wiki
Code
Repository
Branches
Commits
Tags
Repository graph
Compare revisions
Deploy
Releases
Container registry
Model registry
Monitor
Incidents
Analyze
Value stream analytics
Contributor analytics
Repository analytics
Model experiments
Help
Help
Support
GitLab documentation
Compare GitLab plans
Community forum
Contribute to GitLab
Provide feedback
Keyboard shortcuts
?
Snippets
Groups
Projects
Show more breadcrumbs
scicore
cjarchiver
Commits
a8333786
Commit
a8333786
authored
8 years ago
by
Ruben
Browse files
Options
Downloads
Patches
Plain Diff
json file is copied outside of the target folder+somechanges in README and help
parent
0d5e6914
No related branches found
No related tags found
No related merge requests found
Changes
2
Hide whitespace changes
Inline
Side-by-side
Showing
2 changed files
README
+3
-5
3 additions, 5 deletions
README
bin/cjarchiver
+6
-5
6 additions, 5 deletions
bin/cjarchiver
with
9 additions
and
10 deletions
README
+
3
−
5
View file @
a8333786
...
@@ -64,6 +64,7 @@ format <username>_YYYYMMDDThhmmss_<targetfoldername> and the
...
@@ -64,6 +64,7 @@ format <username>_YYYYMMDDThhmmss_<targetfoldername> and the
following extensions:
following extensions:
.log - with the outputs of the script.
.log - with the outputs of the script.
.json - copy of the ARCHIVE_METADATA.json file.
.manifest - with the full list of archived files including
.manifest - with the full list of archived files including
permissions, ownership, size, date, and path.
permissions, ownership, size, date, and path.
.md5sum - with the full list of archived files and their
.md5sum - with the full list of archived files and their
...
@@ -75,14 +76,11 @@ inside of the target directory and, therefore, included in
...
@@ -75,14 +76,11 @@ inside of the target directory and, therefore, included in
the .tar.bz2 file.
the .tar.bz2 file.
After the creation of these files, cjarchiver renames the
After the creation of these files, cjarchiver renames the
target directory as <targetdirectory>.toberemoved/ and
target directory as <targetdirectory>.toberemoved/
automatically moves the .tar.bz2 file to the default archiving
directory: /scicore/archive/<group>/<username>/
As its name indicates, <targetdirectory>.toberemoved/ can be
As its name indicates, <targetdirectory>.toberemoved/ can be
deleted, but prior to that, we strongly recommend to check that
deleted, but prior to that, we strongly recommend to check that
the .tar.bz2 file has been correctly moved to the archiving
the .tar.bz2 file has been created.
directory.
-----------------------------------------------------------
-----------------------------------------------------------
...
...
This diff is collapsed.
Click to expand it.
bin/cjarchiver
+
6
−
5
View file @
a8333786
...
@@ -60,7 +60,7 @@ def checkdirectory(directory,out):
...
@@ -60,7 +60,7 @@ def checkdirectory(directory,out):
logfile
.
flush
()
logfile
.
flush
()
#Checks if metadata exists and has the correct JSON format
#Checks if metadata exists and has the correct JSON format
def
checkmetadata
(
directory
):
def
checkmetadata
(
directory
,
namejson
):
pathfile
=
os
.
environ
[
'
PWD
'
]
+
'
/
'
+
directory
+
'
/ARCHIVE_METADATA.json
'
pathfile
=
os
.
environ
[
'
PWD
'
]
+
'
/
'
+
directory
+
'
/ARCHIVE_METADATA.json
'
exists
=
os
.
path
.
isfile
(
pathfile
)
exists
=
os
.
path
.
isfile
(
pathfile
)
print
"
Searching metadata %s
"
%
(
pathfile
)
print
"
Searching metadata %s
"
%
(
pathfile
)
...
@@ -72,6 +72,9 @@ def checkmetadata(directory):
...
@@ -72,6 +72,9 @@ def checkmetadata(directory):
if
correct
:
if
correct
:
print
"
Metadata format is correct
"
print
"
Metadata format is correct
"
logfile
.
write
(
"
Metadata format is correct
\n
"
)
logfile
.
write
(
"
Metadata format is correct
\n
"
)
os
.
system
(
'
cp
'
+
pathfile
+
'
./
'
+
namejson
)
logfile
.
write
(
"
json file copied
\n
"
)
logfile
.
flush
()
else
:
else
:
print
"
Metadata format is NOT correct. Use -h or --help for an example of JSON format
"
print
"
Metadata format is NOT correct. Use -h or --help for an example of JSON format
"
logfile
.
write
(
"
Metadata format is NOT correct
\n
"
)
logfile
.
write
(
"
Metadata format is NOT correct
\n
"
)
...
@@ -191,9 +194,6 @@ cjarchive.py archives a folder and all its contents in a compressed file
...
@@ -191,9 +194,6 @@ cjarchive.py archives a folder and all its contents in a compressed file
with the name USER_YYYYMMDDHHMMSS_DIRECTORY.tar.bz2. Requires that the
with the name USER_YYYYMMDDHHMMSS_DIRECTORY.tar.bz2. Requires that the
target folder contains a metadata file named ARCHIVE_METADATA.json in
target folder contains a metadata file named ARCHIVE_METADATA.json in
JSON format (see below an example).
JSON format (see below an example).
After the archive has been created it is moved to
/scicore/archive/<group>/<user>/
Developed by Ruben M. Cabezon - sciCORE (University of Basel)
Developed by Ruben M. Cabezon - sciCORE (University of Basel)
ruben.cabezon@unibas.ch
ruben.cabezon@unibas.ch
...
@@ -245,6 +245,7 @@ namelog=name+'.log'
...
@@ -245,6 +245,7 @@ namelog=name+'.log'
namemd5
=
name
+
'
.md5sum
'
namemd5
=
name
+
'
.md5sum
'
nameman
=
name
+
'
.manifest
'
nameman
=
name
+
'
.manifest
'
archivefile
=
name
+
'
.tar.bz2
'
archivefile
=
name
+
'
.tar.bz2
'
namejson
=
name
+
'
.json
'
#*************************************************************
#*************************************************************
#Open logfile
#Open logfile
...
@@ -256,7 +257,7 @@ logfile=open(namelog,'a')
...
@@ -256,7 +257,7 @@ logfile=open(namelog,'a')
#Main code
#Main code
#*************************************************************
#*************************************************************
checkdirectory
(
directory
,
0
)
checkdirectory
(
directory
,
0
)
checkmetadata
(
directory
)
checkmetadata
(
directory
,
namejson
)
if
args
.
exclude
:
if
args
.
exclude
:
for
f
in
excluded
:
for
f
in
excluded
:
checkdirectory
(
directory
+
'
/
'
+
f
,
1
)
checkdirectory
(
directory
+
'
/
'
+
f
,
1
)
...
...
This diff is collapsed.
Click to expand it.
Preview
0%
Loading
Try again
or
attach a new file
.
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Save comment
Cancel
Please
register
or
sign in
to comment