Skip to content

Commit

Permalink
outputdir is a standard parameter on StandardWorkflowComponent as wel…
Browse files Browse the repository at this point in the history
…l as Task
  • Loading branch information
proycon committed Jun 21, 2016
1 parent 8e7046f commit ccddc2a
Show file tree
Hide file tree
Showing 4 changed files with 7 additions and 15 deletions.
15 changes: 6 additions & 9 deletions README.rst
Original file line number Diff line number Diff line change
Expand Up @@ -376,8 +376,9 @@ everything.
#always return the last task(s)
return ucto
------------------------------------
Dynamic dependencies aka Inception
-----------------------------------
------------------------------------

Workflows are static in the sense that based on the format of the input file
and all given parameters, all workflow components and tasks are assembled
Expand Down Expand Up @@ -454,14 +455,10 @@ won't result in parallisation, you must really yield an entire list (or tuple).
Note that we added an ``outputdir`` parameter to the Ucto component which we
hadn't implemented yet. This is necessary to ensure all individual output files
end up in the directory that groups our output. The Ucto component should
simply pass this parameter on to the ``Ucto_txt2tok`` task, and there we
just add it as an optional parameter as follows. The ``outputfrominput()``
method automatically supports this parameter.
.. code-block:: python
class Ucto_txt2tok(Task):
outputdir = Parameter(default="")
simply pass this parameter on to the ``Ucto_txt2tok`` task. The outputdir
parameter is implicitly present on all tasks as well as on
``StandardWorkflowComponent``, the ``outputfrominput()`` method automatically
supports this parameter.
Assuming you have a collecting of text files in a directory ``corpus.txtdir/``,
you can now invoke LuigiNLP as follows and end up with a ``corpus.tokdir/``
Expand Down
1 change: 1 addition & 0 deletions luiginlp/engine.py
Original file line number Diff line number Diff line change
Expand Up @@ -328,6 +328,7 @@ class StandardWorkflowComponent(WorkflowComponent):
"""A workflow component that takes one inputfile"""

inputfile = luigi.Parameter()
outputdir = luigi.Parameter(default="")

class TargetInfo(sciluigi.TargetInfo):
pass
Expand Down
2 changes: 0 additions & 2 deletions luiginlp/modules/ocr.py
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,6 @@ class Tesseract(Task):
executable = 'tesseract'

language = Parameter()
outputdir = Parameter(default="")

in_tiff = None #input slot

Expand All @@ -32,7 +31,6 @@ def run(self):
class OCR_singlepage(StandardWorkflowComponent):
language = Parameter()
tiff_extension=Parameter(default='tif')
outputdir = Parameter(default="")

def autosetup(self):
return Tesseract
Expand Down
4 changes: 0 additions & 4 deletions luiginlp/modules/ucto.py
Original file line number Diff line number Diff line change
Expand Up @@ -49,8 +49,6 @@ class Ucto_folia2folia(Task):
#Parameters for this module (all mandatory!)
language = Parameter()

#Optional parameter
outputdir = Parameter(default="")

in_folia = None #will be linked to an out_* slot of another module in the workflow specification

Expand All @@ -76,7 +74,6 @@ class Ucto(StandardWorkflowComponent):
language = Parameter()
tok_input_sentenceperline = BoolParameter(default=False)
tok_output_sentenceperline = BoolParameter(default=False)
outputdir = Parameter(default="")

def autosetup(self):
return (Ucto_txt2folia, Ucto_folia2folia)
Expand Down Expand Up @@ -136,7 +133,6 @@ class Ucto_dir(StandardWorkflowComponent):
language = Parameter()
tok_input_sentenceperline = BoolParameter(default=False)
tok_output_sentenceperline = BoolParameter(default=False)
outputdir = Parameter(default="")

def autosetup(self):
return (Ucto_txt2folia_dir, Ucto_folia2folia_dir)
Expand Down

0 comments on commit ccddc2a

Please sign in to comment.