Automate LabKey to Snakemake config

Background

Currently there are two scripts that obtain sample metadata from LabKey (see !8 (merged)) and prepare input files (a sample table and Snakemake config) for the workflow (see !10 (merged)), respectively. Currently, the former script only prints out the LabKey data table (as a pandas data frame) to the screen, requiring the user to manually copy and paste the contents in a correctly formatted input file for the latter script. This is cumbersome and error-prone.

Proposed solution

Generating the required inputs could be automated in one of the following ways:

• the two scripts are merged, such that only one script needs to be executed to prepare a workflow run based on a LabKey data table
• the LabKey API client script is modified in such a way that it writes out the data as a tab-separated file in the form that the Snakemake preparation script expects/understands

To choose between the options, it is important to consider whether any of the individual scripts has valid and likely use cases outside of the common one described above. If not, it is likely better to merge the scripts.

In either case, it is highly desirable that the resulting solution allows the user to choose which of the samples on LabKey he/she would like to process. A test for either the modified LabKey API client script or a test for the merged script (replacing the current one available only for the Snakemake preparation script) should be added and the docs should be updated accordingly (removing any script-specific documentation currently available in scripts/).

Additional context

See notes in merge requests !8 (merged) & !10 (merged), issues #2 (closed), #3 (closed), #25 (closed) & #39 (closed), as well as section "Configuring workflow runs via LabKey tables" in README.md. Resolving this issue should also solve #1 (closed) (if not solved already/before).

@boersch @katsanto: FYI

changed the description

The idea for having both was to cover the cases where there is no Labkey table. If there is, an additional call to the API script is made. That was the communication initially.

Cool, thanks. Could you describe the usage of the script in case that there is no LabKey table? Would the user then need to mock up the TSV output of LabKey table (given that the script converts LabKey field identifiers to what is actually used within Snakemake)? It's a bit unclear to me how the script would help me avoiding having to draw up the Snakemake inputs manually (at least the sample table).

In any case, one could still argue that there is not much point in having separate scripts if not both of them have separate use cases, as one could make LabKey access parameters optional in the script (i.e., if we have LabKey, specify the necessary params/options and get the Snakemake input straight from there, otherwise use a TSV input file).

changed the description

changed milestone to %First usable version

added Doing label

assigned to @katsanto

• Create LabKey-like table locally for synthetic data.
• Test input parser for the local labkey table
• Put this table in labkey and fetch it
• Trigger snakelike

mentioned in merge request !23 (merged)

closed via merge request !23 (merged)

mentioned in commit a654943c

removed Doing label