The main interface provided by sevenbridges package is Tool function, it’s basically a R interface similar to Seven Bridges’s graphic user interface to describe tools, which I also highly recommend because it’s very easy.
For R users who want to script in everything here is alternative solution. I also did some work trying to make it simpler to use, any suggestions are welcomed.
So I highly recommend user go over documentation The Tool Editor chapter for cancer genomic cloud to understand how it works, and even try it on the platform with the GUI. This will help use our R interface better and easier.
Or you just want to create tools quickly, please keep reading.
Let’s start from the most simple case
RscriptSome basic arguments used in Tool function.
For more accepted parameter, please check help(CommandLineTool) and help(Tool) for more details. Again we basically conform to CWL standard, you can always check their manual.
In short, hints are not required for execution. We now accept following requirement items cpu, mem, docker, fileDef; and you can easily construct them via requirements() constructor.
The downside of this is clear, you have much less control over your script and command line interface.
Most likely your command line interface accept extra arguments, for example,
So to specify that in your tool, you can use input function, then pass it to the inputs arguments as a list or single item. You can even construct them as data.frame, but in that way, you usually have to provide equal arguments, less typing but less flexible.
input() require arguments id and type. output() require arguments id because type by default is file.
The type could be an array of single type, the most common case is that if your input is a list of files, you can do something like type = ItemArray("File") or as simple as type = "File..." to diffenciate from a single file input.
We also provide an enum type, when you specify the enum, please pass the required name and symbols like this type = enum("format", c("pdf", "html")) then in the interface you will be poped with drop down when you execute the task.
For complete example, please check the “Make the best of bioconductor workflow” tutorial, or if you are advanced user, you can simply check inst/docker folder or check the github example here
If you already have a docker image in mind that provide the functionality you need, you can just use it. The baseCommand is the command line you want to execute in that container. stdout specify the output file you want to capture the standard output and collect it on the platform.
library(sevenbridges)##
## Attaching package: 'sevenbridges'## The following object is masked from 'package:BiocStyle':
##
## outputrbx <- Tool(id = "runif",
label = "runif",
hints = requirements(docker(pull = "rocker/r-base"),
cpu(1), mem(2000)),
baseCommand = "Rscript -e 'runif(100)'",
stdout = "output.txt",
outputs = output(id = "random", glob = "*.txt"))
rbx## sbg:id: runif
## id: '#runif'
## inputs: []
## outputs:
## - type:
## - 'null'
## - File
## label: ''
## description: ''
## streamable: no
## default: ''
## id: '#random'
## outputBinding:
## glob: '*.txt'
## requirements: []
## hints:
## - class: DockerRequirement
## dockerPull: rocker/r-base
## dockerLoad: ''
## dockerFile: ''
## dockerImageId: ''
## dockerOutputDirectory: ''
## - class: sbg:CPURequirement
## value: 1
## - class: sbg:MemRequirement
## value: 2000
## label: runif
## class: CommandLineTool
## baseCommand:
## - Rscript -e 'runif(100)'
## arguments: []
## stdout: output.txtrbx$toJSON()## {"sbg:id":"runif","id":"#runif","inputs":[],"outputs":[{"type":["null","File"],"label":"","description":"","streamable":false,"default":"","id":"#random","outputBinding":{"glob":"*.txt"}}],"requirements":[],"hints":[{"class":"DockerRequirement","dockerPull":"rocker/r-base","dockerLoad":"","dockerFile":"","dockerImageId":"","dockerOutputDirectory":""},{"class":"sbg:CPURequirement","value":1},{"class":"sbg:MemRequirement","value":2000}],"label":"runif","class":"CommandLineTool","baseCommand":["Rscript -e 'runif(100)'"],"arguments":[],"stdout":"output.txt"}By default the tool object shows YAML, but you can simply convert it to JSON and copy it to your seven bridges platform graphic editor by importing JSON.
rbx$toJSON()## {"sbg:id":"runif","id":"#runif","inputs":[],"outputs":[{"type":["null","File"],"label":"","description":"","streamable":false,"default":"","id":"#random","outputBinding":{"glob":"*.txt"}}],"requirements":[],"hints":[{"class":"DockerRequirement","dockerPull":"rocker/r-base","dockerLoad":"","dockerFile":"","dockerImageId":"","dockerOutputDirectory":""},{"class":"sbg:CPURequirement","value":1},{"class":"sbg:MemRequirement","value":2000}],"label":"runif","class":"CommandLineTool","baseCommand":["Rscript -e 'runif(100)'"],"arguments":[],"stdout":"output.txt"}rbx$toJSON(pretty = TRUE)## {
## "sbg:id": "runif",
## "id": "#runif",
## "inputs": [],
## "outputs": [
## {
## "type": ["null", "File"],
## "label": "",
## "description": "",
## "streamable": false,
## "default": "",
## "id": "#random",
## "outputBinding": {
## "glob": "*.txt"
## }
## }
## ],
## "requirements": [],
## "hints": [
## {
## "class": "DockerRequirement",
## "dockerPull": "rocker/r-base",
## "dockerLoad": "",
## "dockerFile": "",
## "dockerImageId": "",
## "dockerOutputDirectory": ""
## },
## {
## "class": "sbg:CPURequirement",
## "value": 1
## },
## {
## "class": "sbg:MemRequirement",
## "value": 2000
## }
## ],
## "label": "runif",
## "class": "CommandLineTool",
## "baseCommand": [
## "Rscript -e 'runif(100)'"
## ],
## "arguments": [],
## "stdout": "output.txt"
## }rbx$toYAML()## [1] "sbg:id: runif\nid: '#runif'\ninputs: []\noutputs:\n- type:\n - 'null'\n - File\n label: ''\n description: ''\n streamable: no\n default: ''\n id: '#random'\n outputBinding:\n glob: '*.txt'\nrequirements: []\nhints:\n- class: DockerRequirement\n dockerPull: rocker/r-base\n dockerLoad: ''\n dockerFile: ''\n dockerImageId: ''\n dockerOutputDirectory: ''\n- class: sbg:CPURequirement\n value: 1\n- class: sbg:MemRequirement\n value: 2000\nlabel: runif\nclass: CommandLineTool\nbaseCommand:\n- Rscript -e 'runif(100)'\narguments: []\nstdout: output.txt\n"If you want to create simple script based on existing image, use fileDef.
## Make a new file
fd <- fileDef(name = "runif.R",
content = "set.seed(1)
runif(100)")
## or simply readLines
.srcfile <- system.file("docker/sevenbridges/src/runif.R", package = "sevenbridges")
fd <- fileDef(name = "runif.R",
content = paste(readLines(.srcfile), collapse = "\n"))
## or read via reader
library(readr)
fd <- fileDef(name = "runif.R",
content = read_file(.srcfile))
rbx <- Tool(id = "runif",
label = "runif",
hints = requirements(docker(pull = "rocker/r-base"),
cpu(1), mem(2000)),
requirements = requirements(fd),
baseCommand = "Rscript runif.R",
stdout = "output.txt",
outputs = output(id = "random", glob = "*.txt")) How bout multiple script?
## or simply readLines
.srcfile <- system.file("docker/sevenbridges/src/runif.R", package = "sevenbridges")
fd1 <- fileDef(name = "runif.R",
content = paste(readLines(.srcfile), collapse = "\n"))
fd2 <- fileDef(name = "runif2.R",
content = "set.seed(1)
runif(100)")
rbx <- Tool(id = "runif_twoscript",
label = "runif_twoscript",
hints = requirements(docker(pull = "rocker/r-base"),
cpu(1), mem(2000)),
requirements = requirements(fd1, fd2),
baseCommand = "Rscript runif.R",
stdout = "output.txt",
outputs = output(id = "random", glob = "*.txt")) ## pass a input list
in.lst <- list(input(id = "number",
description = "number of observations",
type = "integer",
label = "number",
prefix = "--n",
default = 1,
required = TRUE,
cmdInclude = TRUE),
input(id = "min",
description = "lower limits of the distribution",
type = "float",
label = "min",
prefix = "--min",
default = 0),
input(id = "max",
description = "upper limits of the distribution",
type = "float",
label = "max",
prefix = "--max",
default = 1),
input(id = "seed",
description = "seed with set.seed",
type = "float",
label = "seed",
prefix = "--seed",
default = 1))
## the same method for outputs
out.lst <- list(output(id = "random",
type = "file",
label = "output",
description = "random number file",
glob = "*.txt"),
output(id = "report",
type = "file",
label = "report",
glob = "*.html"))
rbx <- Tool(id = "runif",
label = "Random number generator",
hints = requirements(docker(pull = "tengfei/runif"),
cpu(1), mem(2000)),
baseCommand = "runif.R",
inputs = in.lst, ## or ins.df
outputs = out.lst)Here I use data.frame as example for input and output.
in.df <- data.frame(id = c("number", "min", "max", "seed"),
description = c("number of observation",
"lower limits of the distribution",
"upper limits of the distribution",
"seed with set.seed"),
type = c("integer", "float", "float", "float"),
label = c("number" ,"min", "max", "seed"),
prefix = c("--n", "--min", "--max", "--seed"),
default = c(1, 0, 10, 123),
required = c(TRUE, FALSE, FALSE, FALSE))
out.df <- data.frame(id = c("random", "report"),
type = c("file", "file"),
glob = c("*.txt", "*.html"))
rbx <- Tool(id = "runif",
label = "Random number generator",
hints = requirements(docker(pull = "tengfei/runif"),
cpu(1), mem(2000)),
baseCommand = "runif.R",
inputs = in.df, ## or ins.df
outputs = out.df)For advanced users, please read another tutorial “Creating Your Docker Container and Command Line Interface (with docopt)”, “docopt” is more formal way to construct your command line interface, but there is a quick way to make command line interface here using just commandArgs
Suppose I already have a R script like this using position mapping the arguments
fl <- system.file("docker/sevenbridges/src", "runif2spin.R", package = "sevenbridges")
cat(readLines(fl), sep = '\n')#'---
#'title: "Uniform randome number generator example"
#'output:
#' html_document:
#' toc: true
#'number_sections: true
#'highlight: haddock
#'---
#'## summary report
#'
#'This is a randome number generator
#+
args <- commandArgs(TRUE)
r <- runif(n = as.integer(args[1]),
min = as.numeric(args[2]),
max = as.numeric(args[3]))
head(r)
summary(r)
hist(r)Ignore the comment part, I will introduce spin/stich later. My base command will be somethine like
Rscript runif2spin.R 10 30 50I just describe my tool in this way
library(readr)
fd <- fileDef(name = "runif.R",
content = read_file(fl))
rbx <- Tool(id = "runif",
label = "runif",
hints = requirements(docker(pull = "rocker/r-base"),
cpu(1), mem(2000)),
requirements = requirements(fd),
baseCommand = "Rscript runif.R",
stdout = "output.txt",
inputs = list(input(id = "number",
type = "integer",
position = 1),
input(id = "min",
type = "float",
position = 2),
input(id = "max",
type = "float",
position = 3)),
outputs = output(id = "random", glob = "output.txt")) Now copy-paste the json into your project app and run it in the cloud to test it
How about named argumentments? I will still recommend use “docopt” package, but for simple way.
fl <- system.file("docker/sevenbridges/src", "runif_args.R", package = "sevenbridges")
cat(readLines(fl), sep = '\n')Warning in readLines(fl): incomplete final line found on '/tmp/RtmptCGgRL/
Rinst3bc539754275/sevenbridges/docker/sevenbridges/src/runif_args.R'#'---
#'title: "Uniform randome number generator example"
#'output:
#' html_document:
#' toc: true
#'number_sections: true
#'highlight: haddock
#'---
#'## summary report
#'
#'This is a randome number generator
#+
args <- commandArgs(TRUE)
## quick hack to split named arguments
splitArgs <- function(x){
res <- do.call(rbind, lapply(x, function(i){
res <- strsplit(i, "=")[[1]]
nm <- gsub("-+", "",res[1])
c(nm, res[2])
}))
.r <- res[,2]
names(.r) <- res[,1]
.r
}
args <- splitArgs(args)
#+
r <- runif(n = as.integer(args["n"]),
min = as.numeric(args["min"]),
max = as.numeric(args["max"]))
summary(r)
hist(r)
write.csv(r, file = "out.csv")Rscript runif_args.R --n=10 --min=30 --max=50I just describe my tool in this way, note, I use separate=FALSE and add = to my prefix as a hack.
library(readr)
fd <- fileDef(name = "runif.R",
content = read_file(fl))
rbx <- Tool(id = "runif",
label = "runif",
hints = requirements(docker(pull = "rocker/r-base"),
cpu(1), mem(2000)),
requirements = requirements(fd),
baseCommand = "Rscript runif.R",
stdout = "output.txt",
inputs = list(input(id = "number",
type = "integer",
separate = FALSE,
prefix = "--n="),
input(id = "min",
type = "float",
separate = FALSE,
prefix = "--min="),
input(id = "max",
type = "float",
separate = FALSE,
prefix = "--max=")),
outputs = output(id = "random", glob = "output.txt")) Alternative, you can use spin/stich from knitr to generate report directly from a Rscript with special format. For example, let’s use above example
fl <- system.file("docker/sevenbridges/src", "runif_args.R", package = "sevenbridges")
cat(readLines(fl), sep = '\n')Warning in readLines(fl): incomplete final line found on '/tmp/RtmptCGgRL/
Rinst3bc539754275/sevenbridges/docker/sevenbridges/src/runif_args.R'#'---
#'title: "Uniform randome number generator example"
#'output:
#' html_document:
#' toc: true
#'number_sections: true
#'highlight: haddock
#'---
#'## summary report
#'
#'This is a randome number generator
#+
args <- commandArgs(TRUE)
## quick hack to split named arguments
splitArgs <- function(x){
res <- do.call(rbind, lapply(x, function(i){
res <- strsplit(i, "=")[[1]]
nm <- gsub("-+", "",res[1])
c(nm, res[2])
}))
.r <- res[,2]
names(.r) <- res[,1]
.r
}
args <- splitArgs(args)
#+
r <- runif(n = as.integer(args["n"]),
min = as.numeric(args["min"]),
max = as.numeric(args["max"]))
summary(r)
hist(r)
write.csv(r, file = "out.csv")You command is something like this
Rscript -e "rmarkdown::render(knitr::spin('runif_args.R', FALSE))" --args --n=100 --min=30 --max=50And so I describe my tool like this with docker image rocker/hadleyverse this contians knitr and rmarkdown package.
library(readr)
fd <- fileDef(name = "runif.R",
content = read_file(fl))
rbx <- Tool(id = "runif",
label = "runif",
hints = requirements(docker(pull = "rocker/hadleyverse"),
cpu(1), mem(2000)),
requirements = requirements(fd),
baseCommand = "Rscript -e \"rmarkdown::render(knitr::spin('runif.R', FALSE))\" --args",
stdout = "output.txt",
inputs = list(input(id = "number",
type = "integer",
separate = FALSE,
prefix = "--n="),
input(id = "min",
type = "float",
separate = FALSE,
prefix = "--min="),
input(id = "max",
type = "float",
separate = FALSE,
prefix = "--max=")),
outputs = list(output(id = "stdout", type = "file", glob = "output.txt"),
output(id = "random", type = "file", glob = "*.csv"),
output(id = "report", type = "file", glob = "*.html")))You will get a report in the end
Sometimes if you want your output files inherit from particular input file, just use inheritMetadataFrom in your output() call and pass the input file id. If you want to add additional metadata, you could pass metadata a list in your output() function call. For example, I want my output report inherit all metadata from my “bam_file” input node (which I don’t have in this example though) with two additional metadata fields.
out.lst <- list(output(id = "random",
type = "file",
label = "output",
description = "random number file",
glob = "*.txt"),
output(id = "report",
type = "file",
label = "report",
glob = "*.html",
inheritMetadataFrom = "bam_file",
metadata = list(author = "tengfei",
sample = "random")))
out.lst## [[1]]
## type:
## - 'null'
## - File
## label: output
## description: random number file
## streamable: no
## default: ''
## id: '#random'
## outputBinding:
## glob: '*.txt'
##
##
## [[2]]
## type:
## - 'null'
## - File
## label: report
## description: ''
## streamable: no
## default: ''
## id: '#report'
## outputBinding:
## glob: '*.html'
## sbg:inheritMetadataFrom: '#bam_file'
## sbg:metadata:
## author: tengfei
## sample: randomWith API function, you can directly load your Tool into the account. Run a task, for “how-to”, please check the API complete guide
Following section, please for now skip.
a <- Auth(platform = "cgc", username = "tengfei")
p <- a$project("demo")
app.runif <- p$app_add("runif555", rbx)
aid <- app.runif$id
p$task_add(name = "Draft runif simple",
description = "Description for runif",
app = aid,
inputs = list(min = 1, max = 10))
## confirm, show all task status is draft
(tsk <- p$task(status = "draft"))
tsk$run()
tsk$download("~/Downloads/")1. from CLI
While developing tools it is useful to test them locally first. For that we can use rabix - reproducible analyses for bioinformatics, https://github.com/rabix. To test your tool with latest implementation of rabix in Java (called bunny) you could use docker image tengfei/testenv:
docker pull tengfei/testenvDump your rabix tool as json into dir which also contains input data. write(rbx$toJSON, file="<data_dir>/<tool>.json"). Make inputs.json file to declare input parameters in the same directory (you can use relative paths from inputs.json to data). Create container:
docker run --privileged --name bunny -v </path/to/data_dir>:/bunny_data -dit tengfei/testenvExecute tool
docker exec bunny bash -c 'cd /opt/bunny && ./rabix.sh -e /bunny_data /bunny_data/<tool>.json /bunny_data/inputs.json'You’ll see running logs from within container, and also output dir inside
NOTE: tengfei/testenv has R, python, Java… so many tools can work without docker requirement set. If you however set docker requirement you need to pull image inside container first to run docker container inside running bunny docker.
NOTE: inputs.json can also be inputs.yaml if you find it easier to declare inputs in YAML.
2. from R
library(sevenbridges)
in.df <- data.frame(id = c("number", "min", "max", "seed"),
description = c("number of observation",
"lower limits of the distribution",
"upper limits of the distribution",
"seed with set.seed"),
type = c("integer", "float", "float", "float"),
label = c("number" ,"min", "max", "seed"),
prefix = c("--n", "--min", "--max", "--seed"),
default = c(1, 0, 10, 123),
required = c(TRUE, FALSE, FALSE, FALSE))
out.df <- data.frame(id = c("random", "report"),
type = c("file", "file"),
glob = c("*.txt", "*.html"))
rbx <- Tool(id = "runif",
label = "Random number generator",
hints = requirements(docker(pull = "tengfei/runif"),
cpu(1), mem(2000)),
baseCommand = "runif.R",
inputs = in.df, ## or ins.df
outputs = out.df)
params <- list(number=3, max=5)
set_test_env("tengfei/testenv", "mount_dir")
test_tool(rbx, params)Graphic User Interface on Seven Bridges Platform is way more conventient
To create a workflow, we provide simple interface to pipe your tool into a single workflow, it works under situation like
Note for complicated workflow construction, I highly recommend using our graphical interface to do it, there is no better way.
Here I will give a quick example
Here are methods we support
Flow constructor to construct your flow, then pass steps using + sign to connect your object. + only outputs StepList and support operation
%>>% will output Flow not StepList, if you connect tools with %>>% we will create id for you. it always output Workflow
library(sevenbridges)
## A tool that generate a 100 random number
t1 <- Tool(id = "runif new test 3", label = "random number",
hints = requirements(docker(pull = "rocker/r-base")),
baseCommand = "Rscript -e 'x = runif(100); write.csv(x, file = 'random.txt', row.names = FALSE)'",
outputs = output(id = "random",
type = "file",
glob = "random.txt"))
## A tool that take log
fd <- fileDef(name = "log.R",
content = "args = commandArgs(TRUE)
x = read.table(args[1], header = TRUE)[,'x']
x = log(x)
write.csv(x, file = 'random_log.txt', row.names = FALSE)
")
t2 <- Tool(id = "log new test 3", label = "get log",
hints = requirements(docker(pull = "rocker/r-base")),
requirements = requirements(fd),
baseCommand = "Rscript log.R",
inputs = input(id = "number",
type = "file"),
outputs = output(id = "log",
type = "file",
glob = "*.txt"))
## A tool that do a mean
fd <- fileDef(name = "mean.R",
content = "args = commandArgs(TRUE)
x = read.table(args[1], header = TRUE)[,'x']
x = mean(x)
write.csv(x, file = 'random_mean.txt', row.names = FALSE)
")
t3 <- Tool(id = "mean new test 3", label = "get mean",
hints = requirements(docker(pull = "rocker/r-base")),
requirements = requirements(fd),
baseCommand = "Rscript mean.R",
inputs = input(id = "number",
type = "file"),
outputs = output(id = "mean",
type = "file",
glob = "*.txt"))
steplist <- t1 + t2 + t3
steplist## [[1]]## id: '#random_number'
## inputs: []
## outputs:
## - id: '#random_number.random'
## hints: []
## run:
## sbg:id: runif new test 3
## id: '#runif_new_test_3'
## inputs: []
## outputs:
## - type:
## - 'null'
## - File
## label: ''
## description: ''
## streamable: no
## default: ''
## id: '#random'
## outputBinding:
## glob: random.txt
## requirements: []
## hints:
## - class: DockerRequirement
## dockerPull: rocker/r-base
## dockerLoad: ''
## dockerFile: ''
## dockerImageId: ''
## dockerOutputDirectory: ''
## label: random number
## class: CommandLineTool
## baseCommand:
## - Rscript -e 'x = runif(100); write.csv(x, file = 'random.txt', row.names = FALSE)'
## arguments: []## [[2]]## id: '#get_log'
## inputs:
## - id: '#get_log.number'
## source: '#random_number.random'
## outputs:
## - id: '#get_log.log'
## hints: []
## run:
## sbg:id: log new test 3
## id: '#log_new_test_3'
## inputs:
## - type:
## - 'null'
## - File
## label: ''
## description: ''
## streamable: no
## default: ''
## id: '#number'
## required: no
## outputs:
## - type:
## - 'null'
## - File
## label: ''
## description: ''
## streamable: no
## default: ''
## id: '#log'
## outputBinding:
## glob: '*.txt'
## requirements:
## - class: CreateFileRequirement
## fileDef:
## - filename: log.R
## fileContent: "args = commandArgs(TRUE)\n x = read.table(args[1],
## header = TRUE)[,'x']\n x = log(x)\n write.csv(x,
## file = 'random_log.txt', row.names = FALSE)\n "
## hints:
## - class: DockerRequirement
## dockerPull: rocker/r-base
## dockerLoad: ''
## dockerFile: ''
## dockerImageId: ''
## dockerOutputDirectory: ''
## label: get log
## class: CommandLineTool
## baseCommand:
## - Rscript log.R
## arguments: []## [[3]]## id: '#get_mean'
## inputs:
## - id: '#get_mean.number'
## source: '#get_log.log'
## outputs:
## - id: '#get_mean.mean'
## hints: []
## run:
## sbg:id: mean new test 3
## id: '#mean_new_test_3'
## inputs:
## - type:
## - 'null'
## - File
## label: ''
## description: ''
## streamable: no
## default: ''
## id: '#number'
## required: no
## outputs:
## - type:
## - 'null'
## - File
## label: ''
## description: ''
## streamable: no
## default: ''
## id: '#mean'
## outputBinding:
## glob: '*.txt'
## requirements:
## - class: CreateFileRequirement
## fileDef:
## - filename: mean.R
## fileContent: "args = commandArgs(TRUE)\n x = read.table(args[1],
## header = TRUE)[,'x']\n x = mean(x)\n write.csv(x,
## file = 'random_mean.txt', row.names = FALSE)\n "
## hints:
## - class: DockerRequirement
## dockerPull: rocker/r-base
## dockerLoad: ''
## dockerFile: ''
## dockerImageId: ''
## dockerOutputDirectory: ''
## label: get mean
## class: CommandLineTool
## baseCommand:
## - Rscript mean.R
## arguments: []To create a Flow we suggest you using Flow function, so that you can pass id and label to it.
f <- Flow(id = "Random-log-mean-new-test-2",
label = "random log mean new test",
steps = steplist)
f$toJSON()## {"id":"#Random-log-mean-new-test-2","inputs":[],"outputs":[],"requirements":[],"hints":[],"label":"random log mean new test","class":"Workflow","steps":[{"id":"#random_number","inputs":[],"outputs":[{"id":"#random_number.random"}],"hints":[],"run":{"sbg:id":"runif new test 3","id":"#runif_new_test_3","inputs":[],"outputs":[{"type":["null","File"],"label":"","description":"","streamable":false,"default":"","id":"#random","outputBinding":{"glob":"random.txt"}}],"requirements":[],"hints":[{"class":"DockerRequirement","dockerPull":"rocker/r-base","dockerLoad":"","dockerFile":"","dockerImageId":"","dockerOutputDirectory":""}],"label":"random number","class":"CommandLineTool","baseCommand":["Rscript -e 'x = runif(100); write.csv(x, file = 'random.txt', row.names = FALSE)'"],"arguments":[]},"sbg:x":100,"sbg:y":200},{"id":"#get_log","inputs":[{"id":"#get_log.number","source":["#random_number.random"]}],"outputs":[{"id":"#get_log.log"}],"hints":[],"run":{"sbg:id":"log new test 3","id":"#log_new_test_3","inputs":[{"type":["null","File"],"label":"","description":"","streamable":false,"default":"","id":"#number","required":false}],"outputs":[{"type":["null","File"],"label":"","description":"","streamable":false,"default":"","id":"#log","outputBinding":{"glob":"*.txt"}}],"requirements":[{"class":"CreateFileRequirement","fileDef":[{"filename":"log.R","fileContent":"args = commandArgs(TRUE)\n x = read.table(args[1], header = TRUE)[,'x']\n x = log(x)\n write.csv(x, file = 'random_log.txt', row.names = FALSE)\n "}]}],"hints":[{"class":"DockerRequirement","dockerPull":"rocker/r-base","dockerLoad":"","dockerFile":"","dockerImageId":"","dockerOutputDirectory":""}],"label":"get log","class":"CommandLineTool","baseCommand":["Rscript log.R"],"arguments":[]},"sbg:x":350,"sbg:y":200},{"id":"#get_mean","inputs":[{"id":"#get_mean.number","source":["#get_log.log"]}],"outputs":[{"id":"#get_mean.mean"}],"hints":[],"run":{"sbg:id":"mean new test 3","id":"#mean_new_test_3","inputs":[{"type":["null","File"],"label":"","description":"","streamable":false,"default":"","id":"#number","required":false}],"outputs":[{"type":["null","File"],"label":"","description":"","streamable":false,"default":"","id":"#mean","outputBinding":{"glob":"*.txt"}}],"requirements":[{"class":"CreateFileRequirement","fileDef":[{"filename":"mean.R","fileContent":"args = commandArgs(TRUE)\n x = read.table(args[1], header = TRUE)[,'x']\n x = mean(x)\n write.csv(x, file = 'random_mean.txt', row.names = FALSE)\n "}]}],"hints":[{"class":"DockerRequirement","dockerPull":"rocker/r-base","dockerLoad":"","dockerFile":"","dockerImageId":"","dockerOutputDirectory":""}],"label":"get mean","class":"CommandLineTool","baseCommand":["Rscript mean.R"],"arguments":[]},"sbg:x":600,"sbg:y":200}],"sbg:canvas_zoom":1,"sbg:canvas_y":130,"sbg:canvas_x":40}f$toJSON(pretty = TRUE)## {
## "id": "#Random-log-mean-new-test-2",
## "inputs": [],
## "outputs": [],
## "requirements": [],
## "hints": [],
## "label": "random log mean new test",
## "class": "Workflow",
## "steps": [
## {
## "id": "#random_number",
## "inputs": [],
## "outputs": [
## {
## "id": "#random_number.random"
## }
## ],
## "hints": [],
## "run": {
## "sbg:id": "runif new test 3",
## "id": "#runif_new_test_3",
## "inputs": [],
## "outputs": [
## {
## "type": ["null", "File"],
## "label": "",
## "description": "",
## "streamable": false,
## "default": "",
## "id": "#random",
## "outputBinding": {
## "glob": "random.txt"
## }
## }
## ],
## "requirements": [],
## "hints": [
## {
## "class": "DockerRequirement",
## "dockerPull": "rocker/r-base",
## "dockerLoad": "",
## "dockerFile": "",
## "dockerImageId": "",
## "dockerOutputDirectory": ""
## }
## ],
## "label": "random number",
## "class": "CommandLineTool",
## "baseCommand": [
## "Rscript -e 'x = runif(100); write.csv(x, file = 'random.txt', row.names = FALSE)'"
## ],
## "arguments": []
## },
## "sbg:x": 100,
## "sbg:y": 200
## },
## {
## "id": "#get_log",
## "inputs": [
## {
## "id": "#get_log.number",
## "source": ["#random_number.random"]
## }
## ],
## "outputs": [
## {
## "id": "#get_log.log"
## }
## ],
## "hints": [],
## "run": {
## "sbg:id": "log new test 3",
## "id": "#log_new_test_3",
## "inputs": [
## {
## "type": ["null", "File"],
## "label": "",
## "description": "",
## "streamable": false,
## "default": "",
## "id": "#number",
## "required": false
## }
## ],
## "outputs": [
## {
## "type": ["null", "File"],
## "label": "",
## "description": "",
## "streamable": false,
## "default": "",
## "id": "#log",
## "outputBinding": {
## "glob": "*.txt"
## }
## }
## ],
## "requirements": [
## {
## "class": "CreateFileRequirement",
## "fileDef": [
## {
## "filename": "log.R",
## "fileContent": "args = commandArgs(TRUE)\n x = read.table(args[1], header = TRUE)[,'x']\n x = log(x)\n write.csv(x, file = 'random_log.txt', row.names = FALSE)\n "
## }
## ]
## }
## ],
## "hints": [
## {
## "class": "DockerRequirement",
## "dockerPull": "rocker/r-base",
## "dockerLoad": "",
## "dockerFile": "",
## "dockerImageId": "",
## "dockerOutputDirectory": ""
## }
## ],
## "label": "get log",
## "class": "CommandLineTool",
## "baseCommand": [
## "Rscript log.R"
## ],
## "arguments": []
## },
## "sbg:x": 350,
## "sbg:y": 200
## },
## {
## "id": "#get_mean",
## "inputs": [
## {
## "id": "#get_mean.number",
## "source": ["#get_log.log"]
## }
## ],
## "outputs": [
## {
## "id": "#get_mean.mean"
## }
## ],
## "hints": [],
## "run": {
## "sbg:id": "mean new test 3",
## "id": "#mean_new_test_3",
## "inputs": [
## {
## "type": ["null", "File"],
## "label": "",
## "description": "",
## "streamable": false,
## "default": "",
## "id": "#number",
## "required": false
## }
## ],
## "outputs": [
## {
## "type": ["null", "File"],
## "label": "",
## "description": "",
## "streamable": false,
## "default": "",
## "id": "#mean",
## "outputBinding": {
## "glob": "*.txt"
## }
## }
## ],
## "requirements": [
## {
## "class": "CreateFileRequirement",
## "fileDef": [
## {
## "filename": "mean.R",
## "fileContent": "args = commandArgs(TRUE)\n x = read.table(args[1], header = TRUE)[,'x']\n x = mean(x)\n write.csv(x, file = 'random_mean.txt', row.names = FALSE)\n "
## }
## ]
## }
## ],
## "hints": [
## {
## "class": "DockerRequirement",
## "dockerPull": "rocker/r-base",
## "dockerLoad": "",
## "dockerFile": "",
## "dockerImageId": "",
## "dockerOutputDirectory": ""
## }
## ],
## "label": "get mean",
## "class": "CommandLineTool",
## "baseCommand": [
## "Rscript mean.R"
## ],
## "arguments": []
## },
## "sbg:x": 600,
## "sbg:y": 200
## }
## ],
## "sbg:canvas_zoom": 1,
## "sbg:canvas_y": 130,
## "sbg:canvas_x": 40
## }Or use %>>%
f <- t1 %>>% t2 %>>% t3
f$toJSON()## {"id":"#random_number_get_log","inputs":[],"outputs":[],"requirements":[],"hints":[],"label":"random_number get_log","class":"Workflow","steps":[{"id":"#random_number","inputs":[],"outputs":[{"id":"#random_number.random"}],"hints":[],"run":{"sbg:id":"runif new test 3","id":"#runif_new_test_3","inputs":[],"outputs":[{"type":["null","File"],"label":"","description":"","streamable":false,"default":"","id":"#random","outputBinding":{"glob":"random.txt"}}],"requirements":[],"hints":[{"class":"DockerRequirement","dockerPull":"rocker/r-base","dockerLoad":"","dockerFile":"","dockerImageId":"","dockerOutputDirectory":""}],"label":"random number","class":"CommandLineTool","baseCommand":["Rscript -e 'x = runif(100); write.csv(x, file = 'random.txt', row.names = FALSE)'"],"arguments":[]},"sbg:x":100,"sbg:y":200},{"id":"#get_log","inputs":[{"id":"#get_log.number","source":["#random_number.random"]}],"outputs":[{"id":"#get_log.log"}],"hints":[],"run":{"sbg:id":"log new test 3","id":"#log_new_test_3","inputs":[{"type":["null","File"],"label":"","description":"","streamable":false,"default":"","id":"#number","required":false}],"outputs":[{"type":["null","File"],"label":"","description":"","streamable":false,"default":"","id":"#log","outputBinding":{"glob":"*.txt"}}],"requirements":[{"class":"CreateFileRequirement","fileDef":[{"filename":"log.R","fileContent":"args = commandArgs(TRUE)\n x = read.table(args[1], header = TRUE)[,'x']\n x = log(x)\n write.csv(x, file = 'random_log.txt', row.names = FALSE)\n "}]}],"hints":[{"class":"DockerRequirement","dockerPull":"rocker/r-base","dockerLoad":"","dockerFile":"","dockerImageId":"","dockerOutputDirectory":""}],"label":"get log","class":"CommandLineTool","baseCommand":["Rscript log.R"],"arguments":[]},"sbg:x":350,"sbg:y":200},{"id":"#get_mean","inputs":[{"id":"#get_mean.number","source":["#get_log.log"]}],"outputs":[{"id":"#get_mean.mean"}],"hints":[],"run":{"sbg:id":"mean new test 3","id":"#mean_new_test_3","inputs":[{"type":["null","File"],"label":"","description":"","streamable":false,"default":"","id":"#number","required":false}],"outputs":[{"type":["null","File"],"label":"","description":"","streamable":false,"default":"","id":"#mean","outputBinding":{"glob":"*.txt"}}],"requirements":[{"class":"CreateFileRequirement","fileDef":[{"filename":"mean.R","fileContent":"args = commandArgs(TRUE)\n x = read.table(args[1], header = TRUE)[,'x']\n x = mean(x)\n write.csv(x, file = 'random_mean.txt', row.names = FALSE)\n "}]}],"hints":[{"class":"DockerRequirement","dockerPull":"rocker/r-base","dockerLoad":"","dockerFile":"","dockerImageId":"","dockerOutputDirectory":""}],"label":"get mean","class":"CommandLineTool","baseCommand":["Rscript mean.R"],"arguments":[]},"sbg:x":600,"sbg:y":200}],"sbg:canvas_zoom":1,"sbg:canvas_y":130,"sbg:canvas_x":40}If it’s the first time you upload those tools, following script will push both tools and the workflow to your project.
## need id, full, sbg:id
library(sevenbridges)
a <- Auth(platform = "cgc", username = "tengfei")
p <- a$project(id = "tengfei/helloworld")
app.runif <- p$app_add("new_flow", f)
aid <- app.runif$id
p$task_add(name = "Draft flow",
description = "Flow test",
app = aid)
## confirm, show all task status is draft
(tsk <- p$task('Draft flow'))
tsk$run()You can get App from your repos and connect it in R to build flow.
app1 <- p$app(id = "tengfei/helloworld/runif_new_test/0")
app2 <- p$app(id = "tengfei/quickstart/log_new_test_3/0")
app3 <- p$app(id = "tengfei/quickstart/log_new_test_3/0")
f <- app1 %>>% app2 %>>% app3
f$id <- "randome number test flow 2"
f$"sbg:id" <- "tengfei/helloworld/testtesttest"
f$toJSON(pretty = TRUE)** Important **
This flow construction method is easy for linear flow, your best choice is always our graphic user interface and editor.