%\VignetteIndexEntry{1.3 Sequence analysis}

\chapter{Sequencing}

\begin{Exercise}
  Set up your system to do the exercises in this lab. In addition to
  the packages you were asked to install prior to the course, you'll
  need to install the \Rpackage{Morgan2013} package and
  \texttt{BigData} directory of sample data files. Do this following
  the instructions in the lab. The following line MUST succeed
<<test>>=
SYSTEM_OK <- getRversion() > "3.0" && require(Morgan2013, quietly=TRUE)
stopifnot(SYSTEM_OK)
@ 
  %% 
  In addition, some of the exercises require access to data files. These
  should be installed at a convenient place; the exercises assume that
  the files are at the location returned by
<<BigData-path, eval=FALSE>>=
path.expand("~/BigData")
@ 
  %%
  There should be a total of 39 files
<<BigData-length>>=
stopifnot(length(dir("~/BigData", recursive=TRUE)) == 39)
@ 
\end{Exercise}

Recent technological developments introduce high-throughput sequencing
approaches. A variety of experimental protocols and analysis work
flows address gene expression, regulation, and encoding of genetic
variants. Experimental protocols produce a large number (tens to
hundreds of millions per sample) of short (e.g., 35-100's, single or
paired-end) nucleotide sequences. These are aligned to a reference or
other genome. Analysis work flows use the alignments to infer levels
of gene expression (RNA-seq), binding of regulatory elements to
genomic locations (ChIP-seq), or prevalence of structural variants
(e.g., SNPs, short indels, large-scale genomic rearrangements). Sample
sizes range from minimal replication (e.g,. 2 samples per treatment
group) to thousands of individuals.

\section{Technologies}

The most common `second generation' technologies readily available to
labs are
\begin{packeditemize}
\item Illumina single- and paired-end reads. Short ($\sim 100$ per
  end) and very numerous. Flow cell, lane, bar-code.
\item Roche 454. 100's of nucleotides, 100,000's of reads.
\item Life Technologies SOLiD. Unique `color space' model.
\item Complete Genomics. Whole genome sequence / variants / etc as a
  service; end user gets derived results.
\end{packeditemize}
Figure~\ref{fig:pcr} illustrates Illumina and 454
sequencing. \Bioconductor{} has good support for Illumina and Roche
454 sequencing products, and for derived data such as aligned reads or
called variants; use of SOLiD color space reads typically requires
conversion to FASTQ files that undermine the benefit of the color
space model.

\begin{figure}
  \centering
  \begin{minipage}[c]{.4\textwidth}
    \includegraphics[width=\textwidth, height=!]{Solexa-bridge-pcr}
  \end{minipage}
  \hfil
  \begin{minipage}[c]{.4\textwidth}
    \includegraphics[width=\textwidth, height=!]{nbt1485-F1}
  \end{minipage}
  \caption{High-throughput sequencing. Left: Illumina bridge
    PCR~\cite{bentley2008}; mis-call errors. Right: Roche
    454~\cite{rothberg2008}; homopolymer errors.}
  \label{fig:pcr}
\end{figure}

All second-generation technologies rely on PCR and other techniques to
generate reads from samples that represent aggregations of many DNA
molecules.  `Third-generation' technologies shift to single-molecule
sequencing, with relevant players including Pacific Biosciences and
IonTorent. This data is not widely available, and will not be
discussed further.

The most common data in \Bioconductor{} work flows is from Illumina
sequencers. Reads are either single-end or paired-end. Single-end
reads represent $30-\sim100$ nucleotides sequenced from DNA that
has been sheared into $\sim300$ nucleotide fragments. Paired-end
reads represent $30-\sim100$ nucleotide reads that are paired, and
from both ends of the $\sim300$ fragment.

Sequence data can be derived from a tremendous diversity of
experiments. Some of the most common include:
\begin{packeddescription}
\item [RNA-seq] Sequencing of reverse-complemented mRNA from the
  entire expressed transcriptome, typically. Used for
  \emph{differential expression} studies like micro-arrays, or for
  \emph{novel transcript discovery}.
\item [DNA-seq] Sequencing of whole or targeted (e.g., exome) genomic
  DNA. Common goals include \emph{SNP} detection, \emph{indel} and
  other structural polymorphisms, and \emph{CNV} (copy number
  variation). DNA-seq is also used for \emph{de novo} assembly, but
  \emph{de novo} assembly is not an area where \Bioconductor{}
  contributes.
\item [ChIP-seq] ChIP (chromatin immuno-precipitation) is used to
  enrich genomic DNA for regulatory elements, followed by sequencing
  and mapping of the enriched DNA to a reference genome. The initial
  statistical challenge is to identify regions where the mapped reads
  are enriched relative to a sample that did not undergo
  ChIP\cite{pmid19736561}; a subsequent task is to identify
  differential binding across a designed experiment, e.g.,
  \cite{ross2012differential}.
\item [Metagenomics] Sequencing generates sequences from samples
  containing multiple species, typically microbial communities sampled
  from niches such as the human oral cavity. Goals include inference
  of \emph{species composition} (when sequencing typically targets
  phylogenetically informative genes such as 16S) or \emph{metabolic
    contribution}.
\end{packeddescription}

\section{Data}

As a running example, we use the \emph{pasilla} data set, derived from
\cite{pasilla}. The authors investigate conservation of RNA
regulation between \Dmel{} and mammals. Part of their study used RNAi
and RNA-seq to identify exons regulated by Pasilla (\emph{ps}), the
\Dmel{} ortholog of mammalian NOVA1 and NOVA2. Briefly, their
experiment compared gene expression as measured by RNAseq in S2-DRSC
cells cultured with, or without, a 444bp dsRNA fragment corresponding
to the \emph{ps} mRNA sequence. Their assessment investigated
differential exon use, but our worked example will focus on gene-level
differences.  For several examples we look at a subset of the
\emph{ps} data, corresponding to reads obtained from lanes of their
RNA-seq experiment, and to the same reads aligned to a \Dmel{}
reference genome. Reads were obtained from GEO and the Short Read
Archive (SRA), and were aligned to the \Dmel{} reference genome
\emph{dm3} as described in the \Rpackage{pasilla} experiment data
package.

At a very high level, one can envision a work flow that starts with a
challenging biological question (how does \emph{ps} influence gene and
transcript regulation?). The biological question is framed in terms of
wet-lab protocols coupled with an appropriate and all-important
experimental design. There are several well-known statistical issues
at this stage, common to any experimental data. What treatments are
going to be applied? How many replicates will there be of each? Is
there likely to be sufficient power to answer the biologically
relevant question?  Reality is also important at this stage, as
evidenced in the \emph{pasilla} data where, as we will see, samples
were collected using different methods (single versus paired end
reads) over a time when there were rapid technological changes. Such
reality often introduces confounding factors that require appropriate
statistical treatment in subsequent analysis.

The work flow proceeds with data generation, involving both a wet-lab
(sample preparation) component and actual sequencing. It is essential
to acknowledge the biases and artifacts that are introduced at each of
these stages. Sample preparation involves non-trivial amounts of time
and effort. Large studies are likely to have batch effects (e.g.,
because work was done by different lab members, or different batches
of reagent). Samples might have been prepared in ways that are likely
to influence down-stream analysis, e.g., using a protocol involving
PCR and hence introducing opportunities for sample-specific bias. DNA
isolation protocols may introduce many artifacts, e.g., non-uniform
representation of reads across the length of expressed genes in
RNA-seq.  The sequencing reaction itself is far from bias-free, with
known artifacts of called base frequency, cycle-dependent accuracy and
bias, non-uniform coverage, etc. At a minimum, the researcher needs to
be aware of the opportunities for bias that can be introduced during
sample preparation and sequencing.

The informatics component of work flows becomes increasing important
during and after sequence generation.  The sequencer is often treated
as a `black box', producing short reads consisting of 10's to 100's of
nucleotides and with associated quality scores. Usually, the chemistry
and informatics processing pipeline are sufficiently well documented
that one can arrive at an understanding of biases and quality issues
that might be involved; such an understanding is likely to be
particularly important when embarking on questions or using protocols
that are at the fringe of standard practice (where, after all, the
excitement is).  

The first real data seen by users are \emph{fastq} files
(Table~\ref{tab:seq:fileformats}). These files are often simple text
files consisting of many millions of records, and are described in
greater detail in Section~\ref{sec:ShortRead}. The center performing
the sequencing typically vets results for quality, but these quality
measures are really about the performance of their machines. It is
very important to assess quality with respect to the experiment being
undertaken -- Are the numbers of reads consistent across samples? Are
GC content and other observable aspects of the reads consistent with
expectation? Are there anomalies in the sequence results that reflect
primers or other reagents used during sample preparation? Are
well-known artifacts of the experimental protocol evident in the reads
in hand?

\begin{table}
  \centering
  \caption{Common file types (e.g.,
    \url{http://genome.ucsc.edu/FAQ/FAQformat.html}) and
    \Bioconductor{} packages used for input.}
  \label{tab:seq:fileformats}
  \begin{tabular}{lp{.6\textwidth}l}
    File & Description & Package \\
    \hline
    FASTQ & Unaligned sequences: identifier, sequence, and encoded
    quality score tuples & \Biocpkg{ShortRead}\\

    BAM & Aligned sequences: identifier, sequence, reference sequence
    name, strand position, cigar and additional tags &
    \Biocpkg{Rsamtools} \\
    
    GFF, GTF & Gene annotations: reference sequence name, data source,
    feature type, start and end positions, strand, etc. &
    \Biocpkg{rtracklayer}\\
    
    BED & Range-based annotation: reference sequence name, start, end
    coordinates. & \Biocpkg{rtracklayer}\\
    
    WIG, bigWig & `Continuous' single-nucleotide annotation. &
    \Biocpkg{rtracklayer}\\
    
    VCF & Called single nucleotide, indel, copy number, and structural
    variants, often compressed and indexed (with \Biocpkg{Rsamtools}
    \Rfunction{bgzip}, \Rfunction{indexTabix}) &
    \Biocpkg{VariantAnnotation} \\ \hline

  \end{tabular}
\end{table}

The next step in many work flows involves alignment of reads to a
reference genome.  There are many aligners available, including
\href{http://bio-bwa.sourceforge.net/}{BWA} \cite{pmid20080505},
\href{http://bowtie-bio.sourceforge.net/}{Bowtie} /
\href{http://bowtie-bio.sourceforge.net/bowtie2/}{Bowtie2}
\cite{pmid19261174}, and
\href{http://research-pub.gene.com/gmap/}{GSNAP}; merits of these are
discussed in the literature. \Bioconductor{} packages `wrapping' these
tools are increasingly common (e.g., \Biocpkg{Rbowtie},
\Biocpkg{gmapR}; \Biocpkg{cummeRbund} for parsing output of the
\software{cufflinks} transcript discovery pathway). There are also
alignment algorithms implemented in \Bioconductor{} (e.g.,
\Rfunction{matchPDict} in the \Biocpkg{Biostrings} package, and the
\Biocpkg{Rsubread} package); \Rfunction{matchPDict} is particularly
useful for flexible alignment of moderately sized subsets of
data. Most main-stream aligners produce output in `SAM' or `BAM'
(binary alignment) format. BAM files are the primary starting point
for many analyses, and their manipulation and use in \Bioconductor{}
is introduced in Section~\ref{sec:Rsamtools}.

Common analyses often use well-established third-party tools for
initial stages of the analysis; some of these have \Bioconductor{}
counterparts that are particularly useful when the question under
investigation does not meet the assumptions of other facilities. Some
common work flows (a more comprehensive list is available on the
SeqAnswers wiki\footnote{\url{http://seqanswers.com/wiki/RNA-Seq}})
include:
\begin{packeddescription}
\item [ChIP-seq] ChIP-seq experiments typically use DNA sequencing to
  identify regions of genomic DNA enriched in prepared samples
  relative to controls. A central task is thus to identify peaks, with
  common tools including \software{MACS} and
  \software{PeakRanger}.
\item [RNA-seq] In addition to the aligners mentioned above, RNA-seq
  for differential expression might use the
  \software{HTSeq}\footnote{\url{http://www-huber.embl.de/users/anders/HTSeq/doc/overview.html}}
  python tools for counting reads within regions of interest (e.g.,
  known genes) or a pipeline such as the \software{bowtie} (basic
  alignment) / \software{tophat} (splice junction mapper) /
  \software{cufflinks} (esimated isoform abundance)
  (e.g., \footnote{\url{http://bowtie-bio.sourceforge.net/index.shtml}})
  or
  \software{RSEM}\footnote{\url{http://deweylab.biostat.wisc.edu/rsem/}}
  suite of tools for estimating transcript abundance.
\item [DNA-seq] especially variant calling can be facilitated by
  software such as the
  GATK\footnote{\url{http://www.broadinstitute.org/gatk/}} toolkit.
\end{packeddescription}
There are many \R{} packages that replace or augment the analyses
outlined above.

Programs such as those outlined the previous paragraph often rely on
information about gene or other structure as input, or produce
information about chromosomal locations of interesting features. The
GTF and BED file formats are common representations of this
information. Representing these files as \R{} data structures is often
facilitated by the \Biocpkg{rtracklayer} package. We explore these
files in the lab on gene and genome annotation.  Variants are very
commonly represented in VCF (Variant Call Format) files; these are
explored in the lab on variants.

\section{\emph{Bioconductor} for sequence analysis}

Table~\ref{tab:bioc-sequence-pkgs} enumerates some of the packages
available for sequence analysis. The table includes packages for
representing sequence-related data (e.g., \Biocpkg{GenomicRanges},
\Biocpkg{Biostrings}), as well as domain-specific analysis such as
RNA-seq (e.g., \Biocpkg{edgeR}, \Biocpkg{DESeq2}), ChIP-seq
(e.g,. \Biocpkg{ChIPpeakAnno}, \Biocpkg{DiffBind}), variants (e.g.,
\Biocpkg{VariantAnnotation}, \Biocpkg{VariantTools}, and SNPs and copy
number variation (e.g., \Biocpkg{genoset}, \Biocpkg{ggtools}).


\begin{table}
  \centering
  \caption{Selected \Bioconductor{} packages for high-throughput
    sequence analysis.}
  \label{tab:bioc-sequence-pkgs}
  \begin{tabular}{lp{.65\textwidth}}
    Concept & Packages\\
    \hline

    Data representation & \Biocpkg{IRanges}, \Biocpkg{GenomicRanges},
    \Biocpkg{GenomicFeatures}, \Biocpkg{Biostrings},
    \Biocpkg{BSgenome}, \Biocpkg{girafe}.\\

    Input / output & \Biocpkg{ShortRead} (fastq), \Biocpkg{Rsamtools}
    (bam), \Biocpkg{rtracklayer} (gff, wig, bed),
    \Biocpkg{VariantAnnotation} (vcf), \Biocpkg{R453Plus1Toolbox}
    (454).\\

    Annotation & \Biocpkg{GenomicFeatures}, \Biocpkg{ChIPpeakAnno},
    \Biocpkg{VariantAnnotation}.\\

    Alignment & \Biocpkg{gmapR}, \Biocpkg{Rsubread}, \Biocpkg{Biostrings}.\\

    Visualization & \Biocpkg{ggbio}, \Biocpkg{Gviz}.\\

    Quality assessment & \Biocpkg{qrqc}, \Biocpkg{seqbias},
    \Biocpkg{ReQON}, \Biocpkg{htSeqTools}, \Biocpkg{TEQC},
    \Biocpkg{Rolexa}, \Biocpkg{ShortRead}.\\

    RNA-seq & \Biocpkg{BitSeq}, \Biocpkg{cqn}, \Biocpkg{cummeRbund},
    \Biocpkg{DESeq2}, \Biocpkg{DEXSeq}, \Biocpkg{EDASeq},
    \Biocpkg{edgeR}, \Biocpkg{gage}, \Biocpkg{goseq},
    \Biocpkg{iASeq}, \Biocpkg{tweeDEseq}.\\

    ChIP-seq, etc. & \Biocpkg{BayesPeak}, \Biocpkg{baySeq},
    \Biocpkg{ChIPpeakAnno}, \Biocpkg{chipseq}, \Biocpkg{ChIPseqR},
    \Biocpkg{ChIPsim}, \Biocpkg{CSAR}, \Biocpkg{DiffBind},
    \Biocpkg{MEDIPS}, \Biocpkg{mosaics}, \Biocpkg{NarrowPeaks},
    \Biocpkg{nucleR}, \Biocpkg{PICS}, \Biocpkg{PING},
    \Biocpkg{REDseq}, \Biocpkg{Repitools}, \Biocpkg{TSSi}.\\
    
    Variants & \Biocpkg{VariantAnnotation}, \Biocpkg{VariantTools},
    \Biocpkg{gmapR} \\
    
    SNPs & \Biocpkg{snpStats}, \Biocpkg{GWASTools},
    \Biocpkg{hapFabia}, \Biocpkg{GGtools} \\

    Copy number & \Biocpkg{cn.mops}, \Biocpkg{genoset},
    \Biocpkg{CNAnorm}, \Biocpkg{exomeCopy}, \Biocpkg{seqmentSeq}.\\

    Motifs & \Biocpkg{MotifDb}, \Biocpkg{BCRANK}, \Biocpkg{cosmo}, \Biocpkg{cosmoGUI},
    \Biocpkg{MotIV}, \Biocpkg{seqLogo}, \Biocpkg{rGADEM}.\\

    3C, etc. & \Biocpkg{HiTC}, \Biocpkg{r3Cseq}.\\

    Microbiome & \Biocpkg{phyloseq}, \Biocpkg{DirichletMultinomial},
    \Biocpkg{clstutils}, \Biocpkg{manta}, \Biocpkg{mcaGUI}.\\

    Work flows & \Biocpkg{QuasR}, \Biocpkg{easyRNASeq}, 
    \Biocpkg{ArrayExpressHTS}, \Biocpkg{Genominator}, 
    \Biocpkg{oneChannelGUI}, \Biocpkg{rnaSeqMap}.\\

    Database & \Biocpkg{SRAdb}.\\

    \hline
  \end{tabular}
\end{table}
