%\VignetteIndexEntry{1.5 Alignments and Ranges}

\chapter{Ranges and Alignments}
\label{chpt:RangesAndAlignments}

<<setup,echo=FALSE>>=
library(GenomicFeatures)
plotRanges <- function(x, xlim = x, main = deparse(substitute(x)),
                       col = "black", sep = 0.5, ...)
{
    height <- 1
    if (is(xlim, "Ranges"))
        xlim <- c(min(start(xlim)), max(end(xlim)))
    bins <- disjointBins(IRanges(start(x), end(x) + 1))
    plot.new()
    par(mai=c(0.6, 0.2, 0.6, 0.2))
    plot.window(xlim, c(0, max(bins)*(height + sep)))
    ybottom <- bins * (sep + height) - height
    rect(start(x)-0.5, ybottom, end(x)+0.5, ybottom + height, col = col, ...)
    title(main, cex.main=2.8, font.main=1)
    axis(1)
}
@

Ranges describe both features of interest (e.g., genes, exons,
promoters) and reads aligned to the genome. \Bioconductor{} has very
powerful facilities for working with ranges, some of which are
summarized in Table~\ref{tab:ranges-pkgs}.

\begin{table}
  \centering
  \caption{Selected \Bioconductor{} packages for representing and
    manipulating ranges.}
  \label{tab:ranges-pkgs}
  \begin{tabular}{lp{.7\textwidth}}
    Package & Description\\
    \hline

    \Biocpkg{IRanges} & Defines important classes (e.g.,
    \Rclass{IRanges}, \Rclass{Rle}) and methods (e.g.,
    \Rfunction{findOverlaps}, \Rfunction{countOverlaps}) for
    representing and manipulating ranges of consecutive values. Also
    introduces \Rclass{DataFrame}, \Rclass{SimpleList} and other
    classes tailored to representing very large data.\\

    \Biocpkg{GenomicRanges} & Range-based classes tailored to sequence
    representation (e.g., \Rclass{GRanges}, \Rclass{GRangesList}),
    with information about strand and sequence name.\\

    \Biocpkg{GenomicFeatures} & Foundation for manipulating data bases
    of genomic ranges, e.g., representing coordinates and organization
    of exons and transcripts of known genes.\\
    
    \Biocpkg{AnnotationHub} & Access many common large-data resources (e.g., 
    Encode tracks; Ensembl GTF files)  as \Rclass{GRanges} instances.\\

    \hline
  \end{tabular}
\end{table}

\section{Ranges and the \Biocpkg{GenomicRanges} package}

Next-generation sequencing data consists of a large number of short
reads. These are, typically, aligned to a reference genome. Basic
operations are performed on the alignment, asking e.g., how many reads
are aligned in a genomic range defined by nucleotide coordinates
(e.g., in the exons of a gene), or how many nucleotides from all the
aligned reads cover a set of genomic coordinates. How is this type of
data, the aligned reads and the reference genome, to be represented in
\R{} in a way that allows for effective computation?

The \Biocpkg{IRanges}, \Biocpkg{GenomicRanges}, and
\Biocpkg{GenomicFeatures} \Bioconductor{} packages
(Table~\ref{tab:ranges-pkgs}) provide the essential infrastructure for
these operations; we start with the \Rclass{GRanges} class, defined in
\Biocpkg{GenomicRanges}.

\paragraph{\Rclass{GRanges}}

Instances of \Rclass{GRanges} are used to specify genomic
coordinates. Suppose we wish to represent two \Dmel{} genes. The
first is located on the positive strand of chromosome 3R, from
position 19967117 to 19973212. The second is on the minus strand of
the X chromosome, with `left-most' base at 18962306, and right-most
base at 18962925. The coordinates are \emph{1-based} (i.e., the first
nucleotide on a chromosome is numbered 1, rather than 0),
\emph{left-most} (i.e., reads on the minus strand are defined to
`start' at the left-most coordinate, rather than the 5' coordinate),
and \emph{closed} (the start and end coordinates are included in the
range; a range with identical start and end coordinates has width 1, a
0-width range is represented by the special construct where the end
coordinate is one less than the start coordinate).

A complete definition of these genes as \Rclass{GRanges} is:
<<GRanges-genes>>=
genes <- GRanges(seqnames=c("3R", "X"),
                 ranges=IRanges(
                     start=c(19967117, 18962306),
                     end=c(19973212, 18962925)),
                 strand=c("+", "-"),
                 seqlengths=c(`3R`=27905053L, `X`=22422827L))
@
The components of a \Rclass{GRanges} object are defined as vectors,
e.g., of \Rfunarg{seqnames}, much as one would define a
\Rclass{data.frame}. The start and end coordinates are grouped into an
\Rclass{IRanges} instance. The optional \Rfunarg{seqlengths} argument
specifies the maximum size of each sequence, in this case the lengths
of chromosomes 3R and X in the `dm2' build of the \Dmel{} genome. This
data is displayed as
<<GRanges-display>>=
genes
@
For the curious, the gene coordinates and sequence lengths are derived
from the \Biocannopkg{org.Dm.eg.db} package for genes with Flybase
identifiers FBgn0039155 and FBgn0085359, using the annotation
facilities described in the lab on gene and genome annotation.

The \Rclass{GRanges} class has many useful methods defined on
it. Consult the help page
<<GRanges-help, eval=FALSE>>=
?GRanges
@
and package vignettes (especially `An Introduction to
\Biocpkg{GenomicRanges}')
<<GRanges-vignettes, eval=FALSE>>=
vignette(package="GenomicRanges")
@
for a comprehensive introduction. A \Rclass{GRanges} instance can be
subset, with accessors for getting and updating information.
<<GRanges-basics>>=
genes[2]
strand(genes)
width(genes)
length(genes)
names(genes) <- c("FBgn0039155", "FBgn0085359")
genes  # now with names
@ 
\Rfunction{strand} returns the strand information in a compact
representation called a \emph{run-length encoding}, this is introduced
in greater detail below.  The `names' could have been specified when
the instance was constructed; once named, the \Rclass{GRanges}
instance can be subset by name like a regular vector.

As the \Rfunction{GRanges} function suggests, the \Rclass{GRanges}
class extends the \Rclass{IRanges} class by adding information about
\Rfunarg{seqnames}, \Rfunarg{strand}, and other information
particularly relevant to representing ranges that are on genomes. The
\Rclass{IRanges} class and related data structures (e.g.,
\Rclass{RangedData}) are meant as a more general description of ranges
defined in an arbitrary space. Many methods implemented on the
\Rclass{GRanges} class are `aware' of the consequences of genomic
location, for instance treating ranges on the minus strand differently
(reflecting the 5' orientation imposed by DNA) from ranges on the plus
strand.

\paragraph{Operations on ranges}

The \Rclass{GRanges} class has many useful methods from the
\Rclass{IRanges} class; some of these methods are illustrated here. We
use \Rclass{IRanges} to illustrate these operations to avoid
complexities associated with strand and seqnames, but the operations
are comparable on \Rclass{GRanges}. We begin with a simple set of
ranges:
<<ranges-ir>>=
ir <- IRanges(start=c(7, 9, 12, 14, 22:24),
              end=c(15, 11, 12, 18, 26, 27, 28))
@
%%
<<ranges-ir-plot, results=hide, echo=FALSE, fig=FALSE>>=
png("ranges-ir-plot.png", width=800, height=160)
plotRanges(ir, xlim=c(5, 35), main="Original")
dev.off()
png("ranges-shift-ir-plot.png", width=800, height=160)
plotRanges(shift(ir, 5), xlim=c(5, 35), main="Shift")
dev.off()
png("ranges-reduce-ir-plot.png", width=800, height=160)
plotRanges(reduce(ir), xlim=c(5, 35), main="Reduce")
dev.off()
@
%
These and some common operations are illustrated in the upper panel of
Figure~\ref{fig:ranges} and summarized in Table~\ref{tab:range-ops}.

\begin{figure}
\centering
\includegraphics[width=.5\textwidth,height=!]{ranges-ir-plot}\\
\includegraphics[width=.5\textwidth,height=!]{ranges-shift-ir-plot}\\
\includegraphics[width=.5\textwidth,height=!]{ranges-reduce-ir-plot}
\caption{Ranges}
\label{fig:ranges}
\end{figure}

Methods on ranges can be grouped as follows:
\begin{packeddescription}
\item[Intra-range] methods act on each range independently. These
  include \Rfunction{flank}, \Rfunction{narrow}, \Rfunction{reflect},
  \Rfunction{resize}, \Rfunction{restrict}, and \Rfunction{shift},
  among others. An illustration is \Rfunction{shift}, which translates
  each range by the amount specified by the \Rfunarg{shift} argument.
  Positive values shift to the right, negative to the left;
  \Rfunarg{shift} can be a vector, with each element of the vector
  shifting the corresponding element of the \Rclass{IRanges}
  instance. Here we shift all ranges to the right by 5, with the
  result illustrated in the middle panel of Figure~\ref{fig:ranges}.
<<ranges-shift-ir>>=
shift(ir, 5)
@
<<ranges-shift-ir-plot, results=hide, echo=FALSE, fig=FALSE>>=
png("ranges-shift-ir-plot.png", width=800, height=160)
plotRanges(shift(ir, 5), xlim=c(5, 35), main="Shift")
dev.off()
@

\item[Inter-range] methods act on the collection of ranges as a
  whole. These include \Rfunction{disjoin}, \Rfunction{reduce},
  \Rfunction{gaps}, and \Rfunction{range}. An illustration is
  \Rfunction{reduce}, which reduces overlapping ranges into a single
  range, as illustrated in the lower panel of
  Figure~\ref{fig:ranges}.
<<ranges-reduce-ir>>=
reduce(ir)
@
<<ranges-reduce-ir-plot, results=hide, echo=FALSE, fig=FALSE>>=
png("ranges-reduce-ir-plot.png", width=800, height=160)
plotRanges(reduce(ir), xlim=c(5, 35), main="Reduce")
dev.off()
@

  \Rfunction{coverage} is an inter-range operation that calculates how
  many ranges overlap individual positions. Rather than returning
  ranges, \Rfunction{coverage} returns a compressed representation
  (run-length encoding)
<<coverage>>=
coverage(ir)
@    
  The run-length encoding can be interpreted as `a run of length 6 of
  nucleotides covered by 0 ranges, followed by a run of length 2 of
  nucleotides covered by 1 range.

\item[Between] methods act on two (or sometimes more) \Rclass{IRanges}
  instances. These include \Rfunction{intersect}, \Rfunction{setdiff},
  \Rfunction{union}, \Rfunction{pintersect}, \Rfunction{psetdiff}, and
  \Rfunction{punion}.  The \Rfunction{countOverlaps} and
  \Rfunction{findOverlaps} functions operate on two sets of
  ranges. \Rfunction{countOverlaps} takes its first argument (the
  \Rfunarg{query}) and determines how many of the ranges in the second
  argument (the \Rfunarg{subject}) each overlaps. The result is an
  integer vector with one element for each member of
  \Rfunarg{query}. \Rfunction{findOverlaps} performs a similar
  operation but returns a more general matrix-like structure that
  identifies each pair of query / subject overlaps. Both arguments
  allow some flexibility in the definition of `overlap'.
\end{packeddescription}

Common operations on ranges are summarized in Table~\ref{tab:range-ops}.

\begin{table}
  \centering
  \caption{Common operations on \Rclass{IRanges}, \Rclass{GRanges} and
    \Rclass{GRangesList}.}
  \label{tab:range-ops}
  \begin{tabular}{lll}
    Category & Function & Description \\
    \hline

    Accessors & \Rfunction{start}, \Rfunction{end}, \Rfunction{width} & 
        Get or set the starts, ends and widths \\
    & \Rfunction{names} & Get or set the names \\
    & \Rfunction{mcols}, \Rfunction{metadata} & 
        Get or set metadata on elements or object \\
    & \Rfunction{length} & Number of ranges in the vector \\
    & \Rfunction{range} & Range formed from min start and max end \\

    Ordering & \Rfunction{<}, \Rfunction{<=}, \Rfunction{>}, \Rfunction{>=}, 
        \Rfunction{==}, \Rfunction{!=} & 
        Compare ranges, ordering by start then width \\
    & \Rfunction{sort}, \Rfunction{order}, \Rfunction{rank} & 
        Sort by the ordering \\
    & \Rfunction{duplicated} & Find ranges with multiple instances \\
    & \Rfunction{unique} & Find unique instances, removing duplicates \\

    Arithmetic & \Rcode{r + x}, \Rcode{r - x}, \Rcode{r * x} & 
        Shrink or expand ranges \Robject{r} by number \Robject{x} \\
    & \Rfunction{shift} & Move the ranges by specified amount \\
    & \Rfunction{resize} & Change width, anchoring on start, end or mid \\
    & \Rfunction{distance} & Separation between ranges (closest endpoints) \\
    & \Rfunction{restrict} & Clamp ranges to within some start and end \\
    & \Rfunction{flank} & Generate adjacent regions on start or end \\

    Set operations & \Rfunction{reduce} & 
        Merge overlapping and adjacent ranges \\
    & \Rfunction{intersect}, \Rfunction{union}, \Rfunction{setdiff} & 
        Set operations on reduced ranges \\
    & \Rfunction{pintersect}, \Rfunction{punion}, \Rfunction{psetdiff} & 
        Parallel set operations, on each \Rcode{x[i]}, \Rcode{y[i]} \\
    & \Rfunction{gaps}, \Rfunction{pgap} & 
        Find regions not covered by reduced ranges \\
    & \Rfunction{disjoin} & Ranges formed from union of endpoints \\

    Overlaps & \Rfunction{findOverlaps} & 
        Find all overlaps for each \Robject{x} in \Robject{y} \\
    & \Rfunction{countOverlaps} & 
        Count overlaps of each \Robject{x} range in \Robject{y} \\
    & \Rfunction{nearest} & 
        Find nearest neighbors (closest endpoints) \\
    & \Rfunction{precede}, \Rfunction{follow} & 
        Find nearest \Robject{y} that \Robject{x} precedes or follows \\
    & \Rcode{x \%in\% y} & 
        Find ranges in \Robject{x} that overlap range in \Robject{y} \\

    Coverage & \Rfunction{coverage} & Count ranges covering each position \\

    Extraction & \Rcode{r[i]} & Get or set by logical or numeric index \\
    & \Rcode{r[[i]]} & 
        Get integer sequence from \Rcode{start[i]} to \Rcode{end[i]} \\
    & \Rfunction{subsetByOverlaps} & 
        Subset \Robject{x} for those that overlap in \Robject{y} \\
    & \Rfunction{head}, \Rfunction{tail}, \Rfunction{rev}, \Rfunction{rep} & 
        Conventional R semantics \\

    Split, combine & \Rfunction{split} & 
        Split ranges by a factor into a \Rclass{RangesList} \\
    & \Rfunction{c} & Concatenate two or more range objects \\
    \hline
  \end{tabular}
\end{table}

\paragraph{\Rfunction{mcols} and \Rfunction{metadata}}

The \Rclass{GRanges} class (actually, most of the data structures
defined or extending those in the \Biocpkg{IRanges} package) has two
additional very useful data components. The \Rfunction{mcols} function
allows information on each range to be stored and manipulated (e.g.,
subset) along with the \Rclass{GRanges} instance. The element metadata
is represented as a \Rclass{DataFrame}, defined in \Biocpkg{IRanges}
and acting like a standard \R{} \Rclass{data.frame} but with the
ability to hold more complicated data structures as columns (and with
element metadata of its own, providing an enhanced alternative to the
\Biocpkg{Biobase} class \Rclass{AnnotatedDataFrame}).
<<GRanges-mcols>>=
mcols(genes) <- DataFrame(EntrezId=c("42865", "2768869"),
                          Symbol=c("kal-1", "CG34330"))
@
\Rfunction{metadata} allows addition of information to the entire
object. The information is in the form of a list; any data can be
provided.
<<GRanges-metadata>>=
metadata(genes) <-
    list(CreatedBy="A. User", Date=date())
@

\paragraph{\Rclass{GRangesList}}

The \Rfunction{GRanges} class is extremely useful for representing
simple ranges. Some next-generation sequence data and genomic features
are more hierarchically structured. A gene may be represented by
several exons within it. An aligned read may be represented by
discontinuous ranges of alignment to a reference.  The
\Rclass{GRangesList} class represents this type of information. It is
a list-like data structure, which each element of the list itself a
\Rclass{GRanges} instance. The gene FBgn0039155 contains several
exons, and can be represented as a list of length 1, where the element
of the list contains a \Rclass{GRanges} object with 7 elements (the
\Rclass{GRangesList} instance shown here is from a `TranscriptDb'
instance, covered in a later lab).
<<GRangesList-eg-setup, echo=FALSE>>=
library(TxDb.Dmelanogaster.UCSC.dm3.ensGene)
txdb <- TxDb.Dmelanogaster.UCSC.dm3.ensGene # alias
fbgn <- exonsBy(txdb, "gene")["FBgn0039155"]
seqlevels(fbgn) <- "chr3R"
@
<<GRangesList-eg, echo=FALSE>>=
fbgn
@

The \Rclass{GRangesList} object has methods one would expect for lists
(e.g., \Rfunction{length}, sub-setting). Many of the methods
introduced for working with \Rclass{IRanges} are also available,
with the method applied element-wise.

\paragraph{The \Biocpkg{GenomicFeatures} packages}

Many public resources provide annotations about genomic features. For
instance, the UCSC genome browser maintains the `knownGene' track of
established exons, transcripts, and coding sequences of many model
organisms. The \Biocpkg{GenomicFeatures} package provides a way to
retrieve, save, and query these resources. The underlying
representation is as sqlite data bases, but the data are available in
\R{} as \Rclass{GRangesList} objects. The following exercise explores
the \Biocpkg{GenomicFeatures} package and some of the functionality
for the \Rclass{IRanges} family introduced above.

\begin{Exercise}
  Load the \Biocannopkg{TxDb.Dmelanogaster.UCSC.dm3.ensGene}
  annotation package, and create an alias \Robject{txdb} pointing to
  the \Rclass{TranscriptDb} object this class defines.

  Extract all exon coordinates, organized by gene, using
  \Rfunction{exonsBy}. What is the class of this object? How many
  elements are in the object? What does each element correspond to?
  And the elements of each element? Use \Rfunction{elementLengths} and
  \Rfunction{table} to summarize the number of exons in each gene, for
  instance, how many single-exon genes are there?

  Select just those elements corresponding to flybase gene ids
  FBgn0002183, FBgn0003360, FBgn0025111, and FBgn0036449. Use
  \Rfunction{reduce} to simplify gene models, so that exons that
  overlap are considered `the same'.
\end{Exercise}

\begin{Solution}
<<txdb>>=
library(TxDb.Dmelanogaster.UCSC.dm3.ensGene)
txdb <- TxDb.Dmelanogaster.UCSC.dm3.ensGene # alias
ex0 <- exonsBy(txdb, "gene")
head(table(elementLengths(ex0)))
ids <- c("FBgn0002183", "FBgn0003360", "FBgn0025111", "FBgn0036449")
ex <- reduce(ex0[ids])
@
\end{Solution}

%% \begin{Exercise}
%%   It is crucial that annotations and alignments are to the same
%%   reference genome, so \Rclass{GRanges} objects contain information
%%   about the expected sequence names and lengths, and many functions
%%   check to make sure that the sequence information is consistent
%%   between objects involved in, e.g.,
%%   \Rfunction{findOverlaps}. Unfortunately, different naming schemes
%%   are in use to refer to the same reference genome.
%%   \begin{packedenumerate}
%%   \item Use the \Rfunction{supportedSeqnameStyles} function from the
%%     \Biocpkg{GenomicFeatures} package to discover the sequence naming
%%     styles supported for \Dmel.
%%   \item Discover the current sequence name style associated with the
%%     \Robject{txdb} instance, above, using \Rfunction{seqnameStyle}.
%%   \item Use the `setter` version of \Rfunction{seqnameStyle} to change
%%     the naming style on the \Robject{txdb} instance to your favorite,
%%     and view the results for exons grouped by gene.
%%   \end{packedenumerate}
%% \end{Exercise}

%% \begin{Solution}
%%   Load the \Biocpkg{GenomicFeatures} package and query for supported
%%   sequence name styles
%% <<seqnameStyles-discover-available>>=
%% library(GenomicFeatures)
%% names(supportedSeqnameStyles())
%% supportedSeqnameStyles()$Drosophila_melanogaster
%% @ 
%% %% 
%% Discover the current sequence name style on a \Rclass{TranscriptDb}
%% object
%% <<seqnameStyles-discover-txdb>>=
%% head(seqlevels(txdb))
%% seqnamessStyle(txdb)
%% @ 
%% %% 
%% Change the naming style
%% <<seqnamessStyle-update>>=
%% seqnameStyle(txdb) <- "Ensembl"
%% head(seqlevels(txdb))
%% head(exonsBy(txdb, "gene"), 2)
%% @ 

%% \end{Solution}

\section{Alignments and the \Biocpkg{Rsamtools} Package}
\label{sec:Rsamtools}

Most down-stream analysis of short read sequences is based on reads
aligned to reference genomes. 

\paragraph{Alignment formats}

Most main-stream aligners produce output in SAM (text-based) or BAM
format. A SAM file is a text file, with one line per aligned read, and
fields separated by tabs. Here is an example of a single SAM line,
split into fields.
<<SAM>>=
fl <- system.file("extdata", "ex1.sam", package="Rsamtools")
strsplit(readLines(fl, 1), "\t")[[1]]
@

Fields in a SAM file are summarized in Table~\ref{tbl:sam}.
\begin{table}
  \centering
  \caption{Fields in a SAM record. From
    \url{http://samtools.sourceforge.net/samtools.shtml}}
  \medskip
  \label{tbl:sam}
  \begin{tabular}{lll}
    Field & Name & Value \\\hline
    1 & QNAME & Query (read) NAME \\
    2 & FLAG & Bitwise FLAG, e.g., strand of alignment \\
    3 & RNAME & Reference sequence NAME \\
    4 & POS & 1-based leftmost POSition of sequence \\
    5 & MAPQ & MAPping Quality (Phred-scaled) \\
    6 & CIGAR & Extended CIGAR string \\
    7 & MRNM & Mate Reference sequence NaMe \\
    8 & MPOS & 1-based Mate POSition \\
    9 & ISIZE & Inferred insert SIZE \\
    10 & SEQ & Query SEQuence on the reference strand \\
    11 & QUAL & Query QUALity \\
    12$+$ & OPT & OPTional fields, format TAG:VTYPE:VALUE \\\hline
  \end{tabular}
\end{table}
We recognize from the FASTQ file the identifier string, read sequence
and quality. The alignment is to a chromosome `seq1' starting at
position 1. The strand of alignment is encoded in the `flag' field.
The alignment record also includes a measure of mapping quality, and a
CIGAR string describing the nature of the alignment. In this case, the
CIGAR is 36M, indicating that the alignment consisted of 36
\texttt{M}atches or mismatches, with no indels or gaps; indels
(relative to the reference) are represented by \texttt{I} and
\texttt{D}; gaps (e.g., from alignments spanning introns) by
\texttt{N}.

BAM files encode the same information as SAM files, but in a format
that is more efficiently parsed by software; BAM files are the primary
way in which aligned reads are imported in to \R{}. 

\paragraph{Aligned reads in \R{}}

The \Rfunction{readGappedAlignments} function (this is renamed
\Rfunction{readGalignments} in the `devel' version of \Bioconductor)
from the \Biocpkg{GenomicRanges} package reads essential information
from a BAM file in to \R. The result is an instance of the
\Rclass{GappedAlignments} class. The \Rclass{GappedAlignments} class
has been designed to allow useful manipulation of many reads (e.g., 20
million) under moderate memory requirements (e.g., 4 GB); strategies
for dealing with large data are discussed in Chaptar~\ref{chpt:large-data}.
<<readGappedAlignments>>=
alnFile <- system.file("extdata", "ex1.bam", package="Rsamtools")
aln <- readGappedAlignments(alnFile)
head(aln, 3)
@
%% 
The \Rfunction{readGappedAlignments} function takes an additional
argument, \Rfunarg{param}, allowing the user to specify regions of the
BAM file (e.g., known gene coordinates) and characteristics of the
alignments (e.g., `proper' pairs, as identified by the aligner) from
which to extract alignments.

A \Rclass{GappedAlignments} instance is like a data frame, but with
accessors as suggested by the column names. It is easy to query, e.g.,
the distribution of reads aligning to each strand, the width of reads,
or the cigar strings
<<GappedAlignments-accessors>>=
table(strand(aln))
table(width(aln))
head(sort(table(cigar(aln)), decreasing=TRUE))
@ 

\begin{Exercise}
  Use \Rfunction{bigdata}, \Rfunction{file.path} and \Rfunction{dir}
  to obtain file paths to the BAM files. These are a subset of the
  aligned reads, overlapping just four genes.
  
  Input the aligned reads from one file using
  \Rfunction{readGappedAlignments}. Explore the reads, e.g., using
  \Rfunction{table} or \Rfunction{xtabs}, to summarize which chromosome
  and strand the subset of reads is from.
  
  The object \Robject{ex} is created in a later exercises, but we will
  use it now. It contains coordinates of four genes. Use
  \Rfunction{countOverlaps} to first determine the number of genes an
  individual read aligns to, and then the number of uniquely aligning
  reads overlapping each gene. Since the RNAseq protocol was not
  strand-sensitive, set the strand of \Robject{aln} to \code{*}.
  
  Write the sequence of steps required to calculate counts as a simple
  function, and calculate counts on each file. On Mac or Linux, can
  you easily parallelize this operation?
\end{Exercise}

\begin{Solution}
  We discover the location of files using standard \R{} commands:
<<bam-ex-fls>>=
bigdata <- "~/BigData"
fls <- dir(file.path(bigdata, "bam"), ".bam$", full=TRUE) #$
names(fls) <- sub("_subset.bam", "", basename(fls))
@ 
Use \Rfunction{readGappedAlignments} to input data from one of the
files, and standard \R{} commands to explore the data.
<<bam-ex-input>>=
## input
aln <- readGappedAlignments(fls[1])
xtabs( ~ seqnames + strand, as.data.frame(aln))
@
To count overlaps in regions defined in a previous exercise, load
the regions.
<<bam-ex-roi>>=
data(ex)             # from a later exercise
@ 
Many RNA-seq protocols are not strand aware, i.e., reads align to the
plus or minus strand regardless of the strand on which the
corresponding gene is encoded. Adjust the strand of the aligned reads
to indicate that the strand is not known.
<<bam-ex-strand>>=
strand(aln) <- "*"   # protocol not strand-aware
@ 

One important issue when counting reads is to make sure that the
reference names in both the annotation and the read files are
identical.
\begin{Exercise}
  Check the reference name in both the \Robject{ex} and
  \Robject{aln}. If they are not similar, how could you correct them?  
\end{Exercise}

For simplicity, we are interested in reads that align to only a single
gene. Count the number of genes a read aligns to\ldots
<<bam-ex-hits>>=
hits <- countOverlaps(aln, ex)
table(hits)
@ 
%% 
Subset the aligned reads to contain just those aligning once,
\Rcode{aln[hits == 1]}.  Finally, reverse the counting operation to
count the number of times each region of interest aligns to a uniquely
overlapping alignment.
<<bam-ex-cnt>>=
cnt <- countOverlaps(ex, aln[hits==1])
@ 
%% 
A simple function for counting reads (see
\Rfunction{GenomicRangese::summarizeOverlaps} for more realistic
counting schemes) is
<<bam-count-fun>>=
counter <- 
    function(filePath, range)
{
    aln <- readGappedAlignments(filePath)
    strand(aln) <- "*"
    hits <- countOverlaps(aln, range)
    countOverlaps(range, aln[hits==1])
}
@ 
This can be applied to all files using \Rfunction{sapply}
<<bam-count-all>>=
counts <- sapply(fls, counter, ex)
@ 
The counts in one BAM file are independent of counts in another BAM
file. This encourages us to count reads in each BAM file in parallel,
decreasing the length of time required to execute our program. On
Linux and Mac OS, a straight-forward way to parallelize this operation
is:
<<bam-count-mclapply, eval=FALSE>>=
if (require(parallel))
    simplify2array(mclapply(fls, counter, ex))
@ 
\end{Solution}


