%\VignetteIndexEntry{B.2 Genomic Annotation}

\chapter{Genomic Annotation}
\label{chpt:GenomicAnnotation}

\section{\Biocpkg{AnnotationHub}}

\WARNING{This section requires an internet connection, and has been
  removed from the lab.}

\section{Whole genome sequences}

There are a diversity of packages and classes available for
representing large genomes. Several include:
\begin{packeddescription}
\item [\Rpackage{TxDb.*}] For transcript and other genome / coordinate
  annotation.
\item [\Biocpkg{BSgenome}] For whole-genome representation. See
  \Rfunction{available.packages} for pre-packaged genomes, and the
  vignette `How to forge a BSgenome data package' in the
\item [\Biocannopkg{Homo.sapiens}] For integrating \Rpackage{TxDb*} and
  \Rpackage{org.*} packages.
\item [\Rpackage{SNPlocs.*}] For model organism SNP locations derived
  from dbSNP.
\item [\Rfunction{FaFile}] (\Biocpkg{Rsamtools}) for accessing indexed
  FASTA files.
\item [ \Rpackage{SIFT.*}, \Rpackage{PolyPhen}] Variant effect scores.
\end{packeddescription}

\section{Gene models}

\begin{figure}
  \centering
  \includegraphics{GeneModel}
  \caption{Gene model showing exons on three transcripts.}
  \label{fig:anno:genemodel}
\end{figure}

\subsection{\Rpackage{TxDb.*} packages for model organisms}

Genome-centric packages are very useful for annotations involving
genomic coordinates. It is straight-forward, for instance, to discover
the coordinates of coding sequences in regions of interest, and from
these retrieve corresponding DNA or protein coding sequences. Other
examples of the types of operations that are easy to perform with
genome-centric annotations include defining regions of interest for
counting aligned reads in RNA-seq experiments and retrieving DNA
sequences underlying regions of interest in ChIP-seq analysis, e.g.,
for motif characterization.

\begin{Exercise}
  Load the `transcript.db' package relevant to the dm3 build of
  \emph{D.\ melanogaster}. Use \Rfunction{select} and friends to
  select the Flybase gene ids of the top table \Robject{tt} and the
  Flybase transcript names (TXNAME) and Entrez gene identifiers
  (GENEID).
  
  Use \Rfunction{cdsBy} to extract all coding sequences, grouped by
  transcript. Subset the coding sequences to contain just the
  transcripts relevant to the top table. How many transcripts are
  there? What is the structure of the first transcript's coding
  sequence?
  
  Load the `BSgenome' package for the dm3 build of
  \emph{D.\ melanogaster}. Use the coding sequences ranges of the
  previous part of this exercise to extract the underlying DNA
  sequence, using the \Rfunction{extractTranscriptsFromGenome}
  function. Use \Biocpkg{Biostrings}' \Rfunction{translate} to convert
  DNA to amino acid sequences.
\end{Exercise}

\begin{Solution}
  The following loads the relevant Transcript.db package, and creates
  a more convenient alias to the \Rclass{TranscriptDb} instance
  defined in the package.
<<txdb>>=
library(TxDb.Dmelanogaster.UCSC.dm3.ensGene)
txdb <- TxDb.Dmelanogaster.UCSC.dm3.ensGene
@ 
%% 

  We also need the data -- flybase IDs from our differential
  expression analysis.
<<de>>=
library(Morgan2013)
library(edgeR)
data(lrTest)
fbids <- rownames(topTags(lrTest))
@ 
%% 
  We can discover available keys (using \Rfunction{keys}) and columns
  (\Rfunction{cols}) in \Rcode{txdb}, and then use \Rfunction{select}
  to retrieve the transcripts associated with each differentially
  expressed gene. The mapping between gene and transcript is not
  one-to-one -- some genes have more than one transcript.
<<flybase-tx-gn>>=
txnm <- select(txdb, fbids, "TXNAME", "GENEID")
nrow(txnm)
head(txnm, 3)
@ 
%% 
  The \Rclass{TranscriptDb} instances can be queried for data that is
  more structured than simple data frames, and in particular return
  \Rclass{GRanges} or \Rclass{GRangesList} instances to represent
  genomic coordinates. These queries are performed using
  \Rfunction{cdsBy} (coding sequence), \Rfunction{transcriptsBy}
  (transcripts), etc., where the function argument \Rfunarg{by}
  specifies how coding sequences or transcripts are grouped.  Here we
  extract the coding sequences grouped by transcript, returning the
  transcript names, and subset the resulting \Rclass{GRangesList} to
  contain just the transcripts of interest to us. The first transcript
  is composed of 6 distinct coding sequence regions.
<<cdsBy>>=
cds <- cdsBy(txdb, "tx", use.names=TRUE)[txnm$TXNAME]
length(cds)
cds[1]
@ 
%% 

  The following code loads the appropriate BSgenome package; the
  \Robject{Dmelanogaster} object refers to the whole genome sequence
  represented in this package. The remaining steps extract the DNA
  sequence of each transcript, and translates these to amino acid
  sequences. Issues of strand are handled such that gene regions on
  the minus strand are reverse compelemented, so returned 
  in 5' to 3' orientation).
<<BSgenome>>=
library(BSgenome.Dmelanogaster.UCSC.dm3)
txx <- extractTranscriptsFromGenome(Dmelanogaster, cds)
length(txx)
head(txx, 3)
head(translate(txx), 3)
@ 
\end{Solution}

\section{UCSC tracks}

\subsection{Easily creating \Rclass{TranscriptDb} objects from GTF files}

\begin{Exercise}
  \WARNING{Internet connection required}
  (Adapted
  from\footnote{\url{http://www.sph.emory.edu/~hwu/teaching/bioc/bios560R.html}})
  The `track' curated annotations at UCSC are a great resource; this
  exercise creates a \Rclass{TranscriptDb} instance from one such
  track.
  \begin{enumerate}
  \item Load the \Biocpkg{GenomicFeatures} and \Biocpkg{rtracklayer}
    packages.
  \item Discover available genomes with \Rfunction{ucscGenomes}, and
    available tables with the \Rfunction{supportedUCSCtables}.
  \item Use the \Rfunction{makeTranscriptDbFromUCSC} from a suitable
    track, e.g., genome \Rfunarg{genome="ce2",
      tablename="refGene"}. There are some warnings; are these
    something to be concerned about?
  \item Exercise the object that you've created, e.g., exploring the
    basis of the warnings.
  \item Save and load the \Rclass{TranscriptDb} object you've created,
    illustrating how one can make these annotations convenient and
    reproducible.
  \item What's the difference between
    \Rfunction{makeTranscriptDbFromUCSC} and
    \Rfunction{makeFeatureDbFromUCSC}? Where else can transcript and
    feature data bases be made from?
  \end{enumerate}
\end{Exercise}
\begin{Solution}
  Load the \Rpackage{GenomicFeatures} and \Biocpkg{rtracklayer}
  packages and discover available genomes and tables:
<<makeTranscriptDbFromUCSC-discover>>=
library(rtracklayer)
library(GenomicFeatures)

## genomes
gnms <- ucscGenomes()
nrow(gnms)
gnms[grep("elegans", gnms$species),]

## tables
tbls <- supportedUCSCtables()
nrow(tbls)
head(tbls)
@ 
%% 
Make the \Rclass{TranscriptDb} object (this will take a minute)
<<makeTranscriptDbFromUCSC-make, eval=FALSE>>=
## Not run
txdb <- makeTranscriptDbFromUCSC("ce10", "refGene")
saveDb(txdb, file="/path/to/file.sqlite")
@ 
%% 
The warnings during object creation are about unusual lengths for CDS
(coding sequences should be in multiples of 3, since there are three
nucleotide residues per amino acid residue).
\begin{Schunk}
\begin{Soutput}
1: In .extractUCSCCdsStartEnd(cdsStart[i], cdsEnd[i], ... :
  UCSC data anomaly in transcript NM_001129046: the cds cumulative
  length is not a multiple of 3
\end{Soutput}
\end{Schunk}
but we seem to have a useful object with relevant metadata information
for reproducible research:
\begin{Schunk}
\begin{Sinput}
> txdb
\end{Sinput}
\begin{Soutput}
TranscriptDb object:
| Db type: TranscriptDb
| Supporting package: GenomicFeatures
| Data source: UCSC
| Genome: ce10
| Organism: Caenorhabditis elegans
| UCSC Table: refGene
| Resource URL: http://genome.ucsc.edu/
| Type of Gene ID: Entrez Gene ID
| Full dataset: yes
| miRBase build ID: NA
| transcript_nrow: 48714
| exon_nrow: 152542
| cds_nrow: 129947
| Db created by: GenomicFeatures package from Bioconductor
| Creation time: 2013-02-10 10:33:50 -0800 (Sun, 10 Feb 2013)
| GenomicFeatures version at creation time: 1.11.8
| RSQLite version at creation time: 0.11.2
| DBSCHEMAVERSION: 1.0
\end{Soutput}
\end{Schunk}
%% 
Let us investigate the source of the warnings -- what fraction of the
CDS have lengths that are not a multiple of 3? To do this we need to
determine the sum of the widths of the coding sequence exons in each
transcript. Start by extracting all coding sequence exons, grouped by
transcript; verify that this is a \Rclass{GRangesList} with a
reasonable number of entries.
\begin{Schunk}
\begin{Sinput}
> cdsByTx <- cdsBy(txdb, "tx", use.names=TRUE)
> length(cdsByTx)
\end{Sinput}
\begin{Soutput}
[1] 26146
\end{Soutput}
\begin{Sinput}
> cdsByTx[1:2]
\end{Sinput}
\begin{Soutput}
GRangesList of length 2:
$NM_058259 
GRanges with 3 ranges and 3 metadata columns:
      seqnames         ranges strand |    cds_id    cds_name exon_rank
         <Rle>      <IRanges>  <Rle> | <integer> <character> <integer>
  [1]     chrI [11641, 11689]      + |         1        <NA>         1
  [2]     chrI [14951, 15160]      + |         2        <NA>         2
  [3]     chrI [16473, 16585]      + |         3        <NA>         3

$NM_058264 
GRanges with 5 ranges and 3 metadata columns:
      seqnames         ranges strand | cds_id cds_name exon_rank
  [1]     chrI [43733, 43961]      + |      4     <NA>         1
  [2]     chrI [44030, 44234]      + |      5     <NA>         2
  [3]     chrI [44281, 44324]      + |      6     <NA>         3
  [4]     chrI [44372, 44468]      + |      7     <NA>         4
  [5]     chrI [44521, 44677]      + |      8     <NA>         5

---
seqlengths:
     chrI    chrII   chrIII    chrIV     chrV     chrX     chrM
 15072423 15279345 13783700 17493793 20924149 17718866    13794
\end{Soutput}
\end{Schunk}
%% 
Extract the width of each each exon; this is an \Rclass{IntegerList}
instance with one element of the list for each transcript, and one
integer value for each exon.
\begin{Schunk}
\begin{Sinput}
> wd <- width(cdsByTx)
> length(wd)
\end{Sinput}
\begin{Soutput}
[1] 26146
\end{Soutput}
\begin{Sinput}
> head(wd, 3)
\end{Sinput}
\begin{Soutput}
CompressedIntegerList of length 3
[["NM_058259"]] 49 210 113
[["NM_058264"]] 229 205 44 97 157
[["NM_001026606"]] 139 163 183 166
\end{Soutput}
\end{Schunk}
%% 
Use \Rfunction{sum} to add up the widths within each list element;
note that we're using the \Rmethod{sum,CompressedIntegerList-method},
and that this has been specialized to do the summation within list
elements.
\begin{Schunk}
\begin{Sinput}
> head(sum(wd))
\end{Sinput}
\begin{Soutput}
   NM_058259    NM_058264 NM_001026606    NM_058265 NM_001026607    NM_182094 
         372          732          651         1341         1620         1221 
\end{Soutput}
\end{Schunk}
%% 
We now have a standard \R{} vector; a one-liner asking about the
number of transcripts with exons that do not sum to 3 is
\begin{Schunk}
\begin{Sinput}
> table( (sum(width(cdsBy(txdb, "tx"))) %% 3) != 0 )
\end{Sinput}
\begin{Soutput}
FALSE  TRUE 
25676   470 
\end{Soutput}
\end{Schunk}
\end{Solution}

\begin{Exercise}
  (Adapted from\footnote{\url{http://www.sph.emory.edu/~hwu/teaching/bioc/bios560R.html}})
  Suppose you have a list of transcription factor binding sites on
  hg19. How would you obtain (a) the GC content of each site and (b)
  the percentage of gene promoters covered by the binding sites?
\end{Exercise}

\begin{Solution}
  As an outline of a solution, the steps for calculating GC content
  might be
  \begin{packedenumerate}
  \item Represent the list of transcription factor binding sites
    (`regions of interest') as a \Rclass{GRanges} instance,
    \Robject{roi}.
  \item Load \Rpackage{BSgenome} and the appropriate genome package,
    e.g., \Biocannopkg{BSgenome.Hsapiens.UCSC.hg19}.
  \item Use \Rfunction{getSeq} to retrieve the sequences, \Rcode{seqs
    <- getSeq(Hsapiens, gr)}.
  \item Use \Rcode{alphabetFrequency(seqs)} to summarize nucleotide
    use, and simple \R{} functions to determine GC content of each
    region of interest.
  \item Summarize these as density plots, etc. A meaningful extension
    of this exercise might compare the observed GC content to the
    expected content, where expectation is the product of the
    independent G and C frequencies.
  \end{packedenumerate}
  To calculate the percentage of promoters covered by binding sites,
  we might
  \begin{packedenumerate}
  \item Load the reference genome \Rpackage{TxDb} package,
    \Biocannopkg{TxDb.Hsapiens.UCSC.hg19.knownGene}.
  \item Query the package for promoters using the
    \Rfunction{promoters} function, or otherwise manipulating exon or
    transcript coordinates to get a \Rclass{GRanges} or
    \Rclass{GRangesList} representing genomic regions of interest,
    \Robject{groi}.
  \item Use \Rcode{countOverlaps(groi, roi)} to find how many
    transcription factor binding sites overlap each promoter, and from
    there use standard \R{} functions to tally the number of promoters
    that have zero overlaps.
  \end{packedenumerate}
\end{Solution}
