%\VignetteIndexEntry{B.5 Called Variants: Manipulation and Annotation}

\SweaveOpts{keep.source=TRUE}
<<setup, echo=FALSE>>=
options(digits=2)
library(VariantAnnotation)
library(ggplot2)
library(SNPlocs.Hsapiens.dbSNP.20101109)
library(TxDb.Hsapiens.UCSC.hg19.knownGene)
library(BSgenome.Hsapiens.UCSC.hg19)
library(SIFT.Hsapiens.dbSNP132)
@ 

\chapter{Called Variants: Manipulation and Annotation}
\label{chpt:VariantAnnotation}

\section{Variants}

\subsection{Work flows}

The term `variant' applies to a number of different possible analyses:
\begin{itemize}
\item Single nucleotide polymorphism
\item Copy number change
\item Structural variation
\item Long-range interaction
\end{itemize}
Common steps in a variant analysis work flow include:
\begin{itemize}
\item Alignment. requires tools sensitive to variation, e.g.,
  \software{GSNAP}, \software{BWA}; \software{Bowtie} not optimized
  for this.
\item Variant calling, e.g.,
  \software{GATK}\footnote{\url{http://www.broadinstitute.org/gatk/}}
  or \Biocpkg{VariantTools}.
\item Filtering and manipulation.
\item Biological context -- variant annotation.
\item Integrative analysis, e.g., GWAS, genetical genomics, regulatory
  motifs.
\end{itemize}
This chapter explores filtering, manipulation and biological context
of variants; subsequent chapters delve into variant calling work flows
related to tumor / normal pairs (not covered in depth), and to
regulatory motifs.

\subsection{\Bioconductor{} software}

Selected \Bioconductor{} software relevant to DNA-seq work flows are
summarized in Table~\ref{tab:dnaseq}. The \Biocpkg{gmapR} package
provides access to the well-respected \software{GSNAP}
aligner. \Biocpkg{VariantTools} is an emerging tool that works with
aligned reads to call single-sample and sample-specific variants; we
will work with \Biocpkg{VariantTools} as part of this course. Packages
such as \Biocpkg{cn.mops} and \Biocpkg{exomeCopy} identify copy number
variants from high-throughput sequence data.

\Biocpkg{VariantAnnotation} provides facilities for manipulated called
variants stored in VCF files; the facilities are very flexible,
including simple range-based filtering, look-up in dbSNP, and coding
and effect prediction using standard data
bases. \Biocpkg{VariantAnnotation} plays well with
\Biocpkg{ensemblVEP} to feed data through the Ensembl Variant Effect
Predictor perl script, and to \Bioconductor{} facilities for SNP
analysis and genetical genomics like \Biocpkg{snpStats} and
\Biocpkg{GGtools}.

\begin{table}
  \centering
  \caption{Selected \Bioconductor{} packages for DNA-seq analysis.}
  \label{tab:dnaseq}
  \begin{tabular}{ll}
    Package & Description \\\hline
    \Biocpkg{VariantAnnotation} & Manipulating and annotating VCF files \\
    \Biocpkg{ensemblVEP} & Interface to the Ensembl Variant Effect
    Predictor \\

    \Biocpkg{VariantTools} & Single-sample and tumor specific variant calls \\
    \Biocpkg{gmapR} & Alignment (Linux only) \\
    
    \Biocpkg{deepSNV} & Sub-clonal SNVs in deep sequencing experiments \\
    \Biocpkg{cn.mops} & Mixture of Poissons copy number variation estimates \\
    \Biocpkg{exomeCopy} & Hidden Markov copy number variation estimates \\

    \Biocpkg{snpStats} & Down-stream GWAS; also \Biocpkg{GWAStools},
    \Biocpkg{GGtools} \\
    \hline
  \end{tabular}
\end{table}

\section{\Biocpkg{VariantAnnotation} and VCF Files}

A major product of DNASeq experiments are catalogs of called variants
(e.g., SNPs, indels).  We will use the \Biocpkg{VariantAnnotation}
package to explore this type of data.  Sample data included in the
package are a subset of chromosome 22 from the
\href{ftp://ftp-trace.ncbi.nih.gov/1000genomes/ftp/release/20110521/}{1000
  Genomes} project. Variant Call Format (VCF, see\footnote{\url{http://www.1000genomes.org/wiki/Analysis/Variant\%20Call\%20Format/vcf-variant-call-format-version-41}})
%% 
text files contain meta-information lines, a header line with
column names, data lines with information about a position in the
genome, and optional genotype information on samples for each
position.

Important operations on VCF files available with the
\Biocpkg{VariantAnnotation} package are summarized in
Table~\ref{tab:vcf-ops}.

\begin{table}
  \centering
  \caption{Working with \Rclass{VCF} files and data.} 
  \label{tab:vcf-ops}
  \begin{tabular}{lll}
    Category & Function & Description \\
    \hline

    Read & \Rfunction{scanVcfHeader} & Retrieve file header information \\
            & \Rfunction{scanVcfParam} & Select fields to read in \\
            & \Rfunction{readVcf} & Read VCF file into a VCF class \\
            & \Rfunction{scanVcf} & Read VCF file into a list \\
            & \Rfunction{readInfo}$*$ & Read a single `info' column \\
            & \Rfunction{readGeno}$*$ & Read a single `geno' matrix \\
            & \Rfunction{readGT}$*$ & Read `GT' (genotype) fields as matrix \\ 
    Filter  & \Rfunction{filterVcf} & Filter a VCF from one file to another \\
    Write   & \Rfunction{writeVcf} & Write a VCF file to disk \\

    Annotate & \Rfunction{locateVariants} & Identify
        where variant overlaps a gene annotation \\
    & \Rfunction{predictCoding} & Amino acid changes
        for variants in coding regions \\
    & \Rfunction{summarizeVariants} & Summarize variant counts
        by sample \\

    SNPs & \Rfunction{genotypeToSnpMatrix} & Convert genotypes to a SnpMatrix \\
         & \Rfunction{GLtoGP} & Convert genotype likelihoods to genotypes \\ 
         & \Rfunction{snpSummary} & Counts and distribution statistics
         for SNPs \\

    Manipulate & \Rfunction{expand} & Convert CompressedVCF to ExpandedVCF \\
    & \Rfunction{cbind}, \Rfunction{rbind} & Combine by column or row \\
    \hline
    \multicolumn{3}{l}{$*$: available in the `devel' version of \Bioconductor}
  \end{tabular}
\end{table}

\begin{Exercise}
  The objective of this exercise is to compare the quality of called
  SNPs that are located in dbSNP, versus those that are novel.

  Locate the sample data in the file system. Explore the metadata
  (information about the content of the file) using
  \Rfunction{scanVcfHeader}. Discover the `info' fields \Rcode{VT}
  (variant type), and \Rcode{RSQ} (genotype imputation quality).

  Input the sample data using \Rfunction{readVcf}. You'll need to
  specify the genome build (\Rfunarg{genome="hg19"}) on which the
  variants are annotated. Take a peak at the \Rfunction{rowData} to
  see the genomic locations of each variant.

  Data resources often adopt different naming conventions for
  sequences. For instance, dbSNP uses abbreviations such as
  \texttt{ch22} to represent chromosome 22, whereas our VCF file uses
  \texttt{22}. Use \Rfunction{rowData} and \Rfunction{seqlevels<-}
  to extract the row data of the variants, and rename the chromosomes.
\end{Exercise}

\begin{Solution}
%%
  Explore the header:
<<readVcf>>=
library(VariantAnnotation)
fl <- system.file("extdata", "chr22.vcf.gz", package="VariantAnnotation")
(hdr <- scanVcfHeader(fl))
info(hdr)[c("VT", "RSQ"),]
@
%%
  Input the data and peak at their locations:
<<readVcf>>=
(vcf <- readVcf(fl, "hg19"))
head(rowData(vcf), 3)
@
%%
  Rename chromosome levels:
<<renameSeqlevels>>=
seqlevels(vcf, force=TRUE)  <- c("22"="ch22")
@
\end{Solution}

\section{Large-scale filtering}

This section was developed from initial work by Paul Shannon.

One source of VCF files is from whole-genome sequencing and variant
calling; an example data set is from Complete Genomics, and a subset
is available in the \texttt{BigData} folder accompanying this course
<<vcf-AMI, eval=FALSE>>=
vcfDir <- "~/BigData/CompleteGenomics"
dir(vcfDir)
vcfFile <- dir(vcfDir, ".gz$", full=TRUE)
stopifnot(length(vcfFile) == 1)
@ 
%% 
The data were retrieved from the Complete Genomics web site, the first
750,000 of about 14 million variants selected, and the resulting file
compressed and indexed. Indexing makes them accessible to fast queries
using the \Rfunarg{which} argument to \Rfunction{ScanVcfParam}.

\begin{Exercise}
  The objective of this exercise is to filter the larger VCF file to a
  subset of interesting variants that we might wish to study in depth
  at a later date. We use the \Rfunction{filterVcf} function of the
  \Biocpkg{VariantAnnotation} package to perform the filtering.
  
  Start by taking a look at the (complicated) header information
<<eval=FALSE>>=
hdr <- scanVcfHeader(vcfFile)
@ 
%% 
We'll be paying attention to the \texttt{SS} INFO field, and the
\texttt{AD} GENO field. Determine the data types and possible values
for these fields, using commands like
<<eval=FALSE>>=
info(hdr)
geno(hdr)
@ 
%%
What data type would you use to represent the \texttt{SS} field for a
single or several VCF records in \R? What about the \texttt{AD} field,
across all samples?

This VCF file is big. While we can read this into memory all at once,
we will often want to `chunk' through a file, reading many (e.g., a
million) records at a time. We do this by creating a
\Rclass{TabixFile} and specifying a \Rfunarg{yieldSize} representing
the size of the chunk that we would like to read at each iteration. Create a
\Rclass{TabixFile} with \Rfunarg{yieldSize=100000} and verify with a
simple loop that the entire file appears to be processed in chunks of
the specified size, along the lines of\ldots
<<chunks, eval=FALSE>>=
tbx <- TabixFile(vcfFile, yieldSize=10000)
open(tbx)
while (len <- nrow(readVcf(tbx, "hg19")))
    cat("read", len, "rows\n")
@ 
%% 
As you can see from the chunking exercise, it takes quite a bit of
time to process these lines. To filter 14 million variants
effectively, it can pay to do a cheaper `pre-filter'. Specifically,
we're interested in variants tagged `Germline'. If we were to represents
each VCF record as an element of a character vector \Robject{x}, then
we could write a one-liner that returned \Robject{TRUE} if the line
contained the word `Germline':
<<germline-1, eval=FALSE>>=
grepl("Germline", x, fixed=TRUE)
@ 
%% 
This would be fast to read in to \R{}, and fast to perform the
filter. The \Rfunction{filterVcf} function allows us to specify a
pre-filter that works just like this. The filters are constructed
using \Rclass{FilterRules} in \Biocpkg{IRanges}, by
translating our one-liner into a simple function call, and placing the
function call into a list.
<<FitlerRules>>=
isGermline <- function(x)
    grepl("Germline", x, fixed=TRUE)
filters <- FilterRules(list(isGermline=isGermline))
@
%%
The idea is that several filters are chained together. Each filter
returns a logical vector indicating the subset of data to be processed
by the next filter.

Here is our pre-filter in action:
<<prefilter, eval=FALSE>>=
destination <- tempfile()               # temporary location
filterVcf(vcfFile, "hg19", destination, prefilters=filters)
@ 
%% 
This is pretty fast, and drops the number of variants under
consideration quite substantially, to about 110000.

Our next filter is more challenging to write. We're interested in
allelic depth, a summary of the evidence for the variant summarized in
the \texttt{AD} GENO field. There are many variants, each sample has
two values of AD, and there are two samples. This means that
\texttt{AD} is a three-dimensional array. Our filter criteria is that
the ratio of (`alternate allele' of the tumor sample \emph{or} the
`reference allele' of the the normal sample) to total reads is greater
than $0.1$. Here is a function implementing this:
<<allelicDepth>>=
allelicDepth <- function(x) 
{
    ##  ratio of AD of the 'alternate allele' for the tumor sample
    ##  OR 'reference allele' for normal samples to total reads for
    ##  the sample should be greater than some threshold (say 0.1,
    ##  that is: at least 10% of the sample should have the allele
    ##  of interest)
    ad <- geno(x)[["AD"]]
    tumorPct <- ad[,1,2,drop=FALSE] / rowSums(ad[,1,,drop=FALSE])
    normPct <- ad[,2,1, drop=FALSE] / rowSums(ad[,2,,drop=FALSE])
    test <- (tumorPct > 0.1) | (normPct > 0.1)
    !is.na(test) & test
}
@ 
We can add it to our list of filters
<<filters>>=
filters <- FilterRules(list(isGermline=isGermline, 
                            allelicDepth=allelicDepth))
@ 
%% 
To use this filter, we actually need to fully parse the VCF file into
the \Rclass{VCF} instance; the pre-filter trick used for germline
filtering is not enough. \Rfunction{filterVcf} allows us to perform a
filter on the VCF instance, too, and does so after pre-filtering
<<filterVcf, eval=FALSE>>=
destination <- tempfile()
filterVcf(vcfFile, "hg19", destination, prefilters=filters[1],
          filters=filters[2])
@
%% 
And finally input our interesting variants, confirming that we've
done our filtering as desired.
<<filterVariant-final, eval=FALSE>>=
vcf <- readVcf(destination, "hg19")
all(info(vcf)$SS == "Germline")
table(allelicDepth(vcf))
@ 
\end{Exercise}

\section{SNP Annotation}

This section is derived from a vignette by Valerie Obenchain in the
\Biocpkg{VariantAnnotation} package.

Variants can be easily identified according to region such as
\Rcode{coding}, \Rcode{intron}, \Rcode{intergenic}, \Rcode{spliceSite}
etc. Amino acid coding changes are computed for the non-synonymous
variants. SIFT and PolyPhen databases provide predictions of how
severely the coding changes affect protein function. Additional
annotations are easily crafted using the \Biocpkg{GenomicRanges} and
\Biocpkg{GenomicFeatures} software in conjunction with \Bioconductor{}
and broader community annotation resources.

\begin{Exercise}
  The \Biocannopkg{SNPlocs.Hsapiens.dbSNP.20101109} contains
  information about SNPs in a particular build of dbSNP. Load the
  package. 
  
  Review the short helper function \Rfunction{isInDbSNP}, in the
  \Rpackage{Morgan2013} package, to query whether SNPs are in the data
  base (a version of this function will be introduced to
  \Biocpkg{VariantAnnotation} before the next release)
<<isInDbSNP>>=
library(Morgan2013)
isInDbSNP
@ 

  Create a data frame containing the dbSNP membership status and
  imputation quality of each SNP. Create a density plot to illustrate
  the results.
\end{Exercise}

\begin{Solution}
  Discover whether SNPs are located in dbSNP, using our helper
  function.
<<dbSNP, eval=FALSE>>=
library(SNPlocs.Hsapiens.dbSNP.20101109)
inDbSNP <- .isInDbSNP(vcf, "ch22")
table(inDbSNP)
@
%%
  Create a data frame summarizing SNP quality and dbSNP membership:
<<SNP-quality, eval=FALSE>>=
metrics <-
    data.frame(inDbSNP=inDbSNP, RSQ=info(vcf)$RSQ)
@
%%
  Finally, visualize the data, e.g., using \Rpackage{ggplot2}
  (Figure~\ref{fig:variant-annotation:RSQ}).
<<RSQ-plot, fig=TRUE, include=FALSE, eval=FALSE>>=
library(ggplot2)
ggplot(metrics, aes(RSQ, fill=inDbSNP)) +
    geom_density(alpha=0.5) +
    scale_x_continuous(name="MaCH / Thunder Imputation Quality") +
    scale_y_continuous(name="Density") +
    theme(legend.position="top")
@

\begin{figure}
  \centering
  %% \includegraphics[width=.5\textwidth]{VariantAnnotation-RSQ-plot}
  \caption{Quality scores of variants in dbSNP, compared to those not
    in dbSNP.}
  \label{fig:variant-annotation:RSQ}
\end{figure}
\end{Solution}

\subsection{Variants in and around genes}

Variant location with respect to genes can be identified with the
\Rfunction{locateVariants} function. Regions are specified in the
\Rfunarg{region} argument and can be one of the following constructors:
\Rcode{CodingVariants()}, \Rcode{IntronVariants()},
\Rcode{FiveUTRVariants()}, \Rcode{ThreeUTRVariants()},
\Rcode{IntergenicVariants()}, \Rcode{SpliceSiteVariants()}, or
\Rcode{AllVariants()}. Location definitions are shown in
Table~\ref{table:location}.
\begin{table}
  \centering
  \caption{Variant locations}
  \label{table:location}
  \begin{tabular}{lp{.7\textwidth}}
    \hline 
    Location & Details \\ 
    \hline 
    \Rcode{coding} &  Within a coding region \\ 
    \Rcode{fiveUTR} &  Within a 5' untranslated region \\
    \Rcode{threeUTR} &   Within a 3' untranslated region \\
    \Rcode{intron} &   Within an intron region \\
    \Rcode{intergenic} & Not within a transcript associated with a gene \\
    \Rcode{spliceSite} & Overlaps any of the first or last 2 nucleotides of an
    intron \\
    \hline
  \end{tabular}
\end{table}

\begin{Exercise}
  Load the \Biocannopkg{TxDb.Hsapiens.UCSC.hg19.knownGene} annotation
  package, and read in the \texttt{chr22.vcf.gz} example file from the
  \Biocpkg{VariantAnnotation} package.
  
  Remembering to re-name sequence levels, use the
  \Rfunction{locateVariants} function to identify coding variants.
  
  Summarize aspects of your data, e.g., did any coding variants match
  more than one gene? How many coding variants are there per gene ID?
\end{Exercise}

\begin{Solution}
  Here we open the known genes data base, and read in the VCF file.
%% 
<<seqlevels_rename>>=
library(TxDb.Hsapiens.UCSC.hg19.knownGene)
txdb <- TxDb.Hsapiens.UCSC.hg19.knownGene 

fl <- system.file("extdata", "chr22.vcf.gz", package="VariantAnnotation")
vcf <- readVcf(fl, "hg19")
seqlevels(vcf, force=TRUE) <- c("22"="chr22")
@ 
%%
  The next lines locate coding variants.
<<locateVariants_find>>=
rd <- rowData(vcf)
loc <- locateVariants(rd, txdb, CodingVariants())
head(loc, 3)
@ 
% 
  To answer gene-centric questions data can be summarized by gene
  regardless of transcript.
%% 
<<locateVariants_example>>=
## Did any coding variants match more than one gene?
splt <- split(loc$GENEID, loc$QUERYID)
table(sapply(splt, function(x) length(unique(x)) > 1))

## Summarize the number of coding variants by gene ID
splt <- split(loc$QUERYID, loc$GENEID)
head(sapply(splt, function(x) length(unique(x))), 3)
@
\end{Solution}

\subsection{Amino acid coding changes}

\Rfunction{predictCoding} computes amino acid coding changes for
non-synonymous variants. Only ranges in \Rcode{query} that overlap
with a coding region in \Rcode{subject} are considered. Reference
sequences are retrieved from either a \Robject{BSgenome} or fasta file
specified in \Rcode{seqSource}.  Variant sequences are constructed by
substituting, inserting or deleting values in the \Robject{varAllele}
column into the reference sequence. Amino acid codes are computed for
the variant codon sequence when the length is a multiple of 3.

The \Rfunarg{query} argument to \Rfunction{predictCoding} can be a
\Robject{GRanges} or \Robject{VCF}. When a \Robject{GRanges} is
supplied the \Rfunarg{varAllele} argument must be specified. In the
case of a \Robject{VCF} object, the alternate alleles are taken from
\Rcode{alt(<VCF>)} and the \Rfunarg{varAllele} argument is not
specified.

The result is a modified \Rcode{query} containing only variants that
fall within coding regions. Each row represents a variant-transcript
match so more than one row per original variant is possible.

<<predictCoding>>=
library(BSgenome.Hsapiens.UCSC.hg19)
coding <- predictCoding(vcf, txdb, seqSource=Hsapiens)
coding[5:9]
@
%% 
Using variant rs114264124 as an example, we see \Rcode{varAllele}
\Rcode{A} has been substituted into the \Rcode{refCodon} \Rcode{CGG}
to produce \Rcode{varCodon} \Rcode{CAG}. The \Rcode{refCodon} is the
sequence of codons necessary to make the variant allele substitution
and therefore often includes more nucleotides than indicated in the
range (i.e. the range is 50302962, 50302962, width of 1).  Notice it
is the second position in the \Rcode{refCodon} that has been
substituted. This position in the codon, the position of substitution,
corresponds to genomic position 50302962. This genomic position maps
to position 698 in coding region-based coordinates and to triplet 233
in the protein. This is a non-synonymous coding variant where the
amino acid has changed from \Rcode{R} (Arg) to \Rcode{Q} (Gln).

When the resulting \Rcode{varCodon} is not a multiple of 3 it cannot
be translated. The consequence is considered a \Rcode{frameshift} and
\Robject{varAA} will be missing.

<<predictCoding_frameshift>>=
coding[coding$CONSEQUENCE == "frameshift"]
@

\subsection{SIFT and PolyPhen databases}

From \Rfunction{predictCoding} we identified the amino acid coding
changes for the non-synonymous variants. For this subset we can
retrieve predictions of how damaging these coding changes may be. SIFT
(Sorting Intolerant From Tolerant) and PolyPhen (Polymorphism
Phenotyping) are methods that predict the impact of amino acid
substitution on a human protein. The SIFT method uses sequence
homology and the physical properties of amino acids to make
predictions about protein function. PolyPhen uses sequence-based
features and structural information characterizing the substitution to
make predictions about the structure and function of the protein.

Collated predictions for specific dbSNP builds are available as
downloads from the SIFT and PolyPhen web sites. These results have
been packaged into \Rpackage{SIFT.Hsapiens.dbSNP132.db} and
\Rpackage{PolyPhen.Hapiens.dbSNP131.db} and are designed to be
searched by rsid. Variants that are in dbSNP can be searched with
these database packages. When working with novel variants, SIFT and
PolyPhen must be called directly. See references for home pages.

Identify the non-synonymous variants and obtain the rsids.
<<nonsynonymous>>=
nms <- names(coding)
idx <- coding$CONSEQUENCE == "nonsynonymous"
nonsyn <- coding[idx]
names(nonsyn) <- nms[idx]
rsids <- unique(names(nonsyn)[grep("rs", names(nonsyn), fixed=TRUE)])
@
%% 
Detailed descriptions of the database columns can be found with
\Rcode{?SIFTDbColumns} and \Rcode{?PolyPhenDbColumns}. Variants in
these databases often contain more than one row per variant. The
variant may have been reported by multiple sources and therefore the
source will differ as well as some of the other variables.

<<sift>>=
library(SIFT.Hsapiens.dbSNP132)

## rsids in the package 
head(keys(SIFT.Hsapiens.dbSNP132), 3)
## list available columns
cols(SIFT.Hsapiens.dbSNP132)
## select a subset of columns
## a warning is thrown when a key is not found in the database
subst <- c("RSID", "PREDICTION", "SCORE", "AACHANGE", "PROTEINID")
sift <- select(SIFT.Hsapiens.dbSNP132, keys=rsids, cols=subst)
head(sift, 3)
@

PolyPhen provides predictions using two different training datasets
and has considerable information about 3D protein structure. See
\Rcode{?PolyPhenDbColumns} or the PolyPhen web site listed in the
references for more details.

\section{Annotation with \Biocpkg{ensemblVEP}}

\WARNING{This section requires an internet connection and additional
  software, and has been removed from the lab}. As an alternative,
consider working through the \Biocpkg{ensemblVEP} vignette, available
from the \Bioconductor{} web
site\footnote{\url{http://bioconductor.org/packages/release/bioc/html/ensemblVEP.html}}. This
section requires that \software{perl} is installed, and that the
Ensembl VEP perl script is available; see instructions in the
\Biocpkg{ensemblVEP}
README\footnote{\url{http://bioconductor.org/packages/release/bioc/readmes/ensemblVEP/README}}. The
necessary preconditions are tested by the following code;
\Rcode{VEP\_OK} should be \Rcode{TRUE}.

<<ensemblVEP-setup>>=
CONNECTION_OK <- tryCatch({ 
    con <- url("http://google.com", "r"); close(con); TRUE
}, error=function(...) FALSE)

VEP_OK <- CONNECTION_OK && require(ensemblVEP) && tryCatch({
    file.exists(ensemblVEP:::.getVepPath())
}, error=function(...) FALSE)
stopifnot(VEP_OK)
@ 

Functionality of the Ensembl VEP is given
on-line\footnote{\url{http://www.ensembl.org/info/docs/variation/vep/}}
and in \cite{mclaren2010deriving}.
