# Extending the SummarizedExperiment class

Aaron Lun

#### Revised: 30 October, 2018

# 1 Motivation

A large number of Bioconductor packages contain extensions of the standard `SummarizedExperiment` class from the SummarizedExperiment package. This allows developers to take advantage of the power of the `SummarizedExperiment` representation for synchronising data and metadata, while still accommodating specialized data structures for particular scientific applications. This document is intended to provide a developer-level “best practices” reference for the creation of these derived classes.

# 2 Deriving a simple class

## 2.1 Overview

To introduce various concepts, we will start off with a simple derived class that does not add any new slots. This is occasionally useful when additional constraints need to be placed on the derived class. In this example, we will assume that we want our class to minimally hold a `"counts"` assay that contains non-negative values11 1 For simplicity’s sake, we won’t worry about enforcing integer type, as fractional values are possible, e.g., when dealing with expected counts..

## 2.2 Defining the class and its constructor

We name our new class `CountSE` and define it using the `setClass` function from the methods package, as is conventionally done for all S4 classes. We use Roxygen’s `#'` tags to trigger the generation of import/export statements in the `NAMESPACE` of our package.

    #' @export
    #' @import methods
    #' @importClassesFrom SummarizedExperiment SummarizedExperiment
    .CountSE <- setClass("CountSE", contains="SummarizedExperiment")

We define a constructor that accepts a count matrix to create a `CountSE` object. We use `...` to pass further arguments to the `SummarizedExperiment` constructor, which allows us to avoid re-specifying all its arguments.

    #' @export
    #' @importFrom SummarizedExperiment SummarizedExperiment
    CountSE <- function(counts, ...) {
        se <- SummarizedExperiment(list(counts=counts), ...)
        .CountSE(se)
    }

## 2.3 Defining a validity method

We define a validity method that enforces the constraints that we described earlier. This is done by defining a validity function using `setValidity2` from the S4Vectors package22 2 This allows us to turn off the validity checks in internal functions where intermediate objects may not be valid within the scope of the function.. Returning a string indicates that there is a problem and triggers an error in the R session.

    setValidity2("CountSE", function(object) {
        msg <- NULL

        if (assayNames(object)[1] != "counts") {
            msg <- c(msg, "'counts' must be first assay")
        }

        if (min(assay(object)) < 0) {
            msg <- c(msg, "negative values in 'counts'")
        }

        if (is.null(msg)) {
            TRUE
        } else msg
    })

    ## Class "CountSE" [in ".GlobalEnv"]
    ##
    ## Slots:
    ##
    ## Name:            colData            assays             NAMES   elementMetadata
    ## Class:         DataFrame    Assays_OR_NULL character_OR_NULL         DataFrame
    ##
    ## Name:           metadata
    ## Class:              list
    ##
    ## Extends:
    ## Class "SummarizedExperiment", directly
    ## Class "RectangularData", by class "SummarizedExperiment", distance 2
    ## Class "Vector", by class "SummarizedExperiment", distance 2
    ## Class "Annotated", by class "SummarizedExperiment", distance 3
    ## Class "vector_OR_Vector", by class "SummarizedExperiment", distance 3

The constructor yields the expected output when counts are provided:

    CountSE(matrix(rpois(100, lambda=1), ncol=5))

    ## class: CountSE
    ## dim: 20 5
    ## metadata(0):
    ## assays(1): counts
    ## rownames: NULL
    ## rowData names(0):
    ## colnames: NULL
    ## colData names(0):

… and an (expected) error otherwise:

    CountSE(matrix(rnorm(100), ncol=5))

    ## Error in validObject(.Object): invalid class "CountSE" object:
    ##     negative values in 'counts'

## 2.4 Defining a getter method

A generic is a group of functions with the same name that operate on different classes. Upon calling the generic on an object, the S4 dispatch system will choose the most appropriate function to use based on the object class. This allows users and developers to write code that is agnostic to the type of input class.

Let’s say that it is of particular scientific interest to obtain the counts with a flipped sign. We observe that there are no existing generics that do this task, e.g., in BiocGenerics or S4Vectors 33 3 If you have an idea for a generally applicable generic that is not yet available, please contact the Bioconductor core team.. Instead, we define a new generic `negcounts`:

    #' @export
    setGeneric("negcounts", function(x, ...) standardGeneric("negcounts"))

    ## [1] "negcounts"

We then define a specific method for our `CountSE` class44 4 The `...` in the generic function definition means that custom arguments like `withDimnames=` can be provided for specific methods, if necessary.:

    #' @export
    #' @importFrom SummarizedExperiment assay
    setMethod("negcounts", "CountSE", function(x, withDimnames=TRUE) {
        -assay(x, withDimnames=withDimnames)
    })

If any other developers need to compute negative counts for their own classes, they can simply use the `negcounts` generic defined in our package.

## 2.5 Some comments on package organization

It is convention to put all class definitions (i.e., the `setClass` statement) in a file named `AllClasses.R`, all new generic definitions in a file named `AllGenerics.R`, and all method definitions in files that are alphanumerically ordered below the first two. This is because R collates files by alphanumeric order when building a package. It is critical that the collation (and definition) of the classes and generics occurs before that of the corresponding methods, otherwise errors will occur. If alphanumeric ordering is inappropriate, developers can manually specify the collation order using `Collate:` in the `DESCRIPTION` file - see Writing R Extensions for more details.

# 3 Deriving a class with custom slots

## 3.1 Class definition

In practice, most derived classes will need to store application-specific data structures. For the rest of this document, we will be considering the derivation of a class with custom slots to hold such structures. First, we consider 1D data structures:

- `rowVec`: 1:1 mapping from each value to a row of the `SummarizedExperiment`.
- `colVec`: 1:1 mapping from each value to a column of the `SummarizedExperiment`.

Any 1D structure can be used if it supports `length`, `c`, `[` and `[<-`. For simplicity, we will use integer vectors for the `*.vec` slots.

We also consider some 2D data structures:

- `rowToRowMat`: 1:1 mapping from each row to a row of the `SummarizedExperiment`.
- `colToColMat`: 1:1 mapping from each column to a column of the `SummarizedExperiment`.
- `rowToColMat`: 1:1 mapping from each row to a column of the `SummarizedExperiment`.
- `colToRowMat`: 1:1 mapping from each column to a row of the `SummarizedExperiment`.

Any 2D structure can be used if it supports `nrow`, `ncol`, `cbind`, `rbind`, `[` and `[<-`. For simplicity, we will use (numeric) matrices for the `*.mat` slots.

Definition of the class is achieved using `setClass`, using the `slots=` argument to specify the new custom slots55 5 It does no harm to repeat the Roxygen tags, which explicitly specifies the imports required for each class and function..

    #' @export
    #' @import methods
    #' @importClassesFrom SummarizedExperiment SummarizedExperiment
    .ExampleClass <- setClass("ExampleClass",
        slots= representation(
            rowVec="integer",
            colVec="integer",
            rowToRowMat="matrix",
            colToColMat="matrix",
            rowToColMat="matrix",
            colToRowMat="matrix"
        ),
        contains="SummarizedExperiment"
    )

## 3.2 Defining the constructor

The constructor should provide some arguments for setting the new slots in the derived class definition. The default values should be set such that calling the constructor without any arguments returns a valid `ExampleClass` object.

    #' @export
    #' @importFrom SummarizedExperiment SummarizedExperiment
    ExampleClass <- function(
        rowVec=integer(0),
        colVec=integer(0),
        rowToRowMat=matrix(0,0,0),
        colToColMat=matrix(0,0,0),
        rowToColMat=matrix(0,0,0),
        colToRowMat=matrix(0,0,0),
        ...)
    {
        se <- SummarizedExperiment(...)
        .ExampleClass(se, rowVec=rowVec, colVec=colVec,
            rowToRowMat=rowToRowMat, colToColMat=colToColMat,
            rowToColMat=rowToColMat, colToRowMat=colToRowMat)
    }

## 3.3 Creating getter methods

### 3.3.1 For 1D data structures

We define some getter generics for the custom slots containing the 1D structures.

    #' @export
    setGeneric("rowVec", function(x, ...) standardGeneric("rowVec"))

    ## [1] "rowVec"

    #' @export
    setGeneric("colVec", function(x, ...) standardGeneric("colVec"))

    ## [1] "colVec"

We then define the class-specific methods for these generics. Note the `withDimnames=TRUE` argument, which enforces consistency between the names of the extracted object and the original `SummarizedExperiment`. It is possible to turn this off for greater efficiency, e.g., for internal usage where names are not necessary.

    #' @export
    setMethod("rowVec", "ExampleClass", function(x, withDimnames=TRUE) {
        out <- x@rowVec
        if (withDimnames)
            names(out) <- rownames(x)
        out
    })

    #' @export
    setMethod("colVec", "ExampleClass", function(x, withDimnames=TRUE) {
        out <- x@colVec
        if (withDimnames)
            names(out) <- colnames(x)
        out
    })

### 3.3.2 For 2D data structures

We repeat this process for the 2D structures.

    #' @export
    setGeneric("rowToRowMat", function(x, ...) standardGeneric("rowToRowMat"))

    ## [1] "rowToRowMat"

    #' @export
    setGeneric("colToColMat", function(x, ...) standardGeneric("colToColMat"))

    ## [1] "colToColMat"

    #' @export
    setGeneric("rowToColMat", function(x, ...) standardGeneric("rowToColMat"))

    ## [1] "rowToColMat"

    #' @export
    setGeneric("colToRowMat", function(x, ...) standardGeneric("colToRowMat"))

    ## [1] "colToRowMat"

Again, we define class-specific methods for these generics.

    #' @export
    setMethod("rowToRowMat", "ExampleClass", function(x, withDimnames=TRUE) {
        out <- x@rowToRowMat
        if (withDimnames)
            rownames(out) <- rownames(x)
        out
    })

    #' @export
    setMethod("colToColMat", "ExampleClass", function(x, withDimnames=TRUE) {
        out <- x@colToColMat
        if (withDimnames)
            colnames(out) <- colnames(x)
        out
    })

    #' @export
    setMethod("rowToColMat", "ExampleClass", function(x, withDimnames=TRUE) {
        out <- x@rowToColMat
        if (withDimnames)
            rownames(out) <- colnames(x)
        out
    })

    #' @export
    setMethod("colToRowMat", "ExampleClass", function(x, withDimnames=TRUE) {
        out <- x@colToRowMat
        if (withDimnames)
            colnames(out) <- rownames(x)
        out
    })

### 3.3.3 For `SummarizedExperiment` slots

The getter methods defined in SummarizedExperiment can be directly used to retrieve data from slots in the base class. These should generally not require any re-defining for a derived class. However, if it is necessary, the methods should use `callNextMethod` internally. This will call the method for the base `SummarizedExperiment` class, the output of which can be modified as required.

    #' @export
    #' @importMethodsFrom SummarizedExperiment rowData
    setMethod("rowData", "ExampleClass", function(x, ...) {
        out <- callNextMethod()

        # Do something extra here.
        out$extra <- runif(nrow(out))

        # Returning the rowData object.
        out
    })

## 3.4 Defining the validity method

We use `setValidity2` to define a validity function for `ExampleClass`. We use the previously defined getter functions to retrieve the slot values rather than using `@`. This is generally a good idea to keep the interface separate from the implementation66 6 This protects against changes to the slot names, and simplifies development when the storage mode differs from the conceptual meaning of the data, e.g., for efficiency purposes.. We also set `withDimnames=FALSE` in our getter calls, as consistent naming is not necessary for internal functions.

    #' @importFrom BiocGenerics NCOL NROW
    setValidity2("ExampleClass", function(object) {
        NR <- NROW(object)
        NC <- NCOL(object)
        msg <- NULL

        # 1D
        if (length(rowVec(object, withDimnames=FALSE)) != NR) {
            msg <- c(msg, "'rowVec' should have length equal to the number of rows")
        }
        if (length(colVec(object, withDimnames=FALSE)) != NC) {
            msg <- c(
                msg, "'colVec' should have length equal to the number of columns"
            )
        }

        # 2D
        if (NROW(rowToRowMat(object, withDimnames=FALSE)) != NR) {
            msg <- c(
                msg, "'nrow(rowToRowMat)' should be equal to the number of rows"
            )
        }
        if (NCOL(colToColMat(object, withDimnames=FALSE)) != NC) {
            msg <- c(
                msg, "'ncol(colToColMat)' should be equal to the number of columns"
            )
        }
        if (NROW(rowToColMat(object, withDimnames=FALSE)) != NC) {
            msg <- c(
                msg, "'nrow(rowToColMat)' should be equal to the number of columns"
            )
        }
        if (NCOL(colToRowMat(object, withDimnames=FALSE)) != NR) {
            msg <- c(
                msg, "'ncol(colToRowMat)' should be equal to the number of rows"
            )
        }

        if (length(msg)) {
            msg
        } else TRUE
    })

    ## Class "ExampleClass" [in ".GlobalEnv"]
    ##
    ## Slots:
    ##
    ## Name:             rowVec            colVec       rowToRowMat       colToColMat
    ## Class:           integer           integer            matrix            matrix
    ##
    ## Name:        rowToColMat       colToRowMat           colData            assays
    ## Class:            matrix            matrix         DataFrame    Assays_OR_NULL
    ##
    ## Name:              NAMES   elementMetadata          metadata
    ## Class: character_OR_NULL         DataFrame              list
    ##
    ## Extends:
    ## Class "SummarizedExperiment", directly
    ## Class "RectangularData", by class "SummarizedExperiment", distance 2
    ## Class "Vector", by class "SummarizedExperiment", distance 2
    ## Class "Annotated", by class "SummarizedExperiment", distance 3
    ## Class "vector_OR_Vector", by class "SummarizedExperiment", distance 3

We use the `NCOL` and `NROW` methods from BiocGenerics as these support various Bioconductor objects, whereas the base methods do not.

## 3.5 Creating a `show` method

The default `show` method will only display information about the `SummarizedExperiment` slots. We can augment it to display some relevant aspects of the custom slots. This is done by calling the base `show` method before printing additional fields as necessary.

    #' @export
    #' @importMethodsFrom SummarizedExperiment show
    setMethod("show", "ExampleClass", function(object) {
        callNextMethod()
        cat(
            "rowToRowMat has ", ncol(rowToRowMat(object)), " columns\n",
            "colToColMat has ", nrow(colToColMat(object)), " rows\n",
            "rowToColMat has ", ncol(rowToRowMat(object)), " columns\n",
            "colToRowMat has ", ncol(rowToRowMat(object)), " rows\n",
            sep=""
        )
    })

## 3.6 Creating setter methods

### 3.6.1 For 1D data structures

We define some setter methods for the custom slots containing the 1D structures. Again, this usually requires the creation of new generics.

    #' @export
    setGeneric("rowVec<-", function(x, ..., value) standardGeneric("rowVec<-"))

    ## [1] "rowVec<-"

    #' @export
    setGeneric("colVec<-", function(x, ..., value) standardGeneric("colVec<-"))

    ## [1] "colVec<-"

We define the class-specific methods for these generics. Note that use of `validObject` to ensure that the assigned input is still valid.

    #' @export
    setReplaceMethod("rowVec", "ExampleClass", function(x, value) {
        x@rowVec <- value
        validObject(x)
        x
    })

    #' @export
    setReplaceMethod("colVec", "ExampleClass", function(x, value) {
        x@colVec <- value
        validObject(x)
        x
    })

### 3.6.2 For 2D data structures

We repeat this process for the 2D structures.

    #' @export
    setGeneric("rowToRowMat<-", function(x, ..., value)
        standardGeneric("rowToRowMat<-")
    )

    ## [1] "rowToRowMat<-"

    #' @export
    setGeneric("colToColMat<-", function(x, ..., value)
        standardGeneric("colToColMat<-")
    )

    ## [1] "colToColMat<-"

    #' @export
    setGeneric("rowToColMat<-", function(x, ..., value)
        standardGeneric("rowToColMat<-")
    )

    ## [1] "rowToColMat<-"

    #' @export
    setGeneric("colToRowMat<-", function(x, ..., value)
        standardGeneric("colToRowMat<-")
    )

    ## [1] "colToRowMat<-"

Again, we define class-specific methods for these generics.

    #' @export
    setReplaceMethod("rowToRowMat", "ExampleClass", function(x, value) {
        x@rowToRowMat <- value
        validObject(x)
        x
    })

    #' @export
    setReplaceMethod("colToColMat", "ExampleClass", function(x, value) {
        x@colToColMat <- value
        validObject(x)
        x
    })

    #' @export
    setReplaceMethod("rowToColMat", "ExampleClass", function(x, value) {
        x@rowToColMat <- value
        validObject(x)
        x
    })

    #' @export
    setReplaceMethod("colToRowMat", "ExampleClass", function(x, value) {
        x@colToRowMat <- value
        validObject(x)
        x
    })

### 3.6.3 For `SummarizedExperiment` slots

Again, we can use the setter methods defined in SummarizedExperiment to modify slots in the base class. These should generally not require any re-defining. However, if it is necessary, the methods should use `callNextMethod` internally:

    #' @export
    #' @importMethodsFrom SummarizedExperiment "rowData<-"
    setReplaceMethod("rowData", "ExampleClass", function(x, ..., value) {
        y <- callNextMethod() # returns a modified ExampleClass

        # Do something extra here.
        message("hi!\n")

        y
    })

### 3.6.4 Other types of modifying functions

Imagine that we want to write a function that returns a modified `ExampleClass`, e.g., where the signs of the `*.vec` fields are reversed. For example, we will pretend that we want to write a `normalize` function, using the generic from BiocGenerics.

    #' @export
    #' @importFrom BiocGenerics normalize
    setMethod("normalize", "ExampleClass", function(object) {
        # do something exciting, i.e., flip the signs
        new.row <- -rowVec(object, withDimnames=FALSE)
        new.col <- -colVec(object, withDimnames=FALSE)
        BiocGenerics:::replaceSlots(object, rowVec=new.row,
            colVec=new.col, check=FALSE)
    })

We use `BiocGenerics:::replaceSlots` instead of the setter methods that we defined above. This is because our setters perform validity checks that are unnecessary if we know that the modification cannot alter the validity of the object. The `replaceSlots` function allows us to skip these validity checks (`check=FALSE`) for greater efficiency.

## 3.7 Enabling subsetting operations

### 3.7.1 Getting a subset

A key strength of the `SummarizedExperiment` class is that subsetting is synchronized across the various (meta)data fields. This avoids book-keeping errors and guarantees consistency throughout an interactive analysis session. We need to ensure that the values in our custom slots are also subsetted.

    #' @export
    setMethod("[", "ExampleClass", function(x, i, j, drop=TRUE) {
        rv <- rowVec(x, withDimnames=FALSE)
        cv <- colVec(x, withDimnames=FALSE)
        rrm <- rowToRowMat(x, withDimnames=FALSE)
        ccm <- colToColMat(x, withDimnames=FALSE)
        rcm <- rowToColMat(x, withDimnames=FALSE)
        crm <- colToRowMat(x, withDimnames=FALSE)

        if (!missing(i)) {
            if (is.character(i)) {
                fmt <- paste0("<", class(x), ">[i,] index out of bounds: %s")
                i <- SummarizedExperiment:::.SummarizedExperiment.charbound(
                    i, rownames(x), fmt
                )
            }
            i <- as.vector(i)
            rv <- rv[i]
            rrm <- rrm[i,,drop=FALSE]
            crm <- crm[,i,drop=FALSE]
        }

        if (!missing(j)) {
            if (is.character(j)) {
                fmt <- paste0("<", class(x), ">[,j] index out of bounds: %s")
                j <- SummarizedExperiment:::.SummarizedExperiment.charbound(
                    j, colnames(x), fmt
                )
            }
            j <- as.vector(j)
            cv <- cv[j]
            ccm <- ccm[,j,drop=FALSE]
            rcm <- rcm[j,,drop=FALSE]
        }

        out <- callNextMethod()
        BiocGenerics:::replaceSlots(out, rowVec=rv, colVec=cv,
            rowToRowMat=rrm, colToColMat=ccm,
            rowToColMat=rcm, colToRowMat=crm, check=FALSE)
    })

Note the special code for handling character indices, and the use of `callNextMethod` to subset the base `SummarizedExperiment` slots.

### 3.7.2 Assigning a subset

Subset assignment can be similarly performed, though the signature needs to be specified so that the replacement value is of the same class. This is generally necessary for sensible replacement of the custom slots.

    #' @export
    setReplaceMethod("[", c("ExampleClass", "ANY", "ANY", "ExampleClass"),
            function(x, i, j, ..., value) {
        rv <- rowVec(x, withDimnames=FALSE)
        cv <- colVec(x, withDimnames=FALSE)
        rrm <- rowToRowMat(x, withDimnames=FALSE)
        ccm <- colToColMat(x, withDimnames=FALSE)
        rcm <- rowToColMat(x, withDimnames=FALSE)
        crm <- colToRowMat(x, withDimnames=FALSE)

        if (!missing(i)) {
            if (is.character(i)) {
                fmt <- paste0("<", class(x), ">[i,] index out of bounds: %s")
                i <- SummarizedExperiment:::.SummarizedExperiment.charbound(
                    i, rownames(x), fmt
                )
            }
            i <- as.vector(i)
            rv[i] <- rowVec(value, withDimnames=FALSE)
            rrm[i,] <- rowToRowMat(value, withDimnames=FALSE)
            crm[,i] <- colToRowMat(value, withDimnames=FALSE)
        }

        if (!missing(j)) {
            if (is.character(j)) {
                fmt <- paste0("<", class(x), ">[,j] index out of bounds: %s")
                j <- SummarizedExperiment:::.SummarizedExperiment.charbound(
                    j, colnames(x), fmt
                )
            }
            j <- as.vector(j)
            cv[j] <- colVec(value, withDimnames=FALSE)
            ccm[,j] <- colToColMat(value, withDimnames=FALSE)
            rcm[j,] <- rowToColMat(value, withDimnames=FALSE)
        }

        out <- callNextMethod()
        BiocGenerics:::replaceSlots(out, rowVec=rv, colVec=cv,
            rowToRowMat=rrm, colToColMat=ccm,
            rowToColMat=rcm, colToRowMat=crm, check=FALSE)
    })

## 3.8 Defining combining methods

### 3.8.1 By row

We need to define a `rbind` method for our custom class. This is done by combining the custom per-row slots across class instances.

    #' @export
    setMethod("rbind", "ExampleClass", function(..., deparse.level=1) {
        args <- list(...)
        all.rv <- lapply(args, rowVec, withDimnames=FALSE)
        all.rrm <- lapply(args, rowToRowMat, withDimnames=FALSE)
        all.crm <- lapply(args, colToRowMat, withDimnames=FALSE)

        all.rv <- do.call(c, all.rv)
        all.rrm <- do.call(rbind, all.rrm)
        all.crm <- do.call(cbind, all.crm)

        # Checks for identical column state.
        ref <- args[[1]]
        ref.cv <- colVec(ref, withDimnames=FALSE)
        ref.ccm <- colToColMat(ref, withDimnames=FALSE)
        ref.rcm <- rowToColMat(ref, withDimnames=FALSE)
        for (x in args[-1]) {
            if (!identical(ref.cv, colVec(x, withDimnames=FALSE))
                || !identical(ref.ccm, colToColMat(x, withDimnames=FALSE))
                || !identical(ref.rcm, rowToColMat(x, withDimnames=FALSE)))
            {
                stop("per-column values are not compatible")
            }
        }

        old.validity <- S4Vectors:::disableValidity()
        S4Vectors:::disableValidity(TRUE)
        on.exit(S4Vectors:::disableValidity(old.validity))

        out <- callNextMethod()
        BiocGenerics:::replaceSlots(out, rowVec=all.rv,
            rowToRowMat=all.rrm, colToRowMat=all.crm,
            check=FALSE)
    })

We check the other per-column slots across all elements to ensure that they are the same. This protects the user against combining incompatible objects. However, depending on the application, this may not be necessary (or too costly) for all slots, in which case it can be limited to critical slots.

We also use the `disableValidity` method to avoid the validity check in the base `cbind` method. This is because the object is technically invalid when the base slots are combined but before it is updated with the new combined values for the custom slots. The `on.exit` call ensures that the original validity setting is restored upon exit of the function.

### 3.8.2 By column

We similarly define a `cbind` method to handle the custom slots.

    #' @export
    setMethod("cbind", "ExampleClass", function(..., deparse.level=1) {
        args <- list(...)
        all.cv <- lapply(args, colVec, withDimnames=FALSE)
        all.ccm <- lapply(args, colToColMat, withDimnames=FALSE)
        all.rcm <- lapply(args, rowToColMat, withDimnames=FALSE)

        all.cv <- do.call(c, all.cv)
        all.ccm <- do.call(cbind, all.ccm)
        all.rcm <- do.call(rbind, all.rcm)

        # Checks for identical column state.
        ref <- args[[1]]
        ref.rv <- rowVec(ref, withDimnames=FALSE)
        ref.rrm <- rowToRowMat(ref, withDimnames=FALSE)
        ref.crm <- colToRowMat(ref, withDimnames=FALSE)
        for (x in args[-1]) {
            if (!identical(ref.rv, rowVec(x, withDimnames=FALSE))
                || !identical(ref.rrm, rowToRowMat(x, withDimnames=FALSE))
                || !identical(ref.crm, colToRowMat(x, withDimnames=FALSE)))
            {
                stop("per-row values are not compatible")
            }
        }

        old.validity <- S4Vectors:::disableValidity()
        S4Vectors:::disableValidity(TRUE)
        on.exit(S4Vectors:::disableValidity(old.validity))

        out <- callNextMethod()
        BiocGenerics:::replaceSlots(out, colVec=all.cv,
            colToColMat=all.ccm, rowToColMat=all.rcm,
            check=FALSE)
    })

## 3.9 Defining coercion methods

### 3.9.1 Coercion from `SummarizedExperiment`

We define a method to coerce `SummarizedExperiment` objects into our new `ExampleClass` class.

    #' @exportMethods coerce
    setAs("SummarizedExperiment", "ExampleClass", function(from) {
        new("ExampleClass", from,
            rowVec=integer(nrow(from)),
            colVec=integer(ncol(from)),
            rowToRowMat=matrix(0,nrow(from),0),
            colToColMat=matrix(0,0,ncol(from)),
            rowToColMat=matrix(0,ncol(from),0),
            colToRowMat=matrix(0,0,nrow(from)))
    })

… which works as expected:

    se <- SummarizedExperiment(matrix(rpois(100, lambda=1), ncol=5))
    as(se, "CountSE")

    ## class: CountSE
    ## dim: 20 5
    ## metadata(0):
    ## assays(1): ''
    ## rownames: NULL
    ## rowData names(0):
    ## colnames: NULL
    ## colData names(0):

This was not strictly necessary for our previous `CountSE` class as no new slots were added. Of course, developers can still explicitly write a conversion method perform additional work to achieve a “sensible” conversion - for example, one might take the absolute values of all entries of the first matrix to ensure that the `CountSE` is valid for all input `SummarizedExperiment` objects.

### 3.9.2 Deriving from a `RangedSummarizedExperiment`

Note that, if we were deriving from a `RangedSummarizedExperiment` (e.g., for some `ExampleClassRanged`), it would be necessary to define explicit conversions from both `RangedSummarizedExperiment` and `SummarizedExperment` to `ExampleClassRanged`. In theory, we should only have to define a conversion from `RangedSummarizedExperiment` to `ExampleClassRanged` - then, any attempt to convert from a `SummarizedExperiment` to `ExampleClassRanged` would:

1. Use the existing `SummarizedExperiment` to `RangedSummarizedExperiment` converter defined in SummarizedExperiment, and then
2. Use the new `RangedSummarizedExperiment` to `ExampleClassRanged` converter that we just defined.

Unfortunately, in cases involving conversion to non-direct subclasses, the S4 system automatically creates methods for any conversions that are not explicitly defined. This means that the correct “chain” of methods listed above is not used when converting from a `SummarizedExperiment` to a `ExampleClassRanged` object. The automatically generated method is used instead, which may not yield a valid object when the specifics of the conversion are ignored. We avoid this scenario by explicitly defining converters for both `SummarizedExperiment` and `RangedSummarizedExperiment` to `ExampleClassRanged`.

# 4 Unit testing procedures

## 4.1 Overview

We test our new methods using the `expect_*` functions from the testthat package. Each function will test an expression and will raise an error if the output is not as expected. This can be used to construct unit tests for the `tests/` subdirectory of the package. Unit testing ensures that the methods behave as expected, especially after any refactoring that may be performed in the future.

For testing, we will construct an instance of `ExampleClass` that has 10 rows and 7 columns:

    RV <- 1:10
    CV <- sample(50, 7)
    RRM <- matrix(runif(30), nrow=10)
    CCM <- matrix(rnorm(14), ncol=7)
    RCM <- matrix(runif(21), nrow=7)
    CRM <- matrix(rnorm(20), ncol=10)

    thing <- ExampleClass(rowVec=RV, colVec=CV,
        rowToRowMat=RRM, colToColMat=CCM,
        rowToColMat=RCM, colToRowMat=CRM,
        assays=list(counts=matrix(rnorm(70), nrow=10)),
        colData=DataFrame(whee=LETTERS[1:7]),
        rowData=DataFrame(yay=letters[1:10])
    )

We will also add some row and column names, which will come in handy later.

    rownames(thing) <- paste0("FEATURE_", seq_len(nrow(thing)))
    colnames(thing) <- paste0("SAMPLE_", seq_len(ncol(thing)))
    thing

    ## class: ExampleClass
    ## dim: 10 7
    ## metadata(0):
    ## assays(1): counts
    ## rownames(10): FEATURE_1 FEATURE_2 ... FEATURE_9 FEATURE_10
    ## rowData names(2): yay extra
    ## colnames(7): SAMPLE_1 SAMPLE_2 ... SAMPLE_6 SAMPLE_7
    ## colData names(1): whee
    ## rowToRowMat has 3 columns
    ## colToColMat has 2 rows
    ## rowToColMat has 3 columns
    ## colToRowMat has 3 rows

## 4.2 Constructor

We test that the `thing` object we constructed is valid:

    expect_true(validObject(thing))

Another useful set of unit tests involves checking that the default constructors (internal and exported) yield valid objects:

    expect_true(validObject(.ExampleClass())) # internal
    expect_true(validObject(ExampleClass())) # exported

We can also verify that the validity method fails on invalid objects:

    expect_error(ExampleClass(rowVec=1), "rowVec")
    expect_error(ExampleClass(colVec=1), "colVec")
    expect_error(ExampleClass(rowToRowMat=rbind(1)), "rowToRowMat")
    expect_error(ExampleClass(colToColMat=rbind(1)), "colToColMat")
    expect_error(ExampleClass(rowToColMat=rbind(1)), "rowToColMat")
    expect_error(ExampleClass(colToRowMat=rbind(1)), "colToRowMat")

Finally, we check that the coercion method yields a valid object.

    se <- as(thing, "SummarizedExperiment")
    conv <- as(se, "ExampleClass")
    expect_true(validObject(conv))

## 4.3 Getters

Testing the 1D getter methods:

    expect_identical(names(rowVec(thing)), rownames(thing))
    expect_identical(rowVec(thing, withDimnames=FALSE), RV)

    expect_identical(names(colVec(thing)), colnames(thing))
    expect_identical(colVec(thing, withDimnames=FALSE), CV)

Testing the 2D getter methods:

    expect_identical(rowToRowMat(thing, withDimnames=FALSE), RRM)
    expect_identical(rownames(rowToRowMat(thing)), rownames(thing))

    expect_identical(colToColMat(thing, withDimnames=FALSE), CCM)
    expect_identical(colnames(colToColMat(thing)), colnames(thing))

    expect_identical(rowToColMat(thing, withDimnames=FALSE), RCM)
    expect_identical(rownames(rowToColMat(thing)), colnames(thing))

    expect_identical(colToRowMat(thing, withDimnames=FALSE), CRM)
    expect_identical(colnames(colToRowMat(thing)), rownames(thing))

Testing the custom `rowData` method:

    expect_true("extra" %in% colnames(rowData(thing)))

## 4.4 Setters

Testing the 1D setter methods:

    rowVec(thing) <- 0:9
    expect_equivalent(rowVec(thing), 0:9)

    colVec(thing) <- 7:1
    expect_equivalent(colVec(thing), 7:1)

Testing the 2D setter methods:

    old <- rowToRowMat(thing)
    rowToRowMat(thing) <- -old
    expect_equivalent(rowToRowMat(thing), -old)

    old <- colToColMat(thing)
    colToColMat(thing) <- 2 * old
    expect_equivalent(colToColMat(thing), 2 * old)

    old <- rowToColMat(thing)
    rowToColMat(thing) <- old + 1
    expect_equivalent(rowToColMat(thing), old + 1)

    old <- colToRowMat(thing)
    colToRowMat(thing) <- old / 10
    expect_equivalent(colToRowMat(thing), old / 10)

Testing our custom `rowData<-` method:

    expect_message(rowData(thing) <- 1, "hi")

We ensure that we can successfully trigger errors on the validity method:

    expect_error(rowVec(thing) <- 0, "rowVec")
    expect_error(colVec(thing) <- 0, "colVec")
    expect_error(rowToRowMat(thing) <- rbind(0), "rowToRowMat")
    expect_error(colToColMat(thing) <- rbind(0), "colToColMat")
    expect_error(rowToColMat(thing) <- rbind(0), "rowToColMat")
    expect_error(colToRowMat(thing) <- rbind(0), "colToRowMat")

## 4.5 Other modifying functions

We test our new `normalize` method:

    modified <- normalize(thing)
    expect_equal(rowVec(modified), -rowVec(thing))
    expect_equal(colVec(modified), -colVec(thing))

## 4.6 Subsetting methods

Subsetting by row:

    subbyrow <- thing[1:5,]
    expect_identical(rowVec(subbyrow), rowVec(thing)[1:5])
    expect_identical(rowToRowMat(subbyrow), rowToRowMat(thing)[1:5,])
    expect_identical(colToRowMat(subbyrow), colToRowMat(thing)[,1:5])

    # columns unaffected...
    expect_identical(colVec(subbyrow), colVec(thing))
    expect_identical(colToColMat(subbyrow), colToColMat(thing))
    expect_identical(rowToColMat(subbyrow), rowToColMat(thing))

Subsetting by column:

    subbycol <- thing[,1:2]
    expect_identical(colVec(subbycol), colVec(thing)[1:2])
    expect_identical(colToColMat(subbycol), colToColMat(thing)[,1:2])
    expect_identical(rowToColMat(subbycol), rowToColMat(thing)[1:2,])

    # rows unaffected...
    expect_identical(rowVec(subbycol), rowVec(thing))
    expect_identical(rowToRowMat(subbycol), rowToRowMat(thing))
    expect_identical(colToRowMat(subbycol), colToRowMat(thing))

Checking that subsetting to create an empty object is possible:

    norow <- thing[0,]
    expect_true(validObject(norow))
    expect_identical(nrow(norow), 0L)

    nocol <- thing[,0]
    expect_true(validObject(nocol))
    expect_identical(ncol(nocol), 0L)

Subset assignment:

    modified <- thing
    modified[1:5,1:2] <- thing[5:1,2:1]

    rperm <- c(5:1, 6:nrow(thing))
    expect_identical(rowVec(modified), rowVec(thing)[rperm])
    expect_identical(rowToRowMat(modified), rowToRowMat(thing)[rperm,])
    expect_identical(colToRowMat(modified), colToRowMat(thing)[,rperm])

    cperm <- c(2:1, 3:ncol(thing))
    expect_identical(colVec(modified), colVec(thing)[cperm])
    expect_identical(colToColMat(modified), colToColMat(thing)[,cperm])
    expect_identical(rowToColMat(modified), rowToColMat(thing)[cperm,])

Checking that we obtain the same object after trivial assignment operations:

    modified <- thing
    modified[0,] <- thing[0,]
    expect_equal(modified, thing)
    modified[1,] <- thing[1,]
    expect_equal(modified, thing)
    modified[,0] <- thing[,0]
    expect_equal(modified, thing)
    modified[,1] <- thing[,1]
    expect_equal(modified, thing)

We double-check that we can get an error upon invalid assignment:

    expect_error(modified[1,1] <- thing[0,0], "replacement has length zero")

## 4.7 Combining methods

Combining by row:

    combined <- rbind(thing, thing)

    rtwice <- rep(seq_len(nrow(thing)), 2)
    expect_identical(rowVec(combined), rowVec(thing)[rtwice])
    expect_identical(rowToRowMat(combined), rowToRowMat(thing)[rtwice,])
    expect_identical(colToRowMat(combined), colToRowMat(thing)[,rtwice])

    # Columns are unaffected:
    expect_identical(colVec(combined), colVec(thing))
    expect_identical(colToColMat(combined), colToColMat(thing))
    expect_identical(rowToColMat(combined), rowToColMat(thing))

And combining by column. We use `test_equivalent` here for simplicity, as column names are altered to preserve uniqueness.

    combined <- cbind(thing, thing)

    ctwice <- rep(seq_len(ncol(thing)), 2)
    expect_equivalent(colVec(combined), colVec(thing)[ctwice])
    expect_equivalent(colToColMat(combined), colToColMat(thing)[,ctwice])
    expect_equivalent(rowToColMat(combined), rowToColMat(thing)[ctwice,])

    # Rows are unaffected:
    expect_equivalent(rowVec(combined), rowVec(thing))
    expect_equivalent(rowToRowMat(combined), rowToRowMat(thing))
    expect_equivalent(colToRowMat(combined), colToRowMat(thing))

Checking that we get the same object if we combine a single object or an empty object:

    expect_equal(thing, rbind(thing))
    expect_equal(thing, rbind(thing, thing[0,]))

    expect_equal(thing, cbind(thing))
    expect_equal(thing, cbind(thing, thing[,0]))

And checking that the compatibility errors are properly thrown:

    expect_error(rbind(thing, thing[,ncol(thing):1]), "not compatible")
    expect_error(cbind(thing, thing[nrow(thing):1,]), "not compatible")

# 5 Documentation

We suggest creating at least two separate documentation (i.e. `*.Rd`) files. The first file would document the class and the constructor:

    \name{ExampleClass class}

    \alias{ExampleClass-class}
    \alias{ExampleClass}

    \title{The ExampleClass class}
    \description{An overview of the ExampleClass class and constructor.}

    \usage{
    ExampleClass(rowVec=integer(0), colVec=integer(0),
        # etc., etc., I won't write it all out here.
    )
    }

    \arguments{
        \item{rowVec}{An integer vector mapping to the rows, representing
            something important.}

        \item{colVec}{An integer vector mapping to the columns, representing
            something else that's important.}

        % And so on...
    }

    \details{
        % Some context on why this class and its slots are necessary.
        The ExampleClass provides an example of how to derive from the
        SummarizedExperiment class.  Its slots have no scientific meaning and
        are purely for demonstration purposes.
    }

The second file would document all of the individual methods:

    \name{ExampleClass methods}

    % New generics:
    \alias{rowVec}
    \alias{rowVec,ExampleClass-method}
    \alias{rowVec<-}
    \alias{rowVec<-,ExampleClass-method}
    %% And so on...

    % Already have a generic:
    \alias{[,ExampleClass-method}
    \alias{[,ExampleClass,ANY-method}
    \alias{[,ExampleClass,ANY,ANY-method}

    \alias{rbind,ExampleClass-method}
    %% And so on...

    \title{ExampleClass methods}
    \description{Methods for the ExampleClass class.}

    \usage{
    \S4method{rowVec}{ExampleClass}(x, withDimnames=FALSE)

    \S4method{rowVec}{ExampleClass}(x) <- value

    \S4method{[}{ExampleClass}(x, i, j, drop=TRUE)

    \S4method{rbind}{ExampleClass}(..., , i, j, drop=TRUE)

    %% And so on...
    }

    \arguments{
        \item{x}{An ExampleClass object.}

        \item{withDimnames}{A logical scalar indicating whether dimension names
            from \code{x} should be returned.}

        \item{value}{
            For \code{rowVec}, an integer vector of length equal to the number of
            rows.

            For \code{colVec}, an integer vector of length equal to the number of
            columns.
        }

        %% And so on...
    }

    \section{Accessors}{
        % Add some details about accessor behaviour here.
    }

    \section{Subsetting}{
        % Add some details about subsetting behaviour here.
    }

    \section{Combining}{
        % Add some details about combining behaviour here.
    }

# 6 Session information

    sessionInfo()

    ## R version 4.4.0 beta (2024-04-15 r86425)
    ## Platform: x86_64-pc-linux-gnu
    ## Running under: Ubuntu 22.04.4 LTS
    ##
    ## Matrix products: default
    ## BLAS:   /home/biocbuild/bbs-3.19-bioc/R/lib/libRblas.so
    ## LAPACK: /usr/lib/x86_64-linux-gnu/lapack/liblapack.so.3.10.0
    ##
    ## locale:
    ##  [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C
    ##  [3] LC_TIME=en_GB              LC_COLLATE=C
    ##  [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8
    ##  [7] LC_PAPER=en_US.UTF-8       LC_NAME=C
    ##  [9] LC_ADDRESS=C               LC_TELEPHONE=C
    ## [11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C
    ##
    ## time zone: America/New_York
    ## tzcode source: system (glibc)
    ##
    ## attached base packages:
    ## [1] stats4    stats     graphics  grDevices utils     datasets  methods
    ## [8] base
    ##
    ## other attached packages:
    ##  [1] testthat_3.2.1.1            SummarizedExperiment_1.34.0
    ##  [3] Biobase_2.64.0              GenomicRanges_1.56.0
    ##  [5] GenomeInfoDb_1.40.0         IRanges_2.38.0
    ##  [7] S4Vectors_0.42.0            BiocGenerics_0.50.0
    ##  [9] MatrixGenerics_1.16.0       matrixStats_1.3.0
    ## [11] BiocStyle_2.32.0
    ##
    ## loaded via a namespace (and not attached):
    ##  [1] sass_0.4.9              utf8_1.2.4              SparseArray_1.4.0
    ##  [4] lattice_0.22-6          digest_0.6.35           magrittr_2.0.3
    ##  [7] evaluate_0.23           grid_4.4.0              bookdown_0.39
    ## [10] pkgload_1.3.4           fastmap_1.1.1           rprojroot_2.0.4
    ## [13] jsonlite_1.8.8          Matrix_1.7-0            brio_1.1.5
    ## [16] BiocManager_1.30.22     httr_1.4.7              fansi_1.0.6
    ## [19] UCSC.utils_1.0.0        jquerylib_0.1.4         abind_1.4-5
    ## [22] cli_3.6.2               rlang_1.1.3             crayon_1.5.2
    ## [25] XVector_0.44.0          withr_3.0.0             cachem_1.0.8
    ## [28] DelayedArray_0.30.0     yaml_2.3.8              S4Arrays_1.4.0
    ## [31] tools_4.4.0             GenomeInfoDbData_1.2.12 vctrs_0.6.5
    ## [34] R6_2.5.1                lifecycle_1.0.4         zlibbioc_1.50.0
    ## [37] waldo_0.5.2             desc_1.4.3              bslib_0.7.0
    ## [40] pillar_1.9.0            glue_1.7.0              xfun_0.43
    ## [43] knitr_1.46              htmltools_0.5.8.1       rmarkdown_2.26
    ## [46] compiler_4.4.0