Epochs
What is an Epoch?
An epoch is an event or period of time that has been tagged with a name. Epochs are attached to Signal objects. A recording, which contains a collection of signals, has methods that allow us to manipulate the entire collection based on a superset of the epochs in each signal.
An epoch has three values:
start, end, name
where start and end denote the beginning and end of the epoch in
seconds and name is a descriptive string (see how to name
epochs. For events (single points in time), the
start will contain the timestamp of the event and end will be
NaN. Currently, support for events are limited (or nonexistent). As
use-cases arise, we will implement support for these.
Epochs are stored in Signal objects under the
.epochs attribute in a pandas DataFrame containing three columns
(start, end and name).
For the rest of this document, let’s consider some example epoch data in which trials are 40-100 msec long, and there are several TORCs of 20 msec duration played before a tone-in-TORC detection task. In this task, the subject is asked to detect when a tone is played in a sequence of background tokens (known as TORCs).
start end name
0.00 0.060 ExperimentalTrial
0.00 0.05 PreStimSilence
0.00 0.020 TORC_3983
0.00 0.020 Reference
0.015 0.020 PostStimSilence
0.020 0.025 PreStimSilence
0.020 0.040 TORC_572
0.020 0.040 Reference
0.035 0.040 PostStimSilence
0.040 0.045 PreStimSilence
0.040 0.060 TORC_3983
0.040 0.060 PureTone12
0.040 0.060 DetectionTask
0.049 0.049 Licking
0.055 0.060 PostStimSilence
0.060 0.0100 ExperimentalTrial
0.060 0.065 PreStimSilence
0.060 0.080 TORC_444
0.060 0.080 Reference
0.066 0.087 Licking
0.075 0.080 PostStimSilence
0.080 0.0100 TimeOut
In the example above, note that some epochs have the same name, such as “ExperimentalTrial”. If the same name appears several times, we call each appearance an occurrance of a given epoch.
Note also that epochs may overlap. For example, compare the first
TORC_3983 with the first Reference. This is a way of indicating
that TORC_3983 is a Reference token. This approach facilitates
analysis where one may wish to select all reference TORCs and compare
them to all TORCs that occur simultaneously with a pure tone (compare
the second occurence of TORC_3983 with PureTone12).
This set of epochs tells us quite a bit about what’s going on in the
experiment. In the first trial two reference TORCs are played
(TORC_3983, TORC_572), then a tone-in-TORC is played
(TORC_3983, PureTone12), and the animal correctly licks. In the
second trial, the animal licks during the reference sound and gets a
time out.
How signals use epochs
A signal object can use epochs to perform two basic operations:
Mask regions of data. For example, perhaps licking introduces EMG artifacts in to the LFP recordings. In this case, you may want to mask all regions in the LFP recording during a lick so that your analysis isn’t affected by these artifacts:
masked_signal = signal.mask\epoch('Licking')
As you will see later, this masking can also be used to generate subsets
of data for cross-validation when fitting models. Signals also have a
select_epochs method, which is the inverse of mask_epochs:
selected_signal = signal.select_epoch('Reference')
Extract regions of data. For example, perhaps you want to plot the average response to a particular epoch:
torc = signal.extract_epoch('TORC\_3983')
average_torc = np.nanmean(torc, axis=0)
Epoch manipulation
Signal objects offer the following methods:
Getting boundaries of an epoch stored inside the signal using
signal.get_epoch_bounds(epoch_name). This will return a Nx2 array (where N is the number of occurrences, the first column is start time and the second column is end time).Adding epochs to the ones stored inside the signal. You can do this using
signal.add_epoch(epoch_name, epochs).
Fancy manipulation of epochs (e.g., selecting epochs that contain another epoch, resizing epoch boundaries, computing the union of two epochs, etc.) must be done outside the signal object. You can then add the newly-created epochs back to the signal object.
General epoch manipulation
Internally, signal objects store epochs in a DataFrame with three columns (‘start’, ‘end’, ‘name’). However, when working with epochs outside of the signal object, the epochs will be a 2D array of shape Nx2 (where N is the number of occurences of that epoch, the first column is start time and second column is end time). In the example below, we have four occurances of the epoch, with the last epoch running from 300 to 301 msec:
[[0.049 0.049],
[0.066 0.087],
[0.145 0.257],
[0.300 0.301]]
To pull some epochs out for processing, you can use
signal.get_epoch_bounds:
dt_epoch = signal.get_epoch_bounds('DetectionTask')
l_epoch = signal.get_epoch_bounds('Licking')
If we want to take only the correct trials (defined as when the animal licks during a detection task):
from nems0.data.epochs import epoch_contain
correct_epoch = epoch_contain(dt_epoch, l_epoch, mode='start')
Then, mask everything but the correct epochs:
masked_signal = signal.select_epoch(correct_epochs)
Great! You can save that for later by adding it to the epochs in the Signal:
signal.add_epoch('CorrectTrial', correct_epochs)
Then anytime afterward we can simply do:
correct_signal = signal.select_epoch('CorrectTrial')
Manipulating epoch boundaries
You can use set theory to manipulate epoch boundaries by subtracting or adding one epoch to the other:
from nems0.data.epochs import epoch_intersection, epoch_difference
ct_epoch = signal.get_epoch_bounds('CorrectTrial')
prestim_epoch = signal.get_epoch_bounds('PreStimSilence')
# Get only the prestim silence by combining using an intersection operation
only_prestim = epoch_intersection(ct_epoch, prestim_epoch)
# Remove the prestim silence by using a difference operation
no_prestim = epoch_difference(ct_epoch, prestim_epoch)
How do I get the average response to a particular epoch?
Instead of masking data with signal.select_epoch() and
signal.mask_epoch(), you may also extract epochs:
data = signal.extract_epoch('TORC_3983')
average_response = np.nanmean(data, axis=0)
Here, extract_epoch returns a 3D array with the first axis
containing each occurence of TORC_3983. The remaining two axes are
channels and time. In this particular situation, the durations of each
occurence of TORC_3983 are identical. However, in some situations,
the duration of epochs may vary from occurence to occurence. In this
case, shorter epochs will be padded with NaN values so the length
matches the longest occurence. To get the average, use np.nanmean.
How do I get the average response in prestim vs poststim, regardless of behavior?
This might be useful for identifying a baseline that is altered by behavior.
signal.select_epochs('PreStimSilence', inplace=True)
prestim = signal.as_continuous()
prestim_mean = np.nanmean(prestim)
signal.select_epochs('PostStimSilence', inplace=True)
poststim = signal.as_continuous()
poststim_mean = np.nanmean(poststim)
How do I get the average stimulus 300ms before every mistaken lick?
What if we want to know what the animal heard just before it licked accidentally? Or if the TORC was maybe too close to the reference tone?
# Pull out the epoch we want to analyze
trial_epoch = signal.get_epoch_bounds('Trials')
ct_epoch = signal.get_epoch_bounds('CorrectTrials')
# Note the invert=True. This means to return all trial_epoch that do not
# contain a ct_epoch.
bad_trials = epoch_contain(trial_epoch, ct_epoch, invert=True)
# Extend the 'licking' events backward 300ms
lick_epoch = signal.get_epoch_bounds('Licking')
prior_to_licking = adjust_epoch(lick_epoch, -300, 0)
# Now take the intersection of those two selections
before_bad_licks = epoch_intersection(bad_trials, prior_to_licking)
signal.select_epoch(before_bad_licks, inplace=True)
data = signal.as_continous()
some_plot_function(data)
Note that extract_epoch may end up duplicating data. For example, if
the animal licked 10 times a second and you were looking at the 3
seconds prior to each lick, your data will overlap, meaning you just
duplicated your total data about 1/2 * 3 * 10 = 15 times! This may
negatively alter certain computations of the mean in some sense, and in
such circumstances, you may want to use the argument
allow_data_duplication=False for signal.extract_epoch().
How do I use epoch info from two different signals in the same recording?
Like signal objects, recording objects offer mask_epoch and
extract_epoch methods. However, you still need to combine the epochs
manually. In the above examples, we assumed that a single signal will
contain information about both the stimulus and whether the animal
licked or not. However, that may not always be the case. Perhaps the
“stimulus” signal will contain information about the stimulus and trials
while the “lick” signal will contain information about the lick epochs
(i.e., how the animal responded). For example, if we want to find anytim
the animal blinked or licked and treat those as artifacts and mask the
full recording when they occured).
# The recording version of `get_epoch_bounds` takes the signal name as the
# first argument and epoch name as the second argument.
lick_epoch = recording.get_epoch_bounds('lick', 'Licking')
blink_epoch = recording.get_epoch_bounds('pupil', 'blinks')
all_artifacts = epochs_union(blink_epoch, lick_epoch)
recording.mask_signals(all_artifacts)
Epoch naming
Be descriptive. If you give a stimulus a unique name, then when it occurs in other Recordings, you can simply concatenate the two recordings and still select exactly the same data.
Avoid implicit indexes like trial1, trial2, trial3; prefer
using just trial and the folding functionality of
.fold_by('trial'), which gives you a matrix. If you have truly
different stimuli, you may named them stim01, stim02, but
descriptive names like cookoo_bird.wav, and train_horn.wav are
better.
Remember that the idea behind epochs is to tag the content of data, much like HTML marks up text to tell what it is. It’s totally fine to tag the exact same epoch with multiple names, if that will help you perform queries on it later.
What happens with zero-length epochs?
Zero-length epochs are events. They work best with epochs_contain:
trials = signal.get_epochs('Trial')
# Assume a laser is an event (i.e., a zero-length epoch)
laser_pulse = signal.get_epochs('Laser')
laser_trials = epochs_contain(trials, laser_pulse, mode='start')
They will not work with set operations.
Cross-validation and Jackknifes
from nems0.data.epochs import jacknife_epochs
stim = recording.get_signal('stim')
trials = stim.get_epochs('trials')
# Generate 20 jacknife sets
jacknifed_trials = jacknife_epochs(n=20)
results = []
for jacknife in jacknifed_trials:
est = recording.mask_epochs(jacknife)
val = recording.select_epochs(jacknife)
result = fit_model(est, val, model)
result.append(result)
plot_result(result)
publish_paper(result)