LosslessPipeline Class¶

This is the docstring for the Lossless Pipeline class. The methods and attributes detailed below are used throughout the Lossless pipeline.

class pylossless.LosslessPipeline(config_path=None, config=None)¶

Class used to handle pipeline parameters.

Parameters:

config_pathpathlib.Path: path to config file specifying the parameters to be used in the pipeline.

Attributes:

flagsdict: A dictionary of detailing the flagged channels, epochs, and ICs. keys are 'ch', 'epoch', and 'ic', and values are instances of FlaggedChs, FlaggedEpochs, and FlaggedICs, respectively.
config_pathpathlib.Path: path to the config file specifying the parameters to be used in the in the pipeline.
configdict: A dictionary containing the pipeline parameters.
rawmne.io.Raw: An instance of Raw containing that will be processed by the pipeline.
ica1mne.preprocessing.ICA: An instance of ICA. The result of the initial ICA run during the pipeline.
ica2mne.preprocessing.ICA: An instance of ICA. The result of the final ICA run during the pipeline.

Methods

`add_pylossless_annotations`(inds, event_type, ...)	Add annotations for flagged epochs.
`filter`()	Run filter procedure based on structured config args.
`find_breaks`()	Find breaks using mne.preprocessing.annotate_break.
`find_outlier_chs`([epochs, picks])	Detect outlier Channels to leave out of rereference.
`flag_bridged_channels`(data_r_ch)	Flag bridged channels.
`flag_channels_fixed_threshold`([threshold, picks])	Flag channels based on the stdev value across the time dimension.
`flag_epochs_fixed_threshold`([threshold, picks])	Flag epochs based on the stdev value across the time dimension.
`flag_noisy_channels`([picks])	Flag channels with outlying standard deviation.
`flag_noisy_epochs`([picks])	Flag epochs with outlying standard deviation.
`flag_noisy_ics`([picks])	Calculate the IC standard Deviation by epoch window.
`flag_rank_channel`(data_r_ch)	Flag the channel that is the least unique.
`flag_uncorrelated_channels`([picks])	Check neighboring channels for too high or low of a correlation.
`flag_uncorrelated_epochs`([picks])	Flag epochs where too many channels are uncorrelated.
`get_all_event_ids`()	Get a combined event ID dictionary from existing markers and raw annotations.
`get_derivative_path`(bids_path[, derivative_name])	Build derivative path for file.
`get_epochs`([detrend, preload, rereference, ...])	Create mne.Epochs according to user arguments.
`get_events`()	Make an MNE events array of fixed length events.
`get_n_nbr`([picks])	Calculate nearest neighbour correlation for channels.
`load_config`()	Load the config file.
`load_ll_derivative`(derivatives_path)	Load a completed pylossless derivative state.
`run`(bids_path[, save, overwrite])	Run the pylossless pipeline.
`run_dataset`(paths)	Run a full dataset.
`run_ica`(run[, picks])	Run ICA.
`run_staging_script`()	Run a staging script if specified in config.
`run_with_raw`(raw)	Execute pipeline on a raw object.
`save`([derivatives_path, overwrite, format, ...])	Save the file at the end of the pipeline.
`set_montage`()	Set the montage.

add_pylossless_annotations(inds, event_type, epochs)¶

Add annotations for flagged epochs.

Parameters:

indslist | tuple: indices corresponding to artefactual epochs
event_typestr: One of ‘ch_sd’, ‘low_r’, ‘ic_sd1’
epochsmne.Epochs: an instance of mne.Epochs

filter()¶: Run filter procedure based on structured config args.

find_breaks()¶

Find breaks using mne.preprocessing.annotate_break.

Parameters:

kwargsdict: a dict with keys that are valid keyword arguments for mne.preprocessing.annotate_break, and with values that are valid for their respective arguments.

Notes

kwargs=dict(min_break_duration=15.0, t_start_after_previous=5.0) for example would unpack two keyword arguments from mne.preprocessing.annotate_break

find_outlier_chs(epochs=None, picks='eeg')¶

Detect outlier Channels to leave out of rereference.

Parameters:

epochsmne.Epochs | None: An instance of mne.Epochs, or None. If None, then pylossless.LosslessPipeline.raw should be set, and this method will call pylossless.LosslessPipeline.get_epochs() to create epochs to use for outlier detection.
picksstr (default “eeg”): Channels to include in the outlier detection process. You can pass any argument that is valid for the pick() method, but you should avoid passing a mix of channel types with differing units of measurement (e.g. EEG and MEG), as this would likely lead to incorrect outlier detection (e.g. all EEG channels would be flagged as outliers).

Returns:

list: a list of channel names that are considered outliers.

Notes

This method is used to detect channels that are so noisy that they should be left out of the robust average rereference process.

Examples

>>> import mne
>>> import pylossless as ll
>>> config = ll.Config().load_default()
>>> pipeline = ll.LosslessPipeline(config=config)
>>> fname = mne.datasets.sample.data_path() / "MEG/sample/sample_audvis_raw.fif"
>>> raw = mne.io.read_raw(fname)
>>> epochs = mne.make_fixed_length_epochs(raw, preload=True)
>>> chs_to_leave_out = pipeline.find_outlier_chs(epochs=epochs)

flag_bridged_channels(data_r_ch)¶

Flag bridged channels.

Parameters:

data_r_chnumpy.array: an instance of numpy.array

flag_channels_fixed_threshold(threshold=5e-05, picks='eeg')¶

Flag channels based on the stdev value across the time dimension.

Flags channels if the voltage-variance standard deviation is above the given threshold in n_percent of epochs (default: 20%).

Parameters:

thresholdfloat: threshold, in volts. If the standard deviation across time in any channel x epoch indice is above this threshold, then the channel x epoch indices will be considered an outlier. Defaults to 5e-5, or 50 microvolts. Note that here, ‘time’ refers to the samples in an epoch. For each channel, if its std value is above the given threshold in more than 20% of the epochs, it is flagged.
picksstr (default “eeg”): Type of channels to pick.

Returns:

None: If any channels are flagged, those channel names will be logged in the flags attribute of the LosslessPipeline object, under the key 'volt_std', e.g. my_pipeline.flags["ch"]["volt_std"].

Notes

Warning

the default threshold of 50 microvolts may not be appropriate for a particular dataset or data file, as the baseline voltage variance is affected by the impedance of the system that the data was recorded with. You may need to assess a more appropriate value for your own data. You can use the find_bads_by_threshold() function to quickly assess a more appropriate threshold.

See also

find_bads_by_threshold()

Examples

>>> import mne
>>> import pylossless as ll
>>> config = ll.Config().load_default()
>>> config["flag_channels_fixed_threshold"] = {"threshold": 5e-5}
>>> pipeline = ll.LosslessPipeline(config=config)
>>> sample_fpath = mne.datasets.sample.data_path()
>>> fpath = sample_fpath / "MEG" / "sample" / "sample_audvis_raw.fif"
>>> raw = mne.io.read_raw(fpath).pick("eeg")
>>> pipeline.raw = raw
>>> pipeline.flag_channels_fixed_threshold()

flag_epochs_fixed_threshold(threshold=5e-05, picks='eeg')¶

Flag epochs based on the stdev value across the time dimension.

Flags an epoch if the voltage-variance standard deviation is above the given threshold in n_percent of channels (default: 20%).

Parameters:

thresholdfloat: threshold, in volts. If the standard deviation across time in any channel x epoch indice is above this threshold, then the channel x epoch indices will considered an outlier. Defaults to 5e-5, or 50 microvolts. Note that here, ‘time’ refers to the samples in an epoch. For each epoch, if the std value of more than 20% of channels (in that epoch) are above the given threshold, the epoch is flagged.
picksstr (default “eeg”): Type of channels to pick.

Notes

WARNING: the default threshold of 50 microvolts may not be appropriate for a particular dataset or data file, as the baseline voltage variance is affected by the impedance of the system that the data was recorded with. You may need to assess a more appropriate value for your own data.

flag_noisy_channels(picks='eeg')¶

Flag channels with outlying standard deviation.

Calculates the standard deviation of the voltage-variance for each channel at each epoch (default: 1-second epochs). Then, for each epoch, creates a distribution of the stdev values of all channels. Then, for each epoch, estimates a stdev outlier threshold, where any channel that has an stdev value higher than the threshold (in the current epoch) is flagged. If a channel is flagged as an outlier in more than n_percent of epochs (default: 20%), the channel is flagged for removal.

Parameters:

picksstr (default “eeg”): Type of channels to pick.

flag_noisy_epochs(picks='eeg')¶

Flag epochs with outlying standard deviation.

Parameters:

picksstr (default “eeg”): Type of channels to pick.

flag_noisy_ics(picks='eeg')¶

Calculate the IC standard Deviation by epoch window.

Flags windows with too many ICs with outlying standard deviations.

Parameters:

picksstr (default “eeg”): Type of channels to pick.

flag_rank_channel(data_r_ch)¶

Flag the channel that is the least unique.

Flags the channel that is the least unique, the channel to remove prior to ICA in order to account for the rereference rank deficiency.

Parameters:

data_r_chnumpy.array.: an instance of numpy.array.

flag_uncorrelated_channels(picks='eeg')¶

Check neighboring channels for too high or low of a correlation.

Parameters:

picksstr (default “eeg”): Type of channels to pick.

Returns:

data arraynumpy.array: an instance of numpy.array

flag_uncorrelated_epochs(picks='eeg')¶

Flag epochs where too many channels are uncorrelated.

Parameters:

picksstr (default “eeg”): Type of channels to pick.

Notes

Similarly to the neighbor r calculation done between channels this section looks at the correlation, but between all channels and for epochs of time. Time segments are flagged for removal.

get_all_event_ids()¶

Get a combined event ID dictionary from existing markers and raw annotations.

Returns:

dict or None: A combined dictionary of event IDs, including both existing markers and new ones from annotations. Returns None if no events or annotations are found.

get_derivative_path(bids_path, derivative_name='pylossless')¶: Build derivative path for file.

get_epochs(detrend=None, preload=True, rereference=True, picks='eeg')¶

Create mne.Epochs according to user arguments.

Parameters:

detrendint | None (default None): If 0 or 1, the data channels (MEG and EEG) will be detrended when loaded. 0 is a constant (DC) detrend, 1 is a linear detrend.None is no detrending. Note that detrending is performed before baseline correction. If no DC offset is preferred (zeroth order detrending), either turn off baseline correction, as this may introduce a DC shift, or set baseline correction to use the entire time interval (will yield equivalent results but be slower).
preloadbool (default True): Load epochs from disk when creating the object or wait before accessing each epoch (more memory efficient but can be slower).
picksstr (default “eeg”): Type of channels to pick.

Returns:

Epochsmne.Epochs: an instance of mne.Epochs

get_events()¶: Make an MNE events array of fixed length events.

get_n_nbr(picks='eeg')¶

Calculate nearest neighbour correlation for channels.

Parameters:

picksstr (default “eeg”): Type of channels to pick.

load_config()¶: Load the config file.

load_ll_derivative(derivatives_path)¶

Load a completed pylossless derivative state.

Parameters:

derivatives_pathstr | mne_bids.BIDSPath: Path to a saved pylossless derivatives.

Returns:

LosslessPipeline: Returns an instance of LosslessPipeline for the loaded pylossless derivative state.

run(bids_path, save=True, overwrite=False)¶

Run the pylossless pipeline.

Parameters:

bids_pathpathlib.Path: Path of the individual file bids_root.
savebool (default True).: Whether to save the files after completing the pipeline. Defaults to True. if False, files are not saved.
overwritebool (default False).: Whether to overwrite existing files of the same name.

run_dataset(paths)¶

Run a full dataset.

Parameters:

pathslist | tuple: a list of the bids_paths for all recordings in the dataset that should be run.

run_ica(run, picks='eeg')¶

Run ICA.

Parameters:

runstr: Must be ‘run1’ or ‘run2’. ‘run1’ is the initial ICA use to flag epochs, ‘run2’ is the final ICA used to classify components with mne_icalabel.
picksstr (default “eeg”): Type of channels to pick.

run_staging_script()¶: Run a staging script if specified in config.

run_with_raw(raw)¶: Execute pipeline on a raw object.

save(derivatives_path=None, overwrite=False, format='EDF', event_id=None)¶

Save the file at the end of the pipeline.

Parameters:

derivatives_pathNone | mne_bids.BIDSPath: path of the derivatives folder to save the file to.
overwritebool (default False): whether to overwrite existing files with the same name.
formatstr (default “EDF”): The format to use for saving the raw data. Can be "auto", "FIF", "EDF", "BrainVision", "EEGLAB".
event_iddict | None (default None): Dictionary mapping annotation descriptions to event codes.

set_montage()¶: Set the montage.