%% Template for PLoS
%% Version 3.1 February 2015
%%
%% To compile to pdf, run:
%% latex plos.template
%% bibtex plos.template
%% latex plos.template
%% latex plos.template
%% dvipdf plos.template
%%
%% % % % % % % % % % % % % % % % % % % % % %
%%
%% -- IMPORTANT NOTE
%%
%% This template contains comments intended 
%% to minimize problems and delays during our production 
%% process. Please follow the template instructions
%% whenever possible.
%%
%% % % % % % % % % % % % % % % % % % % % % % % 
%%
%% Once your paper is accepted for publication, 
%% PLEASE REMOVE ALL TRACKED CHANGES in this file and leave only
%% the final text of your manuscript.
%%
%% There are no restrictions on package use within the LaTeX files except that 
%% no packages listed in the template may be deleted.
%%
%% Please do not include colors or graphics in the text.
%%
%% Please do not create a heading level below \subsection. For 3rd level headings, use \paragraph{}.
%%
%% % % % % % % % % % % % % % % % % % % % % % %
%%
%% -- FIGURES AND TABLES
%%
%% Please include tables/figure captions directly after the paragraph where they are first cited in the text.
%%
%% DO NOT INCLUDE GRAPHICS IN YOUR MANUSCRIPT
%% - Figures should be uploaded separately from your manuscript file. 
%% - Figures generated using LaTeX should be extracted and removed from the PDF before submission. 
%% - Figures containing multiple panels/subfigures must be combined into one image file before submission.
%% For figure citations, please use "Fig." instead of "Figure".
%% See http://www.plosone.org/static/figureGuidelines for PLOS figure guidelines.
%%
%% Tables should be cell-based and may not contain:
%% - tabs/spacing/line breaks within cells to alter layout or alignment
%% - vertically-merged cells (no tabular environments within tabular environments, do not use \multirow)
%% - colors, shading, or graphic objects
%% See http://www.plosone.org/static/figureGuidelines#tables for table guidelines.
%%
%% For tables that exceed the width of the text column, use the adjustwidth environment as illustrated in the example table in text below.
%%
%% % % % % % % % % % % % % % % % % % % % % % % %
%%
%% -- EQUATIONS, MATH SYMBOLS, SUBSCRIPTS, AND SUPERSCRIPTS
%%
%% IMPORTANT
%% Below are a few tips to help format your equations and other special characters according to our specifications. For more tips to help reduce the possibility of formatting errors during conversion, please see our LaTeX guidelines at http://www.plosone.org/static/latexGuidelines
%%
%% Please be sure to include all portions of an equation in the math environment.
%%
%% Do not include text that is not math in the math environment. For example, CO2 will be CO\textsubscript{2}.
%%
%% Please add line breaks to long display equations when possible in order to fit size of the column. 
%%
%% For inline equations, please do not include punctuation (commas, etc) within the math environment unless this is part of the equation.
%%
%% % % % % % % % % % % % % % % % % % % % % % % % 
%%
%% Please contact latex@plos.org with any questions.
%%
%% % % % % % % % % % % % % % % % % % % % % % % %

\documentclass[10pt,letterpaper]{article}
\usepackage[top=0.85in,left=2.75in,footskip=0.75in]{geometry}
\usepackage[document]{ragged2e}

% Use adjustwidth environment to exceed column width (see example table in text)
\usepackage{changepage}

% Use Unicode characters when possible
\usepackage[utf8]{inputenc}

% textcomp package and marvosym package for additional characters
\usepackage{textcomp,marvosym}

% fixltx2e package for \textsubscript
\usepackage{fixltx2e}

% amsmath and amssymb packages, useful for mathematical formulas and symbols
\usepackage{amsmath,amssymb}

% cite package, to clean up citations in the main text. Do not remove.
\usepackage{cite}

% Use nameref to cite supporting information files (see Supporting Information section for more info)
\usepackage{nameref,hyperref}

% Landscape pages
\usepackage{lscape}

% line numbers
\usepackage[right]{lineno}

% ligatures disabled
\usepackage{microtype}
\DisableLigatures[f]{encoding = *, family = * }

% rotating package for sideways tables
\usepackage{rotating}

% Remove comment for double spacing
%\usepackage{setspace} 
%\doublespacing

% Highlight packages
\usepackage{color,soul}
\soulregister\ref7

% Enum
\usepackage{enumitem}
\usepackage[table]{xcolor}    % loads also »colortbl«
\usepackage{tabularx}  
  
% Text layout
\raggedright
\setlength{\parindent}{0.5cm}
\textwidth 5.25in 
\textheight 8.75in

% Bold the 'Figure #' in the caption and separate it from the title/caption with a period
% Captions will be left justified
\usepackage[aboveskip=1pt,labelfont=bf,labelsep=period,justification=raggedright,singlelinecheck=off]{caption}

% Use the PLoS provided BiBTeX style
\bibliographystyle{plos2015}

% Remove brackets from numbering in List of References
\makeatletter
\renewcommand{\@biblabel}[1]{\quad#1.}
\makeatother

% OS Specifics
\usepackage{ifplatform} % for specifying different locations
\ifmacosx
\newcommand{\papers}{/Users/Yogi/Dropbox/Research/Papers}
\graphicspath{{/Users/Yogi/Dropbox/Research/Papers/tool-rtxi/figures/}}
\fi
\iflinux
\newcommand{\papers}{/home/yapatel/Dropbox/Research/Papers}
\graphicspath{{/home/yapatel/Dropbox/Research/Papers/tool-rtxi/figures/}}
\fi

% Leave date blank
\date{}

% Header and Footer with logo
\usepackage{lastpage,fancyhdr,graphicx}
\usepackage{epstopdf}
\pagestyle{myheadings}
\pagestyle{fancy}
\fancyhf{}
\lhead{\includegraphics[width=2.0in]{PLOS-submission.eps}}
\rfoot{\thepage/\pageref{LastPage}}
\renewcommand{\footrule}{\hrule height 2pt \vspace{2mm}}
\fancyheadoffset[L]{2.25in}
\fancyfootoffset[L]{2.25in}
\lfoot{\sf PLOS}

%% Include all macros below
\newcommand{\lorem}{{\bf LOREM}}
\newcommand{\ipsum}{{\bf IPSUM}}

%% END MACROS SECTION
\begin{document}
\vspace*{0.35in}

%% Title must be 250 characters or less.
%% Please capitalize all terms in the title except conjunctions, prepositions, and articles.
%\begin{flushleft}
%{\Large
%\textbf\newline{Hard real-time closed-loop electrophysiology with the Real-Time eXperiment Interface (RTXI)}}
%\newline
%% Insert author names, affiliations and corresponding author email (do not include titles, positions, or degrees).
%Yogi A. Patel\textsuperscript{1,2},
%Ansel George\textsuperscript{4},
%Alan D. Dorval\textsuperscript{3},
%John A. White\textsuperscript{5},
%David J. Christini\textsuperscript{4,$\ast$},
%Robert J. Butera\textsuperscript{1,2,6,$\ast$}
%\\
%\bigskip
%\bf{1} Bioengineering Graduate Program, Georgia Institute of Technology, Atlanta, GA, USA
%\\
%\bf{2} Dept. of Biomedical Engineering, Georgia Institute of Technology and Emory University, Atlanta, GA, USA
%\\
%\bf{3} Dept. of Bioengineering, University of Utah, Salt Lake City, UT, USA
%\\
%\bf{4} Dept. of Physiology and Biophysics, Weill Cornell Medical College, New York, NY, USA
%\\
%\bf{5} Dept. of Biomedical Engineering, Boston University, Boston, MA, USA
%\\
%\bf{6} School of Electrical and Computer Engineering, Georgia Institute of Technology, Atlanta, GA, USA
%\bigskip
%
%% Insert additional author notes using the symbols described below. Insert symbol callouts after author names as necessary.
%% 
%% Remove or comment out the author notes below if they aren't used.
%%
%% Primary Equal Contribution Note
%%\Yinyang These authors contributed equally to this work.
%
%% Additional Equal Contribution Note
%% Also use this double-dagger symbol for special authorship notes, such as senior authorship.
%%\ddag These authors also contributed equally to this work.
%
%% Current address notes
%%\textcurrency a Insert current address of first author with an address update
%% \textcurrency b Insert current address of second author with an address update
%% \textcurrency c Insert current address of third author with an address update
%
%% Deceased author note
%%\dag Deceased
%
%% Group/Consortium Author Note
%%\textpilcrow Membership list can be found in the Acknowledgments section.
%
%% Use the asterisk to denote corresponding authorship and provide email address in note below.
%$\ast$Co-senior and corresponding authors: dchristi@med.cornell.edu, rbutera@gatech.edu
%
%\end{flushleft}
%
%\justify
%
%%%% Abstract
%% Please keep the abstract below 300 words
%\section*{Abstract}
%The ability to experimentally perturb biological systems has traditionally been limited to static pre-programmed or operator-controlled protocols. In contrast, real-time control allows dynamic probing of biological systems with perturbations that are computed on-the-fly during experimentation. Real-time control applications for biological research are available; however, these systems are costly and often restrict the flexibility and customization of experimental protocols. The Real-Time eXperiment Interface (RTXI) is an open source software platform for achieving hard real-time data acquisition and closed-loop control in biological experiments while retaining the flexibility needed for experimental settings. RTXI has enabled users to implement complex custom closed-loop protocols in single cell, cell network, animal, and human electrophysiology studies. RTXI is also used as a free and open source, customizable electrophysiology platform in open-loop studies requiring online data acquisition, processing, and visualization. RTXI is easy to install, can be used with an extensive range of external experimentation and data acquisition hardware, and includes standard modules for implementing common electrophysiology protocols. %RTXI also increases transparency of hard real-time, closed-loop experiments through standard built-in performance benchmarks and easily shareable user modules and configuration files. The RTXI source code and user-contributed modules can be found on the website or online GitHub repository and are released under the GNU GPLv3 license. RTXI is based on Xenomai, a Real-Time Linux framework, and can be run on any desktop PC using a Live CD (standalone ready-to-use operating system). The Live CD may also be used to install the operating system and RTXI or users may choose to customize and compile the source code manually.  
%
%%%% Required author summary
%% Please keep the Author Summary between 150 and 200 words
%% Use first person. PLOS ONE authors please skip this step. 
%% Author Summary not valid for PLOS ONE submissions.
%\section*{Author Summary}
%Real-time control allows dynamic probing of biological systems with perturbations computed on-the-fly from measurements made during experimentation. Applications enabling real-time control are abundant in biological research; however, few systems enable complete customization of acquisition, processing, and visualization of data in hard real-time without requiring expertise in numerical algorithms or proficiency in computer programming. RTXI fills this void by capitalizing on the strengths of Real-Time Linux. We have developed a turnkey platform with a modular architecture that allows users to create complex, flexible, closed-loop protocols for biological experiments. RTXI allows hard real-time execution of custom user code, rapid parameter screening, and simple implementation of closed-loop and open-loop experimental control. As a complete experimental control, simulation, and data acquisition system, RTXI also features portable data files, user-contributed modules for common algorithms, and experiment configuration files. As a complete experimental control, simulation, and data acquisition system, RTXI also features portable data-file formats, a repository of user-contributed modules for common algorithms, and an assortment of default configuration files for various experimental paradigms.
%\linenumbers
%
%%%% Introduction
%\section*{Introduction}
%Advances in stimulation (electrical, optical, biochemical) and measurement (electrical, biochemical, optical) techniques have increased the spatial and temporal resolution with which researchers can monitor or perturb biological activity. Using such tools and techniques in a closed-loop paradigm, where an acquired signal is used to compute the system output, can enable observation of physiological function, development and validation of computational models, as well as investigation of causal relationships in biological systems. This requires closed-loop control that operates on timescales that are physiologically relevant, which span tens of microseconds at the ion channel level to minutes at the behavioral level. Furthermore, the closed-loop control needs to be hard real-time (RT) - operating on a strict schedule for acquisition, processing, and yielding a computationally determined output with guaranteed performance bounds appropriate for the timescales of interest. Such hard RT, closed-loop control is implemented in industrial applications (e.g, aerospace, robotics, stock markets); however its use in biological research is stymied due to the relative high cost of commercial systems, and lack of flexibility in customizing closed-loop protocols, performance, and features. 
% 
%The Real-Time Experiment Interface (RTXI, \url{http://www.rtxi.org}, Fig. \ref{entire}) is an open source software platform for hard RT, closed-loop data acquisition (DAQ) and experimental control in biological experiments used by over 70 labs worldwide. {This manuscript provides a technical and practical overview of RTXI's architecture and features, as well as highlights select novel applications. Functionality of RTXI's architecture and features has been validated both computationally through load testing and performance characterization, and experimentally in multiple setups of different biological systems and varying time scales.}
%
%%\begin{landscape}
%\begin{figure}[h]
%\begin{center}
%%\includegraphics[keepaspectratio,angle=-90]{entire3.png}
%\caption{Screenshot of arbitrary RTXI workspace with core and custom modules. The workspace is intended to demonstrate the level of user-configurability provided by RTXI, such as channel configurations, combinations of data I/O connections, and saving virtually every programmed element of each module. Additional custom modules are shown (e.g., Spike Detector, FIR Filter, Signal Generator, Neuron Model, etc) to demonstrate the versatility of RTXI's API system for creation of any utility or model to be used in hard RT, closed-loop experiments. We refer the reader to the up-to-date user manual (Supplement 3\ref{itm:user_manual}) and tutorials (Supplement 3\ref{itm:tutorials}) available online for more information on how to navigate the RTXI workspace.}
%\label{entire}
%\end{center}
%\end{figure}
%%\end{landscape}
%
%RTXI is based on Xenomai, a Real-Time Linux framework \cite{Barabanov1996,Xenomai} and can be run or installed on any desktop PC by using the Live CD or by manually compiling the source code. RTXI can interface with an extensive range of external experimentation and data acquisition hardware, and includes standard modules for implementing commonly used electrophysiology protocols. Modules contain function-specific code that can be used in combinations to build custom workflows, experimental protocols, and interfaces, thereby eliminating the need to code all aspects of each experiment protocol from scratch. The power, flexibility, and stability of RTXI has made it possible for users to implement complex custom closed-loop protocols in a variety of cardiac \cite{groenendaal2014voltage,groenendaal2015cell,brown2016illuminating,TJP:TJP12076,Bartolucci2014} and neuronal systems at the single cell \cite{Idoux2011,Nabi2013}, cell network \cite{Wilson2014}, animal, and human electrophysiology \cite{Lustenberger2016} levels. Dozens of publications (Supplement 3\ref{itm:papers}])have used RTXI. Examples include investigation of the contribution of specific ion channels or synaptic receptors to spiking and bursting activity in a variety of neuronal cell types, oscillatory behavior of pacemaker neurons \cite{Norman2016}, the effect of network topology and intrinsic neuronal properties on population activity in a hybrid network \cite{Kispersky2011a}, and effects of transcranial alternating current stimulation (tACS) on cortical activity \cite{Boyle2013,Lustenberger2016}. Each example utilized RTXI to create a custom, hard RT closed-loop protocol with the goal of dynamically probing the target system. 
%
%%%% Design and implementation
%% You may title this section "Methods" or "Models". 
%% "Models" is not a valid title for PLoS ONE authors. However, PLoS ONE
%% authors may use "Analysis" 
%\section*{Design and Implementation of the Real-Time eXperiment Interface}
%\subsection*{System Architecture} 
%An RT control system for closed-loop control typically runs in an iterative computational loop with the smallest possible nominal cycle period, a minimum amount of cycle-to-cycle variation (jitter) in the actual period, and the shortest possible system latency (delay from input to computed output) {determined by the specific application} \cite{bettencourt2008effects}. Standard desktop computer operating systems, including  Linux, are built upon a monolithic kernel whose scheduler is engineered to balance distribution of resources amongst the various active threads and respond to hardware and software triggered interrupts in a resource-efficient manner. This results in standard desktop operating systems providing only ``soft'' RT performance. In soft RT systems, occurrence and timing of data acquisition, processing, and output generation events is neither guaranteed nor bounded. A lack of such guarantees can lead to substantial, yet often unnoticed, effects in experimental control \cite{White2,Butera2001,Raikov2004}. For example, in the case of dynamic clamp \cite{White1,Dorval2001,Prinz2004,Sharp1993}, a soft RT system may occasionally wait so long to compute the injected current that the actual value of the membrane potential has changed significantly in the meantime. The resulting experimental dynamics may look acceptable but still be wrong, since they are based on incorrect assumptions about the state of the cell. In practice, this means that phase-dependent stimulation may be delayed or occur at the incorrect physiologically-relevant time. 
% 
%To achieve hard real-time performance, RTXI uses a Real-Time Operating System (RTOS). An RTOS enables hard RT performance by modifying the operating system's native kernel architecture to enable priority-based pre-emption of processes, allocation of memory, and communication with on-board hardware for data and file I/O. To enable hard RT control, RTXI uses Xenomai, a real-time framework, which installs a micro-kernel alongside the standard Linux kernel. The micro-kernel consists of its own scheduler and interrupt handler and places the standard Linux kernel in a low-priority state, allowing dedicated processes to be prioritized. For example, Xenomai's prioritization of RTXI enables hard RT control of periodic tasks such as sampling from experimental equipment, performing computations, and generating output signals. The use of an RTOS also minimizes system and hardware latencies, resulting in faster sampling rates, computation times, and file I/O. For some experimental designs with closed-loop feedback, a higher sampling rate also improves the stability of the protocol \cite{Preyer2009}.
%
%%In the case of RTXI, the Xenomai \cite{Xenomai} framework micro-kernel, which enables hard real-time scheduling of processes and priority-based allocation of resources, is inserted next to the native Linux kernel. This modification places the native Linux kernel into a low priority state and allows the micro-kernel to use its own RT scheduler and interrupt handler to 
%
%\subsection*{Application Architecture} 
%RTXI is written in C/C++ and utilizes three threads to process: 1) hard RT data acquisition and experimental control, 2) user interactions, and 3) data storage. Figure \ref{system_arch} provides an overview of the complete system architecture. The RT thread is instantiated with the highest system priority and controlled by the micro-kernel. The second user interface and experience (UI/UX) thread, powered by the Qt \cite{qt} and Qwt \cite{qwt}  graphical user interface (GUI) frameworks, processes user inputs to RTXI and online data visualization in soft RT. The final soft RT thread continuously reads and writes data to disk. With the advent of multi-core desktop computers, users are also able to instantiate additional soft RT threads for online data processing. All threads run in the same process address space, making it easy to share data for processing, updating visualizations, and data storage. RTXI's architecture allows it to be used as an open-loop experimental control and data acquisition system \cite{Patel2015,Patel2016,Patel2016a}, or even as a simulation environment.
%
%\begin{figure}[h]
%\begin{center}
%%\includegraphics[width=\textwidth,keepaspectratio]{system_arch.png}
%\caption{System architecture. The bottom block depicts the hardware layer with which RTXI interfaces. RTXI is capable of interfacing with DAQs using either PCI/PCIe, USB, or Ethernet interfaces (see \nameref{sec:hardware} for more information). Hard RT communication with hardware devices is achieved through Analogy, a set of drivers within the Xenomai framework. The top block of the diagram illustrates the core architecture of RTXI. On each cycle of the RT period, the RT Thread wakes up, acquires new data, executes instructions defined within the hard RT function of both core and user modules (see \nameref{sec:user_code_text}), outputs data to the DAQ, and returns to sleep (idle). Transmission of data to/from different modules is handled by the \texttt{IO} class. When the RT Thread is idle, resources are made available to other system applications and functions. The GUI and Data Storage Threads continuously run with a static period to provide a stable balance between hard RT performance and handling of user input, updating of visualizations, and data storage.}
%\label{system_arch}
%\end{center}
%\end{figure}
%
%The RT thread wakes on each clock cycle and executes all DAQ operations, module functions, and RT system functions. Module functions refer to both base system operations, such as transmitting data across modules and writing data to system buffers, and all RT operations implemented within custom modules. During this step, modules can post RT events - which notify other modules about state changes and user-initiated events, e.g., unloading or loading of new modules - to queue. These events are then executed by the RT thread once all module computations are complete.
% 
%{An important advantage of this modular application architecture is the ability to load and unload both core system and user-created modules, change parameters, and modify any system variable online without halting closed-loop execution or affecting hard RT performance. Commercial systems, such as Simulink (MathWorks, Natick, MA), Signal (Cambridge Electronic Design, Cambridge, England), LabVIEW (National Instruments, Austin, TX), Tucker David Technologies (Alachua, FL) typically require halting execution and recompiling the modified loop prior to being able to continue execution. Open-source platforms, such as NeuroRighter \cite{Newman2012}, MANTA \cite{MANTA}, PLDAPS \cite{PLDAPS}, and the Open Ephys GUI \cite{Siegle2017}, provide closed-loop control with varying degrees of flexibility, but none enable hard RT closed-loop control. Such limitations are not ideal for investigation of causal relationships in biological systems.} %experimental conditions where preparation viability and timescales of investigation are critically important factors.
%
%{RTXI enables users to create custom real-time algorithms and protocols for closed-loop processing and visualization. Debugging custom algorithms and protocols can be difficult in a hard real-time environment, especially without expertise or knowledge on how to probe the Linux kernel during execution. To ease the process of developing hard real-time algorithms and protocols, RTXI can be easily configured to run in non-real-time (non-RT) mode with debugging support. This configuration option is provided by the RTXI installation scripts and is outlined on the RTXI website (Supplement 3\ref{itm:website}). When run in non-RT mode, users can obtain a tracelog of events to identify where and why the system crashed, occurrence of race conditions, and system resource usage.}
%
%\subsection*{Compatible Hardware}
%\label{sec:hardware}
%RTXI interfaces with experiments through a variety of hardware interfaces, including PCI/PCIe based DAQs from National Instruments and Sensoray, Ethernet based devices such as cameras and commercial amplifiers, as well as USB-based acquisition devices (Supplement 3\ref{itm:hardware}). The hardware used with RTXI should be chosen based upon the hard RT needs of the custom protocol. Devices interfacing through the PCI/PCIe and Ethernet interfaces are capable of providing hard RT closed-loop performance with the appropriate drivers. PCI/PCIe hardware can  achieve sub-millisecond latencies, while Ethernet devices can provide millisecond latencies. USB devices can provide closed-loop functionality with the appropriate driver, but the non-deterministic bounds of the USB protocol prevent hard RT control.
%
%\subsection*{Portability and Sharing}
%\label{sec:sharing}
%RTXI allows users to move developed and tested modules, algorithms, and entire closed-loop protocols from one computer to another without significant overhead. Once the workflow and protocol have been set up, the entire workspace can be saved to an XML-based workspace settings file. This file saves all core system specifications set by the user, loaded modules along with their parameter and state values, and connections between modules. The settings file can then be used on any other computer with RTXI and the appropriate modules installed to restore the workspace. Furthermore, existing workspace settings files can be used as a starting point for creating new experimental protocols. This reduces the chances of errors when setting up a complex protocol with many modules and provides an easy mechanism for sharing custom protocols. {Users can share all elements of their custom protocols - modules, settings files, screenshots, data, etc - on RTXI's GitHub page (Supplement 3\ref{itm:source_code}}.
%
%\subsection*{Core Modules}
%RTXI's base functionality (data acquisition, processing, and visualization) is achieved through core modules designed to provide hard RT closed-loop performance. A set of core modules with minimal computational overhead are compiled as shared object libraries during the RTXI installation process and linked to RTXI at run-time. This approach leads to lower system overhead and thus greater flexibility for users to define custom protocols that may require greater computational resources.
% 
%\paragraph{System Control Panel.}
%The System Control Panel shown in Fig. \ref{entire} is the primary interface for configuring RT system settings and DAQ I/O channels. The detected on-board DAQs are listed in the Devices drop-down menu (e.g., analogy0). Users can set the Frequency or Period of the RT system, which is used as the data acquisition {(I/O) rate and determines the computation time available per cycle}. All available AI, AO, and DIO channels are automatically detected from the installed DAQ and listed for the user to enable and configure. For example, users can configure the measurement mode (Ground-referenced, Differential, etc), scale the input measurements to account for hardware gains, change the measurement range, and add or remove offset to each channel individually. Users can also configure available Digital I/O channels through this panel. When the channel configurations are ``applied'', an event is registered on the system stack, notifying all open modules of the changes. Users can respond to these changes through the update function of their custom modules {(see Fig. \ref{user_code_fig}D)}.
%
%\begin{figure}[h]
%\begin{center}
%%\includegraphics[width=\textwidth,keepaspectratio]{user_code.png}
%\caption{Example template code available (Supplement 3\ref{itm:user_modules}) to users for writing custom modules, and corresponding module loaded within RTXI. (A) Basic C++ header (\texttt{plugin\_template.h}) for an RTXI module. (B) Users can declare inputs, outputs, states, parameters, and events for custom modules through the DefaultGUIModel struct and (C) implement hard RT code within the execute function. (D) The \texttt{update} function allows execution of state-specific code for the module. (E) The module after it is compiled with the provided Makefile and loaded into RTXI. Each element of the module GUI is tied to a specific line of code. For example, when ``Button A'' is clicked, the \texttt{aBttn\_event()} (line 35, A) code is executed. Similarly, the ``GUI label'' (lines 5-6, B) and ``A State'' (lines 11-13, B) components of the GUI are created by the \texttt{vars} struct and their values are initialized from the \texttt{INIT} block of the \texttt{update()} function (lines 8-9, C). Visit the GitHub repository (Supplement 3\ref{itm:user_modules}) for complete corresponding C++ implementation (\texttt{plugin\_template.cpp}) file.}
%\label{user_code_fig}
%\end{center}
%\end{figure}
%
%%System sampling rates, acquisition parameters, and channel configuration is achieved through the System Control Panel (Fig. \ref{control_panel}). The control panel shows a list of DAQs available on the host PC for configuration and their technical specifications. The number of analog input/output channels, well as digital channels are read from the DAQs onboard chipset and provided for user configuration. Each channels measurement configuration, gain, offset, and decimation can be specified. When parameters such as the system sampling rate are updated or a new channel is enabled, an event is posted to the system notifying all loaded modulated of the change for updating of local variables dependent upon the system time step.
% 
%\paragraph{Data Recorder and HDF5 Files.} The Data Recorder module shown in Fig. \ref{entire} allows saving of virtually anything within RTXI. Users first select the module of interest, and then have the option of saving any module element specified within the \texttt{DefaultGUIModel::vars} struct (see Fig. \ref{user_code_fig}B). Parameters for all modules are automatically registered at the beginning of a recording session. If modified during execution, the updated value is registered to the data file with a timestamp. Users can also ``tag'' various time points throughout their experiment to note relevant events. Events are timestamped and registered as a list of tags within the specified data file. A global downsampling option is available to reduce output file sizes. In between active recording sessions, the size and number of trials present in the specified data file, and length of the previous trial are displayed in the metadata section. If a user chooses an existing file to write data to, the option is presented to either overwrite or append the data file.
%
%Experiment data and metadata saved by the Data Recorder are stored in the Hierarchical Data Format (HDF5). HDF5 \cite{hdf} is an open data model that is increasingly popular for representing large and complex data, data relationships, and their associated metadata. HDF5 file read and write operations are supported by many common analysis frameworks and languages (e.g., MATLAB, Python, Julia, R, etc). Acquired raw and processed data, module states and parameters, system configurations, and almost any other value can be synchronously saved to a single HDF5 file by simply selecting the appropriate signal and adding it to the list of active recording channels. In addition, any computed value or intermediate signal can easily be captured for offline debugging or validation of RT algorithms and processing. When a parameter value is modified on-the-fly during data acquisition, the new value is automatically time stamped and stored into the data file. The Data Recorder also includes the ability to timestamp data with tags to experimental events or making notes and includes DAQ channel configuration details for all active channels. For precise control of data recorder start and stop times, the Data Recorder can easily be coupled with the Sync module, shown in Fig. \ref{entire}, which can be used to control the state of numerous modules and the data recorder all at once.
% 
%Figure \ref{hdf5file} depicts the structure of an RTXI-generated HDF5 file. Scripts and analysis tools are provided for importing RTXI-generated HDF5 files into a single MATLAB structure for post-hoc analysis. RTXI-generated HDF5 files are compatible with many commercial and free software applications for a variety of platforms. There is no required proprietary software for viewing or analyzing data stored in RTXI-generated HDF5 files. Much of the available software also supports editing data in place within the HDF5 file or appending new data to an existing file. This allows users to add associated data such as images, post-processed data, or additional notes. 
%
%\begin{figure}[h]
%\begin{center}
%%\includegraphics[width=\textwidth,keepaspectratio]{hdf5file.png}
%\caption{Hierarchical Data Format data file generated by RTXI. Each trial is represented as an ordered group of data and metadata objects. By default, RTXI saves parameter values from all modules when recording is started into the ``Parameters'' group. All enabled DAQ AI channel configuration settings are also saved into the ``System Settings'' group. HDF5 files can be opened, modified, and appended to from a variety of post-hoc analysis frameworks (e.g., HDFView, MATLAB, R, Python). }
%\label{hdf5file}
%\end{center}
%\end{figure}
%
%\paragraph{Connector.} Passing of data between core and custom modules within the RTXI workspace takes place through the Connector module shown in Fig. \ref{entire}. The module's simple interface allows mapping of outputs from a module or DAQ AI channel to the input of another module in various configurations (one-to-one, one-to-many, many-to-one, etc). The specific input and output signals available to a module are defined in the module's code (see Fig. \ref{user_code_fig}). In cases where a many-to-one connection is formed, the inputs are summed prior to passing the data to the specified input. For example, Fig. \ref{entire} shows the data from DAQ AI channel 0 and the output of the Signal Generator module connected to the input of the Neuron module. Simultaneously, the output of the Neuron module is sent to the Spike Detector module as well as out to hardware via DAQ AO channel 0.
% 
%\paragraph{Oscilloscope.} RTXI's built-in real-time, high-speed, fully configurable Oscilloscope shown in Fig. \ref{entire} module allows visualization of signals, parameters, states, and events from any loaded core or custom module. Similar to the Data Recorder and other system modules, the Oscilloscope can access data of any kind from any loaded module within the workspace. To visualize the signal, the user selects the block of interest on the ``Channel'' tab, which provides a list of available streams along with options to customize the color, style, and vertical scale (per division) of the selected signal. Once the modifications are ``applied'', the enabled signal is visualized with the configured settings and a legend entry is created listing the enabled channels by name along with their respective vertical scale. A second tab, titled ``Display'' allows configuration of the time scale (per division), the Oscilloscope's refresh rate, and specify trigger settings. Screenshots can be saved online during experiments for quick figure generation.
% 
%\paragraph{Real-Time Benchmarks.} Online evaluation of system performance is useful when designing RT algorithms, debugging experimental setups, and even during experiments. The RT Benchmarks module shown in Fig. \ref{entire} (pane with blue top bar) provides key statistics about the RT performance of the system - including the computational time, RT period, and the RT period's jitter - by using timestamps directly from the real-time system's clock. These benchmarks have been shown to be critical in evaluating hard real-time performance \cite{Raikov2004}. The computational time is inclusive of all system and custom modules loaded into the workspace, enabling evaluation of the full hard RT closed-loop execution time of each custom protocol. The RT period and RT period's jitter provide a means for viewing the actual period of the closed-loop system. {The RT period should never exceed twice the desired RT period, otherwise hard RT behavior is compromised.} Furthermore, each statistic reported by the RT Benchmarks module can be saved to an HDF5 file through the Data Recorder for post-hoc validation and verification of hard RT performance during the experiment. As described in the RTXI documentation (Supplement 3\ref{itm:user_manual}), end users are strongly advised to benchmark their hardware and modules to ensure that the performance they require is actually being delivered.
%
%\paragraph{Module Wizard.} There are over 50 modules available to RTXI users through the RTXI GitHub repositories (Supplement 3\ref{itm:user_modules}). The Module Wizard shown in Fig. \ref{entire} was created to ease the process of downloading, installing, and updating new and existing modules. Users can quickly synchronize with the GitHub repositories and view details about each module, including the specific functions, how it works, and its development status. If a specific module is of interest, the user can download and install the module directly from within the Module Wizard and within seconds load the new module into their workspace.
% 
%\paragraph{Users Preferences.} The User Preferences module shown in Fig. \ref{entire} enables customization of default file locations for various file I/O operations. Users can specify folder locations for workspace setfiles and HDF5 data files. In addition, users can specify buffer sizes for the Data Recorder, which can be beneficial when acquiring large amounts of data at high sampling rates. All settings specified within the User Preferences module are set as the default values and saved to the global RTXI configuration file.
%
%\subsection*{Custom Modules and Application Programming Interface (API)}
%\label{sec:user_code_text}
%One core strength of RTXI is the ability to create custom RT modules without being restricted to hardware and software-defined boundaries, which is a common obstacle with commercial systems. RTXI users implement custom experimental protocols by writing all aspects of their RT protocol in C++-based modules. The use of C++ as the language of choice allows users to incorporate a variety of established libraries, such as LAPACK, Boost, and GNU Scientific Library for processing and data visualization, while capitalizing upon the strengths of C++ such as recursion, object-oriented programming, and abstraction. This system for creating custom closed-loop protocols enables complete freedom with respect to customization of closed-loop RT protocols without any virtual limit on what can be implemented.  
% 
%To get started, users are encouraged to reference the Plugin Template, which provides the critical elements required for creating a custom RTXI module. Figure \ref{user_code_fig} provides an overview of the Plugin Template files and code. The Plugin Template files include a Makefile, which is used to define compilation rules on how files should be compiled, which external libraries are to be included, and how to package the individual pieces together to form a shared library object that is RTXI-compatible. In addition, one header (\texttt{plugin\_template.h}, Fig. \ref{user_code_fig}A) and implementation (\texttt{plugin\_template.cpp}) file are included as examples that can be built upon.
%
%All custom modules are abstracted from the \texttt{DefaultGUIModel} class, which includes all elements necessary for interfacing with both the RT and GUI threads. When creating a new module, users create the various I/O elements for that module through the \texttt{vars} structure (Fig. \ref{user_code_fig}B). To run specific code in RT, the code is placed in the \texttt{execute()} function (Fig. \ref{user_code_fig}C), which is common to all system and custom modules and processed at each time step. Functions called from within the \texttt{execute} function are also run in RT, while all code that is not in or called by the \texttt{DefaultGUIModel::execute} function is run on the soft RT GUI thread. This enables a simple mechanism for separating hard RT and soft RT tasks without significant programming overhead for the end user. 
% 
%Users are able to customize the GUI of the module by including a variety of labels for model parameter and state variables, buttons and other elements for module control, as well as plotting elements for online analysis and visualization through the \texttt{customizeGUI()} function. By default, each module consists of three automatically generated buttons for starting or pausing execution, updating user-set parameters, and unloading itself from the workspace. Customization of the GUI is achieved by using the Qt framework, which provides a variety of GUI elements such as buttons, lists, text boxes, etc. Users can include signals and slots, which is a mechanism for calling specific functions based upon the occurrence of an event (e.g., button press $\rightarrow$ meet threshold criteria $\rightarrow$ specific animal behavior). For example, users can include a radio button in their module that has a signal-slot connection to a plot. When a user clicks the radio button, a signal is emitted and tells the module to display a scatter plot of data, which is otherwise hidden. The same signal-slot mechanism can be used to create buttons that drive outputs via the DAQ, call specific protocol functions, or generate notifications when an event occurs. Users are also encouraged to check the listing of available modules on the RTXI GitHub page for examples or potentially suitable modules that can easily be adapted to meet individual needs. 
%
%Updates to module variables, responding to changes in RT system settings applied through the system control panel, and controlling the state of each module takes place through the \texttt{update()} function (Fig. \ref{user_code_fig}D). Each module can be in one of five states - which are responsible for initializing all module elements (\texttt{INIT}), registering UI/UX commands to the module (\texttt{MODIFY}), responding to changes in the RT system frequency (\texttt{PERIOD}), and controlling whether or not the module is running (\texttt{UNPAUSED}) or not (\texttt{PAUSED}). Users can customize any state for experiment-specific needs.
%
%%%% Results and discussion
%% For figure citations, please use "Fig." instead of "Figure"
%% Results and Discussion can be combined.
%\section*{Results and Discussion}
%\subsection*{System Characterization and Load Testing}
%We evaluated RTXI's performance in different I/O configurations and computational loads (Fig. \ref{io}). Tests were performed by providing a randomly triggered input signal (square wave, 50\% duty cycle, 5 Vpp) to one of the DAQ AI channels. Each test case was run for 30 minutes (1800 total input events). In Case 1 (top), 1 AI channel was used to measure the input signal, and 1 AO channel was used to output the same signal, with no processing in the loop to evaluate system performance in a {single-input, single-output (SISO)} configuration. Case 2 (middle) used the same configuration as Case 1, but included 2 AI and 2 AO channels. Only one AI channel received the input signal. The other AI channel was left floating and directly connected to the second AO channel, allowing characterization of system performance in a {multi-input, multi-output (MIMO)} configuration. Case 3 used the same configuration as Case 2, but with added processing within the loop. The Hodgkin-Huxley Neuron model was configured to receive the test input signal and the output of the Neuron model (membrane potential, V$_m$) was connected to the Spike Detector module. The Spike Detector module determines if an action potential has occurred by checking for a threshold-crossing event. When an action potential is detected, the Spike Detector outputs a TTL pulse that is then sent out via the AO channel.
%
%%\begin{landscape}
%\begin{figure}[h]
%\begin{center}
%%\includegraphics[height=0.65\textwidth,keepaspectratio]{io.png}
%\caption{System performance under different computational loads and closed-loop configurations. (A) System configurations tested. (B) System I/O latency does not exceed twice the system period (50 $\mu$s) nor are I/O events missed in all test cases, demonstrating hard RT performance and a delay of no more than one sample. (C) Distribution of measured RT periods in each tests case by the Real-Time Benchmarks module. The RT period should never exceed twice the desired RT period, otherwise hard RT behavior is compromised. The increase in RT period variability with increasing computational loads is predominantly from the initial time frame after which a protocol starts execution. This is represented by the RT period jitter data (D), which demonstrates the RT period stabilizing within the first few minutes of execution. (E) Time spent processing the \texttt{execute()} function of all loaded module on each RT loop cycle.}
%\label{io}
%\end{center}
%\end{figure}
%%\end{landscape}
%
%A separate data acquisition system (MATLAB, NI USB-6341) was used to sample (250 KHz) the AI and AO signals from each case to quantify the I/O latency with each configuration. The RT period, RT period jitter, and computation time are measured with the Real-Time Benchmarks module and recorded with the Data Recorder. All tests were performed with an RTXI (v2.1) system frequency of 20 KHz on a desktop computer with an Intel Core i5 Quad Core Processor (3.40 GHz), 32GB of physical RAM, 10,000 RPM hard drive, Radeon HD 8570 graphics card, and an NI PCIe-6259 DAQ. All system and application daemons were killed prior to performance evaluation (recommended) because such background processes can prevent resource allocation or occupy significant amounts of CPU time. All analysis was performed using MATLAB 2016 (MathWorks, Natick, MA, USA). Data shown in Fig. \ref{io} is from one RTXI installation, however additional performance characterization results are available online from different installations {(Supplement 3\ref{itm:website})}. Each test case demonstrates I/O processing of 1800 events with the desired RT period, demonstrating hard RT performance and a delay of no more than one sample.  
%
%{Fig. \ref{io}A summarizes each case scenario used to characterize system performance. With a system frequency of 20 kHz, the system has 50 us to complete input, processing, and output events. Histograms of latencies measured by an external data acquisition system are shown in Fig. \ref{io}B. For all test cases, the system latencies are distributed around 50$\mu$s. If a latency greater than 100$\mu$s had been measured (twice the system period), the {hard} aspect of the real-time system would be lost due to the system missing deadlines for events (e.g., input not acquired, output not generated).
%
%Fig. \ref{io}C-E of show system performance metrics measured and reported by the core RT Benchmarks module. Fig. \ref{io}C demonstrates that the measured RT period is unimodal and tightly distributed around 50$\mu$s in all I/O configurations and system loads. As the system load increases, the variance of the RT period increases, but the mean stays closely centered at 50$\mu$s. The RT period's jitter is shown in Fig. \ref{io}D and is consistent throughout RT execution, demonstrating that RTXI has access to sufficient resources, is prioritized by the real-time scheduler, and can provide the requested performance. If RTXI was unable to meet the performance demands of the configuration, the RT period, and thus the jitter, would vary significantly over the course of an experiment. The final column of Fig. \ref{io} represents the time taken to complete the input, processing, and output events. As the system configuration and load increase, the computation time also increases but without losing hard real-time performance (as shown by Fig. \ref{io}C-D).}
%
%\subsection*{Use Cases}
%\subsubsection*{Use Case 1: Replication and suppression of learning-induced membrane and synaptic plasticity}
%RTXI's most common application is for the dynamic-clamp technique (of the many examples, some include: \cite{Prinz2004,Sharp1993,Dorval2001,Lin2010a}). This has led to significant advances in fundamental concepts in neuroscience and related fields. 
% 
%Feeding behavior in the mollusk Aplysia is modified by various forms of associative learning, including classical and operant conditioning, and alter the central decision-making processes related to feeding actions. Experimental modification of intrinsic excitability and electrical synapses of neurons in the Aplysia feeding central pattern-generating network have been shown to correlate with compulsive-like motor output expressions induced by \textit{in vivo} operant conditioning. Despite this correlation between plasticity and operant conditioning-induced changes, a causal relationship was not shown.  
% 
%Using RTXI, Sieling et al \cite{Sieling2014} used the dynamic-clamp technique, in which simulated membrane and synaptic currents are artificially added or subtracted from neurons, to examine whether selective changes in single conductances governing cell excitability and electrical coupling are responsible for the associative modification of feeding circuit output and behavior (Fig. \ref{use_case_1_image}). Using \textit{in vitro} preparations of buccal ganglia isolated from naive and operantly trained animals, Sieling et al either enhanced or diminished neuronal excitability and coupling strengths in RT to test for causation and determine respective contributions of synaptic and non-synaptic processes by which associative learning leads to expression of compulsive behavior.  
%
%\begin{figure}[h]
%\begin{center}
%%\includegraphics[width=\textwidth,keepaspectratio]{Sieling-01.png}
%\caption{Artificially modulating neuronal excitability mimics learning-induced increase in frequency, but not the regularity, of buccal motor pattern (BMP) genesis. (A) Experimental protocol and equivalent electrical circuit for the addition of a dynamic-clamp-defined leak conductance (G$_{leak}$) to the natural input conductance (G$_{in}$) of an individual neuron using RTXI. (B) Introduction of an artificial G$_{leak}$ of -60 nS (shaded panel) increased the excitability of a target B63 neuron (indicated by a decrease in spike threshold) compared with that arising from the natural leak conductance alone (i.e., G$_{leak}$: 0 nS). Horizontal and vertical scale bars represent 2 s and 20 mV, respectively. In (C), G$_{leak}$ was introduced into one of B63, B30, or B65. In (D), G$_{leak}$ was introduced into a B63 and current pulses for measuring coupling coefficients were injected into either postjunctional B30 or B65. (For details, see \cite{Sieling2014} Fig. S1.) (E) In a control preparation, the frequency, but not the regularity, of spontaneous BMP genesis and associated spike bursts in B63/B30/B65 increased in response to a dynamic-clamp-defined G$_{leak}$ of -60 nS (shaded panel) introduced simultaneously into the three neurons. Horizontal and vertical scale bars represent 30 s and 25 mV, respectively. (F and G) Quantification of changes in frequency (F), but not irregularity (G), of BMP generation for different values of artificial G$_{leak}$ added simultaneously to the three neurons. Group data show means $\pm$ SEM and individual sample sizes. All figures obtained and modified with permission from \cite{Sieling2014}.}
%%(C and D) Relationship between G$_{leak}$ and spike threshold of B63/B30/B65 (C; data fitting, B63: r$^2$ = 0.977, p $<$ 0.0001; B30: r$^2$ = 0.998, p $<$ 0.0001; B65: r$^2$ = 0.998, p $<$ 0.0001) and coupling coefficients for the B63/B65 and B63/B30 cell pairs (D; data fitting, B63/B65: r$^2$ = 0.995, p $<$ 0.0001; B63/B30: r$^2$ = 0.981, p $<$ 0.0001). 
%\label{use_case_1_image}
%\end{center}
%\end{figure}
%
%\subsubsection*{Use Case 2: Real-time distorted auditory feedback for control of song bird timing}
%In addition to the widely used dynamic clamp, RTXI's flexibility and robustness are useful for a variety of investigations testing causal relationships. One such area is Distorted Auditory Feedback (DAF), which is routinely used to assess the effects of auditory input on vocal production and underlying neural activity in songbirds. Some bird species' songs have been shown to be highly sensitive to acoustic input, showing immediate effects on the timing and acoustic structure of the produced song. Investigation of immediate changes taking place after DAF presentation often requires having a system capable of providing RT acoustic processing and feedback. Commercial systems for RT acoustic processing are available but are expensive and contain a significant number of constraints that must be considered during experimental design. Furthermore, typical experiments using acoustic feedback use detection of a certain template signal, either a certain frequency or more complex combination or sweeps of frequencies, and feedback is delivered upon template detection. Template detection tasks are computationally intensive, requiring spectrogram-based or feature-based techniques \cite{Tchernichovski2000,Leonardo2005}, comparison across detected elements and templates, and if successful, generate the appropriate feedback signal. These computations have to be done fast enough to enable RT performance while not interfering with other processes such as data acquisition, storage, and visualization. 
%
%Skocik and Kozhevinokov \cite{Skocik2012} capitalized on the strengths of RTXI and modified it for experimental use in RT acoustic signal processing and feedback. Acoustic measurements were made by interfacing a microphone and amplifier to the analog inputs present on a National Instruments PCIe-6251 DAQ with a  system frequency of 30 KHz. Acoustic feedback was generated by custom RTXI modules and delivered via the analog outputs on the DAQ. To minimize processing latency, the Data Recorder module was customized to provide two high-speed modes of operations – triggered (active) and non-triggered (idle). The complete  process flow for the customized Data Recorder is shown in Fig. \ref{use_case_2_image}. A circular buffer is inserted into the Data Recorder's processing loop to enable buffering of acquired acoustic signals. The root mean square (RMS) is continuously quantified for the last 10ms of acquire data at a rate of 1 KHz. When the RMS exceeds the user-specified threshold, the system automatically switches to triggered mode operation.
%
%\begin{figure}[h]
%\begin{center}
%%\includegraphics[width=\textwidth,keepaspectratio]{Skocik-01.png}
%\caption{Hard RT distorted auditory feedback system. (A) Simplified diagram of the acoustic feedback system. When not triggered (top), the system computes the root mean square (RMS) of the input signal. When the RMS exceeds the threshold, the system is triggered. When triggered (bottom), the system computes the spectrogram of the most recent 20ms of signal and computes the correlation coefficient of this spectrogram with the spectrogram of the template sound (e.g., song syllable). The template sound is detected when the correlation coefficient exceeds a threshold value; in this case, acoustic feedback can be generated. Both the input and the acoustic output are saved to the computer hard drive. (B) Spectrogram of the song of a Bengalese finch and the times of occurrence of one of the song syllables. The system was programmed to only detect the occurrences of the target syllable in real time, no acoustic feedback was generated. The detection times are shown as vertical red lines. Bottom: the system is detecting the target syllables (vertical red lines) and is generating acoustic feedback after detection. The acoustic feedback waveform is shown below. The feedback signal is one of the birdsong syllables; the acoustic feedback pickup by the microphone is visible on the spectrogram. The zoomed-in spectrogram of the template is shown on the right. (C) DAF increases the duration of the time interval between Bengalese finch song syllables. Histogram depicts the time intervals between two subsequent syllables in the song in the presence of DAF (blue) and without DAF (red). The means are: ${\Delta}t_{mean}$ = 74.8ms (control, \textit{N} = 637 syllables) and ${\Delta}t_{mean}$  = 75.7ms (feedback, \textit{N} = 97 syllables), the difference is statistically significant (\textit{p} = 0.001, two-way Kolmogorov–Smirnov test). All figures obtained and modified with permission from \cite{Skocik2012}.}
%\label{use_case_2_image}
%\end{center}
%\end{figure} 
% 
%Performance was also evaluated in experiments to detect specific syllables in the song of Bengalese finches. The finch song consists of a sequence of syllables separated by silences (inter-syllable gaps) (Fig. \ref{use_case_2_image}), with variability introduced by the specific sequence of syllables in each song. Experiments were conducted to detect specific target syllables, with successful crossing of the correlation coefficient resulting in generation of acoustic feedback (either white noise or the song syllable). Figure \ref{use_case_2_image}B provides sample data demonstrating RTXI's performance in both open-loop (detection only) and closed-loop modes (detection and feedback). System performance was characterized by comparing online syllable detection to off-line detection using spectrogram-based techniques. Over 92\% of the target syllables (n = 659) were correctly identified by RTXI, with zero false positives. The undetected 8\% are believed to be due to natural variability in acoustic structure that are not accounted for by the detection algorithm. Additional tests in Zebra finches resulted in a detection rate up to 96\%(n = 756), with less than 1\% false positives. In a final round of experiments, detection of target syllables was followed by DAF for modulation of timing between song syllables. Without DAF, the mean interval is 74.8 ms (n = 637), while with DAF the mean interval increases to 75.7 ms (n = 97). 
%
%These observations are consistent with previously published observations on the effects of RT feedback on song structure\cite{Sakata}. Moreover, they demonstrate the robust hard-RT capabilities of RTXI and the advantages of open-source software. Using RTXI enabled the authors to set up interfaces with different I/O hardware and modify the RTXI source code to run custom RT protocols.
% 
%\subsubsection*{Use Case 3: EEG feedback-controlled transcranial Alternating Current Stimulation}
%In addition to \textit{in vitro} and \textit{in vivo} closed-loop hard RT electrophysiology, RTXI's flexibility, customizability, and hard RT performance has been utilized to investigate the effects of feedback-driven tACS. Brain stimulation using transcranial Alternating Current Stimulation (tACS) has gained significant momentum as an alternative to pharmacological methods for treatment of neurological and psychiatric disorders. By electrically stimulating the brain, aberrant network dynamics can be targeted with potentially higher efficacy, increasing therapeutic efficacy while minimizing undesired off-target effects \cite{Frohlich2014Review,Frohlich2015Review}. Unfortunately, most clinical investigations using tACS are conducted in a feedforward, open-loop manner. Given the success of feedback (closed-loop) control in a variety of engineering and neuroscience applications, Boyle et al\cite{Boyle2013} investigated the ability of electroencephalography (EEG)-measurement driven delivery of tACS to provide better control of cerebral cortex dynamics, which is implicated in several psychiatric illnesses. 
%
%tACS applies a weak sinusoidal electrical current to the scalp, resulting in a change in polarization of a large number of neurons, thus effectively altering neuronal network activity. Evidence from previous studies suggests that sinusoidal stimulation waveforms can be used to selectively modulate cortical oscillations at different frequencies commonly associated with different cognitive states. In their study, Boyle et al combined 40 Hz tACS with EEG measurements to control visual cortex state dynamics and modulate high- and low-alpha states induced by opening and closing the eyes.  
% 
%RTXI was used in this investigation to measure and process incoming EEG activity, compute the appropriate output stimulation waveform in RT, and control timing of the experimental protocol. EEG measurements were made by interfacing with a Biopac EEG amplifier connected to a National Instruments PCI-6221 DAQ with a system frequency of 2 KHz. Measured EEG activity was processed online with custom C++ modules. Stimulation timing and waveforms were generated by RTXI and delivered to an isolation unit through the DAQ's analog output channel. Figure \ref{use_case_3_image} depicts the custom closed-loop, RT protocol engineered for this investigation.  
%
%\begin{figure}[h]
%\begin{center}
%%\includegraphics[width=\textwidth,keepaspectratio]{Frohlich-01.png}
%\caption{Optimization of therapeutic benefit of tACS via closed-loop EEG feedback-controlled tACS. (A) Hard real-time closed-loop protocol used for feedback-controlled delivery of tACS. EEG are measured and amplified prior to sampling with RTXI while the subject opens and closes their eyes every 30 seconds. EEG data from international 10-20 system sites O2, A1, and A2 are processed in hard RT using a custom module written to measure the alpha band power within a one second window. The computed power is validated against a threshold criteria, which determines the tACS amplitude to be delivered. The output from RTXI is connected to an external current-controlled stimulation and isolation unit. Sample alpha-filtered EEG traces with and without feedback-controlled tACS are shown in (B). Feedback-controlled tACS (C) almost completely suppressed alpha band power in a targeted way (Ratio EC$_\alpha$ to EO$_\alpha$ = 1.03, \textit{p} = 0.041). Dose-matched random tACS (D) also suppressed alpha band power (normalized random-No Stim = -0.31, \textit{p} = 0.0183), but was less effective than feedback-controlled tACS (normalized feedback$_\alpha$ - random$_\alpha$ = -0.10, \textit{p} = 0.0145). All figures obtained and modified with permission from \cite{Boyle2013}.}
%\label{use_case_3_image}
%\end{center}
%\end{figure}
%
%All data processing and analysis was conducted in RT with custom modules written in C++. The incoming EEG data was filtered with a 6\textsuperscript{th} order Butterworth band-pass filter with a bandwidth of 8 – 12 Hz to isolate the alpha band. Power was then computed for the alpha band as the mean oscillation power with a 1 second window width. During the calibration period in each experiment, the module built a distribution with the computed alpha power values to determine the mean alpha power for both eyes-open and eyes-closed conditions. Stimulation thresholds were set to 1.05 of the average of the two mean alpha power values. The protocol then continued to assess mean alpha power at a rate of 0.25 Hz and modulated the stimulation depending on the measured alpha power. If alpha power was higher than the stimulation threshold, tACS was delivered to the subject for the first 2 seconds of the next epoch, otherwise stimulation was turned off. Each subject completed a total of four 12 minute recording sessions. Each recording started with a 2 minute calibration period. For the first 60 seconds, the subjects were asked to relax and be still with their eyes open. For the next 60 seconds, the subjects were asked to relax, be still, and close their eyes. After calibration, two recordings were conducted in each subject. First, subjects were asked to keep their eyes open (EO) and received EEG feedback-controlled tACS. For the second recording, subjects were asked to open and close their eyes when told by the experimenter every 30 seconds (EOEC) while receiving EEG-feedback driven tACS. These two recordings were repeated, however subjects received dose-matched randomly administered tACS.  
% 
%The results of this investigation found that EEG feedback-controlled tACS can successfully suppress alpha power as well as state transitions caused by opening and closing of eyes. Subsequent investigations by the same group using RTXI have also demonstrated the ability of closed-loop tACS to boost sleep spindle activity and sleep-dependent motor memory consolidation\cite{Lustenberger2016}. These results demonstrate the customizability, modularity, and robust closed-loop RT performance of RTXI.
%
%%%% Future directions/availability
%\section*{Availability}
%\label{sec:availability}
%RTXI is available under the GNU GPLv3 license and is publicly accessible online via GitHub (Supplement 3\ref{itm:source_code}) along with over 50 custom modules contributed by the RTXI community. This repository enables tracking of user-submitted bugs and requests, and makes the continued development of RTXI publicly viewable. GitHub also allows users to watch the progress and development of RTXI, receive notifications as new updates or patches are submitted, and easily share custom modules built by any RTXI user. Users are encouraged to use this platform to communicate with the RTXI team with questions, comments, feature requests, and troubleshooting advice. 
% 
%In addition to GitHub, a website (Supplement 3\ref{itm:website}) is maintained and routinely updated with new troubleshooting tips and FAQs (Supplement 3\ref{itm:tutorials}) as well as a user manual and documentation (Supplement 3\ref{itm:user_manual}). Publications utilizing RTXI are indexed and listed for users to access key details associated with experimental setups, module use, and system parameters - enabling greater reproducibility of shared data. The RTXI team is dedicated to helping users with any aspect of RTXI – whether it is installation, creating custom modules, or integration of addition experimental hardware.
%
%%\begin{table}[ht]
%%\centering
%%\rowcolors{2}{gray!25}{white}
%%\footnotesize
%%    \def\arraystretch{1.3}
%%    \begin{tabularx}{\textwidth}{m{2.75cm}|m{5cm}|m{2cm}|m{1.75cm}}
%%    \rowcolor{gray!50}
%%    \textbf{System} & \textbf{Technology \& Limitations} & \textbf{Control} & \textbf{Plugin API}\\
%%
%%    DynClamp2/4 \cite{Pinto:2001ul} & 
%%    Windows; only for DigiData card &
%%    Soft-RT & 
%%    No \\ 
%%    
%%    StdpC \cite{Kemenes:2011bt} & 
%%    Windows; only for DigiData card &
%%    Soft-RT & 
%%    No \\ 
%%    
%%    G-clamp \cite{Kullmann:2004vl} & 
%%    Windows; LabView RT with NI RT card &
%%    Hard-RT & 
%%    No \\ 
%%
%%    SM-1; SM-2 \cite{Robinson:2008dj} & 
%%    Requires custom analog hardware &
%%    Hard-RT & 
%%    No \\ 
%%
%%    QuB \cite{Milescu:2008uq} & 
%%    Windows XP; NI DAQ Drivers & 
%%    Soft-RT & 
%%    No \\ 
%%
%%    mafDC \cite{Yang2015} & 
%%    Windows; NI DAQs &
%%    Soft-RT  & 
%%    No \\
%%    
%%    RTBiomanager \cite{Muniz:2009vj} & 
%%    Linux; Comedi; No online presence or download available &
%%    Hard-RT & 
%%    No \\ 
%%
%%    BCI2000 \cite{schalk:2004a} & 
%%    Windows; latency $>$1ms & 
%%    non-RT  & 
%%    No \\ 
%%
%%    RELACS & 
%%    Linux; requires deprecated dependencies &
%%    Hard-RT  & 
%%    No \\ 
%%    
%%    MEABench \cite{wagenaar_meabench:_2005} & 
%%    Windows; Multi-Channel Systems amplifiers and PCI cards; \textgreater 100ms latencies; custom hardware required & 
%%    Soft-RT  & 
%%    No \\ 
%%
%%    LCG \cite{Linaro2014} & 
%%    RT Linux; minimal flexibility; command line interface (not user-friendly) &
%%    Hard-RT  & 
%%    No \\ 
%%
%%    NeuroRighter \cite{Newman2012} & 
%%    Windows; multiple NI cards; custom hardware required; pre-programmed protocols only & 
%%    Soft-RT  & 
%%    Yes \\ 
%%
%%    NeurOnline \cite{stefan_neuronline:_2011} & 
%%    Python; C++; Comedi; Commercial multi-electrode arrrays  &
%%    Soft-RT  & 
%%    No \\
%%
%%    Open Ephys & 
%%    Windows, Mac, and Linux, FPGA-based &
%%    Soft-RT  & 
%%    Yes \\
%%        
%%    MATLAB \cite{clausen_matlab_2013} &
%%    MATLAB-RT, Windows, NI card, dedicated hardware, pre-programmed protocols only & 
%%    Hard-RT  & 
%%    No \\ 
%%    
%%    Tucker-Davis \\Technologies &
%%    Proprietary hardware, pre-programmed protocols only &
%%    Hard-RT &
%%    Yes \\
%%    
%%    National Instruments &
%%    LabViewRT, Proprietary hardware, pre-programmed protocols only &
%%    Hard-RT &
%%    Yes \\
%%
%%    \end{tabularx}
%%    \caption{Existing commercial and open-source alternatives for closed-loop electrophysiology {\label{others_table}}}
%%\end{table}
%
%%{The attractiveness of RTXI to scientists is apparent when compared to alternative commercial and open-source systems for closed-loop experimental control (Table \ref{others_table}). None of the systems listed offer RTXI's combination of 1) guaranteed hard real-time performance with microsecond closed-loop latencies, 2) a flexible API that enables a diverse range of applications, and 3) a mature and developed user and code base.}
%
%\section*{Future Directions}
%\label{sec:future}
%Advances in measurement and stimulation technologies over the last decade have increased both the spatial and temporal resolution at which scientists can investigate biological systems. Enhancements in micro-fabrication technologies have enabled development of high density silicon probes \cite{Scholvin2016,Frey2010}, allowing large-scale spatial and temporal over-sampling of biological activity \cite{Buzsaki2004}. Development of ASICs has led to wide adoption of systems such as the low-cost and easy-to-use Intan headstages \cite{Harrison2007,Harrison2015P}. Furthermore, recent developments in imaging technologies have made it possible to consider closed-loop applications in which the measured signal is an image representative of underlying electrical activity, rather than direct measurement of electrical potentials. 
%
%On-going development efforts within RTXI are focused on incorporating new measurement modalities (e.g., imaging) and acquiring from high channel count interfaces -- all with hard RT closed-loop performance. For example, a image acquisition and processing module (GenICam) and a Ethernet-based data acquisition module (EthernetAcq) are now available for use within RTXI (Supplement 3\ref{itm:user_modules}). In many cases, hard RT closed-loop control with image or high channel count data processing can require more computation time than is available per cycle. On-going RTXI development efforts are also focused on providing API calls for distributing computational loads across dedicated processor cores and GPUs, with the goal of requiring little to no technical know-how on the user's end.
%
%The advent of optogenetics has enabled tight spatial and temporal control of stimulation of biological tissues \cite{Zhang2006o,Yizhar2011,Arrenberg2010}. These developments have spawned a significant interest in closed-loop control of biological systems \cite{Grosenick2015}, such as optical clamping of network activity \cite{Newman2015} and optical shortening of cardiac action potential durations \cite{Govorunova2016}. In most cases, delivery of optogenetic stimuli via a laser or diode is controlled and modulated by a voltage signal. For example, users can generate a standard stimulus waveform using the Signal Generator module or a custom stimulus waveform by providing an ASCII file to the Waveform Maker module and use it to control a laser or diode to achieve hard RT optogenetic stimulation. Such re-utilization of existing code and custom modules simplifies the transition to hard RT closed-loop control.
%
%%%% Misc
%\section*{Acknowledgments}
%All the work presented in this manuscript is a result of the effort put in by many individuals. First, we would like to thank Calin Culianu, Jonathan Bettencourt, Risa Lin, and Francis Ortega for their significant and valuable contributions to the development of RTXI that continue to be a part of its core. Next, we would like to thank authors of the highlighted use cases, Michale Skocik, Romuald Nargeot, Michael Boyle, Caroline Lustenberger, Sankar Alagapan, and Flavio Frohlich for openly providing data, figures, and feedback on this manuscript. In addition, the authors also thank Laura O'Farrell, Brian Kim, and Will Rountree for comments on the manuscript and how to communicate highly technical concepts in a simpler and effective manner. Most importantly, the authors would like to thank the RTXI user-community and greater open-source hardware/software communities for continued feedback, interest, and use of RTXI. Their contributions have helped develop a long-term, stable product that enables new paradigms of scientific investigations. 
%
%\section*{Author Contributions}
%Contributed to the development of RTXI: YAP, AG. Wrote the paper: YAP, AG, ADD, JAW, DJC, and RJB.
%

%%% Supplemental Information
% Include only the SI item label in the subsection heading. Use the \nameref{label} command to cite SI items in the text.
\section*{Supporting Information}
\subsection*{S1 Terminology}
\label{sec:terms}
\begin{itemize}
\item Hard real-time - guaranteed performance of I/O within specified bound; failure to achieve bounded performance results in a system crash\label{itm:hardrt}
\item Soft real-time - guaranteed performance of I/O within specified bounds; failure to achieve bounded performance is acceptable and does not effect system stability\label{itm:softrt}
\item Open-loop - a process that computes an output based upon an input only (no feedback signal)\label{itm:openloop}
\item Closed-loop - a process that computes an output based upon an input and feedback signal\label{itm:closedloop}
\item Kernel - core of an operating system that enables bi-directional communication between software and hardware\label{itm:kernel}
\item Thread - a sequence of machine instructions for execution\label{itm:thread}
\item Scheduler - an operating system component that manages allocation of system resources\label{itm:scheduler}
\item Synchronous - unbuffered data acquisition or processing\label{itm:sync}
\item Asynchronous - buffered data acquisition  or processing\label{itm:async}
\item Jitter - deviation from the desired real-time period\label{itm:jitter}
\item Real-time period - duration between the real-time thread waking up (becoming active) and going to sleep (becoming idle)\label{itm:rtperiod}
\item Live CD - a complete bootable operating system with applications and drivers ready to run from a computer's memory; requires no installation\label{itm:livecd}
\item Daemon - a computer process that runs as a background process (e.g., Dropbox, Skype but not Web Browsers or File Managers).\label{itm:daemon}
\end{itemize}

%\subsection*{S2 Abbreviations}
%\label{sec:abrev}
%\begin{itemize}
%\item RTXI - Real-Time eXperiment Interface
%\item RTOS - Real-Time Operating System
%\item GUI - Graphical User Interface
%\item I/O - Input/Output
%\item DAQ - Data Acquisition
%\item ADC - Analog-to-Digital Converter
%\item DAC - Digital-to-Analog Converter
%\item AI - Analog Input
%\item AO - Analog Output
%\item DIO - Digital Input/Output
%\item SISO - Single-Input Single-Output
%\item SIMO - Single-Input Multiple-Output
%\item MISO - Multiple-Input Single-Output
%\item MIMO - Multiple-Input Multiple-Output
%\item FLOPS - Floating-Point Operations per Second
%\item HDF - Hierarchical Data Format
%\item EEG - Electroencephalography
%\item tACS - Transcranial Alternating Current Stimulation
%\item ASIC - Application Specific Integrated Circuit
%\item CPU - Central Processing Unit
%\item GPU - Graphics Processing Unit
%\item RPM - Revolutions per minute
%\item GB - Gigabytes
%\item GHz - Gigahertz
%\item KHz - Kilohertz
%\item RAM - Random Access Memory
%\end{itemize}
%
%\subsection*{S3 Useful Links}
%\label{sec:links}
%\begin{enumerate}[label=\textnormal{\Alph*}]
%\item Website: \url{https://rtxi.org}\label{itm:website}
%\item Source code: \url{https://github.com/RTXI/rtxi}\label{itm:source_code}
%\item Compatible Hardware: \url{http://rtxi.org/docs/manual/#daqs}\label{itm:hardware}
%\item User manual: \url{http://rtxi.org/docs/manual}\label{itm:user_manual}
%\item Tutorials and FAQs: \url{http://rtxi.org/docs/}\label{itm:tutorials}
%\item User contributed modules: \url{http://rtxi.org/modules/}\label{itm:user_modules}
%\item Support: \url{http://rtxi.org/contact/}\label{itm:contact}
%\item Publications using RTXI: \url{http://rtxi.org/papers/}\label{itm:papers}
%\end{enumerate}

%\nolinenumbers
%\pagebreak
%
%%%% Refs
%% OS Specifics
%%\ifmacosx
%\bibliography{rtxi-revised.bib}
%%\bibliography{/Users/Yogi/Dropbox/Documents/library.bib,rtxi.bib}
%%\fi
%%\iflinux
%%\bibliography{/home/yapatel/Dropbox/Documents/%library.bib,rtxi.bib}
%%\fi
%
%%\begin{landscape}
%%\begin{figure}
%%\begin{center}
%%\includegraphics[keepaspectratio,angle=-90]{entire3.png}
%%\caption{Screenshot of arbitrary RTXI workspace with core and custom modules. The workspace is intended to demonstrate the level of user-configurability provided by RTXI, such as channel configurations, combinations of data I/O connections, and saving virtually every programmed element of each module. Additional custom modules are shown (e.g., Spike Detector, FIR Filter, Signal Generator, Neuron Model, etc) to demonstrate the versatility of RTXI's API system for creation of any utility or model to be used in hard RT, closed-loop experiments. We refer the reader to the up-to-date user manual (Supplement 3\ref{itm:user_manual}) and tutorials (Supplement 3\ref{itm:tutorials}) available online for more information on how to navigate the RTXI workspace.}
%%\label{entire}
%%\end{center}
%%\end{figure}
%%\end{landscape}
%
%%\begin{figure}
%%\begin{center}
%%\includegraphics[width=\textwidth,keepaspectratio]{system_arch.png}
%%\caption{System architecture. The bottom block depicts the hardware layer with which RTXI interfaces. RTXI is capable of interfacing with DAQs using either PCI/PCIe, USB, or Ethernet interfaces (see \nameref{sec:hardware} for more information). Hard RT communication with hardware devices is achieved through Analogy, a set of drivers within the Xenomai framework. The top block of the diagram illustrates the core architecture of RTXI. On each cycle of the RT period, the RT Thread wakes up, acquires new data, executes instructions defined within the hard RT function of both core and user modules (see \nameref{sec:user_code_text}), outputs data to the DAQ, and returns to sleep (idle). Transmission of data to/from different modules is handled by the \texttt{IO} class. When the RT Thread is idle, resources are made available to other system applications and functions. The GUI and Data Storage Threads continuously run with a static period to provide a stable balance between hard RT performance and handling of user input, updating of visualizations, and data storage.}
%%\label{system_arch}
%%\end{center}
%%\end{figure}
%
%%\begin{figure}
%%\begin{center}
%%\includegraphics[width=\textwidth,keepaspectratio]{hdf5file.png}
%%\caption{Hierarchical Data Format data file generated by RTXI. Each trial is represented as an ordered group of data and metadata objects. By default, RTXI saves parameter values from all modules when recording is started into the ``Parameters'' group. All enabled DAQ AI channel configuration settings are also saved into the ``System Settings'' group. HDF5 files can be opened, modified, and appended to from a variety of post-hoc analysis frameworks (e.g., HDFView, MATLAB, R, Python). }
%%\label{hdf5file}
%%\end{center}
%%\end{figure}
%
%%\begin{figure}
%%\begin{center}
%%\includegraphics[width=\textwidth,keepaspectratio]{user_code.png}
%%\caption{Example template code available (Supplement 3\ref{itm:user_modules}) to users for writing custom modules, and corresponding module loaded within RTXI. (A) Basic C++ header (\texttt{plugin\_template.h}) for an RTXI module. (B) Users can declare inputs, outputs, states, parameters, and events for custom modules through the DefaultGUIModel struct and (C) implement hard RT code within the execute function. (D) The \texttt{update} function allows execution of state-specific code for the module. (E) The module after it is compiled with the provided Makefile and loaded into RTXI. Each element of the module GUI is tied to a specific line of code. For example, when ``Button A'' is clicked, the \texttt{aBttn\_event()} (line 35, A) code is executed. Similarly, the ``GUI label'' (lines 5-6, B) and ``A State'' (lines 11-13, B) components of the GUI are created by the \texttt{vars} struct and their values are initialized from the \texttt{INIT} block of the \texttt{update()} function (lines 8-9, C). Visit the GitHub repository (Supplement 3\ref{itm:user_modules}) for complete corresponding C++ implementation (\texttt{plugin\_template.cpp}) file.}
%%\label{user_code_fig}
%%\end{center}
%%\end{figure}
%
%%\begin{landscape}
%%\begin{figure}
%%\begin{center}
%%\includegraphics[height=0.65\textwidth,keepaspectratio]{io.png}
%%\caption{System performance under different computational loads and closed-loop configurations. (A) System configurations tested. (B) System I/O latency does not exceed twice the system period (50 $\mu$s) nor are I/O events missed in all test cases, demonstrating hard RT performance and a delay of no more than one sample. (C) Distribution of measured RT periods in each tests case by the Real-Time Benchmarks module. The RT period should never exceed twice the desired RT period, otherwise hard RT behavior is compromised. The increase in RT period variability with increasing computational loads is predominantly from the initial time frame after which a protocol starts execution. This is represented by the RT period jitter data (D), which demonstrates the RT period stabilizing within the first few minutes of execution. (E) Time spent processing the \texttt{execute()} function of all loaded module on each RT loop cycle.}
%%\label{io}
%%\end{center}
%%\end{figure}
%%\end{landscape}
%
%%\begin{figure}
%%\begin{center}
%%\includegraphics[width=\textwidth,keepaspectratio]{Sieling-01.png}
%%\caption{Artificially modulating neuronal excitability mimics learning-induced increase in frequency, but not the regularity, of buccal motor pattern (BMP) genesis. (A) Experimental protocol and equivalent electrical circuit for the addition of a dynamic-clamp-defined leak conductance (G$_{leak}$) to the natural input conductance (G$_{in}$) of an individual neuron using RTXI. (B) Introduction of an artificial G$_{leak}$ of -60 nS (shaded panel) increased the excitability of a target B63 neuron (indicated by a decrease in spike threshold) compared with that arising from the natural leak conductance alone (i.e., G$_{leak}$: 0 nS). Horizontal and vertical scale bars represent 2 s and 20 mV, respectively. In (C), G$_{leak}$ was introduced into one of B63, B30, or B65. In (D), G$_{leak}$ was introduced into a B63 and current pulses for measuring coupling coefficients were injected into either postjunctional B30 or B65. (For details, see \cite{Sieling2014} Fig. S1.) (E) In a control preparation, the frequency, but not the regularity, of spontaneous BMP genesis and associated spike bursts in B63/B30/B65 increased in response to a dynamic-clamp-defined G$_{leak}$ of -60 nS (shaded panel) introduced simultaneously into the three neurons. Horizontal and vertical scale bars represent 30 s and 25 mV, respectively. (F and G) Quantification of changes in frequency (F), but not irregularity (G), of BMP generation for different values of artificial G$_{leak}$ added simultaneously to the three neurons. Group data show means $\pm$ SEM and individual sample sizes. All figures obtained and modified with permission from \cite{Sieling2014}.}
%%%(C and D) Relationship between G$_{leak}$ and spike threshold of B63/B30/B65 (C; data fitting, B63: r$^2$ = 0.977, p $<$ 0.0001; B30: r$^2$ = 0.998, p $<$ 0.0001; B65: r$^2$ = 0.998, p $<$ 0.0001) and coupling coefficients for the B63/B65 and B63/B30 cell pairs (D; data fitting, B63/B65: r$^2$ = 0.995, p $<$ 0.0001; B63/B30: r$^2$ = 0.981, p $<$ 0.0001). 
%%\label{use_case_1_image}
%%\end{center}
%%\end{figure}
% 
%%\begin{figure}
%%\begin{center}
%%\includegraphics[width=\textwidth,keepaspectratio]{Skocik-01.png}
%%\caption{Hard RT distorted auditory feedback system. (A) Simplified diagram of the acoustic feedback system. When not triggered (top), the system computes the root mean square (RMS) of the input signal. When the RMS exceeds the threshold, the system is triggered. When triggered (bottom), the system computes the spectrogram of the most recent 20ms of signal and computes the correlation coefficient of this spectrogram with the spectrogram of the template sound (e.g., song syllable). The template sound is detected when the correlation coefficient exceeds a threshold value; in this case, acoustic feedback can be generated. Both the input and the acoustic output are saved to the computer hard drive. (B) Spectrogram of the song of a Bengalese finch and the times of occurrence of one of the song syllables. The system was programmed to only detect the occurrences of the target syllable in real time, no acoustic feedback was generated. The detection times are shown as vertical red lines. Bottom: the system is detecting the target syllables (vertical red lines) and is generating acoustic feedback after detection. The acoustic feedback waveform is shown below. The feedback signal is one of the birdsong syllables; the acoustic feedback pickup by the microphone is visible on the spectrogram. The zoomed-in spectrogram of the template is shown on the right. (C) DAF increases the duration of the time interval between Bengalese finch song syllables. Histogram depicts the time intervals between two subsequent syllables in the song in the presence of DAF (blue) and without DAF (red). The means are: ${\Delta}t_{mean}$ = 74.8ms (control, \textit{N} = 637 syllables) and ${\Delta}t_{mean}$  = 75.7ms (feedback, \textit{N} = 97 syllables), the difference is statistically significant (\textit{p} = 0.001, two-way Kolmogorov–Smirnov test). All figures obtained and modified with permission from \cite{Skocik2012}.}
%%\label{use_case_2_image}
%%\end{center}
%%\end{figure}
%
%%\begin{figure}
%%\begin{center}
%%\includegraphics[width=\textwidth,keepaspectratio]{Frohlich-01.png}
%%\caption{Optimization of therapeutic benefit of tACS via closed-loop EEG feedback-controlled tACS. (A) Hard real-time closed-loop protocol used for feedback-controlled delivery of tACS. EEG are measured and amplified prior to sampling with RTXI while the subject opens and closes their eyes every 30 seconds. EEG data from international 10-20 system sites O2, A1, and A2 are processed in hard RT using a custom module written to measure the alpha band power within a one second window. The computed power is validated against a threshold criteria, which determines the tACS amplitude to be delivered. The output from RTXI is connected to an external current-controlled stimulation and isolation unit. Sample alpha-filtered EEG traces with and without feedback-controlled tACS are shown in (B). Feedback-controlled tACS (C) almost completely suppressed alpha band power in a targeted way (Ratio EC$_\alpha$ to EO$_\alpha$ = 1.03, \textit{p} = 0.041). Dose-matched random tACS (D) also suppressed alpha band power (normalized random-No Stim = -0.31, \textit{p} = 0.0183), but was less effective than feedback-controlled tACS (normalized feedback$_\alpha$ - random$_\alpha$ = -0.10, \textit{p} = 0.0145). All figures obtained and modified with permission from \cite{Boyle2013}.}
%%\label{use_case_3_image}
%%\end{center}
%%\end{figure}
%
\end{document}
