%!TEX root = main.tex

We now evaluate the performance of our covert channels
with respect to the transmission rate and robustness against application interference.

\subsection{Bit error rate}

In Section~\ref{sec:powerprofs}, we show that it is possible to distinguish between power consumption
profiles of the \texttt{memset} when the size of the memory chunks use are sufficiently different.
But, to implement a binary channel, we only need to distinguish between two DRAM power consumption
states: doing nothing and \texttt{memset}
some sufficiently large memory chunk. The size of this memory chunk must be small enough to enable
high rate communication, but also large enough to be distinguishable from the baseline DRAM power consumption.

To find good sizes for the memory chunk, we ran Trojans simulations using memory chunks of different
sizes considering the notebook and desktop setups. For the notebook, we considered
2, 5, and 10 times its L3 size. For the desktop, we considered two sequential \texttt{memset} operations on
memory chunks of sizes 0.8, 1, and 1.5 times its L3 cache size.

The simulations consisted in a Trojan process transmitting 10 different 1000 bits random messages at increasing
rates. The spy used the first 100 bits for synchronization using the maximum likelihood algorithm discussed
in Section~\ref{sec:sync}., and then decoded the remaining bits.
The decoding threshold $\overline{P}$ is dynamically computed as the average of the power
samples.\footnote{Notice that this works because the binary messages are random, thus we expected them to have a similar number of 0's and 1's.}
The first column of Figure~\ref{fig:result_mem} (Baseline) shows the results of our experiments.


\begin{figure}[!ht]
  \centering
  \includegraphics[scale=1]{graphs/eval/mem_bit_error_benchmarks_syncing_highrates.pdf}

  \caption{Error rates for DRAM-based Trojans transmitting at increasing rates when the underlying system
    is under different workloads. L3\textsubscript{D} and L3\textsubscript{N} denote the L3 cache sizes of the
    desktop and notebook platforms, respectively.}
  \label{fig:result_mem}
\end{figure}

We can see that the larger memory chunks are useful when the transmission
rate is lower, because it makes it easier to distinguish between 0's and 1's.
However, for higher transmission rates, larger sizes do not work very well because operations on them are
slow, which cause synchronization problems.

This simple channel implementation is capable of achieving high transmission rates,
without big differences between the two setups.
For example, it can transmit at 400 bps with almost 0\% error rate,
and at 600 bps with less than 10\% error rate.


\subsection{Robustness of the channels}

To assess the robustness of the covert channels with respect to application interference, we considered
the channels' performance under 3 different workloads: MRBench~\cite{kim2008mrbench}, Terasort~\cite{o2008terabyte},
and LINPACK~\cite{dongarra1993performance}.
For each workload, we tested the DRAM-based Trojans using the
parameters considered in the previous section.

The results are shown in the corresponding columns
of Figure~\ref{fig:result_mem}.
Under the desktop setup, the channel tolerates MRBench and Terasort with around 15\%
error rate at 300 bps, but LINPACK makes the channel very unreliable.
With respect to the notebook setup, the DRAM-based Trojan appears to be fairly robust,
achieving 500 bps at error rates close to 10\% under MRBench and Terasort, and 300 bps
under the LINPACK workload with less than 15\% error.

It is interesting to see that LINPACK is the one
to affect the channel the most, which is not surprising given its
power consumption profile presented in Section~\ref{sec:powerprofs:bench}.


