TripleCross/docs/chapters/chapter4.tex

\chapter{Design of a malicious eBPF rootkit}
In the previous chapter, we discussed the capabilities of eBPF programs from a security standpoint, detailing which helpers and program types are particularly useful for developing malicious programs, and analysing some techniques (stack scanning, overwriting packets together with TCP retransmissions) which helps us circumvent some of the limitations of eBPF.

Taking as a basis these capabilities, this chapter is now dedicated to a comprehensive description of our rootkit, including the techniques and functionalities implemented, thus showing how these capabilities can lead to the creation of a real malicious application. As we mentioned during the project objectives, our goals for our rootkit include the following:
\begin{itemize}
\item Hijacking the execution of user programs while they are running, injecting libraries and executing malicious code, without impacting their normal execution.
\item Featuring a command-and-control module powered by a network backdoor, which can be operated from a remote client. This backdoor should be controlled with stealth in mind, featuring similar mechanisms to those present in rootkits found in the wild.
\item Tampering with user data at system calls, resulting in running malware-like programs and for other malicious purposes.
\item Achieving stealth, hiding rootkit-related files from the user.
\item Achieving rootkit persistence, the rootkit should run after a complete system reboot.
\end{itemize}

We will firstly present an overview on the rootkit architecture and design. Afterwards, we will be exploring each functionality individually, offering a comprehensive view on how each of the systems work.


\section{Rootkit architecture}
Figure \ref{fig:rootkit} shows an overview of the rootkit modules and components which have been built for this research work.

\begin{figure}[htbp]
	\centering
	\includegraphics[width=15.5cm]{rootkit.png}
	\caption{Overview of the rootkit subsystems and components.}
	\label{fig:rootkit}
\end{figure}

As we can observe in the figure, we can distinguish 6 different rootkit modules, along with a rootkit client which provides remote control of the rootkit over the network from the attacker machine. Also, there exists a rootkit user space process, which is listening for commands issued from the kernel-side, transmitted through a ring buffer.
\begin{itemize}
\item The \textbf{user space process} of the rootkit is in charge of loading and attaching the eBPF rootkit in the kernel, and creating the eBPF maps needed for their operations. For this, it uses the eBPF programs configurator, an internal structure that manages the eBPF modules at runtime, being able to attach or deattach them after a command to do so is received.

The user space process also listens to any data received at the ring buffer, an special map which the eBPF program at the kernel will use to communicate with the user-side, issuing commands and triggering actions from it. Between others actions, the rootkit user space process can spawn TLS clients, execute malicious programs or use the eBPF program configurator for managing the eBPF programs.

\item The \textbf{library injection} module is in charge of hijacking the execution of target processes by injecting a malicious library. For this, it uses a set of eBPF tracepoints in the kernel side, and a code caver module in the user side in charge of scanning user processes and injecting shellcode, apart from the malicious library itself, which is prepared to communicate with the attacker's remote client.

\item The \textbf{execution hijacking} module is in charge of hijacking the execution of programs right before the process is even created, modifying the kernel function arguments in such a way that the a new malicious program is called, but the original information is not lost so that the malicious program can still create the original process. Therefore, it hijacks the creation of processes by transparently injecting the creation of one additional malicious process on top of the intended one.

\item The \textbf{privilege escalation} module is in charge of ensuring that any user process spawned by the rootkit will maintain full privilege in the system. Therefore, it hijacks any call to the sudoers file (on which privileged users are listed) so that the user on which the rootkit is loaded is always treated as root. Note that we have not listed this module as one of the main project objetives mainly because it acts as a helper to other modules, such as the execution hijacking one.

\item The \textbf{backdoor} is one of the most critical modules in the rootkit. It has full control over incoming traffic with an XDP program, and outgoing traffic with a TC egress program. As we will see, both the XDP and TC programs are loaded in different eBPF programs, so they use a shared eBPF map to communicate between them.

The backdoor maintains a Command and Control (C2) system that is prepared to listen for specially-crafted network triggers which intend to be stealthy and go unnoticed by network firewalls. These triggers transmit information and commands to the XDP program at the network border, which the backdoor is in charge of interpreting and issuing the corresponding actions, either by writing data at an eBPF map in which other eBPF programs are reading, or issuing an action request via the ring buffer. On top of that, the TC program interprets the data parsed by the XDP program and shapes the outgoint traffic, being able to inject secret messages into packets. 

\item The \textbf{rootkit stealth} module is in charge of implementing measures to hide the rootkit from the infected host. For this, it hijacks certain system calls so that rootkit-related files and directories are hidden from the system.

\item The \textbf{rootkit persistence} module is in charge of ensuring that the rootkit will stay loaded even after a complete reboot of the infected system. For this, it injects secret files at the \textit{cron} system (which will launch the rootkit after a reboot) and at the sudo system (which maintains the privileged permissions of the rootkit after the reboot).

\item The \textbf{rootkit client} is a command-line interface (CLI) program that enables the attacker to remotely control the rootkit at the infected machine. For this, it incorporates multiple operation modes that launch different commands and network triggers. These network triggers, and any other packet sent to the backdoor, are customly designed TCP packets sent over a raw socket, enabling to avoid the noisy TCP 3-way handshake and to control every detail of the packet fields. Each of the messages generated by the client (and sent by the backdoor) follow a custom rootkit protocol, that defines the format of the messages and allows both the client and the backdoor to identify those packets belonging to this malicious traffic. In order to craft these packets, the rootkit client uses a raw sockets library (RawTCP\_Lib) that we have developed for this purpose \cite{rawtcp_lib}.

The RawTCP\_Lib library incorporates packets building, raw socket packet transmissions, and a sniffer for incoming packets. This sniffer is particularly relevant since the client will need to listen for responses by the rootkit backdoor and quickly detect those that follow the rootkit protocol format.

Apart from the network triggers, upon receiving a response by the backdoor the rootkit client can start pseudo-shells connections (commands can be sent to the backdoor and the backdoor executes them, but no shell process is spawned in the client), or spawn TLS servers that establish an encrypted connection with the backdoor. This connection, internally, still uses the custom rootkit protocol to act as a pseudo-shell, enabling to execute commands remotey.
\end{itemize}


With respect to how the rootkit implementation is distributed into multiple programs, we can find that, overall, there exist 4 main components, as shown in figure \ref{fig:rootkit_files}.

\begin{figure}[htbp]
	\centering
	\includegraphics[width=15cm]{rootkit_files.jpg}
	\caption{Rootkit programs and scripts.}
	\label{fig:rootkit_files}
\end{figure}

As we can observe in the figure, the rootkit modules we have overviewed previously are distributed into different files:
\begin{itemize}
\item The program \textit{\textbf{injector}} comprises the rootkit client and the shared library RawTCP\_Lib. This program is to be launched from the attacker machine after a successful infection of a host.
\item The program \textit{\textbf{tc}} contains the TC program needed for managing the egress network traffic. The reason why it is loaded separately is because the libbpf library does not currently incorporate support for integrating TC programs easily as with XDP or tracepoints.

This program is also responsible of creating the shared map which the backdoor will use, and therefore it must be the first part of the rootkit loaded.
\item The program \textit{\textbf{kit}} contains most of the rootkit functionality, spawning the user process and the kernel-side eBPF programs and maps.
\item The \textit{\textbf{packager.sh}} and \textit{\textbf{deployer.sh}} files are scripts which an attacker, upon gaining access to a machine, can use to quickly set up the rootkit and infect the machine:
\begin{itemize}
	\item \textit{packager.sh} compiles the rootkit and prepares the \textit{injector}, \textit{kit} and \textit{tc} files in an output directory to be used (this directory is hidden by the rootkit once it is loaded).
	\item \textit{deployer.sh} uses the output directory to launch the rootkit files in order (first \textit{tc}, then \textit{kit}). It also injects the necessary files into the sudoers.d and cron.d directories (which will be later hidden by the rootkit) to maintain persistence.
\end{itemize}
\end{itemize}



\section{Library injection module}
In this section, we will discuss how to hijack an user process running in the system so that it executes arbitrary code instructed from an eBPF program. For this, we will be injecting a library which will be executed by taking advantage of the fact that the GOT section in ELFs is flagged as writable (as we introduced in section \ref{subsection:elf_lazy_binding} and using the stack scanning technique covered in section \ref{subsection:bpf_probe_write_apps}. This injection will be stealthy (it must not crash the process), and will be able to hijack privileged programs such as systemd, so that the code is executed as root.

We will also research how to circumvent the protections which modern compilers have set in order to prevent similar attacks (when performed without eBPF), as we overview in section \ref{subsection:hardening_elf}.

This technique has some advantages and disadvantages to the one described by Jeff Dileo at DEFCON 27 \cite{evil_ebpf_p6974}, which we will briefly cover before presenting ours. Both techniques will be later compared in chapter \ref{chapter:related_work}.


\subsection{ROP with eBPF} \label{subsection:rop_ebpf}
In 2019, Jeff Dileo presented in DEFCON 27 the first technique to achieve arbitrary code execution using eBPF \cite{evil_ebpf_p6974}. For this, he used the ROP technique we described in section \ref{subsection:rop} to inject malicious code into a process. We will present an overview on his technique, in order to later compare it to the one we will develop for our rootkit, and find advantages and disadvantages. Note that this is a summary and some aspects have been simplified, however we will go in full detail during the explanation of our own technique.

Figure \ref{fig:rop_evil_ebpf_1} shows an overview on the process memory and the eBPF programs loaded. For this injection, we will use the stack scanning technique (section \ref{subsection:bpf_probe_write_apps}) using the arguments of a system call whose arguments are passed using the stack (sys\_timerfd\_settime, which receives two structs utmr and otmr). Therefore, a kprobe is attached to the system call, so that it can start to scan for the return address of the system call, which we know is the original value of register rip which was pushed into the stack (ret).

%TODO This figure needs a remodel. I tried to keep it simple to explain the main concepts on the technique described afterwards, but after writing the next section I realised it gets some things wrong:
% - It does not show .got and .plt sections.
% - It shows the RBP register in an incorrect place.
\begin{figure}[htbp]
	\centering
	\includegraphics[width=15cm]{rop_evil_ebpf_1.jpg}
	\caption{Initial setup for the ROP with eBPF technique.}
	\label{fig:rop_evil_ebpf_1}
\end{figure}

%TODO I don't quite like this. Maybe the glibc bit, because of its importance, is better somewhere else
An additional aspect must be introduced now (we will cover it more in detail in section \ref{TODO}): system calls are not directly called by the instructions in the .text section, but rather user programs in C make use of the C Standard Library to delegate the actual syscall, which in this case is the GNU Standard Library (glibc) \cite{glibc}. Therefore, a program calls a function in glibc (in this case timerfd\_settime) in which the syscall is performed, and the kernel executes it.

This means that, during the stack scanning technique, if we start from struct utmr and scan forward in the stack, what we will find in ret is the return address of the PLT stub that calls the function at glibc, and not directly that of the syscall to the kernel. Therefore, our goal is, for every data in the stack while scanning forward, check whether it is the real return address of the PLT stub we are looking for. For an address to be the real return address, we will follow the next steps:
\begin{enumerate}
\item Take an address from the stack. If that is the return address (the saved rip), then the instruction that called the PLT stub that jumps to the function in glibc must be the previous instruction (rip - 1).
\item We now have a \textit{call} instruction, that directs us to the PLT stub. We take the address stored at the GOT section and jump to the function at glibc.
\item We scan forward, inside timerfd\_settime of glibc, until we find a \textit{syscall} instruction. That is the point where the flow of execution moves to the kernel, so we have checked that the return address we found in the stack truly is the one we are looking for.
\end{enumerate}

Now that we have found the return address, we save a backup of the stack (to recover the original data later) and we proceed to overwrite the stack using bpf\_probe\_write\_user(), setting it for the ROP technique. For this, some gadgets (G0, G1 ... GN) have been previously discovered in the glibc library. Figure \ref{fig:rop_evil_ebpf_2} shows process memory after this overwrite:

\begin{figure}[H]
	\centering
	\includegraphics[width=15cm]{rop_evil_ebpf_2.jpg}
	\caption{Process memory after syscall exits and ROP code overwrites the stack.}
	\label{fig:rop_evil_ebpf_2}
\end{figure}

As we can see in the figure, the function has already exited, and ret has been popped into register rip. As we explained in section \ref{subsection:rop}, the attacker places in that position the address of the first ROP gadget. After that, the attacker can execute arbitrary code. Jeff Dileo, for instance, loads a malicious library into the process (we will do the same and explain this process in the next sections).

Once the attacker has finished executing the injected code, the stack must be restored to the original position so that the program can continue without crashing. A simplified view of this procedure consists of attaching a kprobe to a random system call (in this case, sys\_close()) so that, from the ROP code, we can alert the eBPF program when it is time to remove the ROP code and restore the original stack. Figure \ref{fig:rop_evil_ebpf_3} shows this final step:

\begin{figure}[H]
	\centering
	\includegraphics[width=15cm]{rop_evil_ebpf_3.jpg}
	\caption{Stack data is restored and program continues its execution.}
	\label{fig:rop_evil_ebpf_3}
\end{figure}

As we can see, eBPF writes back the original stack and thus the execution can continue. Note that, in practice, some final gadgets must also be executed in order to restore the state of rip and rsp, the stack data for this is written in the free memory zone, so that it does not need to be removed.


%TODO Eligible to writing more. This was merged with the explanation of each feature before, so it was more extense, but now it might need some more info??
\subsection{Bypassing hardening features in ELFs} \label{subsection:hardening_bypass}
During section \ref{subsection:hardening_elf}, we presented multiple  security hardening measures that have been introduced to prevent common exploitation techniques (such as stack buffer overflows) and that nowadays can be incorporated, usually by default, in ELF binaries generated using modern compilers. We will now explore how to bypass these features, so that we can design an injection technique that can target any process in the system, independently on whether it was compiled using these mitigations.

\textbf{Stack canaries}\\
Since stack canaries will be checked after the vulnerable function returns, an attacker seeking to overwrite the stack must ensure that the value of the canary remains constant. In the context of a buffer overflow attack, this can be achieved by leaking the value of the canary and incorporating it into the overflowing data at the stack, so that the same value is written on the same address \cite{canary_exploit}.

In our rootkit, unlike in the ROP technique presented in section \ref{subsection:rop_ebpf}, we will avoid overwriting the value of the saved rip in the stack completely. Therefore, as long as our eBPF program leaves all registers and stack data in the same state as before calling the function, we will not trigger any alerts.

\textbf{DEP/NX}\\
The only alternative for an attacker upon a non-executable stack is either injecting shellcode at any other executable memory address, or the use of advanced techniques like ROP that fully circumvent this mitigation since the data at the stack is not directly executed at any step.

In our rootkit, we will choose the first option, scanning the process virtual memory for an executable page where we will inject our shellcode. This process is usually known as finding 'code caves'.

\textbf{ASLR}\\
In order to bypass ASLR, attackers must take into account that, although the address at which, for instance, a library is loaded is random, the internal structure of the library remains unchanged, with all symbols in the same relative position, as figure \ref{table:aslr_offset} shows.

%TODO Add the .data section here
\begin{figure}[htbp]
	\centering
	\includegraphics[width=13cm]{aslr_offset.jpg}
	\caption{Two runs of the same executable using ASLR, showing a library and two symbols.}
	\label{fig:alsr_offset}
\end{figure}

As we can observe in the figure, although glibc is loaded at a different base address each run, the offset between the functions it implements, malloc() and free(), remains constant. Therefore, a method for bypassing ASLR is to gather information about the absolute address of any symbol, which can then easily lead to knowing the address of any other if the attacker decompiles the executable and calculates the offset between a pair of addresses where one is known. This is the chosen method for our technique.

\textbf{PIE}\\
Similarly to ASLR, although the starting base address of each memory section is random, the internal structure of each section remains the same. Therefore, if an attacker is able to leak the address of some symbol in a section, and by knowing the offset at which it is located with respect to the base address of the section, then the address of any other symbol in the same section can be calculated \cite{pie_exploit}. This is the technique we will incorporate in our rootkit.

\textbf{RELRO}\\
If an executable was compiled using Partial RELRO, then the value of GOT can still be overwritten. If in turn it was compiled using Full RELRO, this stops any attempt of GOT hijacking, unless an attacker finds an alternative method for writing into the virtual memory of a process that bypasses the read-only flag. 

In our rootkit, we will directly write using eBPF the value of GOT if it was compiled with Partial RELRO, and use an alternative technique for writing into the virtual memory of a process whenever it was compiled using Full RELRO.


\subsection{Library injection via GOT hijacking} \label{subsection:got_attack}
Taking into account the previous background and that about stack attacks, ELF's lazy binding and hardening features for binaries we presented in section \ref{section:elf}, we will now present the exploitation technique incorporated in our rootkit to inject a malicious library into a running process. 

This attack is based on the possibility of overwriting the data at the GOT section. As we have mentioned previously, this section is marked as writeable if the program was compiled using Partial RELRO, meaning that we will be able to overwrite its value from an eBPF program using the helper bpf\_probe\_write\_user(). After modifying the value of GOT, a PLT stub will take the new value as the jump address (as we explained in section \ref{subsection:elf_lazy_binding}), effectively hijacking the flow of execution of the program. In the case that a program was compiled with Full RELRO (which will be the case of many programs running by default in a Linux system such as systemd), we will make use of the /proc filesystem for overwriting this value.

The rootkit will inject the library once an specific syscall is called by a process, but the library injection will only happen after the second syscall, since we need to wait for the GOT address to be loaded by the dynamic linker. This is a necessary step because eBPF will need to validate that it really is the GOT section to overwrite.

This technique works both in compilers with low hardening fetaures by default (Clang) and also on a compiler with all of them active (GCC), see table \ref{table:compilers}. On each of the steps, we will detail the different existing methods depending on the compiler features.

For this research work, the rootkit is prepared to perform this attack on any process that makes use of either the system call sys\_openat or sys\_timerfd\_settime, which are called by the standard library glibc.

We will now describe the multiple exploitation stages for our technique. Appendix \ref{annexsec:lib_injection} shows a flow diagram with the complete process.


\textbf{Stage 1: eBPF tracing and scan the stack}\\
We load and attach a tracepoint eBPF program at the \textit{enter} position of syscall sys\_timerfd\_settime. Firstly, we must ensure that the process calling the tracepoint is one of the processes to hijack.

We will then proceed with the stack scanning technique, as we explained in section \ref{subsection:bpf_probe_write_apps}. In this case, we will take one of the syscall parameters and scan forward in the stack. For each iteration, we must check if the data at the stack corresponds to the saved return address of the PLT stub that jumps to glibc where the syscall sys\_timerfd\_settime is called. Figure \ref{fig:lib_stage1} shows an overview of how these call instructions relate each memory section. 


\begin{figure}[htbp]
	\centering
	\includegraphics[width=13cm]{plt_got_glibc_flow.jpg}
	\caption{Overview of jump and return instructions from the program instructions to the syscall at the kernel.}
	\label{fig:lib_stage1}
\end{figure}

The following are the steps we will follow to perform check some data at the stack is the saved return address:
\begin{enumerate}
\item Check that the previous instruction is a call instruction, by checking the instruction length and opcodes (call instructions always start with e8, and the length is 5 bytes, see figure \ref{fig:firstcall}).
\begin{figure}[htbp]
	\centering
	\includegraphics[width=13cm]{sch_firstcall.png}
	\caption{Call to the glibc function, using objdump.}
	\label{fig:firstcall}
\end{figure}
\item Now that we know we localized a call instruction, we take the address at which it jumps. That should be an address in a PLT stub.
\item We analyse the instructions at the PLT stub. If the program was compiled with GCC, the first instruction will be an \textit{endbr64} instruction followed by the PLT jump instruction using the address at GOT (see figure \ref{fig:plt_gcc}), since it generates Intel CET-compatible programs. Otherwise, if using Clang, which does not generate Intel CET instructions, the first instruction is the PLT jump (see figure \ref{fig:plt_clang}).

We analyse the jump instruction and, again, take the address at which it jumps. This time, it should be the address of the function at glibc.
\begin{figure}[htbp]
	\centering
	\includegraphics[width=14cm]{sch_plt_gcc.png}
	\caption{PLT stub generated with gcc compiler, using objdump.}
	\label{fig:plt_gcc}
\end{figure}
\begin{figure}[htbp]
	\centering
	\includegraphics[width=14cm]{sch_plt_clang.png}
	\caption{PLT stub generated with clang compiler, using objdump.}
	\label{fig:plt_clang}
\end{figure}

\item We now have the address of timerfd\_settime at glibc, from where the syscall will be called. From eBPF, we continue to scan the first opcodes and compare them to those we expect to find at glibc. Specifically, the function would have to contain the instruction opcodes shown in figure \ref{fig:settime_glibc}. Note that, in our version of Ubuntu, we will find Glibc compiled with GCC.

\begin{figure}[htbp]
	\centering
	\includegraphics[width=14cm]{sch_settime_glibc.png}
	\caption{Timerfd\_settime function at glibc, using objdump.}
	\label{fig:settime_glibc}
\end{figure}

\end{enumerate}

Once we ensured we reached the correct glibc function, we are now sure that the data we found at the stack is the return address of the PLT stub that jumped to glibc and called the syscall sys\_timerfd\_settime. Most importantly, we know the address of the GOT section which we want to overwrite.

Our rootkit also incorporates an alternative scanning technique for processes calling the syscall sys\_openat(). This technique enables to scan the stack even when the system call does not incorporate any arguments from the userspace (and thus we cannot take them from our eBPF tracing program to use them as a foothold in the stack).

As we explained in section \ref{subsection:tracing_arguments}, tracepoint programs receive an struct pt\_regs pointer as an argument. We can take this struct and use the value of register rbp as our starting point for scanning the stack. As we can see on figures \ref{fig:plt_clang}, \ref{fig:plt_gcc} and \ref{fig:settime_glibc}, the  PLT does not contain any function prologue (it does not modify the value of rsp) and the function at glibc does not change this value either. Therefore, in our eBPF program, since we are hooking the syscall at the beginning of its execution, the value of rbp will be the original frame pointer before calling the PLT, and therefore we can use it as our starting address for stack scan, proceeding to scan forward until we find the saved return address.

\textbf{Stage 2: Programming shellcode}\\
Once that we have the address of the GOT section, we need to prepare our shellcode to be injected into the process memory. We will overwrite the value at GOT and redirect the flow of execution to the address at which our shellcode is stored in memory. 

Since we want our shellcode to be able to load a library, it will need to call the function \_\_libc\_dlopen\_mode, which can be found in glibc. This function expects to receive as an argument a string with the file path of the malicious library, and therefore the shellcode will also need to call \_\_libc\_malloc to allocate space for the argument. Tables \ref{table:libc_malloc} and \ref{table:libc_dlopen_mode} explain the expected arguments and return value of each function in detail.

\begin{table}[htbp]
\begin{tabular}{|>{\centering\arraybackslash}p{4cm}|>{\centering\arraybackslash}p{10cm}|}
\hline
Register & Value\\
\hline
\hline
edi & Number of bytes to allocate. \\
\hline
rax & Return value, contains the address at which the requested bytes were allocated\\
\hline
\end{tabular}
\caption{Arguments and return value of function \_\_libc\_malloc.}
\label{table:libc_malloc}
\end{table}

\begin{table}[htbp]
\begin{tabular}{|>{\centering\arraybackslash}p{4cm}|>{\centering\arraybackslash}p{10cm}|}
\hline
Register & Value\\
\hline
\hline
rsi & 0x1, indicating flag RTLD\_LAZY\\
\hline
rdi & Address where to read path of library to load\\
\hline
\end{tabular}
\caption{Arguments of function \_\_libc\_dlopen\_mode.}
\label{table:libc_dlopen_mode}
\end{table}

The programs were compiled having ASLR active, and therefore we cannot know the virtual address at which these functions are loaded into the process memory. However, since we have leaked the address of timerfd\_settime at glibc with the previous eBPF scan, we can calculate the address of the other functions, as we introduced in section \ref{subsection:hardening_bypass}. Figure \ref{fig:aslr_bypass_example} shows an example of this process.

\begin{figure}[htbp]
	\centering
	\includegraphics[width=10cm]{aslr_bypass_example.png}
	\caption{Functions at glibc with ASLR active.}
	\label{fig:aslr_bypass_example}
\end{figure}

We will use the example of the figure to illustrate how to calculate the address of the functions:
\begin{enumerate}
\item Decompile using objdump the glibc diagram and calculate the constant offset between the timerfd\_settime function (whose address we will know at runtime) and a reference function usually found in the first addresses of glibc, in this case \_\_libc\_start\_main (this step can be avoided, but it is recommended when searching for many functions and to avoid working with negative offsets). In the example, this offset is 0x30000.
\item Calculate the offset from the reference function \_\_libc\_start\_main to \_\_libc\_dlopen\_mode and \_\_libc\_malloc. In the example, this is 0x20000 and 0x5000 respectively by looking at  decompiled glibc.
\item During runtime, although the ASLR offset will be applied, it will skew all functions inside glibc by the same amount, and therefore the offsets previously calculated will be maintained. By using the previously, calculated offsets, we get that:
\begin{itemize}
	\item \_\_libc\_start\_main = timerfd\_settime - 0x30000
	\item \_\_libc\_dlopen\_mode = \_\_libc\_start\_main + 0x50000
	\item \_\_libc\_malloc = \_\_libc\_start\_main + 0x20000
\end{itemize}
\end{enumerate}

Once we know the address of the functions we want our shellcode to call, we can start to develop it. We will program an x86\_64 assembly program, from which we will extract its opcodes. The shellcode will follow the next algorithm:
\begin{enumerate}
\item Backup the value of all registers, including rbp and rsp. We must ensure that the stack frame is not modified after the shellcode ends, otherwise we may trigger a stack canary alert.
\item Allocate memory for the pathname of the library at the heap using \_\_libc\_malloc.
\item Write into the allocated memory the pathname of our library to load.
\item Call \_\_libc\_dlopen\_mode indicating the allocated memory with the library pathname. Before doing this, we found that reserving an additional stack frame reduces the chances of the process crashing, since apparently the function modifies the stack. By moving rbp and rsp, we prevent the function from modifying any pre-existing data.
\item Restore the original value of the registers, and jump back to the original system call which the glibc function intended to call.
\end{enumerate}

The complete developed shellcode and its opcodes can be found in Appendix \ref{annex:shellcode}.


\textbf{Stage 3: Injecting shellcode in a code cave}\\
Once we have developed our shellcode, and before overwriting the value of GOT, we need to find a memory section where to write our shellcode, so that we can executing the necessary instructions to inject our malicious library. This area must be large enough to fit our shellcode, and it must be marked as executable. 

Because of DEP/NX, we cannot use the stack for executing code. On top of that, as we can observe in the section header dump at Appendix \ref{annexsec:readelf_sec_headers}, for security reasons all sections are nowadays marked either writeable or executable, but never both simultaneously.

Therefore, we will use the proc filesystem which we introduced in section \ref{section:proc_filesystem}. By using the file under \textit{/proc/<pid>/maps}, we will easily identify the address range of those memory sections marked as executable, and by using the file \textit{/proc/<pid>/mem}, we will write our shellcode into that memory section, bypassing the absence of a write flag.

Although we may write freely into any virtual address using this technique, as we saw in section \ref{subsection:proc_maps} executable memory usually corresponds to the .text section. Therefore, we are at risk of overwriting critical instructions of the program. This is the reason why we must search for empty memory spaces inside the virtual memory, called code caves.

We will consider an appropiate code cave as a continuous memory space inside the .text section that consists of a series of NULL bytes (opcode 0x00). Although in principle this may seem like a rare occurence, it is a common find in most processes due to how memory access control is implemented.

In figure \ref{fig:proc_maps_sample}, we can observe how virtual memory sections have a length of 0x1000, or are a multiple of it. This is not an arbitrary number, but rather it is because memory sections must always be of length multiple of the system page length (4 KB = 0x1000 bytes). Therefore, the minimum granularity of a set of permissions over a memory section is of 0x1000 bytes.

Since sections must occupy a multiple of 1000 bytes, this leads to multiple sections which leave lots of empty, NULL bytes, unocuppied without any instructions. This is the reason why we will, quite probably, find a code cave in most processes.

Therefore the steps to find a code cave and inject our shellcode are the following:
\begin{itemize}
\item Send a command from eBPF to the rootkit user space program, indicating that we want to find a code cave in process with an specific PID.
\item Iterate over each entry of \textit{/proc/<pid>/maps}, looking for a sufficiently large code cave in an executable memory section.
\item Inject the shellcode into the code cave using \textit{/proc/<pid>/mem}.
\end{itemize}

Note that, although we used the \textit{/proc/<pid>/maps} file for finding a code cave, this can still be done using the helper bpf\_probe\_read (by taking the return address at the stack and scanning forward in the .text section) or, in the case of programs compiled without PIE, finding an static code cave at the .text section by decompiling the program (since the .text section will be loaded at the same position on every program execution). Still, we would have needed to use \textit{/proc/<pid>/mem} for bypassing the write access prevention.

\textbf{Stage 4: Overwriting GOT}\\
Once the shellcode is loaded at the code cave, eBPF can proceed to overwrite the GOT value with the address of the code cave. As we mentioned, this address is writable using the helper bpf\_probe\_write\_user() if the program was compiled using Partial RELRO, but it cannot be modified if Full RELRO was used. 

Therefore, our rootkit will modify GOT using bpf\_probe\_write\_user() with the address of an static code cave for those programs compiled with Clang (Partial RELRO, no PIE), and use \textit{/proc/<pid>/mem} for modifying GOT with the value of code cave found using \textit{/proc/<pid>/maps} for those programs compiled using GCC (Full RELRO, PIE active).

\textbf{Stage 5: Second syscall, execution of the library}\\
Once we have overwriten GOT with the address of our code cave, the next time the same syscall is called, the PLT stub will jump to our code cave and execute our shellcode. As instructed by it, the malicious library will be loaded and afterwards the flow of execution jumps back to the original glibc function.

%Explain reverse shell?
With respect to the malicious library, it forks the process (to keep the malicious execution in the background) and spawns a simple reverse shell which the attacker can use to execute remote commands.


\section{Privilege escalation module}
In this section we will discuss how the rootkit tampers with the access control permissions in the system, so that unprivileged programs gain root access. Although it is based on a simple technique, it will be used to support other modules launching malicious programs with full privilege (such as the execution hijacking module).

Therefore, the purpose of this section is that, without having to introduce any password, programs executed by an unprivileged user can enjoy privileged access in a infected system.

\subsection{Sudoers file}
Sometimes, unprivileged users need to run a program requiring privileged access. For this, Linux systems incorporate the sudo security policy module, which sets a 'sudo' privilege on users and user groups, allowing them to run a program as root. 

The most widespread and default sudo security policy module is the 'sudoers' policy module, which sets the available sudo permissions of users and groups in the \textit{/etc/sudoers} file \cite{sudoers_man}. In this file, the system administrator can determine the specific permissions of each entity and set different options, including whether they need to introduce the user password when using the 'sudo' command, which is particularly relevant for us. Figure \ref{fig:sudoers} shows the \textit{/etc/sudoers} file of the host we will infect with our rootkit.

\begin{figure}[htbp]
	\centering
	\includegraphics[width=10cm]{sch_sudoers.png}
	\caption{/etc/sudoers file of infected host.}
	\label{fig:sudoers}
\end{figure}

As we can observe in the figure, members of the sudo group are allowed to execute any command as root. Figure \ref{fig:groupfile} shows the users which belong to this group.

\begin{figure}[htbp]
	\centering
	\includegraphics[width=10cm]{sch_groupfile.png}
	\caption{/etc/group file in the infected host.}
	\label{fig:groupfile}
\end{figure}

As we can appreciate, the user osboxes (the default user in the host) is included in this group, and therefore this user is allowed to use sudo and run commands as root.

Any user can check its current sudo privileges by running the command \lstinline{sudo -l} \lstinline{}. Figure \ref{fig:sudol} shows this for the osboxes user.

\begin{figure}[htbp]
	\centering
	\includegraphics[width=10cm]{sch_sudol.png}
	\caption{Sudo privileges of user osboxes, with sudo -l.}
	\label{fig:sudol}
\end{figure}

The value of these entries is taken from the parameters set in figure \ref{fig:sudoers}, where each of the ALL values mean:
\begin{itemize}
\item First ALL: Any user of the group
\item Second ALL: Any host
\item Third ALL: As any user
\item Fourth ALL: Any command
\end{itemize}

Therefore, user osboxes, as part of the sudo group, may run any command as any user in any host as sudo. The host part is not relevant for our us, since it is used when a single sudoers file is distributed betweem multiple machines, but we still have to follow the appropiate format when writing an entry in the \textit{/etc/sudoers} file.

Each time we execute a command with sudo, a process named 'sudo' will open and read the \textit{/etc/sudoers} file, interpreting the contents and allowing or rejecting the action. Note that, although once an user introduces the sudo password it may not be asked again for a period of time, the sudo process will still open and read the \textit{/etc/sudoers} file for each time sudo is used. This aspect is particularly relevant for our technique.


\subsection{Hijacking sudoers read accesses}
We will now discuss how our rootkit tampers with the sudoers policy module. The technique we will present is based on modifying the content that the sudo process reads from the \textit{/etc/sudoers} file, so that what the user process receives is different than that contained in the file. By crafting some special entries in the file, we can grant automatic password-less access to any process we want.

In order to read the contents from the \textit{/etc/sudoers} file, the sudo process will need to perform the following actions:
\begin{itemize}
\item Open the file, using the syscall sys\_openat.
\item Read the file, using the syscall sys\_read.
\end{itemize}

Note that some intermediate or additional syscalls such as sys\_newfstatat, sys\_lseek or sys\_close are also called, but we are not considering them for simplicity.

Table \ref{table:sudoers_syscall} shows the parameters expected by these system calls, based on \cite{syscall_reference}.

\begin{table}[htbp]
\begin{tabular}{|c|>{\centering\arraybackslash}p{8cm}|}
\hline
System call & Arguments\\
\hline
\hline
\multirow{4}{*}{sys\_openat} & \multicolumn{1}{c|}{int dfd}\\
\cline{2-2}
& \multicolumn{1}{c|}{const char \_\_user *filename}\\
\cline{2-2}
& \multicolumn{1}{c|}{inf flags} \\
\cline{2-2}
& \multicolumn{1}{c|}{umode\_t umode} \\
\hline
\multirow{3}{*}{sys\_read} & \multicolumn{1}{c|}{unsigned int fd}\\
\cline{2-2}
& \multicolumn{1}{c|}{char \_\_user *buf} \\
\cline{2-2}
& \multicolumn{1}{c|}{size\_t count} \\
\hline
\end{tabular}
\caption{Arguments of syscalls used by sudo process.}
\label{table:sudoers_syscall}
\end{table}

The table shows that there exist two arguments marked as \textit{\_\_user}, which, as we explained in section \ref{subsection:bpf_probe_write_apps}, can be overwritten from an eBPF tracing program using the helper bpf\_probe\_write\_user(). Therefore, there exist two different attack vectors:
\begin{itemize}
\item Modify the argument \textit{filename}, so that the sudo process opens a fake, crafted sudoers file. In this file we would write the entries needed for our user to have sudo privilege without a password. Since the sys\_open syscall returns a file descriptor, which is later used by sys\_read, that is the only argument needed to be modified.
\item Modify the buffer \textit{buf} in the sys\_read syscall so that it returns specially crafted data to the sudo program.
\end{itemize}

Although the first option is easier, the second technique can not only apply to reading files, but also to any system calls that loads data into an user buffer. Therefore, the privilege escalation module will incorporate the second technique to show the potential of eBPF in this area.

Figure \ref{fig:privilege_esc_module} shows the complete process of the technique we will use.
\begin{figure}[htbp]
	\centering
	\includegraphics[width=15cm]{privilege_esc_module.png}
	\caption{Buffer overwrite technique for the privilege escalation module.}
	\label{fig:privilege_esc_module}
\end{figure}

As we can observe in the figure, we will use three eBPF tracepoints. The reason for this is that, although we are able to write into the user buffer at any tracepoint attached to sys\_read, we would lack information with only one tracepoint:
\begin{itemize}
\item An \textit{enter} tracepoint at sys\_openat knows the file being opened, but it does not have access to the user buffer.
\item An \textit{enter} tracepoint at sys\_read has access to the user buffer, but does not know the name of the file (it only has a file descriptor). Also, if it writes into the buffer now, it will be overwritten later when the kernel reads the \textit{/etc/sudoers} file.
\item An \textit{exit} tracepoint at sys\_read only receives the return value as a parameter (as we explained in section \ref{subsection:tracing_arguments}), but it can freely write to the user buffer if it had access to it, since the kernel already finished writing on it.
\end{itemize}

Taking the above into account, we designed the privilege escalation technique as follows:
\begin{enumerate}
\item We load and attach three eBPF tracepoint programs, and an eBPF map:
\begin{itemize}
	\item An \textit{enter} tracepoint attached to sys\_openat (sys\_enter\_openat).
	\item An \textit{enter} tracepoint attached to sys\_read (sys\_enter\_read).
	\item An \textit{exit} tracepoint attached to sys\_read (sys\_exit\_read).
	\item An eBPF map (fs\_open) that stores fs\_open\_data structs, composed of:
	\begin{itemize}
	\item A process name.
	\item A filename.
	\end{itemize}
	The key of the map fs\_open is the PID of the user process from which the call to an eBPF program originated, this can be obtained using the bpf\_get\_current\_pid\_tgid() helper (see section \ref{subsection:ebpf_helpers}).
\end{itemize}
\item A malicious program we executed from user "osboxes" requests sudo privileges. Our goal is to let it run with privileged permissions without having to introduce a password. Note that, although in the system we are using osboxes is an user in the \textit{/etc/sudoers} file already (although requiring a password for running as sudo), this process also works if we used an user not included on it in the first place.

The sudo process opens the \textit{/etc/sudoers} file. The syscall is called and the sys\_enter\_openat tracepoint is called before the syscall is executed. We check that the syscall was called by the sudo process using the helper bpf\_get\_current\_comm() (see section \ref{subsection:ebpf_helpers}) and, if it is, write the filename into the fs\_open map. After that, the tracepoint exists and the syscall is executed.

\item The sudo process now reads from the file descriptor of the file \textit{/etc/sudoers}. The sys\_enter\_read tracepoint is executed right before the syscall is called. In the tracepoint, we check if we can find an entry with a filename in the fs\_open map using the process PID as key (which is the same for all tracepoints, since they originated from the same sudo process). We now write address of the buffer supplied by the sudo process into the map.

\item The sys\_read syscall is executed and, when it is about to exit, our tracepoint sys\_exit\_read is executed. We take the filename and the address of the user buffer from the fs\_open map, and overwrite the data at the user buffer which contained the bytes read from \textit{/etc/sudoers} using bpf\_probe\_write\_user(). The data we will write resembles a real entry of the \textit{/etc/sudoers} file:
\begin{verbatim}
osboxes ALL=(ALL:ALL) NOPASSWD:ALL #
\end{verbatim}

Injecting that string into the read file will grant us with password-less sudo privileges. There are two particularly relevant details on it:
\begin{itemize}
\item The NOPASSWD option instructs sudo not to request a password.
\item A \# symbol is included at the end so that any data not overwritten at that line is considered a comment (see figure \ref{fig:sudoers}).
\end{itemize}

Although the previous is sufficient for tricking the sudo process into believing we have sudo privileges, it can happen that an user (in this case, osboxes) already has an entry in the \textit{/etc/sudoers} file. When this happens, the sudo process usually chooses the last entry that appears on the file or fails. 

Although not the most elegant solution, the solution for this issue incorporated in our rootkit is that the tracepoint program will continue writing \# symbols until an error happens (thus indicating we reached the end of the file).

\end{enumerate}


\section{Execution hijacking module}
This section describes how the rootkit can hijack the execution of programs. Although in principle eBPF in the kernel cannot start the execution of a program by itself, this module shows how a malicious rootkit may take advantage of benign programs in order to execute malicious code in the user space. Therefore, we aim to achieve two main goals:
\begin{itemize}
\item Execute a malicious user program taking advantage of other program's execution.
\item Be transparent to the user space, that is, if we hijack the execution of a program so that another is run, the original program should be executed too with the least delay.
\end{itemize}

This technique is based on the modification of the arguments of the system call sys\_execve, used to execute programs. When it is called, it causes the program that is currently being run to be completely replaced by the new executed program \cite{execve_man}. Its arguments are listed in table \ref{table:execve_args}

\begin{table}[htbp]
\begin{tabular}{|c|>{\centering\arraybackslash}p{7cm}|}
\hline
Argument & Description\\
\hline
\hline
const char \_\_user *filename & Path and filename of the file to execute\\
\hline
const char \_\_user *const \_\_user *argv & NULL-terminated array with arguments passed to the program\\
\hline
const char \_\_user *const \_\_user *envp & NULL-terminated array with the environment variables associated to the executed program \cite{environ}\\
\hline
\end{tabular}
\caption{Arguments of system call sys\_execve.}
\label{table:execve_args}
\end{table}

As we can observe in the table, all of the arguments of the syscall are marked with the keyword \_\_user, and therefore as we explain in section \ref{subsection:bpf_probe_write_apps} these arguments can be overwritten using the eBPF helper bpf\_probe\_write\_user(). This opens for us the possibility of modifying these arguments so that another file is modified.

Figure \ref{fig:summ_execve_hijack} summarizes the results of an attack using this rootkit module. As we can observe in the figure, we will hijack the execution of sys\_execve to run our own program, but as we mentioned we must execute the original program too in order not to raise concerns in the user space. Therefore, the malicious program must be able to access the original arguments of the sys\_execve call to execute the original program.

\begin{figure}[htbp]
	\centering
	\includegraphics[width=14cm]{summ_execve_hijack.png}
	\caption{Overview of execution hijacking attack.}
	\label{fig:summ_execve_hijack}
\end{figure}

As we will discuss, apart from running the original program, the malicious program will run itself as sudo (taking advantage of the privilege escalation module) and then connecting to the rootkit client.


\subsection{Overwriting sys\_execve}
We have mentioned the possibility of overwriting the parameters of the sys\_execve syscall. However, after loading an eBPF \textit{enter} tracepoint attached to sys\_execve and writing into any of this buffers, we found three scenarios:
\begin{itemize}
\item The helper successfully overwrites the user buffers.
\item The helper fails to overwrite all or some of the buffers.
\item The helper successfully overwrites a buffer but, with a single write operation, it has also modified the value of some other user buffer.
\end{itemize}

The reason for this is that, as we covered in section \ref{subsection:bpf_probe_write_apps}, the bpf\_probe\_write\_user() helper fails to write any data in the occurence of a page fault. As we explained in section \ref{subsection:mem_faults}, minor memory faults are particularly common when executing a fork() of a process, since the child process will not get its page table completely copied from the parent, but will request the mapping once it is attempted to be read.

Because of the fact that programs calling sys\_execve will be completely replaced by the new program, we can find this function used commonly in two contexts:
\begin{itemize}
\item User programs which execute a new program as a child, but they do not want to be terminated themselves. For this, they call a fork() and then execute execve() (which calls the sys\_execve syscall) in the child process.
\item Programs that are run by the user in the command-line interface. Once a command is introduced, the program corresponding to the command is searched, and the bash process (or any other shell being used) will fork() itself and execute the new program.
\end{itemize}

Therefore, when modifying the arguments of sys\_execve, we will find that most calls are from programs which had executed fork() previously, thus having a high probability of failing. Note that the exact reason why writing one buffer with bpf\_probe\_write\_user() modifies multiple buffers simultaneouslly is unknown, but it is a situation we must account for, since we cannot trust in the helper not returning an error, we must check the result of this write accesses.

\subsection{Hiding data in a system call}
Apart from having to take into account that the bpf\_probe\_write\_user helper may fail in unexpected manners as we described, we also need to give special attention to how we will preserve the original information of the program being executed via sys\_execve after we modify the arguments of this call. As we showed in figure \ref{fig:summ_execve_hijack}, the malicious program executed using the hijacked syscall must be able to execute the original program. For this, the program will fork() and create a child process, on which execve() will be called with the original program arguments. Therefore, the main issue would be how to recover the original arguments once they were overwritten by eBPF.

In order to achieve this, we will hide the original arguments in those passed to the malicious program. Table \ref{fig:execve_args_hide} shows how this process works with a sample sys\_execve call. Environment variables have been omitted for simpleness, but we can usually find a large array of them.

\begin{table}[H]
\begin{tabular}{|>{\centering\arraybackslash}p{2cm}|>{\centering\arraybackslash}p{3cm}|}
\hline
\multicolumn{2}{|c|}{Original arguments}\\
\hline
\hline
filename & "/bin/ls"\\
\hline
argv[0] & "ls"\\
\hline
argv[1] & "-l"\\
\hline
argv[2] & NULL\\
\hline
envp[0] & NULL\\
\hline
\end{tabular}
\quad
\begin{tabular}{|>{\centering\arraybackslash}p{2cm}|>{\centering\arraybackslash}p{3cm}|}
\hline
\multicolumn{2}{|c|}{Modified arguments}\\
\hline
\hline
filename & "/home/osboxes/execve\_hijack"\\
\hline
argv[0] & "/bin/ls"\\
\hline
argv[1] & "-l"\\
\hline
argv[2] & NULL\\
\hline
envp[0] & NULL\\
\hline
\end{tabular}
\caption{Hiding data in sys\_execve arguments.}
\label{table:execve_args_hide}
\end{table}

As we can observe in the table, we will modify the value of \tetxit{filename} with the malicious program filename, and save the original filename into argv[0]. Performing this substitution means losing little information since the argv[0] argument contains the name of the program \cite{c_standard_main}, information that can also be taken from the filename (thus it can be recovered later). Only in very specific use cases the argv[0] argument is different from the file included in the filename argument (like in Busybox \cite{busybox_argv}). 

After the above substitution, the malicious program (in the table, "execve\_hijack") will be called, whose main function receives the following arguments:

\begin{verbatim}
int main (int argc, char *argv[], char *envp[]){}
\end{verbatim}

Hence, the malicious program will use the argv[] and envp[] arrays to make another sys\_execve call with the original arguments, running the original program. 

\subsection{Hijacking a program execution}
Once we have analysed the two fundamental issues regarding this module (bpf\_probe\_write\_user fails and hiding information in the syscall arguments) we will now analyze the execution hijacking module in detail using a sample program execution.

Figure