FDE – EXPLOINSIGHTS, INC. Sys-Admin

Image result for luks key list

So we have seen some postings online that suggested you can’t encrypt an lxd zpool, such as this GitHub posting here, which correctly explains that an encrypted zpool that doesn’t mount at startup disappears WITHOUT WARNING from your lxd configuration.

It’s not the whole picture as it IS possible to so encrypt an lxd zpool with luks (the standard full disk encryption option for Linux Ubuntu) and have it work out-of-the-box at startup, but perhaps it’s not as straightforward as everyone would like

WARNING WARNING – THE INSTRUCTIONS BELOW ARE NOT GUARANTEED. WE USE COMMANDS THAT WILL WIPE A DRIVE SO GREAT CARE IS NEEDED AND WE CANNOT HELP YOU IF YOU LOSE ACCESS TO YOUR DATA. DO NOT TRY THIS ON A PRODUCTION SERVER. SEEK PROFESSIONAL HELP INSTEAD, PLEASE!

With that said…this post is for those who, for example, have a new clean system that they can always do-over if this tutorial does not work as advertised. Ubuntu OS changes and so the instructions might not work on your particular system.

Firstly, we assume you have your ubuntu 16.04 installed on a luks encrypted drive (i.e. the standard ubuntu instal using the “encrypt HD” option). This of course requires you to enter a password at boot-up to decrypt your system, something like: Image result for ubuntu full disk encryption

We assume you have a second drive that you want to use for your linux lxd containers. That’s how we roll our lxd.

So, to setup an encrypted zpool, select your drive to be used (we assume it’s /dev/sdd here, and we assume it’s a newly created partition that is not yet formatted – your drive might be /dev/sda, /dev/sdb or something quite different – MAKE SURE YOU GET THAT RIGHT).

Go through the normal luks procedure to encrypt the drive:

sudo cryptsetup -y -v luksFormat /dev/sdd

Enter the password and NOTE THE WARNING – this WILL destroy the drive contents. #YOUHAVEBEENWARNED

Then open it:

sudo cryptsetup luksOpen /dev/sd?X sd?X_crypt

Normally, you would create your normal file system now, such as an ext4, but we don’t do that. Instead, create your zpool (we are calling ours ‘lxdzpool’ – feel free to change that to ‘tank’ or whatever pool name you prefer):

sudo zpool create -f -o ashift=12 -O normalization=formD -O atime=off -m none -R /mnt -O compression=lz4 lxdzpool  /dev/mapper/sdd_crypt

And there you have an encrypted zpool. Add it to lxd using the standard ‘sudo lxd init’ procedure that you need to go through to create lxc containers, then start launching your containers and voila, you are using an encrypted zpool.

So, we are not done yet. We can’t let the OS boot up without decrypting the zpool drive, lest our containers disappear and lxd goes back to using a directory for its zpool, per the GitHub posting referred to above. That would not be good. So how do we make sure this is auto-decrypted at boot-up (which is needed for lxc containers to launch)?

Well, we have to create a keyfile that is used to decrypt this drive after you decrypt the main OS drive (so you do still need to decrypt your PC at bootup as usual – as above):

sudo dd if=/dev/urandom of=/root/.keyfile bs=1024 count=4
sudo chmod 0400 /root/.keyfile
sudo cryptsetup luksAddKey /dev/sdd /root/.keyfile

This creates keyfile at /root/.keyfile. This file is used to decrypt the zpool drive. Just answer the prompts that these commands generate (self explanatory).

Now find out your disks UUID number with:

sudo blkid

This should give you a list of your drives with various information. We need the long string that comes after “UUID=…” for your drive, e.g.:

/dev/sdd: UUID=”971bf7bc-43f2-4ce0-85aa-9c6437240ec5″ TYPE=”crypto_LUKS”

Note we need the UUID – not the PARTUUID or anything else. It must say “UUID=…”.

Now edit /etc/crypttab as root:

sudo nano /etc/crypttab

And add an entry like this:

#Add entry to aut-unlock the encrypted drive at boot-up,
#after the main OS drive has been unlocked
sdd_crypt UUID=971bf7bc-43f2-4ce0-85aa-9c6437240ec5 /root/.keyfile luks,discard

And now reboot. You should see your familiar boot-up screen for decrypting your ubuntu OS. And once you enter the correct password, the encrypted zfs zpool drive will be automatically decrypted and will allow lxd to access it as your zpool. Here’s an excerpt from our ‘lxc info’ output AFTER a reboot. We highlighted the most important bit for this tutorial:

$ lxc info
config:
storage.zfs_pool_name: lxdzpool
api_extensions:
– id_map
– id_map_base
– resource_limits
api_status: stable
api_version: “1.0”
auth: trusted
auth_methods: []
public: false
driver: lxc
driver_version: 2.0.8
kernel: Linux
kernel_architecture: x86_64
kernel_version: 4.15.0-34-generic
server: lxd
storage: zfs

Note we are using our ‘lxdzpool’.

We hope this is useful. GOOD LUCK!

Useful additional reference materials are here (or at least, they were here when we posted this article):

Encrypting a second hard drive on Ubuntu (post-install)

Setting up ZFS on LUKS

So we’ve been asked ‘How do you share files from Nextcloud securely’?

It’s a fair question because if you END-TO-END encrypt a file, you can’t actually share the file unless you also share a means of decrypting it. And we don’t share our end-to-end encryption keys, EVER.

So how do we manage sharing? How do we protect our files as we hand them over to our customers? The short answer is with LAYERS of security:

Our files are protected by one of several levels of independent security:

Robust SSL/TLS connectivity (HTTPS all digital coms)

End-to-end encryption for files

Server-side encryption for files

Full Disk Encryption (FDE) for computer drives (server or desktop)

We can ignore FDE here as it doesn’t really come into play (it’s only really applicable here to a stolen PC).

For normal at-rest files, we have end-to-end AND server-side encryption in play. So our files are protected against threats of all kinds. You just can’t get at our data without TWO separate sets of credentials, neither of which are connected, neither of which are stored in a file.

When we share a file, we have to remove one of these layers (the end-to-end encryption), and that of course makes the protection weaker during the process of sharing FOR THAT FILE, since it only has server-side encryption to keep it safe. Note that this is not “bad” – the file is still encrypted, and it’s safe from even quite determined hackers. But you know, we like to be a bit paranoid and very responsible, so we bring other measures in play.

Firstly, here’s a real file from our server we created with end-to-end encryption still applied:

It’s very exciting. #notReally. You can see, the name is complete gibberish. And for the record, if you download this ‘blob’ and view it in Notepad, this is what you get:

“îe¤ÎÈCÀ’0pMi%?/£`ÕÕäÔ}CÇ+>ê3Ø½[Æ~©HïïþÑàß·”M*ÓÝì?L‚L*x)š!ýU\nW£7zšK‚ubÎ
91Ïií=¡l‰¸üN#åS›¬Æàçý*¨”PÔãºÕë*,8ƒÇŠûìÒ†Áè‹™p\\©Jšs|Ö_ž›”ÒÿEÊ ÈÈõÁÁ”

And as you will see later, the real file is actually a simple text message.

Once the file has end-to-end encryption removed, we can now more easily find it on the server AS THE SYS-ADMIN (or as a hacker that has somehow penetrated our system). It takes a while, but you can find it:

Wow, there it is. Plain as day. This file is actually present in an lxc container. If you can ‘see’ this, you are ither a SysAdmin or an unauthorized hacker. But the good news, you are not at the system root (you just think you are), so your damage potential is still very limited.

The file we created especially for this demo is shown among other directories. The server ADMIN or hacker (not necessarily the owner of the file) can “see” the file, so it seems. But before everyone panics, let’s take a close look at what he or she can actually “see”. There’s a file name and permissions and modification time and apparent size (meta-data if you will). You can also see file/folder names too. Gosh, does that mean the SYS-ADMIN or hacker can snoop on my files? Let’s look at the contents using cat:

Well that doesn’t look terribly useful. As you can see, the DATA are still encrypted. Albeit this time via the SERVER keys. A hacker and even the SysAdmin can’t look at file contents (but it’s true, he or she can see the name and other file meta data). So, the file is still quite well protected against hacking and theft.

In fact, the ONLY way you can view these data is if you unlock the server-side encryption that’s now in effect for that file. And for this, you either need an exploit that works against Nextcloud’s encryption (very unlikely) OR you need to somehow secure the credentials needed to login to the Nextcloud account which will then unlock these files. For that, you need a Nextcloud password AND 2FA code (one that changes every 30 seconds), so that too we think is quite safe from unauthorized disclosure. But when you do enter those data, the file becomes fully decrypted and is then more visible, albeit over an HTTPS connection only (our final level of protection):

So “finally”, this is a file we can read: HelloWorld.txt, that says “Hello everyone. Have a nice day”. So the Nextcloud user can read the file (just as he/she should).

When it comes to SHARING this file, we actually share an HTTPS:// link for that file that includes the decryption credentials for that file. In Nextcloud, this is what that looks like:

You can see on the bottom right that a sharing link has been created. We also add a PASSWORD to that link and an EXPIRATION DATE for the link, so that the link will not be active for long AND you need the extra password to access the file. We send the LINK to the file via regular email. Regular email is not terribly safe. It can be intercepted. We know that. But let’s continue:

This link looks like this:

https://cloudvault.exploinsights.com/index.php/s/aeX7irmbWfneipn

(The link won’t exist for long, so it’s probably not worth trying, sorry!).

If you click on the link and it’s still active, it will bring up THIS page:

Which is still not the file. You need that password. We do not send the password with the link. We don’t even send the password via email. We send that via another means, say WhatsApp, or Signal or even SMS. So, to get to this file, you need the link (from an email sent to you) AND the password (sent in a different way) sent to you. Finally, if you get the link and the password, you can click/enter the top secret password assigned to protect this file, you get rewarded in this case with this:

The file. In decrypted form. Finally! You can download the file but ONLY via https:// (SSL/TLS) connection (our final layer), so even then it remains encrypted-in-transit until it hits your download folder. Only then do we relinquish our management –> OVER TO YOU! 🙂

So that’s how we share our data. it is ALWAYS protected with up to four layers of encryption: FDE, End-to-End, Server-side and lastly SSL/TLS.

These “layers” are gradually stripped off as we get closer and closer to sharing a file with you, but only when it lands on your machine is our last (but strong) layer of encryption removed. We use 2FA credentials to share our data and even the Sys-Admin (or unauthorized hacker) has very little access to our data: they NEVER see raw decrypted data, but occasionally they can sometimes view a file name and some basic meta-data.

Is this perfect? No. We are always looking to improve it. This system is our latest and greatest implementation, but it won’t be our last. Is this pretty good? We think so! What do you think?

So this is how we ROLL our file-sharing; how we securely share data with our customer. What do you do?

Comments by email only – [email protected] or you can catch us on Twitter @SysAdminEI

Tag: FDE

Encrypting and auto-boot-decryption of an LXC zpool on Ubuntu with LUKS

Securely Sharing Files from Nextcloud