Using cloud services as glorified drive: iSCSI caveats (part VII)

Previous posts described a proof-of-concept using cloud computing to run Windows Server VMs and configure iSCSI targets to create private cloud storage encrypted by BitLocker-To-Go. In this update we will look at the fine-print and limitations of using iSCSI.

Concurrent access

One problem glossed over until now has been accessing data from multiple locations. Suppose the user has mounted the iSCSI target as local disk from one machine at home. Later they want to access the same data in the cloud from their laptop. This does not pose any problems for Bitlocker-To-Go decryption since the smart card protecting the volume is inherently mobile. (Also note the card is only used once to unlock the volume. It does not have to remain inserted in the reader or otherwise present near the machine for continued access.) But it does pose a problem with iSCSI model: the same underlying NTFS filesystem is now being manipulated by multiple machines with no synchronization. That is a recipe for data corruption.

Contrast this to the encrypted VHD model sketched earlier: in that case synchronization is done implicitly by software running locally on each machine, responsible for uploading that VHD file to the cloud. When uploads are done concurrently from multiple machines, it is up to the cloud provider to decide which one wins– most likely the one with most recent timestamp. But no attempt is done to “merge” files. If the user mounted VHD from one machine and added file A, then mounted it from another machine to add file B, the cloud provider does not magically collate those into a single virtual disk containing both additions. (On the bright side, whatever disk image is eventually uploaded is consistent. There is no possibility of ending up with corrupt “mash-up” combining sectors from both VHDs.)

If two machines are accessing the same iSCSI target and making modifications, there is no such guarantee. In both cases, each initiator eg some instance of Windows, will be directly manipulating file-system structures on the assumption that it alone is in charge of NTFS on that volume. The result is similar to what could happen if VHD image associated with a running virtual machine is also locally mounted by the host OS: too many cooks in the kitchen messing with the filesystem.

Solving this would require carefully coordinating access between multiple clients. One could envision a local agent on each machine, responsible for disconnecting or remounting the volume as read-only (but this is tricky) when another client connects to the same volume. The problem is that such a change will be disruptive to any application with files opened on that volume, akin to yanking a USB drive in the middle of using it.

Authentication and integrity pitfalls

The simplified flow described earlier did not use any authentication for iSCSI initiators. Of course this is not a viable option when the VM is running remotely on a cloud platform such as Azure; anyone on the Internet would be able to access the iSCSI target. Bitlocker encryption provides confidentiality but it can not provide integrity or availability. Without some manner of authentication, nothing prevents any other person from repeating the same steps to mount the remote disk and delete all data.

The problem with CHAP

At first it looks like this can be addressed by using CHAP (Challenge Handshake Authentication Protocol) which is officially part of the iSCSI standard. It is also supported by Windows. CHAP can provide mutual authentication between initiators and targets. Revisiting the steps from the previous post for configuring an iSCSI target, there is an authentication option on the Windows Server wizard to limit access to specific initiators in possession of a particular secret. These credentials are specified using the “Advanced” option on the initiator side. Limiting access in this manner mitigates the basic risk: random people hitting the IP address of our remote storage VM and discovering the iSCSI targets can no longer connect and overwrite data at will.

Suspending disbelief that such access control is scalable– since each device accessing the iSCSI volume would have its own initiator ID, it requires an entry in the target configuration– there is still another problem: CHAP does not protect the payload itself. This can be easily observed by getting a network trace of raw TCP traffic between an initiator and target. Below is an example from MSFT Network Monitor which comes with a built-in parser recognizing iSCSI traffic:

iSCSI traffic between initiator and target

iSCSI traffic between sample initiator and target

It shows an iSCSI response fragment transmitted when accessing a Powerpoint file. BitLocker is not turned on for the target volume– to make it easier to observe raw data in the trace– but CHAP authentication is enabled for the connection. Highlighted region of the IP packet shows distinctly recognizable plain-text data. A similar experiment would show that modifying parts of the payload is not detectable by either side. RFC3720 describing iSCSI says as much under security considerations:

The authentication mechanism alone (without underlying IPsec) should only be used when there is no risk of eavesdropping, message insertion, deletion, modification, and replaying.

When BitLocker-To-Go is used, eavesdropping problem goes away but tampering risks remain. An attacker can not recover the data or even make controlled changes such as flipping one bit. Under ordinary circumstances, standard encryption modes such as CBC would allow making such changes on ciphertext. But disk encryption schemes are designed to resist controlled bit-fiddling attacks. No scheme can actually prevent or detect changes completely, because there is no extra space to store an integrity check– one sector worth of plaintext must encrypt to exactly one sector ciphertext. The best outcome one can aim for is to ensure that any change to ciphertext will result in decrypting to effectively random data, uncorrelated with the original. That is still a problem. For example when the user is uploading a document to the cloud, an attacker intercepting network traffic can replace those BitLocker-protected blocks with all zeroes. When those blocks are later retrieved from the cloud to reconstruct the document, the modified ciphertext will still decrypt to something other than what the user originally uploaded.

Protecting the payload itself requires some type of transport-layer mechanism such as IPSec or establishing an ad-hoc ssh tunnel between initiator and target. This is yet another moving piece on top of an already complex setup. IPSec in particular is closely tied to Active Directory on Windows and requires having both sides being members of the same forest. That is a tall order, considering we want to access the cloud data from any Windows device, or even any device capable of acting as iSCSI initiator.

Verdict on iSCSI + BitLocker

Because of these limitations and general lack of support for iSCSI on platforms outside Windows, this proof-of-concept combining iSCSI targets in cloud VM with BitLocker-To-Go is far from being widely-usable.

The final post in this series will tackle the question of what an ideal solution for cloud storage with local encryption could look like.