Encrypting Enterprise Data Storage

Given recent revelations that the NSA is reading what appears to be the bulk of internet traffic, companies are looking to encryption to try to keep prying eyes away.  American cloud vendors are worried that Europeans are not going to want to store their data in data centers located in the USA as the government may be reading their mail.

The American government also has some regulatory requirements that say you must encrypt data in case it falls into the wrong hands.  Among these are Payment Card Industry Data Security Standard (PCI DSS) and Health Insurance Portability and Accountability Act (HIPAA) that require encryption of financial and health-related data and transactions.
Encryption works by using the centuries old, unsolved problem of trying to determine whether a number, n, is a prime number, meaning divisible only by itself and 1.  To know for sure you have to try every number up to  to see if any of them are factors of n.  If that number is large, that could take weeks or even years.  There are routines that can estimate whether a number is “probably prime,” but no one has nailed that problem completely.
To think of this in terms of IT, that number is a private key by which you can decrypt data sent to you which was encrypted using your public key, which is yet another number.
There is strong encryption that uses shorter keys, but is still based on the basic idea of using prime numbers.  It is called elliptic curve encryption.
There are two types of data to encrypt: data-at-rest and data-in-motion. Data-at-rest is data stored on a disk drive. Data-in-motion is data which is in transit from the disk to the application.  Data-in-motion is particularly susceptible to being hacked, because at some point it has to be decrypted to be read by the application.  Memory is an obvious place to look for such unencrypted data.
The Apple iPad and iPhone encrypt data-at-rest using the serial number of the phone as the private key.  Apple even has a technique to encrypt data-in-transit (memory).  Android devices let the user enter their own private key.  Forget the key and you have to wipe the phone.
For SAN storage, HP offer an HP Encryption SAN Switch, which is designed to encrypt data-at-rest as it passes through the SAN switch.
Amazon’s cloud storage (Amazon S3) lets you upload encrypted data from the client side or you can use Amazon’s encryption tools to encrypt the data there.
EMC PowerPath encrypts data-at-rest and data-in-transit.  It also foils such clever tactics to read encrypted files as physically removing a disk from a storage array.
LDAP is often used to store passwords in a database.  When that data is read by the application, it is secured with the password, protected by a simple encoding mechanism (not the same as encryption) that can be easily foiled. To plug that gap, LDAP supports the LDAPS protocol to encrypt data passing from the LDAP storage to the application.
The Oracle database and Microsoft SQL server offers the same approach.  They use SSL encryption to protect data-in-transit.  SSL is the same technique used to encrypt traffic to and from a website and LDAPS.
In order to prevent privileged users, or a virus, from reading data-at-rest it is necessary to encrypt data.  This is called file encryption or it could be full disk-level encryption.  Not all operating or file systems support encryption.  Not all types of data need to be encrypted.  For example, financial information should be encrypted, but not the weather forecast.
There is a downside to encryption and that is speed and overhead.  Encrypting and decrypting data requires running mathematical computations that are supposed to be difficult to solve.  Web and database servers can do this themselves or you can add some kind of hardware SSL accelerator to offload some of that work.
Some programming languages support encrypting data in memory, so that a virus lurking there cannot read it or make it tougher to do so.  Microsoft .NET lets the programmer add commands to encrypt and decrypt data in memory.  Java provides the Java Cryptographic Extension to support encryption.
There clearly is a need to encrypt some but not all data.  Data needs to be encrypted on the file system itself and, where it is most vulnerable, when it is in transit.