Skip to main content

Tokenization - Replacing Sensitive Data with Tokens

About 2 min read

Tokenization is a technique that protects sensitive data such as credit card numbers and personal identification numbers by replacing them with meaningless substitute strings (tokens). The original data is stored in a token vault (a secure mapping table), and only authorized systems can recover the original value from a token. The PCI SSC published guidelines on replacing card numbers with tokens in August 2011 as a supplement to version 2.0 of PCI DSS.

What It Takes for No Original Values to Remain

The question that remains after adopting a replacement scheme is how to keep the original values from surviving anywhere. Changing where the data is stored does not by itself remove copies that were made along the way. Operational records written at the moment the input is accepted, temporary copies left behind while a failure is investigated, and extracts pulled together for reporting or analysis all fall into this category, and because they are created in places that stay out of sight while everything is working, they are easily missed when the list of things to replace is drawn up. Being able to say that nothing remains anywhere is therefore not the result of changing one storage location; it follows only from enumerating the path a value travels from the moment it arrives to the moment it leaves. Second, points that handle the value itself will always remain on either side of the replacement. The point that receives the value immediately before conversion, and the point that holds the authority to turn it back when that is required, cannot be removed by their very nature. What is gained is not that the value stops being handled but that the number of places handling it is narrowed, and whether the narrowed state persists depends on continuing to check that no new entry points have been added since. Third, a design that keeps the correspondence outside your own systems carries an assumption about changing providers. What is held locally is only a meaningless string, while the information that ties it back to the original value sits with the party it was entrusted to. If that party is changed, the strings can be carried out but the correspondence cannot, so either the original values have to be collected again or a handover procedure has to be arranged between the two sides. Because this question surfaces years later rather than right after adoption, it rarely appears among the criteria compared at selection time.

The Tokenization Flow

The user enters their card number
The tokenization service generates a token
The original data is securely stored in the token vault
Only the token is stored in the merchant database
At payment time, the original data is restored from the token vault and processed

The Difference from Data Masking

Data masking irreversibly transforms the original data and cannot be reversed, whereas tokenization differs in that the original data can be restored through the token vault. In payment processing, the card number is tokenized at the time of purchase, and at the time of actual billing the original number is retrieved from the token vault and processed. With this mechanism, the merchant's system does not need to store card numbers. The PCI SSC guidelines state that tokens should be unusable to an attacker even if a system storing only tokens is compromised.

Choosing Between Tokenization and Encryption

Encryption transforms data with a mathematical algorithm, so if the key is leaked, the data encrypted with that key is exposed to decryption. The PCI SSC tokenization guidelines list token generation methods that include not only random numbers and index assignment but also cryptographic functions that derive the token from the original value, and they require that, whichever method is used, the original card number cannot feasibly be computed from the token alone. However, the same guidelines describe the data vault, which holds both card numbers and tokens, as often the most attractive target for attackers, so protecting the vault requires key management with an HSM and strict access control.

Key Points for Real-World Adoption

Mobile payments such as Apple Pay and Google Pay are examples of device tokenization. A device-specific token is used for payment instead of the real card number. The "save card information" feature on e-commerce sites can also be built to store a token instead of the card number. Protecting payment administration screens with strong random passwords is one measure against unauthorized access to the token vault.

Was this article helpful?

Related Terms