Tokenization - Replacing Sensitive Data with Tokens

About 2 min read

Tokenization is a technique that protects sensitive data such as credit card numbers and personal identification numbers by replacing them with meaningless random strings (tokens). The original data is stored in a token vault (a secure mapping table), and only authorized systems can recover the original value from a token. It is widely adopted in PCI DSS-compliant payment systems, and its adoption is accelerating further with the full enforcement of PCI DSS v4.0 in 2025.

What It Takes for No Original Values to Remain

The question that remains after adopting a replacement scheme is how to keep the original values from surviving anywhere. Changing where the data is stored does not by itself remove copies that were made along the way. Operational records written at the moment the input is accepted, temporary copies left behind while a failure is investigated, and extracts pulled together for reporting or analysis all fall into this category, and because they are created in places that stay out of sight while everything is working, they are easily missed when the list of things to replace is drawn up. Being able to say that nothing remains anywhere is therefore not the result of changing one storage location; it follows only from enumerating the path a value travels from the moment it arrives to the moment it leaves. Second, points that handle the value itself will always remain on either side of the replacement. The point that receives the value immediately before conversion, and the point that holds the authority to turn it back when that is required, cannot be removed by their very nature. What is gained is not that the value stops being handled but that the number of places handling it is narrowed, and whether the narrowed state persists depends on continuing to check that no new entry points have been added since. Third, a design that keeps the correspondence outside your own systems carries an assumption about changing providers. What is held locally is only a meaningless string, while the information that ties it back to the original value sits with the party it was entrusted to. If that party is changed, the strings can be carried out but the correspondence cannot, so either the original values have to be collected again or a handover procedure has to be arranged between the two sides. Because this question surfaces years later rather than right after adoption, it rarely appears among the criteria compared at selection time.

The Tokenization Flow

The Difference from Data Masking

Data masking irreversibly transforms the original data and cannot be reversed, whereas tokenization is fundamentally different in that the original data can be restored through the token vault. In payment processing, the card number is tokenized at the time of purchase, and at the time of actual billing the original number is retrieved from the token vault and processed. Thanks to this mechanism, no card numbers are ever stored in the merchant's system, greatly reducing the damage in the event of a data breach.

Choosing Between Tokenization and Encryption

Encryption transforms data with a mathematical algorithm, so if the key is leaked there is a risk that all data can be decrypted. Tokenization has no mathematical relationship between the token and the original data, so even if only the token is obtained, the original data cannot be inferred. However, because the token vault itself becomes a single point of failure, protecting the vault requires key management with an HSM and strict access control.

Key Points for Real-World Adoption

Mobile payments such as Apple Pay and Google Pay are representative examples of device tokenization. By using a device-specific token instead of the real card number, the card information remains safe even if the device is stolen. The "save card information" feature on e-commerce sites also, in most cases, actually stores a token. Protect your payment administration screens with strong random passwords to prevent unauthorized access to the token vault.