Tokenization

When working with microservices, chances are that sensitive data must be passed through several services before reaching the target service. For example, a credit card number is collected in the shopping cart service and needs to pass through order service to reach the payment service. To make the application PCI compliant, the credit card number should be encrypted or tokenized at the shopping cart service so that only the payment service can decrypt or detokenize it.

Another business scenario is that when information is passed to the Internet, sensitive information needs to be protected for confidentiality. For example, the account number in the URL needs to be hidden so that nobody can dig it from log files. Again, there are two approaches for it: encryption and tokenization.

Light has an encryptor and decryptor for data encryption as plugins that support custom implementations. It covers some use cases for data confidentiality, and tokenization covers the rest.

Tokenization is a reversible security method that replaces sensitive data with fake data that looks and feels just like the real thing while making it worthless to potential thieves. Tokenization can provide equal or better security than encryption while retaining the vital usability of data for analytics and other business processes.

Flexible, format-preserving token types, including numeric, alphanumeric, date, time, address, and other structured tokens can be created with “bleed through” with parts of the original data exposed for business purposes, preserving privacy when applications require only part of the sensitive data for processing.

In Light, tokenization service is provided as part of the enterprise package and hosting service. It is built on top of light-4j and light-rest-4j frameworks with very high throughput and very low latency as most operations are done in the cache. It also supports multi-tenancy based on the client_id in JWT token. A group of clients can share the same token vault, or each client can have its vault.

If you are interested in deploying this service in your data center or signing up our hosting service, please contact [email protected] for details.

  • Getting Started with our test service and tutorial
  • API Document to learn the details on how it works
  • Artifact find release artifacts and deploy options
  • Configuration for different configurations based on your situations
  • Performance for production like environment with both TLS and OAuth 2.0 enabled.
  • Reference gives users more information about the standards and other implementations.
  • Roadmap shows which areas that need to be enhanced in the future.

Tokenization Roadmap

Database Encryption For some organizations, they need to encrypt the value in the token_vault table so that even the database is compromised, the values linked to tokens are still useless for hackers. As tokenization service is read heavy and we have an in-memory cache that contains clear text. The read service performance won’t be impacted by the database encryption. The token creation may be a little bit slower than clear text as the encryption is part of the process. Read More »

Tokenization Reference

While implementing our tokenization service, we have evaluated several commercial products and open source offerings. Our philosophy is to pick the right open source product that meets our requirement and can be easily integrated into our ecosystem. When there is no open source project suitable for our needs, then we consider commercial offerings. Our last resort is to implement it on our own. Open Source There is an open source project Tokenator available on GitHub. Read More »

Tokenization Performance

The tokenization service is designed to be a hosting service that can support thousands of clients to access concurrently. If this service is installed in-house, it is supposed to be used in a high throughput and low latency enterprise environment. The performance of the service is critical. The following section described how the performance test is done locally to measure if the service is suitable for your use case. Prepare Environment A remote instance and tokenization instance are in a docker-compose file to start them together. Read More »

API Document

Format Schemes Token schemes need to be flexible enough to handle multiple formats for the sensitive data they accept — such as personally identifiable information, Social Insurance Numbers, and credit card numbers. In some cases additional format constraints must be honored. As an example, a token representing a Social Insurance Number in a customer service application may need to retain the real last 4 digits. This enables customer service representatives to verify user identities without access to the rest of the SIN. Read More »

Tokenization Service Configuration

In most of the cases, the only configuration file you want to change is datasouce.yml file. The default datasource.yml file packaged into the docker container: tokenization:jdbcUrl:jdbc:mysql://mysqldb:3306/tokenization?useSSL=falseusername:mysqluserpassword:mysqlpwparameters:maximumPoolSize:10useServerPrepStmts:truecachePrepStmts:truecacheCallableStmts:trueprepStmtCacheSize:40prepStmtCacheSqlLimit:20vault000:jdbcUrl:jdbc:mysql://mysqldb:3306/vault000?useSSL=falseusername:mysqluserpassword:mysqlpwparameters:maximumPoolSize:10useServerPrepStmts:truecachePrepStmts:truecacheCallableStmts:trueprepStmtCacheSize:40prepStmtCacheSqlLimit:20You might want to change the database jdbcUrl for both connections. Moreover, you might want to add a new vault for a brand new LOB. Read More »

Artifact

The tokenization service is shipped in three different forms to give customers maximum flexibility to consume it. Docker Image The docker image can be found at https://hub.docker.com/r/networknt/tokenization-1.0.0/ FatJar The fat jar can be downloaded from maven central with all the dependencies included for customers who can not use Docker. Tokenization as a Service The service is in beta testing and can be accessed from lightapi.net portal. Read More »

Getting Started

There are several endpoints for tokenization service. You can start the server at local or access our testing service to get familiar with the tokenization service. The following assumes to use our test instance in the cloud. Please don’t delete existing token unless it is created by yourself. Get all format scheme curl -k -H 'Authorization: Bearer eyJraWQiOiIxMDAiLCJhbGciOiJSUzI1NiJ9.eyJpc3MiOiJ1cm46Y29tOm5ldHdvcmtudDpvYXV0aDI6djEiLCJhdWQiOiJ1cm46Y29tLm5ldHdvcmtudCIsImV4cCI6MTgzNzEwODM3NCwianRpIjoiNUVIWDFzXzlDZkhXQjY3dk1XOHRudyIsImlhdCI6MTUyMTc0ODM3NCwibmJmIjoxNTIxNzQ4MjU0LCJ2ZXJzaW9uIjoiMS4wIiwidXNlcl9pZCI6InN0ZXZlIiwidXNlcl90eXBlIjoiRU1QTE9ZRUUiLCJjbGllbnRfaWQiOiJmN2Q0MjM0OC1jNjQ3LTRlZmItYTUyZC00YzU3ODc0MjFlNzIiLCJzY29wZSI6WyJ0b2tlbi5yIiwidG9rZW4udyIsInNjaGVtZS5yIl19.r_3u4HAJpCztcX8HhV5kihSj6V2gBbqfB4Bdjr3arRKHKJdncaaoDRcYgXihdtutBsA7QVRimu576HL6FwV9iurpqEEA-uy-rzfuCfXJYP4s4F5C_PFaeroGi9siG_dc34p-iFh6eA6dksa6pwBiko9Pb1eBO8XfIV7ndNUmqTUbEvjV6J_Nv_aVPoDgOz00laMDDgj3bOtkz3lGTrfZCQloAhagthfMcUzyj04qe_bKZFKcrbCxXfBjelItUBwdt1td8FBpiSQPI0FXVud69TFmDmDZT6UXci8qJVOb0vuADJcPFe5PEWXxIORoduU8an0Mtey5svQx3c0W_Gqvcg' https://198.55.49.187:6888/v1/scheme Result: [{"name":"UUID","description":"UUID Version 4 Universally Unique Identifier. This format requires no validation and will return a UUID as the token. Read More »