Using the Vault API

Nearly two years ago, Idera acquired apilayer, a collection of various cloud-based APIs. Idera being Embarcadero's parent company (and Embarcadero being the publisher of my favorite development tool, Delphi), I was interested to see what this was all about and looked at the handful of APIs available. One caught my eye and I made a note to come back and check it out more when the time was right.

Recently, I started developing a couple of products that needed to store remote configurations. I briefly considered writing my own API or simply using FTP and storing config files on my website but there are several reasons why I didn't want to take either of those routes. Then I remembered this one API I had looked at previously--the time was right to look more deeply.

Vault API - Overview

The Vault API is actually quite simple: it stores encrypted data organized in a Unix-like folder structure of named content items. Every piece of data you submit is encrypted. You can use a default generated encryption key for the account or, if you have a paid account, use your own encryption keys. This is nice because even if someone somehow gets your API key, they cannot see any of the data you've stored unless they also know the encryption key used to store that piece of data--and every content item can be encrypted with a different encryption key.

My Needs

As I mentioned, I need to remotely store license keys and program configuration settings. Remotely storing license keys is a common need and there are APIs available specifically for that purpose. For example, Keygen has an elaborate licensing structure to handle a huge array of customer and product types, licensing schemes multiple versions, self-activation and validation, expiration, and so forth. It's really sophisticated but a bit of an overkill for what I need--and it gets expensive quickly.

With the Vault API, I can structure my data and store anything I need in any string format: CSV, XML, JSON, or something else I create. This means I create my own license keys, attach expiration dates, product versions, and more, and store it in Vault. Then my app can get that data and validate it to activate the software. If need be, I can update parameters that might activate another part of the program, change an option, extend the expiration date, or deactivate the license the next time the program validates with the API--if I build the program to act that way.

Most applications I distribute have a local screen or app that allows configuration of the program and stores a local .INI. Recently, I've been writing some automated database exports that run unattended for a small vendor that sells business information services nationally to store owners who often have very little knowledge of the database field names or internet protocols required to set up the proper queries and communications. The vendor wants to manage options for these customers remotely; their current approach is to remote in to the customer's server but that often requires a locally hired IT person to go on-site to let them in. It's awkward to relay instructions to the IT person as they aren't familiar with the configuration needs and the vendor sometimes needs to try different settings to fit particular scenarios.

So my solution is to write a replacement program that will install a Windows service to store and periodically check the settings stored online in Vault; additionally, the vendor will get a copy of a remote configuration program that will access those settings. This way, the vendor can manipulate the configurations without anyone needing to manage the software at the store; the next time the Windows service checks Vault, it'll get the changed configuration and make the necessary changes to successfully export the data in the proper format.

The Vault API Endpoints

There are only three different URLs but with combinations of HTTP methods, there are a total of 7 actions:

• Generate a new encryption key
  
  End Point: /key
  HTTP Method: POST
  Parameters: None
  
  First and foremost in Vault is Encryption Key Generation. You cannot store any content in Vault without an encryption key. You can create content using your own encryption key if you have a paid account; if not, you must generate an encryption key which is immediately stored with your account and becomes the "default" encryption key for all content stored without a custom encryption key.
  
  Vault's default encryption keys are kept in a stack of "current" keys. If you call this end-point again, it generates a new key but remembers the previous key in case you delete the new one.
  
  Sample Request URL:
  https://api.apilayer.com/vault/key
  
  Sample Response:

  {
      "key": "FNKerqcdZLeUFnrP6qbkSAZYeRfznqkb8t5O4ljgAIY=",
      "stored": true
  }	
  	

   
• Delete the encryption key
  
  End Point: /key
  HTTP Method: DELETE
  Parameters: None
  
  You can delete the currently stored encryption key. When you do, the previously generated encryption key gets automatically reinstated as the default encryption key for the account. If there are no more encryption keys, then you are left with an account without a default encryption key.
  
  Note: Calling POST /key, then DELETE /key, then POST /key again does not regenerate the same key you had previously--in my testing, generated encryption keys are always new and unique.
  
  Sample Request URL:
  https://api.apilayer.com/vault/key
  
  Sample Response:

  {
      "message": "Previous key was deleted, encryption key changed. Please note that items encrypted with the previous key can not be decrypted anymore.",
      "key": "F_DaK4S269ARR8HgYgCHExQt9xlXCfST30NfrD2glLE="
  }

   
• Get the encryption key
  
  End Point: /key
  HTTP Method: GET
  Parameters: None
  
  Calling this end-point simply returns the current default encryption key for the account. When you generate a new encryption key, the new key is returned for you to keep and store and use but in case you lose it or need to check which key is currently active, this end-point is useful.
  
  Sample Request URL:
  https://api.apilayer.com/vault/key
  
  Sample Response:

  {
      "key": "F_DaK4S269ARR8HgYgCHExQt9xlXCfST30NfrD2glLE="
  }

   
• Insert or update content
  
  End Point: /content
  HTTP Method: POST
  Parameters: body (required), path (optional), encryption_key (optional)
  
  Posting content adds a new content item if one by that name and on the given path does not exist; if it does, it's overwritten. Every new and updated content replaces the entire data block each time--there is no support for changing part of it or only sending a patch of changes. The size of each named content item is unlimited.
  
  The path parameter is sent in the query string of the API call. If this parameter is given, the content is stored in the path specified. If the path didn't previously exist, it is automatically created. There is no end-point to manipulate paths--they only exist as part of the identification of stored content items. You can store an unlimited number of named content items in a path; any name conflict is assumed to be the same content item and overwrites the previous content by the same path/name.
  
  The encryption_key parameter is sent in the header strings of the API call. If there is a default encryption key associated with the account, you do not have to supply an encryption key--the data will be encrypted with the account's encryption key. If you have a paid account, you can provide your own encryption key and it will be used to encrypt the stored content regardless whether there is a default encryption key or not.
  
  The body parameter is sent in the body of the API call as the "POST" data. It is unstructured string data--completely managed by your application.
  
  Sample Request URL:
  https://api.apilayer.com/vault/content?path=/ProductA/MyDataExporter
  
  Sample Response:

  {
      "message": "Content securely stored into the vault"
  }

   
• Delete Content
  
  End Point: /content
  HTTP Method: DELETE
  Parameters: path (required)
  
  To delete stored content, use it's path and content name to identify it; if it was stored without a path, the path is on the "root" path, or '/'. The deletion occurs immediately and is non-recoverable.
  
  Sample Request URL:
  https://api.apilayer.com/vault/content?path=/ProductA/MyDataExporter
  
  Sample Response:

  {
      "message": "Content deleted successfully"
  }

   

• Get Content
  
  End Point: /content
  HTTP Method: GET
  Parameters: path (required), encryption_key (optional)
  
  To retrieve content previously stored, call this end-point using it's path and content name to identify it. If it was stored with a custom encryption key, you must send that as well otherwise the content will be inaccessible. If you don't use custom encryption keys, the default account encryption key that was in effect when the content was created must still be active otherwise the content will be inaccessible until you send the encryption key that was used as a custom encryption key--content is NOT re-encrypted when a new account-level encryption key is created. If you don't use custom encryption keys and never change the account-level encryption key, you do not have to send an encryption key. Indeed, if you're using the free account, you can't use custom keys so sending one is pointless--just be sure to never change the default encryption key if there's still data you want to access.
  
  Sample Request URL:
  https://api.apilayer.com/vault/content?path=/ProductA/MyDataExporter
  
  Sample Response:

  {
      "data": "my test data"
  }

• Browse for Content
  
  End Point: /browse
  HTTP Method: GET
  Parameters: path (required)
  
  To get a list of paths and content items (a.k.a. "files") that are stored in Vault, call this end point. The required path is a Unix-like structure starting with '/' at the "root" level.
  
  Sample request:
  https://api.apilayer.com/vault/browse?path=/
  
  Sample response:

  {
      "files": [
          "TestData1"
      ],
      "folders": [
          "Client1",
          "ProductA"
      ]
  }

Rate Limits and Pricing

The number of times your API key can be used within a month to call any of the end-points listed above for this API is determined by the subscription level at which you've subscribed. At the time of this writing there are three price levels:

• FREE - limit of 3,000 requests per month (~100/day)
• $5.99/month - limit of 150,000 requests per month (~5,000/day)
• $39.99/month - limit of 1,500,000 requests per month (50,000/day)

The current rate limits in effect and the number of requests you have left based on your subscription is returned in the headers of each API call:

RateLimit-Limit=5000
RateLimit-Remaining=4908
RateLimit-Reset=61728
X-RateLimit-Limit-Day=5000
X-RateLimit-Limit-Month=150000
X-RateLimit-Remaining-Day=4908
X-RateLimit-Remaining-Month=149320

Open Source Delphi Implementation

To start using this API in my Delphi apps, I created a free account and started testing with the REST Debugger. Using the Copy Components button, I quickly created a data module, then spent the next couple of days working out the nuances of each call and handling errors.

In the process of building and testing the data module, a small app grew around this, in the end becoming a complete Vault API Manager. It allows you to enter your API key (which is saved locally for your convenience) and then try out all the different API calls with a simple interface. You can use it with default or custom encryption keys (depending on your subscription) to create, update, browse, and delete content and keys in your own account.

After polishing it up a little, I have released it as open source on GitHub--there's a screenshot below and more on GitHub. It's written in Delphi 11 and uses Firemonkey--I've run it on both Windows and MacOSX. I hope others find this useful; I welcome comments.