ASN.1 Format Configuration

Intro to ASN.1

In our last article "Damn that Data Format Hell" we talked about how layline.io deals with the multitude of complex structural data formats by using a grammar language to describe almost any such format, and enabling you to instantly work with the data.

We also discussed, that this doesn't work for ALL formats you may be working with, but maybe 80% of all structured formats.

There are data exchange formats out there in the wild, however, which adhere to their own standard and come with their own grammar language. One of which is Abstract Syntax Notation One, aka ASN.1.

layline.io Showcase

In this article we will showcase how our users can define ASN.1 based formats using layline.io.

layline.io is an opinionated event data processor. It wraps the power of reactive stream management in a framework which makes setup, deployment and monitoring of large scale event data processing requirements easy and straightforward.

TD.57 - Transferred Account Procedure (TAP)

One popular area where ASN.1 protocol format plays an important role is in telecoms roaming and the ensuing TAP exchange process. TAP (Transferred Accounts Procedure) handles the financial settlement process between mobile networks for their roaming subscribers.

In a nutshell: If you use your cell phone in a visiting network (e.g. other country or simply other network), then the visited network needs to invoice your home network for the service provided. The information about who used their phone in a visited network, how long and who they called, etc. is all exchanged in TAP and RAP files.

Our example: TAP3.11 file format configuration

TAP3.11 Grammar

This is what the beginning of the official TAP3.11 grammar looks like:

-- TD57_3.11.2
--
-- The following ASN.1 specification defines the abstract syntax for
--        Data Record Format Version 03
--                           Release 11
...

TAP-0311  DEFINITIONS IMPLICIT TAGS  ::=

BEGIN

DataInterChange ::= CHOICE
{
    transferBatch TransferBatch,
    notification  Notification,
...
}
...
TransferBatch ::= [APPLICATION 1] SEQUENCE
{
    batchControlInfo       BatchControlInfo            OPTIONAL,
    accountingInfo         AccountingInfo              OPTIONAL,
    networkInfo            NetworkInfo                 OPTIONAL,
    ...
}

Create ASN.1 Asset

Let's set up TAP3.11 using layline.io. In the Project Asset View (1) pick Formats (2) and then ASN.1 Format (3).

The Asset editor has two tabs "Configuration" and "Modules". On the "Configuration" tab (1) we name the Asset "TAP3.11" (2):

Define Message Types

TAP3.11 defines a number of different "root" message types. We are only going to identify three of them to work with in this example:

In layline.io we map them to "Header", "Detail", and "Trailer" respectively:

We are later going to debug output the data we read. Because some data is BCD-encoded and for readability's sake we use a built-in converter to decode them:

Adding the TAP3.11 grammar

Switch to the Modules tab (2) of the ASN.1 Asset we have added (1).

An ASN.1 grammar can be made-up of multiple modules. In our case it is just one. We have added it and named it "TAP.311-Module" (3/4).

You can find the original ASN.1 grammar for TAP3.11 in the written specification:

Let's copy and paste it to our ASN.1 Asset (1) (it's about 1,422 lines):

Interactive Testing of the ASN.1 Grammar

layline.io provides the ability to instantly check whether format setup matches the actual data. You can upload a sample file (1) directly into the Configuration Center (2) to perform a cross-check.

layline.io will instantly:

1. Generate a parser for the ASN.1 grammar
2. Parse the sample file
3. Feed the result back (3 & 4)

Switch to the "Sample Messages" tab (2) where we can get a more readable view on the file content:

Interactive testing means, that you now could go back to the grammar, change it, and then instantly see how this affects how this affects the interpretation of your sample data.

Sample Project

We put together a complete sample Project together with test-data which is ready to run in layline.io. The example simply reads and writes a TAP file and outputs its contents to the console in a JSON-format:

Summary

Using layline.io you can not only define complex, distributed, scalable event-processing scenarios, but pretty much work with any data format. In this example we demonstrated how to deal with ASN.1 structured data.

Most other solutions will support specific formats only, as their parsers have been hard-coded to match a specific use-case. In layline.io there are literally millions of use-cases. That's why formats are never hard-coded, but configurable.

There are many popular ASN.1 protocols around the world of which we would like to list some below:

• Ericsson AIR / AXE10 / CCN / CCR / MTAS / R3-R13 / SDP
• Huawei IMS
• 4G UMTS / 5G IMS / 3GPP
• TD.35 NRTRDE - Near Real Time Roaming Data Exchange
• TD.57 / TAP2 / TAP3 Transferred Account Procedure
• X.500 Directory Services
• Lightweight Directory Access Protocol (LDAP)
• PKCS Cryptography Standards
• Simple Network Management Protocol (SNMP)
• Signalling System No. 7 (SS7)
• Long-Term Evolution (LTE)
• And many more...

Resources

• Read more about layline.io here.
• Contact us at hello@layline.io.