ASN.1 Format Configuration

Intro to ASN.1

In our last article “/Damn that Data Format Hell/” we talked about how layline.io deals with the multitude of complex structural data formats by using a grammar language to describe almost any such format, and enabling you to instantly work with the data.

We also discussed, that this doesn’t work for ALL formats you may be working with, but maybe 80% of all structured formats.

There are data exchange formats out there in the wild, however, which adhere to their own standard and come with their own grammar language. One of which is Abstract Syntax Notation One, aka ASN.1. The grammar notation is described in various standardization documents. To describe them is beyond the scope of this piece, but you can research them with the link provided below.

layline.io Showcase

In this article we will showcase how our users can define ASN.1 based formats using layline.io.

layline.io is an opinionated event data processor. It wraps the power of reactive stream management in a framework which makes setup, deployment and monitoring of large scale event data processing requirements easy and straightforward. layline.io provides everything you would expect from a product-base solution including upgradability, carrier-grade resilience and scalability, among other features.

TD.57 - Transferred Account Procedure (TAP)

One popular area where ASN.1 protocol format plays an important role is in telecoms roaming and the ensuing TAP exchange process. TAP (Transferred Accounts Procedure) handles the financial settlement process between mobile networks for their roaming subscribers.

In a nutshell: If you use your cell phone in a visiting network (e.g. other country or simply other network), then the visited network needs to invoice your home network for the service provided. The information about who used their phone in a visited network, how long and who they called, etc. is all exchanged in TAP and RAP files. These file types are both based on ASN.1. Their grammar is well-defined in the TD.57 specification which you can find here.

Simplified TD.57 TAP procedure

(Image by

Our example: TAP3.11 file format configuration

In our example, we will configure the quite complex format of TAP3.11 which travels on line (1) in the image above (remember, it could be any other ASN.1-based format).

TAP3.11 Grammar

This is what the beginning of the official TAP3.11 grammar looks like:

-- TD57_3.11.2
--
-- The following ASN.1 specification defines the abstract syntax for
--        Data Record Format Version 03
--                           Release 11
...

TAP-0311  DEFINITIONS IMPLICIT TAGS  ::=

BEGIN

DataInterChange ::= CHOICE
{
    transferBatch TransferBatch,
    notification  Notification,
...
}
...
TransferBatch ::= [APPLICATION 1] SEQUENCE
{
    batchControlInfo       BatchControlInfo            OPTIONAL, -- *m.m.
    accountingInfo         AccountingInfo              OPTIONAL,
    networkInfo            NetworkInfo                 OPTIONAL, -- *m.m.
    messageDescriptionInfo MessageDescriptionInfoList  OPTIONAL,
    callEventDetails       CallEventDetailList         OPTIONAL, -- *m.m.
    auditControlInfo       AuditControlInfo            OPTIONAL, -- *m.m.
...
}

Create ASN.1 Asset

Given the above, let’s set up TAP3.11 using layline.io. (We will not explain how to fully set up a Project in the article. If you are interested in this then please check the respective documentation in the resource links at the end of this blog).

In the Project Asset View (1) pick Formats (2) and then ASN.1 Format (3).

Create ASN.1 Asset

(Image by

The Asset editor has two tabs “Configuration” and “Modules”. On the “Configuration” tab (1) we name the Asset “TAP3.11” (2):

Naming the new Asset

(Image by

Define Message Types

TAP3.11 defines a number of different “root” message types. We are only going to identify three of them to work with in this example:

Available Message Types

(Image by

In layline.io we map them to “Header”, “Detail”, and “Trailer” respectively it looks like this (1):

Defining Message Types in layline.io

(Image by

We are later going to debug output the data we read. Because some data is BCD-encoded and for readability’s sake we use a built-in converter to decode them and make them readable:

BCD En-/Decoding

(Image by

Adding the TAP3.11 grammar

We are now ready to add the actual TAP3.11 grammar. Switch to the Modules tab (2) of the ASN.1 Asset we have added (1).

An ASN.1 grammar can be made-up of multiple modules. In our case it is just one. We have added it and named it “TAP.311-Module” (3/4).

Adding ASN.1 grammar module

(Image by

You can find the original ASN.1 grammar for TAP3.11 in the written specification:

GSMA's TD.57 TAP3.11 ASN.1 Grammar Notation

(Image by

Let’s copy and paste it to our ASN.1 Asset (1) (it’s about 1,422 lines):

Pasting ASN.1 Grammar into layline.io

(Image by

Interactive Testing of the ASN.1 Grammar

layline.io provides the ability to instantly check whether format setup matches the actual data. You can upload a sample file (1) directly into the Configuration Center (2) to perform a cross-check.

Uploading ASN.1 Sample File

(Image by

Let’s do this (2 above):

Viewing ASN.1 Sample File

(Image by

layline.io will instantly

1. Generate a parser for the ASN.1 grammar
2. Parse the sample file
3. Feed the result back (3 & 4)

The file content will be displayed in a token tree (3) as well as in a hex-dump style display (4). You can mark individual tokens (3) to see the corresponding file content highlighted (4).

Switch to the “Sample Messages” tab (2) where we can get a more readable view on the file content:

Decoded ASN.1 Sample File Viewing

(Image by

Interactive testing means, that you now could go back to the grammar, change it, and then instantly see how this affects how this affects the interpretation of your sample data.

Sample Project

We put together a complete sample Project together with test-data which is ready to run in layline.io. The example simply reads and writes a TAP file and outputs its contents to the console in a JSON-format:

Sample Project TAP3 Workflow

(Image by

Sample Project Debug Output

(Image by

Any other complex operation on the data would work as well, of course and is likely to be what would be happening in a real Project context. But for the purpose of our sample, the above will suffice.

You can find the sample Project in the resource links at the end of this article.

Summary

Using layline.io you can not only define complex, distributed, scalable event-processing scenarios, but pretty much work with any data format. In this example we demonstrated how to deal with ASN.1 structured data.

Most other solutions will support specific formats only, as their parsers have been hard-coded to match a specific use-case. In layline.io there are literally millions of use-cases. That’s why formats are never hard-coded, but configurable.

There are many popular ASN.1 protocols around the world of which we would like to list some below. And there are - of course - countless custom ASN.1 protocols as well.

Protocol

• Ericsson AIR / AXE10 / CCN / CCR / MTAS / R3 / R4 / R7 / R8 / R9 / R10 / R11 / R12 / R13 / SDP / Tele2 TIP
• Huawei IMS
• 4G UMTS / 5G IMS / 3GPP
• MetaSolv GPRS
• TD.35 NRTRDE - Near Real Time Roaming Data Exchange
• TD.57 / TAP2 / TAP3 Transferred Account Procedure
• Interledger Protocol
• NTCIP 1103 - Transport Management Protocols
• X.500 Directory Services
• Lightweight Directory Access Protocol (LDAP)
• PKCS Cryptography Standards
• X.400 Message Handling
• T.120 Multimedia conferencing
• Simple Network Management Protocol (SNMP)
• Common Management Information Protocol (CMIP)
• Signalling System No. 7 (SS7)
• ITU H-Series Multimedia Protocols
• BioAPI Interworking Protocol (BIP)
• Common Biometric Exchange Formats Framework (CBEFF)
• Authentication Contexts for Biometrics (ACBio)
• Computer-supported telecommunications applications (CSTA)
• Dedicated short-range communications (DSRC)
• IEEE 802.11p (IEEE WAVE)
• Intelligent Transport Systems (ETSI ITS)
• Global System for Mobile Communications (GSM)
• General Packet Radio Service (GPRS) / Enhanced Data rates for GSM Evolution (EDGE)
• Universal Mobile Telecommunications System (UMTS)
• Long-Term Evolution (LTE)
• Common Alerting Protocol (CAP)
• Controller–pilot data link communications (CPDLC)
• Space Link Extension Services (SLE)
• Manufacturing Message Specification (MMS)
• File Transfer, Access and Management (FTAM)
• Remote Operations Service Element protocol (ROSE)
• Association Control Service Element (ACSE)
• Building Automation and Control Networks Protocol (BACnet)

More

There is a lot more to tell here. For example what the hell is layline.io anyway? In the links below you can learn about layline.io and also dive deeper into the documentation of how this specific grammars work in layline.io.

You can also grab your free copy of layline.io from our website and work with a sample Project which you can also download.

Thanks for reading!

Resources

• Read more about layline.io here.
• Contact us at hello@layline.io.