db-vendo-client/docs/writing-a-profile.md

Writing a profile

Per endpoint, hafas-client has an endpoint-specific customisation called profile which may for example do the following:

• handle the additional requirements of the endpoint (e.g. authentication),
• extract additional information from the data provided by the endpoint,
• guard against triggering bugs of certain endpoints (e.g. time limits).

This guide is about writing such a profile. If you just want to use an already supported endpoint, refer to the API documentation instead.

Note: If you get stuck, ask for help by creating an issue! We want to help people expand the scope of this library.

0. How do the profiles work?

A profile contains of three things:

• mandatory details about the HAFAS endpoint
  • endpoint: The protocol, host and path of the endpoint.
  • locale: The BCP 47 locale of your endpoint (or the area that your endpoint covers).
  • timezone: An IANA-time-zone-compatible timezone of your endpoint.
• flags indicating that features are supported by the endpoint – e.g. journeyRef
• methods overriding the default profile

As an example, let's say we have an Austrian endpoint:

const myProfile = {
	endpoint: 'https://example.org/bin/mgate.exe',
	locale: 'de-AT',
	timezone: 'Europe/Vienna'
}

Assuming the endpoint returns all lines names prefixed with foo , We can strip them like this:

// get the default line parser
const createParseLine = require('hafas-client/parse/line')

const createParseLineWithoutFoo = (profile, opt, data) => {
	const parseLine = createParseLine(profile, opt, data)

	// wrapper function with additional logic
	const parseLineWithoutFoo = (l) => {
		const line = parseLine(l)
		line.name = line.name.replace(/foo /g, '')
		return line
	}
	return parseLineWithoutFoo
}

profile.parseLine = createParseLineWithoutFoo

If you pass this profile into hafas-client, the parseLine method will override the default one.

1. Setup

Note: There are many ways to find the required values. This way is rather easy and has worked for most of the apps that we've looked at so far.

1. Get an iOS or Android device and download the "official" app for the public transport provider that you want to build a profile for.
2. Configure a man-in-the-middle HTTP proxy like mitmproxy.
   • Configure your device to trust the self-signed SSL certificate, as outlined in the mitmproxy docs.
   • Note: This method does not work if the app uses public key pinning. In this case (the app won't be able to query data), please create an issue, so we can discuss other techniques.
3. Record requests of the app.
   • There's a video showing this step.
   • Make sure to cover all relevant sections of the app, e.g. "journeys", "departures", "live map". Better record more than less; You will regret not having enough information later on.
   • To help others in the future, post the requests (in their entirety!) on GitHub, e.g. in as format like this. This will also let us help you if you have any questions.

2. Basic profile

• Identify the endpoint. The protocol, host and path of the endpoint, but not the query string.
  • Note: hafas-client for now only supports the interface providing JSON (generated from XML), which is being used by the corresponding iOS/Android apps. It supports neither the JSONP, nor the XML, nor the HTML interface. If the endpoint does not end in mgate.exe, it mostly likely won't work.
• Identify the locale. Basically guess work; Use the date & time formats as an indicator.
• Identify the timezone. This may be tricky, a for example Deutsche Bahn returns departures for Moscow as +01:00 instead of +03:00.
• Copy the authentication and other meta fields, namely ver, ext, client and lang.
  • You can find these fields in the root of each request JSON. Check a VBB request and the corresponding VBB profile for an example.
  • Add a function transformReqBody(body) to your profile, which assigns them to body.
  • Some profiles have a checksum parameter (like here) or two mic & mac parameters (like here). If you see one of them in your requests, jump to Appendix A: checksum, mic, mac. Unfortunately, this is necessary to get the profile working.

You may want to use the profile boilerplate code.

3. Products

In hafas-client, there's a difference between the mode and the product field:

• The mode field describes the mode of transport in general. Standardised by the Friendly Public Transport Format 1.0.1, it is on purpose limited to a very small number of possible values, e.g. train or bus.
• The value for product relates to how a means of transport "works" in local context. Example: Even though S-Bahn and U-Bahn in Berlin are both trains, they have different operators, service patterns, stations and look different. Therefore, they are two distinct products subway and suburban.

Specify products that appear in the app you recorded requests of. For a fictional transit network, this may look like this:

const products = [
	{
		id: 'commuterTrain',
		mode: 'train',
		bitmasks: [16],
		name: 'ACME Commuter Rail',
		short: 'CR',
		default: true
	},
	{
		id: 'metro',
		mode: 'train',
		bitmasks: [8],
		name: 'Foo Bar Metro',
		short: 'M',
		default: true
	}
]

Let's break this down:

• id: A sensible, camelCased, alphanumeric identifier. Use it for the key in the products array as well.
• mode: A valid Friendly Public Transport Format 1.0.1 mode.
• bitmasks: HAFAS endpoints work with a bitmask that toggles the individual products. It should be an array of values that toggle the appropriate bit(s) in the bitmask (see below).
• name: A short, but distinct name for the means of transport, just precise enough in local context, and in the local language. In Berlin, S-Bahn-Schnellzug would be too much, because everyone knows what S-Bahn means.
• short: The shortest possible symbol that identifies the product.
• default: Should the product be used for queries (e.g. journeys) by default?

If you want, you can now verify that the profile works; We've prepared a script for that. Alternatively, submit a Pull Request and we will help you out with testing and improvements.

Finding the right values for the bitmasks field

As shown in the video, search for a journey and toggle off one product at a time, recording the requests. After extracting the products bitmask (example) you will end up with values looking like these:

toggles                     value  binary  subtraction     bit(s)
all products                31     11111   31 - 0
all but ACME Commuter Rail  15     01111   31 - 2^4        2^4
all but Foo Bar Metro       23     10111   31 - 2^3        2^3
all but product E           25     11001   31 - 2^2 - 2^1  2^2, 2^1
all but product F           30     11110   31 - 2^0        2^0

4. Additional info

We consider these improvements to be optional:

• Check if the endpoint supports the journey legs call.
  • In the app, check if you can query details for the status of a single journey leg. It should load realtime delays and the current progress.
  • If this feature is supported, add journeyLeg: true to the profile.
• Check if the endpoint supports the live map call. Does the app have a "live map" showing all vehicles within an area? If so, add radar: true to the profile.
• Consider transforming station & line names into the formats that's most suitable for local users. Some examples:
  • M13 (Tram) -> M13. With Berlin context, it is obvious that M13 is a tram.
  • Berlin Jungfernheide Bhf -> Berlin Jungfernheide. With local context, it's obvious that Jungfernheide is a train station.
• Check if the endpoint has non-obvious limitations and let use know about these. Examples:
  • Some endpoints have a time limit, after which they won't return more departures, but silently discard them.

---

Appendix A: checksum, mic, mac

As far as we know, there are three different types of authentication used among HAFAS deployments.

unprotected endpoints

You can just query these, as long as you send a formally correct request.

endpoints using the checksum query parameter

checksum is a message authentication code: hafas-client will compute it by hashing the request body and a secret salt. This secret can be read from the config file inside the app bundle. There is no guide for this yet, so please open an issue instead.

endpoints using the mic & mac query parameters

mic is a message integrity code, the hash of the request body.

mac is a message authentication code, the hash of mic and a secret salt. This secret can be read from the config file inside the app bundle. There is no guide for this yet, so please open an issue instead.